AI Strategy7 min read

AI Architecture and Governance Terms for Business Leaders

Part 5 of a plain-English AI terms series: RAG, fine-tuning, inference, cloud vs edge, data residency, human-in-the-loop, AI use policies and AI KPIs.

Haojun See

Founder & Director, On The Ground

Updated 22 October 2026

Why architecture and governance terms matter

Leaders don't need to design AI systems, but they do sign off on the choices inside them: where data goes, who approves what, and how success is measured. This is the final part of our plain-English series on AI terms. The terms follow the vocabulary of the Singtel AI.dea – AI bridge programme, run with SIM Academy (Programme Architect: Michelle Yeo), which includes legal input from Drew & Napier. The explanations are our own and are not legal advice.

Training, fine-tuning and inference

Training builds a model from scratch on huge amounts of data. Fine-tuning adjusts an existing model on a smaller, specific set of examples. Inference is simply using a model to get an answer. Training a large model costs far more than an SME will ever spend, so it is left to the big AI labs. Fine-tuning is within reach when you need a model to follow a particular style or task consistently. Almost everything a business does day to day is inference: every prompt you type is one. Related is the context window: how much text the model can consider at once, including your instructions and any documents you give it. Ask: does this project really need fine-tuning, or would good instructions and the right documents do the job?

RAG: answering from your own documents

Retrieval-augmented generation (RAG) makes an AI look up relevant passages in documents you choose, then answer from them. A staff assistant that answers questions from your HR handbook and SOPs is a typical example. Web search inside an AI chat tool works the same way, with the web as the document library. RAG is usually cheaper and easier to keep current than fine-tuning: update the document and the answers change. It reduces made-up answers but doesn't remove them, and it is only as good as the documents behind it. Ask: which documents will it answer from, who keeps them up to date, and can users see the source behind each answer?

Cloud AI, edge AI and hybrid

Cloud AI runs in a provider's data centre. Edge AI runs on a device on your premises, such as a camera, a machine or a local server. Hybrid uses both. Cloud is easy to start, scales up and down, and is paid for by usage, but data leaves your premises and every request travels over the internet. Edge responds faster, keeps working offline and keeps data on site, but needs hardware bought up front. Many SMEs end up hybrid: quick decisions made on site, heavier analysis in the cloud. Ask: does this need an instant response, must the data stay on our premises, and what will each option cost over three years?

Data residency: where your data actually lives

Data residency is the physical location where your data is stored and processed. Many AI tools process data overseas, and so may their subcontractors, even when the vendor is local. This matters for three reasons: contracts, especially government and enterprise ones, may require data to stay in Singapore; the PDPA has specific rules for transferring personal data overseas; and data held abroad falls under that country's laws too. Ask: in which countries will our data be stored and processed, including by the vendor's subcontractors, and does that fit our contracts and the PDPA?

Human-in-the-loop: who approves what

Human-in-the-loop means a named person reviews or approves AI output at set points before it takes effect. The more an AI can act on its own, the more this matters, especially for payments, prices, hiring decisions and anything a customer will rely on. Singapore's Model AI Governance Framework for Agentic AI, launched by IMDA in January 2026 and updated in May 2026, puts human accountability at the centre of deploying AI agents. Ask: at which points does a person approve this AI's work, and is it one named person rather than a department?

AI use policy: rules people can actually follow

An AI use policy tells staff which AI tools they may use, for what, and with which data. The useful ones are short and specific. They say which tools are approved, which data stays out of public tools, which decisions always need a person, and when customers should be told AI was involved, and they have one owner who updates them as tools change. Under the PDPA, mishandling personal data can lead to fines of up to S$1 million, or 10% of annual turnover in Singapore for organisations with turnover above S$10 million. A policy staff have actually read is one of the cheapest controls available. Ask: if a staff member read our policy today, would they know exactly what they can and can't do with AI tomorrow?

AI KPIs: leading and lagging indicators

Lagging indicators confirm results after the fact, such as cost per order or revenue. Leading indicators show early whether you are on track, such as how many staff use the tool in their actual workflow. Good AI scorecards mix both and add guardrails, such as accuracy, error rates and how often people override the AI. A weak KPI is vague or counts activity: 'everyone uses AI weekly'. A strong one names the process, the baseline, the target, the deadline and the guardrail: issue quotations within one working day instead of three by March, with no rise in quoting errors. Ask: what will we measure in the first month to know this is working, and what must not get worse while it does?

The full AI terms series

That completes the series. The five parts are: • Part 1: Hyperscalers, agents and observability • Part 2: Types of AI explained • Part 3: AI risk terms • Part 4: AI strategy terms • Part 5: AI architecture and governance terms (this post) To learn these with your own team, you can register your interest on the AI.dea programme page.

Frequently asked questions

What is the difference between RAG and fine-tuning?

RAG lets an AI look up passages in documents you choose and answer from them, so updating a document updates the answers. Fine-tuning changes the model itself using a set of examples, which suits a consistent style or task. For most business Q&A, RAG is cheaper, faster to set up and easier to keep current.

What is data residency and why does it matter for AI?

Data residency is the physical location where data is stored and processed. Many AI tools process data overseas, sometimes through subcontractors. This matters because contracts may require data to stay in Singapore, the PDPA has rules for overseas transfers of personal data, and data held abroad is also subject to that country's laws.

What should an AI use policy include?

A practical AI use policy lists approved tools and how to request new ones, names the data that must never go into public AI tools, sets out decisions AI may draft but not make, explains when AI use must be disclosed, and names one owner who reviews the policy regularly. It should be short enough for staff to read.

AI Strategy

Types of AI Explained: Predictive, Generative, Vision and More

AI Strategy

AI Terms for Leaders: Hyperscalers, Agents and Observability

AI Strategy

AI Strategy Terms: Ambition, Readiness, Quick Wins and RICE

PreviousAI Risk Terms: Hallucination, Workslop, Shadow AI and More Next GeBIZ Tenders with AI: What Singapore SMEs Should Automate