The short answer
A general AI chatbot answers from training and knows nothing about your leave policy, product specifications or contracts. Retrieval-augmented generation, usually shortened to RAG, adds a search step: the system finds the most relevant passages in your approved documents and the model answers from them, with citations. Done well, a long search becomes a cited answer in seconds. It reduces made-up answers but does not eliminate them.
How RAG works, step by step
- CollectApproved policies, manuals, procedures, contracts from SharePoint, file shares, document systems, databases.
- Split and indexPassages indexed by words and embeddings, with permissions, owner and date.
- RetrieveThe most relevant passages, filtered to what this person may see.
- GenerateThe model answers only from those passages, citing them.
- Show sourcesLinks to the passages used, so people can check the original.
Most quality comes from steps 1 to 3: if the right passage is never retrieved, the best model in the world cannot use it.
More detail: embeddings and the known challenges
An embedding is a numerical representation of meaning that lets "time off" match a question about "annual leave". Microsoft's Azure AI Search documentation describes the same pattern as grounding a model's responses in your own content, with challenges that include understanding vague questions, reaching content across several systems, sending only the most relevant passages and enforcing access control (Microsoft Learn).
Why it reduces made-up answers, but does not eliminate them
The model works from your text rather than memory, and citations let people check. OWASP lists RAG as a way to reduce hallucination and misinformation, alongside cross-checking trusted sources and human oversight (OWASP). It still fails in predictable ways:
Wrong passage
Last year's version of a policy, or a similar clause from another contract.
Missed passage
Different wording in the question, or an answer spread across documents.
Filled gaps
The model blends in general knowledge or misreads a table.
Bad source
Wrong or outdated text is faithfully repeated.
Permissions and keeping sources current
An assistant that searches everything for everyone is a data leak with a friendly interface. OWASP warns weak retrieval access controls can reveal sensitive content to the wrong user, and documents can carry hidden instructions; it recommends permission-aware retrieval and validating sources before they are added (OWASP). In practice:
- The source system's access rights apply, checked on every question.
- HR files, salary data and legal matters are excluded or separately controlled.
- Documents are checked before indexing, including for hidden text.
- Sources are re-indexed on a schedule or on change; retired documents are removed.
- Every source has an owner; admins see what is included and when updated.
Stale content is the quality problem we see most: two versions of one procedure, both indexed, giving two answers.
When Microsoft 365 Copilot is enough
Microsoft 365 Copilot grounds answers in your emails, chats and documents, and Microsoft states that it only surfaces organisational data each user has at least view permission to see (Microsoft Learn). So tidy permissions before rollout, as our Copilot readiness guide explains.
| Situation | Usually the better fit |
|---|---|
| Content in SharePoint, OneDrive and Teams; staff want help across everyday work | Microsoft 365 Copilot, after a permissions clean-up |
| Answers only from an approved set of documents | A Copilot agent scoped to those sources, or a custom assistant |
| Content outside Microsoft 365, in an ERP, document system or database | A custom assistant connected to those systems |
| Assistant inside your own portal, intranet or application | A custom assistant |
| Control over the model, where data is processed (for example, keeping certain data in Nigeria) and what is logged | A custom assistant |
Many organisations use both: Copilot for everyday productivity, a custom assistant for one high-value document collection.
Rather talk it through? If your staff keep asking the same questions of the same documents, we can show you what a cited, permission-aware assistant would look like on your own content. Talk to a Promatics specialist
What good evaluation looks like
- Real questions collected from future users, including awkward and ambiguous ones
- Expected answer and source agreed per question by a business owner
- Scored for right answer, right citation and an honest "not found"
- Testers cannot get answers from documents they may not open
- Prompt injection, other users' data and off-topic requests tested
- Accuracy threshold and go or no-go decision agreed before testing
- After launch: answers sampled; re-run when sources, prompts or model change
"Not found" questions are useful in their own right: they show where your documentation has gaps.
Typical first use cases
A policy library
HR, finance, travel and IT policy answers linked to the clause. High volume, low risk, easy to check.
Manuals and procedures
Specifications, troubleshooting and procedures, without chasing the one person who knows.
Contract clauses
Renewal, termination, liability or privacy clauses quoted and linked; a lawyer still interprets.
When to bring in help
A small off-the-shelf trial on non-sensitive documents is a fine way to learn; with content in Microsoft 365 and tidy permissions, Copilot may be all you need. Bring in a professional when the assistant reaches confidential or personal data, permissions are complex, sources span several systems, or people will act on its answers. The Nigeria Data Protection Act 2023 (NDPA) applies to personal data in the documents an assistant searches. Most hosted AI services process data outside Nigeria, so sending passages to the model is usually a transfer needing a lawful basis and often a data protection impact assessment under the NDPC's GAID 2025 (see AI and the NDPA; general information, not legal advice).
Our generative AI development service defines questions, sources and permissions with you, builds access-aware retrieval that cites every answer, tests it on your real questions before launch and monitors accuracy and running costs (usually billed in US dollars) after: an assistant staff can check, and a team accountable for keeping it right. RAG works the same way with an open-weight model on your own servers when documents must not leave your environment.
Sources and further reading
Product capabilities and guidance change. These are the primary sources this article relies on, checked on the review date above.
- RAG and generative AI (Azure AI Search), Microsoft Learn
- Data, privacy and security for Microsoft Copilot, Microsoft Learn
- LLM08:2025 Vector and Embedding Weaknesses, OWASP Gen AI Security Project
- LLM09:2025 Misinformation, OWASP Gen AI Security Project
This article is general information, not legal, accounting or security advice for your specific situation. Examples are hypothetical unless stated otherwise.