On this page
An AI assistant that works with your own data is a system that finds the relevant section in your company’s documents before answering a question, and bases its answer on that section. What sets it apart from a general chat tool is that it “knows” your price list, your procedures and your past work, and can show which document an answer came from.
These assistants are among the most requested AI applications right now. They’re also among the most disappointing. The reason usually isn’t the technology but questions nobody asked before the build: which questions will it answer, which documents will it look at, who will be able to see what?
Why doesn’t a general language model know your company?
General-purpose language models are trained on very large amounts of public text. Your internal procedure, your current price list and the returns policy that changed last month aren’t in that text. Asked about them, the model either speaks in generalities or invents a plausible but wrong answer.
The answer isn’t retraining the model. For most businesses, the right method is to show the model the relevant documents at the moment a question is asked. In technical literature this is called retrieval-augmented generation (RAG). The logic is simple: search first, then answer.
How does the assistant work behind the scenes?
When a question comes in, the system follows three steps:
- Search. The question is searched against an index prepared from your documents, and the most relevant sections are found.
- Context. Those sections are passed to the language model together with the question. The model is asked to answer only on the basis of those sections.
- Answer and source. The model produces the answer and shows which document it used. If the answer isn’t in the documents, it’s expected to say “I don’t know”.
This has two practical benefits: answers are grounded in your real content, and because the source is shown, they can be checked. When you update a document, the assistant’s knowledge updates too; there’s no need to retrain the model.
Off-the-shelf tool or custom build?
The business versions of some general-purpose assistants let you upload documents and answer from them. For a small team and a limited number of documents this is often enough, and much faster. A custom build makes sense when one of the situations in the right-hand column below applies.
| Situation | Off-the-shelf tool is enough | Custom build needed |
|---|---|---|
| Number of documents | Few, rarely changing | Many, frequently updated |
| Access rights | Everyone can see everything | Different access by department or role |
| Data source | Files can be uploaded by hand | Documents live in different systems and need automatic syncing |
| Where it’s used | In the assistant’s own interface | Inside your existing software, website or messaging tool |
| Data sensitivity | General internal information | Data must not leave the organisation |
If the left-hand column describes you, I’d suggest trying an off-the-shelf tool first. Custom development should only happen when it’s genuinely needed.
Building it in seven steps
1. Write down the scope and a list of questions
List the real questions the assistant will answer. Collecting the questions the team has asked over the past few weeks is a good start. The list sets the scope and later doubles as the test set.
2. Take a data inventory
List the documents the answers live in. At this stage, weed out documents that are out of date, contradict each other or exist in several versions. The assistant can’t be more accurate than the documents it’s given.
3. Decide access and permissions
Should everyone see every document? Should a finance document appear as a source when the sales team asks something? Permission rules are hard to add later; they should be designed from the start. This is also when it’s written down which data will never be included. The data classification in How to Write an AI Usage Policy for Your Team is a good framework for this.
4. Prepare and index the documents
Documents are split into sections of a size that makes search accurate and added to a searchable index. Documents with proper headings and clear sections give much better results at this step. How the index is updated as documents change is set up here too.
5. Test with a narrow prototype
The first version should be narrow: one department, a limited set of documents. Test it with the question list from step one and judge each answer three ways: is it right, is the source shown, and if the answer isn’t in the documents, does the assistant say so?
6. Design in sources and human approval
Language models can give wrong answers in a confident tone; it’s a known characteristic of the technology. Showing the source makes it easier for the user to check. Where answers go to customers or lead to critical decisions, a human approval step belongs in the flow.
7. Roll out, measure and hand over
Once the assistant is live, track which questions come in and which ones it can’t answer well. Unanswered questions usually point to a missing or outdated document. Code, accounts and documentation should stay with you so that your team, or another developer, can keep it going.
Common mistakes
- Not tidying the documents first. An assistant built on messy, contradictory documents gives messy, contradictory answers.
- Trying to cover everything at once. A build that covers the whole company in one go can’t be tested. Start with one department.
- Skipping the test set. Nobody can tell whether an assistant works well if it isn’t measured against real questions.
- Not allowing “I don’t know”. When the answer isn’t in the documents, the assistant saying so isn’t a bug; it’s a condition of trust.
Frequently asked questions
Will our data be used to train the model?
That depends on the tool chosen and its contract. Some business versions commit not to use submitted data for model training; where data is sensitive, setups that keep data inside your organisation can also be considered. We decide this together before any code is written, based on the type of data and your legal obligations.
What happens if the assistant gives a wrong answer?
The possibility can’t be removed entirely, but it can be managed. Because sources are shown, users can compare the answer with the document. For critical uses, a human approval step stops a wrong answer going straight to a customer or into a decision.
How long does the build take?
A narrow prototype usually takes a few weeks. What really sets the timeline isn’t development but the state of the data: if the documents are organised, it’s shorter; if they’re scattered, most of the time goes into tidying them.
How do we maintain it without a technical team?
Adding or updating documents can be designed so it needs no technical knowledge. At handover I explain how to do it and leave written documentation. Because the code and accounts are yours, you can also continue with another developer later.
Planning your own assistant
Let’s check whether an off-the-shelf tool is enough first
On the Custom AI Solutions page I explain the development process, my ownership principles and when I don’t recommend custom development.
Updated:


