A language model on its own answers from what it absorbed in training. It has never seen your policies, your contracts or last week’s price change. Ask it about them and it will produce something plausible. That is the root of most wrong answers people call “hallucinations”.
Retrieval: give it the page before it answers
Retrieval-augmented generation, usually shortened to RAG, changes the order of events. When a question arrives, the system first searches your own documents for the relevant passages, then asks the model to answer using only those passages, and to show which ones it used.
Three things improve at once. Answers reflect your current information. Each answer can cite its source, so a person can check it. And the system can be instructed to say “I don’t have that” when the search comes back empty.
Where plain retrieval struggles
Search finds passages that look similar to the question. It is weaker when the answer depends on how things relate. “Which of our suppliers are affected by this regulation, through any subsidiary?” has no single paragraph that answers it. The answer lives in the connections between documents.
GraphRAG: retrieval that follows relationships
A knowledge graph stores facts as things and the links between them: this policy covers that product, this person approved that exception, this company owns that one. GraphRAG lets the system walk those links while gathering evidence, so it can answer questions that span several documents and show which facts and links the answer was built from.
Which one do you need?
- Questions answered by one passage in one document: plain retrieval is enough, and simpler to run.
- Questions about relationships, ownership, dependencies or history: a graph usually pays for itself.
- Regulated settings where you must show how an answer was reached: the graph’s explicit path is a real advantage.
What grounding does not do
It reduces wrong answers; it does not remove them. A grounded system can still misread a passage or retrieve the wrong one. That is why the test set matters as much as the architecture: a fixed list of questions, including ones the documents cannot answer, scored every time something changes.
Be wary of anyone who says their system cannot be wrong. Ask instead how they test it, and what it does when it does not know.
