GraphRAG vs Vector RAG: The Question Types Decide, Not the Technology

What does each approach actually retrieve?

Both approaches answer the same question shape — "what should the model see" — and they fill the context window with structurally different things.

The last two rows are the ones that matter commercially. Vector RAG fails visibly, because a retrieved passage that does not support the claim is usually detectable by a reader. Graph RAG can fail invisibly: a mis-resolved entity produces a graph in which a false relationship is a first-class fact, and the traversal returns it with complete confidence. Graph quality therefore has to be a governed artefact with an owner, not a build step someone ran once.

Where does vector RAG genuinely fail?

Vector RAG does not fail because the embedding is weak. It fails for structural reasons, and the diagnosis matters because each failure has a specific fix.

• The evidence is not in one passage. The answer requires combining a contract term with a schedule and an amendment. Each piece may be individually retrievable, but the ranking that surfaces the most semantically similar piece buries the others. This is a ranking failure, and reranking and fusion address it — not a graph.
• The question needs a count. "How many" cannot be answered from the top-k passages, because counting requires the whole set. This is the clearest case for a structured or graph retrieval path.
• The relation is implicit. Two documents refer to the same entity without naming it identically, so no passage contains both. Nothing to retrieve, because the bridge is not text. This is the case where a graph genuinely wins — provided the entity resolution was done properly.
• The corpus is too large to retrieve from. At a certain scale, top-k over a vector index is dominated by the most semantically similar and least authoritative documents. A structured filter — by entity, by date, by document type, by permission — has to come before the similarity ranking, and that is a hybrid design.
• Authority is wrong. The most similar passage is a superseded version, a draft, or a document from a business unit that does not own the policy. No retrieval tuning fixes a missing authority model. This is a governance problem that the semantic layer solves.

That last point is worth dwelling on, because it recurs. A large share of enterprise "RAG is not accurate enough" cases are not retrieval problems at all. They are the absence of a single source of truth, expressed as a retrieval symptom. Adding a graph to that estate adds machinery on top of an unresolved semantic question, which is why we always diagnose before recommending architecture.

What does a hybrid retrieval setup look like?

Most production estates we work on are hybrid, and the pattern is consistent enough to describe.

1. Classify the question first. A router — which can itself be a small model — decides whether the question is a lookup, a relationship question, an aggregation, or a synthesis. This is the single highest-leverage component in a hybrid system, because it determines which path runs.
2. Run structured filters before similarity. Entity, date, document type, jurisdiction, permission. Filtering first prevents a semantically perfect but unauthoritative document from ever reaching the candidate set, and it makes permission enforcement exact rather than approximate.
3. Run the retrieval paths in parallel. Vector search for the semantic neighbourhood, graph traversal for the relationship path, and a structured query for anything countable or filtered.
4. Fuse and rerank. Merge the candidate sets with a reciprocal-rank style fusion, then rerank the merged set with a scoring model, so the final context window contains the best evidence from all paths rather than the union of them.
5. Keep the trace. Record which path produced which passage. This is what makes a bad answer diagnosable, and it is what tells you over time whether the graph path is earning its maintenance cost.

Step five is where most organisations fail, and it is what makes hybrid systems hard to operate. Without per-path provenance you cannot tune the system, retire an unproductive path, or defend the cost of the graph index. The knowledge and context capability page describes how we keep retrieval provenance and source traceability attached to every answer in delivery.

Where do we draw the boundary?

We do not recommend a graph architecture because it is the sophisticated choice, and we will decline to build one where a tuned vector pipeline answers the workload. GraphRAG earns its cost on multi-hop, relationship, dependency and aggregation questions, and on nothing else. If your question set does not contain those, the honest recommendation is to tune what you have — which is the majority of the estates we assess in Singapore, Hong Kong SAR and across Asia Pacific.

We also do not treat a graph as a substitute for a settled definition of the business. A graph built over an ambiguous corpus will faithfully encode the ambiguity in a form that is harder to spot, because the structure lends it an authority it has not earned. Where definitions are unresolved, the work is semantic, and it belongs with the business teams — a point we make at length in our article on what enterprise ontology is and why agents depend on it.

And we are clear about the failure we see most often: an enterprise adopts a graph retrieval approach, sees the multi-hop question type improve, and concludes the whole retrieval problem is solved. It is not. The single-document questions, the authority questions and the permission questions are still where most of the failures live.