What Is GraphRAG, and Why Does Compressing the Graph Matter?

GraphRAG is a way of answering questions with a language model by first organising your documents into a knowledge graph of entities and relationships, then retrieving from that graph. It handles questions that need several facts connected together, but the graph is costly to build and to query, which is why compressing it is worth studying.

This is the area of my own research, so this article explains the idea in plain terms and then says why the cost problem interests me.

What is ordinary RAG?

Retrieval-augmented generation, or RAG, is the common way to make a language model answer from your own documents. You split the documents into chunks, turn each chunk into a vector, and store them. When a question arrives, you find the chunks whose vectors are closest to the question and hand them to the model along with the question.

This works well when the answer sits inside one or two chunks. Ask what the refund policy is, and the chunk about refunds comes back.

Where does ordinary RAG struggle?

It struggles when the answer is spread out. Suppose you ask which suppliers are affected by a delay at a particular factory. One document says the factory makes a component. Another says which products use that component. A third lists which suppliers depend on those products. No single chunk contains the answer, and the chunks that do matter may not look similar to the question at all.

It also struggles with questions about the whole collection, such as asking for the main themes across a thousand reports. Similarity search returns a handful of passages. It does not return an overview.

How does GraphRAG help?

GraphRAG adds structure before retrieval. A model reads the documents and extracts entities, such as people, companies, products and places, along with the relationships between them. Those become the nodes and edges of a graph.

At question time the system can follow edges. Starting from the factory, it steps to the component, then to the products, then to the suppliers. This is often called multi-hop retrieval, and it is exactly what the similarity search could not do.

Some approaches go further and group the graph into communities of closely related nodes, then write a summary for each group. Those summaries are what let the system answer broad questions about the whole collection.

Why is it expensive?

Building the graph means running a language model over every document to extract entities and relationships, which costs far more than computing vectors. The graph that comes out is large, and much of it is redundant: the same entity under slightly different names, edges that say almost the same thing, nodes that are never useful for any question.

The cost returns at query time. The piece of the graph you retrieve has to be turned into text and placed in the model's context. A bigger subgraph means more tokens, a slower answer and a higher bill, and past a certain point the extra context makes the answer worse instead of better.

Why compress the graph?

If a smaller graph can answer the same questions, everything downstream gets cheaper. The interesting question is what to remove. Delete too little and you save nothing. Delete the wrong edge and a multi-hop answer silently breaks, because the path it depended on is gone.

A fixed rule, such as dropping every node with few connections, is easy to write and easy to get wrong, since a rarely connected node can still be the only bridge between two parts of the graph. My research looks at making that decision adaptive, using reinforcement learning to learn which parts of a graph can be merged or pruned while keeping answer quality. It is work in progress, and I will write about results when there are results to show.

There is a small interactive toy of the idea in the lab section of this site. It is not the research itself, only a way to feel the trade-off: remove nodes and watch what still connects.

Should you use GraphRAG?

If your questions are simple lookups, ordinary RAG is cheaper and usually enough. If your users ask questions that connect several facts, or want summaries across a large collection, a graph earns its cost. Start with the simpler system, collect the questions it fails on, and let those failures tell you whether you need the graph.