GraphRAG in Practice: How to Build Cost-Efficient, High-Recall Retrieval Systems

article, Do You Really Need GraphRAG? A Practitioner’s Guide Beyond the Hype, I outlined the core ideas of GraphRAG design and launched an augmented retrieval-and-generation pipeline that mixes graph search with vector search. I additionally mentioned why constructing a superbly full graph—one which captures each entity and relation within the corpus—will be prohibitively advanced, particularly at scale.

On this article, I develop on these concepts with concrete examples and code, demonstrating the sensible constraints encountered when constructing and querying actual GraphRAG techniques. I additionally illustrate that the retrieval pipeline helps stability price and implementation complexity with out sacrificing accuracy. Particularly, we are going to cowl:

1. Constructing the graph: Ought to entity extraction occur on chunks or full paperwork—and the way a lot does this selection really matter?
2. Querying relations with no dense graph: Can we infer significant relations utilizing iterative search-space optimisation as an alternative of encoding each relationship within the graph explicitly?
3. Dealing with weak embeddings: Why alphanumeric entities break vector search and the way graph context fixes it.

GraphRAG pipeline

To recall from the earlier article, the GraphRAG embedding pipeline used is as follows. The Graph node and relations and their embeddings are saved in a Graph database. Additionally, the doc chunks and their embeddings are saved within the database.

The proposed retrieval and response technology pipeline is as follows:

As will be seen, the graph outcome just isn’t immediately used to reply to consumer question. As an alternative it’s used within the following methods:

1. Node metadata (significantly doc_id) acts as a robust classifier, serving to establish the related paperwork earlier than vector search. That is essential for big corpora the place naive vector similarity can be noisy.
2. Context enrichment of the consumer question to retrieve probably the most related chunks. That is essential for sure kinds of question with weak vector semantics akin to IDs, car numbers, dates, and numeric strings.
3. Iterative search area optimisation, first by choosing probably the most related paperwork, and inside these, probably the most related chunks (utilizing context enrichment). This permits us to maintain the graph easy, whereby all relations between the entities needn’t be essentially extracted into the graph for queries about them to be answered precisely.

To exhibit these concepts, we are going to use a dataset of 10 synthetically generated police experiences, GPT-4o because the LLM, and Neo4j because the graph database.

Constructing the Graph

We will probably be constructing a easy star graph with the Report Id because the central node and entities linked to the central node.  The immediate to construct that may be as follows:

custom_prompt = ChatPromptTemplate.from_template("""
You might be an info extraction assistant.
Learn the textual content beneath and establish vital entities.

**Extraction guidelines:**
- At all times extract the **Report Id** (that is the central node).
- Extract **individuals**, **establishments**, **locations**, **dates**, **financial quantities**, and **car registration numbers** (e.g., MH12AB1234, PK-02-4567, KA05MG2020).
- Don't ignore any individuals names; extract all talked about within the doc, even when they appear minor or function not clear.
  Deal with all of kinds of automobiles (eg; vehicles, bikes and so forth) as the identical form of entity referred to as "Automobile".

**Output format:**
1. Record all nodes (distinctive entities).
2. Establish the central node (Report Id).
3. Create relationships of the shape:
   (Report Id)-[HAS_ENTITY]->(Entity),
4. Don't create another kinds of relationships.                                            

Textual content:
{enter}

Return solely structured information like:
Nodes:
- Report SYN-REP-2024
- Honda bike ABCD1234
- XYZ Faculty, Chennai
- NNN Faculty, Mumbai
- 1434800
- Mr. John

Relationships:
- (Report SYN-REP-2024)-[HAS_ENTITY]->(Honda bike ABCD1234)
- (Report SYN-REP-2024)-[HAS_ENTITY]->(XYZ faculty, Chennai)
- ...
""")

Word that on this immediate, we’re not extracting any relations akin to accused, witness and so forth. within the graph. All nodes could have a uniform “HAS_ENTITY” relation with the central node which is the Report Id. I’ve designed this as an excessive case, for example the truth that we will reply queries about relations between entities even with this minimal graph, based mostly on the retrieval pipeline depicted within the earlier part. In the event you want to embrace just a few vital relations, the immediate will be modified to incorporate clauses akin to the next:

3. For individual entities, the relation ought to be based mostly on their function within the Report (e.g., complainant, accused, witness, investigator and so forth).
    eg: (Report Id) -[Accused]-> (Particular person Identify)
4. For all others, create relationships of the shape:
   (Report Id)-[HAS_ENTITY]->(Entity),

llm_transformer = LLMGraphTransformer(
    llm=llm,
    # allowed_relationships=["HAS_ENTITY"],
    immediate= custom_prompt,
)

Subsequent we are going to create the graph for every doc by making a Langchain doc from the total textual content after which offering to Neo4j.

# Learn whole file (no chunking)
with open(file_path, "r", encoding="utf-8") as f:
    text_content = f.learn()

# Create LangChain Doc
doc = Doc(
    page_content=text_content,
    metadata={
        "doc_id": doc_id,
        "supply": filename,
        "file_path": file_path
    },
)
attempt:
    # Convert to graph (whole doc)
    graph_docs = llm_transformer.convert_to_graph_documents([document])
    print(f"✅ Extracted {len(graph_docs[0].nodes)} nodes and {len(graph_docs[0].relationships)} relationships.")

    for gdoc in graph_docs:
        for node in gdoc.nodes:
            node.properties["doc_id"] = doc_id

            original_id = node.properties.get("id") or getattr(node, "id", None)
            if original_id:
                node.properties["entity_id"] = original_id

    # Add to Neo4j
    graph.add_graph_documents(
        graph_docs,
        baseEntityLabel=True,
        include_source=False
    )
besides:
...

This creates a graph comprising 10 clusters as follows:

Key Observations

1. The variety of nodes extracted varies with LLM used and even for various runs of the identical LLM. With gpt-4o, every execution extracts between 15 to 30 nodes (relying upon the scale of the doc) for every of the paperwork for a complete of 200 to 250 nodes. Since every is a star graph, the variety of relations is one lower than the variety of nodes for every doc.
2. Prolonged paperwork end in consideration dilution of the LLMs, whereby, they don’t recall and extract all the desired entities (individual, locations and so forth) current within the doc.

To see how extreme this impact is, lets see the graph of one of many paperwork (SYN-REPORT-0008). The doc has about 4000 phrases. And the ensuing graph has 22 nodes and appears like the next:

Now, lets attempt producing the graph for this doc by chunking it, then extracting entities from every chunk and merging them utilizing the next logic:

1. The entities extraction immediate stays identical as earlier than, besides we ask to extract entities aside from the Report Id.
2. First extract the Report Id from the doc utilizing this immediate.

report_id_prompt = ChatPromptTemplate.from_template("""
Extract ONLY the Report Id from the textual content.

Report Ids sometimes appear like:
- SYN-REP-2024

Return strictly one line:
Report: <report_number_here>

Textual content:
{enter}
""")

Then, extract entities from every chunk utilizing the entities immediate.

def extract_entities_by_chunk(llm, textual content, chunk_size=2000, overlap=200):
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=overlap
    )

    chunks = splitter.split_text(textual content)
    all_entities = []

    for i, chunk in enumerate(chunks):
        print(f"🔍 Processing chunk {i+1}/{len(chunks)}")
        uncooked = run_prompt(llm, entities_prompt, chunk)

        pairs = re.findall(r"- (.*?)s*|s*(w+)", uncooked)
        all_entities.prolong([(e.strip(), t.strip()) for e, t in pairs])

    return all_entities

c. De-duplicate the entities

d. Construct the graph by connecting all of the entities to the central Report Id node.

The impact is sort of exceptional. The graph of SYN-REPORT-0008 now appears to be like like the next. It has 78 nodes, 3X occasions the depend earlier than. The trade-off in constructing this dense graph are the time and utilization incurred for the iterations for chunk extraction.

What are the implications?

The impression of the variation in graph density is within the means to reply questions associated to the entities immediately and precisely; i.e if an entity or relation just isn’t current within the graph, a question associated to it can’t be answered from the graph.

An strategy to minimise this impact with our sparse star graph can be to create a question such that there’s a reference to a outstanding associated entity more likely to be current within the graph.

For example, the investigating officer is talked about comparatively fewer occasions than town in a police report, and there’s a larger likelihood of town to be current within the graph moderately than the officer. Subsequently, to seek out out the investigating officer, as an alternative of claiming “Which experiences have investigating officer as Ravi Sharma?”, one can say “Among the many Mumbai experiences, which of them have investigating officer as Ravi Sharma?”, whether it is recognized that this officer is from Mumbai workplace. Our retrieval pipeline will then extract the experiences associated to Mumbai from the graph, and inside these paperwork, find the chunks having the officer identify precisely. That is demonstrated within the following sections.

Dealing with weak embeddings

Contemplate the next comparable queries which might be more likely to be incessantly requested of this information.

“Inform me in regards to the incident involving Person_3”

“Inform me in regards to the incident in report SYN-REPORT-0008”

The small print in regards to the incident within the report can’t be discovered within the graph as that holds the entities and relations solely, and subsequently, the response must be derived from the vector similarity search. 

So, can the graph be ignored on this case?

In the event you run these, the primary question is more likely to return an accurate reply for a comparatively small corpus like our check dataset right here, whereas the second is not going to. And the reason being that the LLMs have an inherent understanding of individual names and phrases as a consequence of their coaching, however discover arduous to connect any semantic that means to alphanumeric strings akin to report_id, car numbers, quantities, dates and so forth. And subsequently, the embedding of an individual’s identify is way stronger than that of alphanumeric strings.  So the chunks retrieved within the case of alphanumeric strings utilizing vector similarity have a weak correlation to the consumer question, leading to an incorrect reply.

That is the place the context enrichment utilizing Graph helps. For a question like “Inform me in regards to the incident in SYN-REPORT-0008”, we get all the small print from the star graph of the central node SYN-REPORT-0008 utilizing a generated cypher, then have the LLM use this to generate a context (interpret the JSON response in pure language). The context additionally incorporates the sources for the nodes, which on this case returns 2 paperwork, certainly one of which is the proper doc SYN-REPORT-0008. The opposite one SYN-REPORT-00010 is because of the truth that one of many hooked up nodes –metropolis is widespread (Mumbai) for each the experiences.

Now that the search area is refined to solely 2 paperwork, chunks are extracted from each utilizing this context together with the consumer question. And since the context from the graph mentions individuals, locations, quantities and different particulars current within the first report however not within the second, it allows the LLM to simply perceive within the response synthesis step that the proper chunks are those extracted from SYN-REPORT-0008 and never from 0010. And the reply is fashioned precisely. Right here is the log of the graph question, JSON response and the pure language context depicting this.

Generated Cypher:
cypher
MATCH (r:`__Entity__`:Report)
WHERE toLower(r.id) CONTAINS toLower("SYN-REPORT-0008")
OPTIONAL MATCH (r)-[]-(e)
RETURN DISTINCT 
    r.id AS report_id, 
    r.doc_id AS report_doc_id,
    labels(e) AS entity_labels,
    e.id AS entity_id, 
    e.doc_id AS entity_doc_id

JSON Response:
[{'report_id': 'Syn-Report-0008', 'report_doc_id': 'SYN-REPORT-0008', 'entity_labels': ['__Entity__', 'Person'], 'entity_id': 'Mr. Person_12', 'entity_doc_id': 'SYN-REPORT-0008'}, {'report_id': 'Syn-Report-0008', 'report_doc_id': 'SYN-REPORT-0008', 'entity_labels': ['__Entity__', 'Place'], 'entity_id': 'New Delhi', 'entity_doc_id': 'SYN-REPORT-0008'}, {'report_id': 'Syn-Report-0008', 'report_doc_id': 'SYN-REPORT-0008', 'entity_labels': ['__Entity__', 'Place'], 'entity_id': 'Kottayam', 'entity_doc_id': 'SYN-REPORT-0008'}, {'report_id': 'Syn-Report-0008', 'report_doc_id': 'SYN-REPORT-0008', 'entity_labels': ['__Entity__', 'Person'], 'entity_id': 'Person_4', 'entity_doc_id': 'SYN-REPORT-0008'}, {'report_id': 'Syn-Report-0008', 'report_doc_id': 'SYN-REPORT-0008', 'entity_labels':… truncated 

Pure language context:
The context describes an incident involving a number of entities, together with people, locations, financial quantities, and dates. The next particulars are extracted:

1. **Individuals Concerned**: A number of people are talked about, together with "Mr. Person_12," "Person_4," "Person_11," "Person_8," "Person_5," "Person_6," "Person_3," "Person_7," "Person_10," and "Person_9."

2. **Locations Referenced**: The locations talked about embrace "New Delhi," "Kottayam," "Delhi," and "Mumbai."

3. **Financial Quantities**: Two financial quantities are famous: "0.5 Million" and "43 Hundreds."

4. **Dates**: Two particular dates are talked about: "07/11/2024" and "04/02/2025."

Sources: [SYN-REPORT-0008, SYN-REPORT-00010]

Can relations be efficiently discovered?

What about discovering relations between entities? We’ve got ignored all particular relations in our graph and simplified it such that there’s just one relation “HAS_ENTITY” between the central report_id node and remainder of the entities. This could suggest that querying for entities not current within the graph and relations between entities shouldn’t be attainable. Let’s check our iterative search optimisation pipeline towards quite a lot of such queries. We are going to contemplate two experiences from Kolkata, and the next queries for this check.

• The place the referred relation just isn’t current within the graph. Eg; “Who’s the investigating officer in SYN-REPORT-0006?” Or “Who’re the accused in SYN-REPORT-0006?”
• Relation between two entities current within the graph.  Eg; “Is there a relation between Ravi Verma and Rakesh Prasad Verma?”
• Relation between any entities associated to a 3rd entity. Eg; “Are there brothers in experiences from Kolkata?”
• Multi-hop relations: “Who’s the investigating officer within the experiences the place brothers from Kolkata are accused?”

Utilizing our pipeline, all of the above queries yield correct outcomes. Lets have a look at the method for the final multi-hop question which is probably the most advanced one. Right here the cypher doesn’t yield any outcome, so the move falls again to semantic matching of nodes. The entities are extracted (Place: Kolkata) from the consumer question, then matched to get references to all of the experiences linked to Kolkata, that are SYN-REPORT-0005 and SYN-REPORT-0006 on this case. Based mostly on the context that the consumer question is inquiring about brothers and investigating officers, probably the most related chunks are extracted from each the paperwork. The resultant reply efficiently retrieves investigating officers for each experiences.

Right here is the response:

“The investigating officer within the experiences the place the brothers from Kolkata (Mr. Rakesh Prasad Verma, Mr. Ravi Prasad Verma, and Mr. Vijoy Kumar Varma) are accused is Ajay Kumar Tripathi, Inspector of Police, CBI, ACB, Kolkata, as talked about in SYN-REPORT-0006. Moreover, Praveen Kumar, Deputy Superintendent of Police, EOB Kolkata, is famous because the investigating officer in SYN-REPORT-0005.

Sources: [SYN-REPORT-0005, SYN-REPORT-0006]”

> Getting into new GraphCypherQAChain chain...
2025-12-05 17:08:27 - HTTP Request: ... LLM referred to as
Generated Cypher:
cypher
MATCH (p:`__Entity__`:Particular person)-[:HAS_ENTITY]-(r:`__Entity__`:Report)-[:HAS_ENTITY]-(pl:`__Entity__`:Place)
WHERE toLower(pl.id) CONTAINS toLower("kolkata") AND toLower(p.id) CONTAINS toLower("brother")
OPTIONAL MATCH (r)-[:HAS_ENTITY]-(officer:`__Entity__`:Particular person)
WHERE toLower(officer.id) CONTAINS toLower("investigating officer")
RETURN DISTINCT 
    r.id AS report_id, 
    r.doc_id AS report_doc_id, 
    officer.id AS officer_id, 
    officer.doc_id AS officer_doc_id

Cypher Response:
[]
2025-12-05 17:08:27 - HTTP Request: ...LLM referred to as

> Completed chain.
is_empty: True
❌ Cypher didn't produce a assured outcome.
🔎 Working semantic node search...
📋 Detected labels: ['Place', 'Person', 'Institution', 'Date', 'Vehicle', 'Monetary amount', 'Chunk', 'GraphNode', 'Report']
Consumer question for node search: investigating officer within the experiences the place brothers from Kolkata are accused
2025-12-05 17:08:29 - HTTP Request: ...LLM referred to as
🔍 Extracted entities: ['Kolkata']
2025-12-05 17:08:30 - HTTP Request: ...LLM referred to as
📌 Hits for entity 'Kolkata': [Document(metadata={'labels': ['Place'], 'node_id': '4:5b11b2a8-045c-4499-9df0-7834359d3713:41'}, page_content='TYPE: PlacenCONTENT: KolkatanDOC: SYN-REPORT-0006')]
📚 Retrieved node hits: [Document(metadata={'labels': ['Place'], 'node_id': '4:5b11b2a8-045c-4499-9df0-7834359d3713:41'}, page_content='TYPE: PlacenCONTENT: KolkatanDOC: SYN-REPORT-0006')]
Expanded node context:
 [Node] This can be a __Place__ node. It represents 'TYPE: Place
CONTENT: Kolkata
DOC: SYN-REPORT-0006' (doc_id=N/A).
[Report Syn-Report-0005 (doc_id=SYN-REPORT-0005)] --(HAS_ENTITY)--> __Entity__, Establishment: Mrs.Sri Balaji Forest Product Non-public Restricted (doc_id=SYN-REPORT-0005)
[Report Syn-Report-0005 (doc_id=SYN-REPORT-0005)] --(HAS_ENTITY)--> __Entity__, Date: 2014 (doc_id=SYN-REPORT-0005)
[Report Syn-Report-0005 (doc_id=SYN-REPORT-0005)] --(HAS_ENTITY)--> __Entity__, Particular person: Mr. Pallab Biswas (doc_id=SYN-REPORT-0005)
[Report Syn-Report-0005 (doc_id=SYN-REPORT-0005)] --(HAS_ENTITY)--> __Entity__, Date: 2005 (doc_id=SYN-REPORT-0005).. truncated
[Report Syn-Report-0006 (doc_id=SYN-REPORT-0006)] --(HAS_ENTITY)--> __Entity__, Establishment: M/S Jkjs & Co. (doc_id=SYN-REPORT-0006)
[Report Syn-Report-0006 (doc_id=SYN-REPORT-0006)] --(HAS_ENTITY)--> __Entity__, Particular person: B Mishra (doc_id=SYN-REPORT-0006)
[Report Syn-Report-0006 (doc_id=SYN-REPORT-0006)] --(HAS_ENTITY)--> __Entity__, Establishment: Vishal Engineering Pvt. Ltd. (doc_id=SYN-REPORT-0006).. truncated

Key Takeaways

• You don’t want an ideal graph. A minimally structured graph—even a star graph—can nonetheless help advanced queries when mixed with iterative search-space refinement.
• Chunking boosts recall, however will increase price. Chunk-level extraction captures way more entities than whole-document extraction, however requires extra LLM calls. Use it selectively based mostly on doc size and significance.
• Graph context fixes weak embeddings. Entity varieties like IDs, dates, and numbers have poor semantic embeddings; enriching the vector search with graph-derived context is crucial for correct retrieval.
• Semantic node search is a robust fallback, to be exercised with warning. Even when Cypher queries fail (as a consequence of lacking relations), semantic matching can establish related nodes and shrink the search area reliably.
• Hybrid retrieval delivers correct response on relations, with no dense graph. Combining graph-based doc filtering with vector chunk retrieval permits correct solutions even when the graph lacks specific relations.

Conclusion

Constructing a GraphRAG system that’s each correct and cost-efficient requires acknowledging the sensible limitations of LLM-based graph building. Massive paperwork dilute consideration, entity extraction is rarely good, and encoding each relationship rapidly turns into costly and brittle.

Nonetheless, as proven all through this text, we will obtain extremely correct retrieval with no totally detailed data graph. A easy graph construction—paired with iterative search-space optimization, semantic node search, and context-enriched vector retrieval—can outperform extra advanced and costly designs.

This strategy shifts the main target from extracting every thing upfront in a Graph to extracting what’s cost-effective, fast to extract and important, and let the retrieval pipeline fill the gaps. The pipeline balances performance, scalability and value, whereas nonetheless enabling subtle multi-hop queries throughout messy, real-world information.

You may learn extra in regards to the GraphRAG design ideas underpinning the ideas demonstrated right here at Do You Really Need GraphRAG? A Practitioner’s Guide Beyond the Hype

Join with me and share your feedback at www.linkedin.com/in/partha-sarkar-lets-talk-AI

_{All photos and information used on this article are synthetically generated. Figures and code created by me}