My thesis was on knowledge graph embeddings, so when GraphRAG started trending I was genuinely excited. Finally, knowledge graphs getting the attention they deserve in the LLM era. But having lived in that world, I also know what people aren’t talking about: the cost of actually building and maintaining a knowledge graph from scratch.
What Is a Knowledge Graph?
A knowledge graph is a graph where nodes are entities and edges are relations between them. The canonical example is a triple: (Albert Einstein, bornIn, Ulm). Millions of these triples form a structured representation of knowledge — Wikidata has over 100 billion triples. The key property is that knowledge is explicit and traversable. You can follow links, reason over paths, and answer multi-hop questions that would be impossible with flat text.
graph LR
E["Einstein"] -->|bornIn| U["Ulm"]
E -->|field| P["Physics"]
E -->|won| NP["Nobel Prize 1921"]
NP -->|category| P
U -->|country| DE["Germany"]
style E fill:#264653,stroke:#264653,color:#fff
style U fill:#2a9d8f,stroke:#2a9d8f,color:#fff
style P fill:#e9c46a,stroke:#e9c46a,color:#000
style NP fill:#f4a261,stroke:#f4a261,color:#000
style DE fill:#2d6a4f,stroke:#2d6a4f,color:#fff
Why Traditional RAG Falls Short on Global Questions
Traditional RAG works like this: chunk documents, embed them into vectors, retrieve the top-k most similar chunks for a query. It works well for factual lookups — “what does this API return?” or “what’s the refund policy?” But it falls apart on questions that require synthesizing information across many documents. Try asking “what are the main themes in this dataset?” or “how are these companies connected?” — vector similarity doesn’t help because no single chunk contains the answer.
The GraphRAG Approach
This is exactly what GraphRAG (Edge et al., 2024) [1] addresses. The core idea: build a knowledge graph from your corpus, detect communities of related entities using the Leiden algorithm [3], generate summaries for each community, then use those summaries to answer global questions via map-reduce.
graph LR
DOC["Documents"] --> CHUNK["Chunk"]
CHUNK --> EXT["Extract Entities"]
EXT --> KG["Knowledge Graph"]
KG --> COM["Detect Communities"]
COM --> SUM["Summarize Communities"]
SUM --> QA["Map-Reduce QA"]
style DOC fill:#264653,stroke:#264653,color:#fff
style CHUNK fill:#e76f51,stroke:#e76f51,color:#fff
style EXT fill:#f4a261,stroke:#f4a261,color:#000
style KG fill:#e9c46a,stroke:#e9c46a,color:#000
style COM fill:#2a9d8f,stroke:#2a9d8f,color:#fff
style SUM fill:#40916c,stroke:#40916c,color:#fff
style QA fill:#2d6a4f,stroke:#2d6a4f,color:#fff
The results are compelling. On their benchmarks (~1M token datasets), GraphRAG outperformed vector RAG on comprehensiveness (72-83% win rate) and diversity (62-82% win rate) using LLM-as-judge evaluation. Vector RAG still won on directness — it gives more concise, pointed answers for specific questions. This makes sense: vector RAG is great at finding the needle, GraphRAG is great at describing the haystack.
| Metric |
GraphRAG vs Vector RAG |
What it means |
| Comprehensiveness |
72-83% win |
Covers more aspects of the answer |
| Diversity |
62-82% win |
Provides more varied perspectives |
| Directness |
Vector RAG wins |
More concise for specific questions |
| Empowerment |
Mixed |
Depends on whether quotes or summaries help more |
The Hidden Cost of Building the Knowledge Graph
Here’s what it actually takes to go from documents to a usable knowledge graph:
Step 1: Entity and Relationship Extraction. An LLM reads every chunk and extracts entities and their relationships. The Neo4j implementation [2] using LangChain’s LLMGraphTransformer with GPT-4o extracted ~13,000 entities and ~16,000 relationships from 2,000 news articles. Cost: ~$30, time: ~35 minutes with 10 parallel workers. And this is a small dataset.
# LangChain's LLMGraphTransformer — simplified
from langchain_experimental.graph_transformers import LLMGraphTransformer
transformer = LLMGraphTransformer(llm=ChatOpenAI(model="gpt-4o"))
graph_documents = transformer.convert_to_graph_documents(documents)
# Each document → nodes (entities) + relationships (edges)
Step 2: Entity Resolution. This is the step the original paper mentions but doesn’t ship code for — and it’s arguably the hardest. The same entity appears with different names: “Silicon Valley Bank”, “Silicon_Valley_Bank”, “SVB”. You need to deduplicate them. The Neo4j blog’s approach: compute text embeddings, build a KNN graph (cosine similarity > 0.95), find connected components, filter by edit distance, then LLM verification for final merge decisions. Even with all that, it still has failure modes for dates, abbreviations, and domain-specific terms.
Step 3: Community Detection. Run the Leiden algorithm to partition the graph into hierarchical communities — clusters of closely related entities. The paper’s podcast dataset produced communities ranging from 34 (coarsest) to 1,310 (finest). Not every level adds meaningful information — the Neo4j blog found levels 3 and 4 differed by only 4 communities.
Step 4: Community Summarization. An LLM generates natural language summaries for each community, bottom-up through the hierarchy. That’s potentially thousands of LLM calls. The paper’s indexing step took 281 minutes for ~1M tokens of source documents.
| Pipeline Step |
What it does |
Cost/Complexity |
| Entity Extraction |
LLM reads every chunk |
~$30 for 2K articles (GPT-4o) |
| Entity Resolution |
Deduplicate entities |
Multi-step pipeline, domain-dependent |
| Community Detection |
Cluster related entities |
Needs graph DB (Neo4j [5] + GDS plugin) |
| Community Summarization |
LLM summarizes each community |
Potentially thousands of LLM calls |
| Total indexing time |
End to end |
~281 min for ~1M tokens (paper) |
The Maintenance Problem — What Happens When Knowledge Changes
If you have a fixed, curated knowledge graph — like Wikidata or a domain-specific ontology that rarely changes — GraphRAG works beautifully. The graph is your ground truth, you run community detection once, generate summaries, and you’re done. Query-time is efficient: root-level community summaries use 9-43x fewer tokens than processing raw text.
But if you’re building the knowledge graph from your own documents — which is the whole point of GraphRAG for most use cases — every update is painful:
Adding new documents means re-extracting entities, but now you need to resolve them against the existing graph. Is “OpenAI” in the new document the same “OpenAI” already in the graph? Probably yes, but what about “GPT-5” vs “GPT 5” vs “the new GPT model”? Every new entity needs link prediction against the full graph.
Entity deduplication gets harder as the graph grows. With 13,000 entities, pairwise comparison is already expensive. At 100K+ entities, you need approximate methods (LSH, blocking strategies), each with its own failure modes.
Community structure shifts. Adding a few hundred nodes can completely reorganize communities, invalidating existing summaries. Do you re-run Leiden on the full graph? Only on affected subgraphs?
Summary staleness. Even if you detect which communities changed, regenerating summaries means more LLM calls. And if higher-level summaries depend on lower-level ones (they do — the paper uses bottom-up summarization), a change at the leaf level cascades through the entire hierarchy.
graph LR
NEW["New Documents"] --> EXT["Re-extract Entities"]
EXT --> RES["Resolve Against\nExisting Graph"]
RES --> LINK["Link Prediction"]
LINK --> DEDUP["Entity Dedup\n(full graph)"]
DEDUP --> RECOM["Re-detect\nCommunities"]
RECOM --> RESUM["Re-summarize\n(cascade)"]
style NEW fill:#e76f51,stroke:#e76f51,color:#fff
style EXT fill:#f4a261,stroke:#f4a261,color:#000
style RES fill:#e9c46a,stroke:#e9c46a,color:#000
style LINK fill:#e9c46a,stroke:#e9c46a,color:#000
style DEDUP fill:#f4a261,stroke:#f4a261,color:#000
style RECOM fill:#e76f51,stroke:#e76f51,color:#fff
style RESUM fill:#e76f51,stroke:#e76f51,color:#fff
This is the fundamental tension: GraphRAG converts unstructured data into structured data, and structured data is harder to update than unstructured data. With vector RAG, adding a new document is trivial — chunk it, embed it, append to the index. With GraphRAG, adding a new document means potentially restructuring your entire knowledge representation.
What About Knowledge Graphs for Source Code?
There’s another place where knowledge graphs have been proposed recently: source code. Projects like CodexGraph (Liu et al., 2024) [9] and GraphCoder [10] build knowledge graphs from codebases — extracting entities like functions, classes, and modules, with edges for calls, imports, inheritance, and type relationships — then use graph retrieval to give LLMs better repository-level context.
The idea sounds appealing: code is full of relationships, and understanding a function often means understanding what it calls, what calls it, and what types it uses. A knowledge graph could capture all of that.
But here’s my issue: code is already structured data. Unlike natural language documents, source code has a formal grammar. We already have tools that parse it perfectly:
| Tool |
What it provides |
Maintenance cost |
| AST (tree-sitter) |
Complete syntactic structure of every file |
Zero — deterministic parse |
| LSP |
Go-to-definition, find-references, call hierarchy, type info |
Zero — runs on-demand from source |
| Package managers |
Dependency graphs (pip, npm, cargo) |
Zero — reads lockfiles |
| CodeQL / Semgrep |
Data flow, taint tracking, control flow graphs |
Near-zero — static analysis |
These tools give you the exact same entities and relations that a code knowledge graph would extract — functions, classes, call graphs, import chains, type hierarchies — but with perfect accuracy, zero LLM cost, and no maintenance burden. An AST is a lossless representation of code structure. An LLM-extracted knowledge graph is a lossy, probabilistic approximation of the same thing.
Claude Code’s LSP integration [11] is a good example: it can jump to definitions, find all references, and traverse call hierarchies in real time, directly from the source. No graph database, no entity extraction pipeline, no community detection. Just the language server doing what it was designed to do.
graph LR
CODE["Source Code"] --> AST["AST\n(tree-sitter)"]
CODE --> LSP["LSP\n(definitions, refs)"]
CODE --> DEP["Dependency\nGraph"]
CODE --> SA["Static Analysis\n(CodeQL)"]
KG_APPROACH["LLM-Extracted\nCode KG"]
AST -->|"lossless, free"| SAME["Same Relations"]
LSP -->|"real-time, free"| SAME
DEP -->|"exact, free"| SAME
SA -->|"precise, free"| SAME
KG_APPROACH -->|"lossy, $$$"| SAME
style CODE fill:#264653,stroke:#264653,color:#fff
style AST fill:#2a9d8f,stroke:#2a9d8f,color:#fff
style LSP fill:#2a9d8f,stroke:#2a9d8f,color:#fff
style DEP fill:#2a9d8f,stroke:#2a9d8f,color:#fff
style SA fill:#2a9d8f,stroke:#2a9d8f,color:#fff
style KG_APPROACH fill:#e76f51,stroke:#e76f51,color:#fff
style SAME fill:#e9c46a,stroke:#e9c46a,color:#000
And the maintenance problem is even worse for code than for documents. Codebases change constantly — every commit modifies functions, adds files, changes call patterns. If maintaining a knowledge graph for a slowly-changing document corpus is already painful, imagine maintaining one for a codebase with dozens of commits per day. Every refactor, every rename, every new dependency would require re-extraction and re-resolution.
The one argument for code KGs is cross-repository or cross-language reasoning — “which services depend on this shared library?” or “how does the Python backend connect to the TypeScript frontend?” LSP doesn’t cross language boundaries, and package managers don’t trace internal function calls across repos. But even here, tools like Sourcegraph [12] solve this with SCIP-based code intelligence [13] — deterministic, not probabilistic.
My take: knowledge graphs make sense when you’re dealing with unstructured data that has no inherent structure. Documents, research papers, news articles — these genuinely benefit from having structure imposed on them. But code already has structure. Building a knowledge graph on top of code is building a lossy approximation of something you can already access losslessly. It’s solving a problem that’s already solved.
When to Use GraphRAG
| Scenario |
Recommendation |
Why |
| Static knowledge base, global queries |
GraphRAG |
Upfront cost amortized, global queries excel |
| Rapidly changing documents |
Vector RAG |
Update cost too high for GraphRAG |
| Specific factual lookups |
Vector RAG |
No need for global synthesis |
| Existing curated KG (Wikidata, domain ontology) |
KG-augmented RAG |
Skip the construction step entirely |
| Mixed: some global, some specific |
Hybrid |
Vector for specific, graph for thematic |
The honest answer for most teams: if you already have a knowledge graph, absolutely use it for retrieval. If you need to build one from scratch for GraphRAG, think very carefully about whether the maintenance cost is worth the improvement over vector RAG. The benchmarks are real — GraphRAG genuinely outperforms on global questions. But benchmarks run on static datasets. Production systems don’t stay static.
Tools like LangChain’s LLMGraphTransformer [4], Neo4j [5], and Microsoft’s GraphRAG library [6] have made the initial construction more accessible. But accessible construction doesn’t mean accessible maintenance.
My take: the future of GraphRAG is in incremental graph updates — methods that can add new knowledge without restructuring the entire graph. Some work is happening here (incremental community detection [7], streaming knowledge graph construction [8]), but it’s still early. Until incremental updates are solved, GraphRAG is best suited for corpora that change infrequently and are queried frequently with global, thematic questions.
What’s your experience with knowledge graphs in production? Are you building them from scratch or leveraging existing ones?
References:
[1] Edge et al. “From Local to Global: A Graph RAG Approach to Query-Focused Summarization.” arXiv 2024.
[2] Bratanic, T. “Implementing ‘From Local to Global’ GraphRAG with Neo4j and LangChain.” Neo4j Blog 2024.
[3] Traag, V. A. et al. “From Louvain to Leiden: guaranteeing well-connected communities.” Scientific Reports 2019.
[4] “How to construct knowledge graphs.” LangChain Documentation.
[5] “Neo4j Graph Database.” Neo4j.
[6] “GraphRAG.” Microsoft GitHub.
[7] Banerjee, P. et al. “Incremental Community Detection in Distributed Dynamic Graph.” arXiv 2023.
[8] Chuang, Y. et al. “Streaming Knowledge Graph Construction.” arXiv 2023.
[9] Liu et al. “CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases.” arXiv 2024.
[10] Liu et al. “GraphCoder: Enhancing Repository-Level Code Completion via Code Context Graph-based Retrieval and Language Model.” arXiv 2024.
[11] “Claude Code Overview.” Anthropic.
[12] “Sourcegraph — Code Intelligence Platform.” Sourcegraph.
[13] “Announcing SCIP — a better code indexing format.” Sourcegraph Blog.
]]>