← Back

HippoRAG v1 / v2, and Why

Retrieval augmented generation, or 'RAG', has been around for years. The crux of RAG is embedding a corpus of text, then using and embedding of the (user) query to find similar content, aiming to contextualize an LLM's answer. The original paper, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" published in 2021 by Lewis et. al.[1] set in motion a cascade of development to eek out architectural and system gains in LLM systems. At the time of writing, RAG is roughly equidistant to the publication of the original "Attention is All You Need" paper of 2017 as it is to today, and roughly marks my introduction to AI engineering.

The period between '21 and '23 was a kind of fever dream of prompt engineering methods bubbling to the surface as "SOTA" in the mainstream and being subsequently embedded in then-nascent tools like Langchain. It was fun, fast, and never felt as formal[2] as the context and harness engineering of today, though it directly contributed to critical methods we now see everywhere: better retrieval, -aware prompting, eval harnesses, RL-tuned behavior, and system-level LLM engineering.

Normal RAG, based on vector embeddings, covers semantic similarity. It can tell that some text is roughly "about the same thing" as another, but can't capture relational information. If you made a vague or complicated query, it's unlikely that the chunks of text surfaced by pure similarity would contain sufficient context to allow an LLM to answer completely. For example, let's say that we have a chat system specializing in movies, IMBD information, and the like. If a user asks:

"What university did the screenwriter of the highest-grossing film directed by Guillermo del Toro attend?"

A vector db would surface Guillermo del Toro film descriptions, articles about high grossing films, (some of possibly relating to Guillermo del Toro), some additional information about schools and programs bet for screenwriting, and likely some additional noise. It's possible that scattered across these chunks the LLM will have enough context to synthesize a correct answer, but more likely we will be missing critical details and produce either a variant of "I don't know", or worse, a confabulation of the information present in that context.

Many methods exist for improving the information quality coming out of a RAG pipeline:

  • Filter vectors using query metadata, eg: "filter": {"director" == "Guillermo del Toro"} [3]
  • Vector reranking to order retrieved vectors by relevance
  • Additional keyword search like BM25
  • Iterative retrieval using partial results to guide future searches

Often, a combination of the three methods produce strong results, especially if tuned to a specific domain. But, most forms wind up relying on the semantic similarity. They may improve retrieval surface, but cannot natively express complicated relationships within the text.

To capture rich relationships beyond semantic similarity in a corpus, we need to look to knowledge graphs and graph databases.

GraphRAG methods have growing in popularity for exactly this reason. To set up GraphRAG you need a knowledge graph (KG), where nodes are entities, concepts, or text, and edges represent relationships between them. At query time, retrieval executes as a traversal over this graph to collect context that is structurally connected to the query, not just similar to it.

But there's a catch: the construction of a knowledge graph is expensive[4], requiring myriad LLM calls, deduplication, normalization, and ongoing maintenance to the knowledge graph. Just as well, the process of traversing a knowledge graph during the retrieval step can quickly become computationally expensive, making production GraphRAG expensive and high latency. Even though graph traversal is a class of problem about as old as computer science itself, to do so in a way that maximizes collection of valuable information with respect to some query is a completely different beast.

As one examines the GraphRAG landscape, motifs emerge:

  • NLP methods for KG construction
  • Classic graph traversal algorithms like PageRank during retrieval
  • LLM-based reasoning and traversal of the KG

Often, we find that a combination of these methods produce some of the most effective results - a beautiful example being HippoRAG[5]. Mind you that's Hippo as in hippocampus, not hippopotamus, and is the core inspiration for the method.

HippoRAG is inspired by hippocampal memory indexing theory (HMIT), which posits that memory relies on the interactions of the neocortex and hippocampus, with the former storing actual memory representations, and the latter holding a set of connections which "point" to the memories in the neocortex itself[6]. HippoRAG's authors explicitly map this theory onto RAG, using an LLM to to act as neocortex, a knowledge graph (and algorithms we'll get to in a moment) as the hippocampal index, and a retrieval encoder (embedding model) as the parahippocampal regions connecting the two.

The theory has two goals:

  1. Pattern separation - keeping distinct experience representations unique, and
  2. Pattern completion - allowing complete memory reconstruction based on partial stimuli

With these goals in mind, the authors of the HippoRAG paper use this as the basis for their encoding and decoding pipeline. More specifically, we have offline indexing and online retrieval in HippoRAG v1, mapping to encoding and decoding respectively.

HippoRAG v1

Offline indexing takes passages from the corpus and takes an (instruction-tuned) LLM to extract OpenIE triples (subject, predicate, object) to construct a knowledge graph. Importantly, the extraction is schema-less, meaning no need for a pre-defined relationship ontology. Keeping without our filmmaker example, one such triple might be:

("Pan's Labyrinth", "Directed by", "Guillermo del toro")

Once the KG is complete for the corpus, we also compute the cosine similarity between all entity representations, adding an additional "synonymy" edge between them if they are above a threshold (0.8 in the paper). This synonym edge is to ensure representations of the same entity are allowed to centralize their respective relationships, even if they aren't exactly the same, for example adding a synonymy edge between extracted entities "MIT" and "Massachusetts Institute of Technology". In addition to the knowledge graph, we also build an |N| x |P| matrix P containing the number of times each noun phrase in the KG appears in the original passages.

Finally, we calculate node "specificity". This specificity vector is essentially a graph-native proxy for inverse document frequency, calculated for each node as 1 / (number of passages from which that entity was extracted).

Online retrieval takes a given query and first exteracts named entities from it. These entities are then embedded by the same retrieval encoder used to establish synonymy connections, and used to find the top k most similar semantically similar nodes in the KG. These nodes are used as the seed (personalization vector) for Personalized PageRank, wherein each of the top k similar nodes are initially seeded with equal probability 1 / k, and all other nodes set to probability of zero. Before running, this sparse vector is multiplied by our specificity vector, further weighting each node according to its relative rarity in the initial corpus.

By running PPR over this graph, we see probability mass settle on nodes in and around the joint neighborhoods of those initial top k. With this new set of nodes and their post-PPR probabilities, we multiply this against the |N| x |P| matrix P to obtain a ranking score for each of the passages, which are finally retrieved and collated in that ranked order for use by the LLM connected to the artificial hippocampus!

Where v1 Struggles:

V1 is strong as a multi-hop question answerer, but the biggest issue as identified by the researchers in their v2 paper[7] is "entity-centricity". Because v1 uses entities at the core of indexing and retrieval, we run up against context loss and trouble with the semantic matching normal RAG is good at. This translates to v1 being good with concepts, but sometimes lacking the exact context which isn't captured well enough by pagerank over a graph based on named entities.

HippoRAG V2

HippoRAG v2 keeps much of the core idea regarding triple extraction, KG construction, and Personalized PageRank. Where v2 makes improvements is with respect to passage, triple, and query interactions.

Specifically:

  • Passages are treated as a part of the graph itself, not just the outputs ranked after graph search
  • Passages and triples are scored against the query
  • Seed node selection is made more context aware
  • An LLM is used to filter irrelevant triples during online retrieval

Without reiterating the algorithm with the v2 tweaks, v2 searches a graph that includes both conceptual nodes and contextual passage nodes. This ultimately makes the graph less brittle, allowing PPR to follow entity relationships but still preserve original passage context. The v2 paper reports results showing that this design improves factual, sensemaking, and associative memory tasks, instead of solely improving on multi-hop QA (which it does as well).

Reproduction Results

I implemented v1 a few months after the original paper, recently returning to the code to add v2 and benchmarking logic. I use NetworkX for the knowledge graph, deployed to EC2 against the two multihop benchmarks below, using BM25 as a baseline:

MuSiQue @5

Method Our Result Paper Result
BM25 0.3746 41.2
HippoRAG v1 0.4783 52.1
HippoRAG v2 0.5346 74.7

HotpotQA @5

Method Our Result Paper Result
BM25 0.6900 72.2
HippoRAG v1 0.7235 76.2
HippoRAG v2 0.7915 96.3

Directionally, we see the right results, but the gap from v1 to v2 likely boils down to model quality and static synonymy threshold. I swapped the triple extraction model to GPT-4o-mini, and the embedding model for text-embedding-3-small. OpenIE quality is critical when it comes to KG construction, and we kept the synonymy threshold at 0.8 (tuned specifically for working with the contriever / ColBERTv2 retrieval encoders), we see that because our embedding model is different, we do wind up with a substantially less dense graph, resulting in worse performance @5 on either dataset, especially from v1 to v2.

A 'todo' for me would be a parameter sweep over a subset of each benchmark to determine the appropriate synonymy threshold for 4o-mini or other models. My goal here was correct implementation of the underlying algorithms, not optimization for exactly aligned benchmark results, so I'm leaving the hyperparam sweep for another time.

Implementation with deployment scripts for your own use can be found on GitHub or on the project page.

In Sum

Non-graph RAG – usually a combination of hybrid search, reranking, and metadata filtering – is probably still the best RAG system for most direct QA tasks. When an answer lives across a few small chunks, hybrid with rerank is going to be much cheaper, much faster, and easier to maintain than building and querying a graph.

GraphRAG becomes a compelling option when the questions an LLM sees frequently are more complex, being grounded in sources that might be causally or contextually related. Here, graph based methods like GraphRAG from Microsoft[8], HippoRAG, and CausalRAG[9] all serve different facets of the retrieval problem. Microsoft's GraphRAG for broad corpus understanding and summarization, HippoRAG for multihop question answering, and CausalRAG for preserving cause and effect structure. Each has it's place as an enhanced form of retrieval suited to a specific aim, but as mentioned, none of this is free. Graph construction has a high up-front cost, traversal adds latency, and maintenance of that graph can be cumbersome when the corpus updates.

Practically speaking, the question to ask when considering GraphRAG is not "is GraphRAG better?", rather it's: "is the GraphRAG overhead worth the difficulty of my task?"

Notes & Sources

  1. https://arxiv.org/pdf/2005.11401 ↩
  2. Subjectively that is. Many research groups from well renowned labs and instiutions were working on this, but it was still "early", even among the tech forward. ↩
  3. Very simple example of filtering with specifically Pinecone's syntax. The power of filtering comes from composition of operators, which I recommend any reader familiarize themselves with in their own time. ↩
  4. Very expensive. Proof of concept is cheap, but creating and maintaining an enterprise KG (for humans and AI) can run into the multimillions for complicated ERP data. ↩
  5. https://arxiv.org/pdf/2405.14831 ↩
  6. T. J. Teyler and P. Discenna. The hippocampal memory indexing theory. Behavioral neuroscience, 100 2:147–54, 1986. URL https://pubmed.ncbi.nlm.nih.gov/3008780/. ↩
  7. https://arxiv.org/pdf/2502.14802 ↩
  8. https://arxiv.org/pdf/2404.16130 ↩
  9. https://arxiv.org/pdf/2503.19878 ↩