Hanzo AI

Research Papers

All papers

Hops Do Not Fix Multi-Hop Retrieval

A Negative Result on Long-Horizon Conversational Memory, and What the Failure Actually Is

Hanzo AI Research

MemoryRetrievalLoCoMoBenchmarksNegative Result

Abstract

On LoCoMo, a dense retriever answering single-hop questions finds every required turn 78.2% of the time at k=20. On multi-hop questions the same retriever over the same index finds every required turn 22.7% of the time — but finds at least one of them 80.1% of the time, essentially the single-hop rate. It reliably finds a relevant turn and reliably misses the rest. The textbook fix is to search more than once and let the first result inform the second; we implemented four retrieval policies over one index — a single search, pseudo-relevance feedback, a union over neighbours of the top hits, and a two-deep chain — so that the only variable is the policy. None of them fixes it: the best gained 0.7 percentage points on multi-hop at double the latency, two were worse than a single search, and the one that gained on multi-hop lost 3.1 points on single-hop. The diagnostic says why. Multi-hop questions need 3.1 gold turns on average (up to 19), and 95.4% of that evidence spans more than one conversation session; the median gold turn sits at rank 24 and the median worst gold turn at rank 67, beyond k=20 for 77.2% of questions and beyond k=100 for 43.1%. Additional hops re-search the same similarity space, and a turn that is far away in that space stays far however many times it is queried. The missing signal is linkage, not similarity — a structure to be built, not a search to be repeated. We publish this because the negative result is the useful part: it rules out the cheap fix and names the expensive one.

Preview unavailable in this browser. Download the PDF.