Hanzo AI

Research Papers

17 papers

Hanzo AIResearch Papers

Owned, self-hostable AI and post-quantum cryptography — sovereign infrastructure for regulated industries, critical infrastructure, and government. Technical papers, measured case studies, and formal proofs.

17 papers

Linkage, Not Similarity

What long-term agent memory has to retrieve, and how it is measured

Vector similarity retrieves what sounds related; long-term memory has to retrieve what is connected. On LoCoMo a cosine index recovers every annotated turn for only 22.7% of multi-hop questions at k=20 while recovering at least one for 80.1% — the signature of a linkage failure that a better embedding does not fix. An index of atomic facts, each citing the turn it came from, plus two structural bonuses, raises complete evidence to 36.2% with nothing learned and under a millisecond per query. On MemoryAgentBench FactConsolidation the typed search scores 98.3 single-hop and 84.5 multi-hop pooled with no reader at all, against CAR's 94.8 / 51.5 with gpt-4o and Naive Brain's published 91 / 57 on a matched gpt-4o-mini backbone. Every figure carries a declared dev/test split, a frozen commit, bootstrap intervals, and its own chance floor; the ablations include two plausible additions that make the result worse.

Zach Kelling
BenchmarksMemoryRetrievalLoCoMo
PDFSource
Benchmark suite and every runRESULTS.md — regenerated from runs/

Hanzo AI Chain (AIVM)

Useful-Work Mining, the Native AI Coin, and Post-Quantum Omnichain Settlement

The canonical Proof-of-AI architecture: a real workload is admitted before mining, any conforming CPU or GPU executes its deterministic integer profile, and A-Chain alone validates the canonical transcript, consumes the global work nullifier, and authorizes AI. Z-Chain compresses finalized receipts through P3Q while Lux Quasar supplies post-quantum finality; destination chains consume receipts and never become mining authorities.

Hanzo AI Research
Proof of AIA-ChainP3QAI Mining
PDFSource
Lux AIVM settlement paperHanzo Network

Cloud Economics of a Sovereign OSS Stack

A Regulated Capital-Markets Migration Case Study

Cut cloud spend ~90% — from ~$35K/mo to ~$3.5K/mo (~$380,000/yr) — migrating a regulated capital-markets platform off a sprawling public-cloud footprint (200+ microservices, 185 projects on GCP) onto a consolidated, self-hostable open-source stack on owned hardware. Decomposes where the savings come from (no metered model markup, no hyperscaler margin, efficient transport, consolidation) and shows the same mechanism applies to any organization carrying a metered-API plus hyperscaler-sprawl cost structure.

Hanzo AI Research
Sovereign AICloud EconomicsSovereign StackOSS
PDFSource

One Native Stack for Private, Continuously-Learning AI

Inference at the Bandwidth Wall, On-Device QLoRA, One Engine

A synthesis paper: train and serve collapse into a single Rust engine with one tensor type, one quantization core, and one device backend — eliminating the format-break tax of training in PyTorch, exporting to GGUF/ONNX, and serving on a third engine. Formalizes a unified bandwidth-budget theorem and QLoRA bit-exactness, with every claim sourced to a measured ROCm campaign on consumer hardware. Scrupulously scoped: claims no speed win over NVIDIA and no zero-leakage guarantee.

Hanzo AI Research
EngineInferenceOn-DevicePrivacy
PDFSource

Native ROCm Inference on a Consumer RDNA3.5 APU

Reaching llama.cpp Decode Parity via a Unified 1-bit-to-Full Quant Core

A measurement-driven engineering campaign that took native-ROCm LLM inference on a consumer AMD Ryzen AI Max+ 395 "Strix Halo" (Radeon 8060S, gfx1151) from the slowest of five backends to decode parity (0.98×) and 0.86× prefill versus llama.cpp on the same GPU. A lane-strided decode kernel reaches 245.8 GB/s — 97% of the memory-bandwidth wall. Documents negative results in detail and derives the physical ~25 tok/s ceiling that bounds both engines.

Hanzo AI Research
EngineROCmInferenceBenchmarks
PDFSource

Edge LLM Inference Across Commodity Accelerators

Measured AMD, NVIDIA, and Apple Performance

Measured single-GPU inference for one 8B dense transformer at Q8_0 across three commodity edge accelerators — AMD "Strix Halo", NVIDIA GB10 "Grace Blackwell", and Apple M4 Max — comparing the Hanzo engine to llama.cpp on identical model and metric. Two findings: decode parity on all three backends, and the fastest box flips by workload (NVIDIA wins prefill, Apple wins decode ~2.5×). Includes a full reproducibility appendix with exact hardware, commands, and git SHA.

Hanzo AI Research
EngineInferenceEdgeBenchmarks
PDFSource
The numbers, live: hanzo.ai/benchmarks/inference

Native Training in the Hanzo Engine

One Engine, One Quant Format, One Backend

The architecture for training language models natively inside the Hanzo Rust inference stack, eliminating the conversion boundary between a Python training stack and a separate serving engine. Shows the QLoRA primitive already exists in the engine's reverse-mode autograd, and formalizes QLoRA parameter/memory accounting plus additive LoRA-soup composition. Unusually candid: an entire section names the stack's own non-functional stubs to replace.

Hanzo AI Research
EngineTrainingQLoRAOn-Device
PDFSource

Continuously-Learning Private AI

Self-Improvement and Privacy in the Hanzo Native Stack

A private continuously-learning model as three loops: an inference loop that serves and captures its own interactions, a curation loop that filters/redacts/de-duplicates into safe training data, and a training loop that folds it back into cheap swappable adapters. Formalizes a privacy invariant (no PII token enters the training set under a sound gate) and bounded-forgetting (frozen base → zero base drift), and is honest about model-collapse and delta-leakage failure modes and which components are still research.

Hanzo AI Research
EnginePrivacyContinual LearningOn-Device
PDFSource

Economical Serving and the hanzo.ai Platform

A Bandwidth-Optimal Unified Train-and-Serve Runtime

The serving economics, platform architecture, and on-device thesis that follow from a native Rust train-and-inference engine whose decode kernel runs at the memory-bandwidth wall. Derives a cost law (tokens/$/s ∝ B / (q·N_active)) showing quantization — not bigger accelerators — is the primary cost lever, and proves paged-attention tenant concurrency and a structural egress bound (raw data never crosses the wire). Honestly marks train/personalize/federate as stub/roadmap.

Hanzo AI Research
EngineEconomicsPlatformOn-Device
PDFSource

The Unified Sovereign-Tenant Cloud

Linear Shared-Nothing Scaling via Per-Tenant SQLite and In-Process Composition

The measured memory-and-scaling case for running an entire multi-tenant cloud as one Go binary — 56 in-process subsystems over shared-nothing storage, where every tenant is its own post-quantum-encrypted SQLite file and writes shard embarrassingly with no global lock. Measured through the production open path: an actively-served tenant costs ≈0.25 MB of RAM and an idle tenant essentially zero, so memory scales with concurrency, not registrations — one billion registered tenants at 1% peak concurrency fits ≈20 commodity 128 GB nodes. Measures the shared-Postgres microservice-mesh alternative at 11–32× the memory footprint, and is explicit about writer-pinning failover, hot-tenant skew, and cross-tenant query limits.

Hanzo AI Research
CloudMulti-TenancySQLiteBenchmarks
PDFSource

Refutation-Driven Performance Engineering

An Empirical Study of a Multi-Week GPU Kernel Campaign

An N=1 observational study of a multi-week campaign to close the llama.cpp inference gap on three accelerators (AMD gfx1151 RDNA3.5, NVIDIA GB10 Blackwell, Apple M4 Max) inside a one-source GPU kernel DSL. The primary artifact is not a kernel but a refutation log: twenty-five numbered hypotheses, most plausible, each killed by a cheap decisive experiment. Reports a Vulkan-prefill win ladder (212→1734 tok/s, 8.2× of the gap closed in one ~36-hour cadence), a CUDA decode gap closed to parity by a single model-eligibility entry with zero kernel work, and an instrument fix that collapsed prefill benchmark noise from ±42% to ±0.8% (52×). Formalizes three recurring algorithms — a multi-fidelity Gate Ladder, a decisive-experiment selection rule, and refutation-log-as-prior — and a measurement doctrine of same-run, sustained-only ratios. Honest about standing: two wins, one parity, three open gaps.

Hanzo AI Research
EngineGPU KernelsBenchmarksMethodology
PDFSource
Companion: Evolutionary Schedule SearchThe numbers, live: hanzo.ai/benchmarks/inference

Evolutionary Schedule Search in a One-Source Kernel DSL

And the Continual-Improvement Loop It Anchors

A one-source kernel DSL exposes each operation's schedule — tile dimensions, buffering depth, vector width, per-shape selection — as compile-time knobs, so schedule search becomes a discrete search over monomorphized, bit-exact kernels. Building on an existing per-(device, op, shape) winner cache, this paper extracts the constants such a search must respect from the campaign's ~15 hand-evaluated configurations (free static rejection, a ±0.4% per-op fitness signal, strongly non-additive lever interaction, and hard device non-transfer), specifies a multi-fidelity evolutionary schedule search (~300 configurations/night against ~15 by hand), and embeds it in a continual-improvement loop where a calibrated judge flock — weighted by measured ground truth rather than preference — selects training signal under a Goodhart tripwire, framed as the small-N precursor of a mean-field game. The search is specified, not yet run; that boundary is marked throughout.

Hanzo AI Research
EngineAutotuningEvolutionary SearchKernels
PDFSource
Companion: Refutation-Driven Performance Engineering

Hanzo Router

Memory-Aware, Local-First LLM Routing with a Learned SLO-Constrained Policy

A production request stream is dominated by its easiest queries — short chats, classification, formatting, boilerplate — yet frontier pricing charges the hardest-query rate on every token. Describes the router hanzo-node uses to place each request across a swappable pool of local and cloud models, decomplecting mechanism from brain: the mechanism is pure logic over value inputs (a model registry, a memory snapshot, the set of already-loaded models, a per-request SLO) that prefers reusing a resident model at zero marginal cost, then loading a local model that fits available memory, then the cheapest usable cloud model; the brain is a swappable policy behind a one-method seam — a rule-based cold-start policy, a learned bilinear utility with per-user online LinUCB adaptation, or a small trainable routing encoder. Gives the SLO-constrained objective the learned policy optimizes (quality − λ·cost − μ·latency under hard feasibility ceilings), the memory-aware fit check that makes local-first placement deterministic and unit-testable, and a usage-plane signal that biases routing away from providers near their rate limits. Claims no measured benchmark saving: instead an itemized cost model that goes beyond API token prices — serving simple traffic locally for the price of electricity rather than a rented GPU — whose arithmetic yields roughly a 90% reduction in total AI spend for a mixed workload.

Hanzo AI Research
RouterRoutingEconomicsOn-Device
PDFSource

What a Dormant Agent Costs

State, Execution and Isolation Measured Separately at a Fleet of One Million

An agent platform is usually costed as though an agent were a process, which sets the per-agent floor at a container runtime's minimum reservation — 128–512 MB on commodity platforms. At that floor a fleet of a million agents is a datacenter, and every design decision downstream of the estimate is made in its shadow. We take the estimate apart. An idle agent is three separable things: the state it will resume from, the execution context that holds its loop, and the isolation boundary its code runs behind. These have no reason to cost the same, and measured on one laptop they differ by five orders of magnitude. State at rest is a row: 477 bytes on disk, so a million dormant agents occupy 454.6 MiB, written at 309,789 agents/s and resumed in 0.034 ms in-process. Execution at rest is a goroutine: 601 bytes of heap, identical on every run, so a million live loops are 573.0 MB and each starts in 125 ns. Isolation is the expensive one, and it is expensive for a reason: an in-process V8 context costs 0.13 ms but runs only JavaScript, while a microVM costs 309 ms — 2377× more — and is the only one of the two that can run a test suite, a package manager or a shell. The conclusion is not that containers are wasteful. It is that a fleet pays for isolation per running agent, not per registered one, while the industry estimate charges the isolation price to every agent whether or not it is doing anything. We give the capacity model this implies, and we are explicit about what 601 bytes is: a floor for an agent holding nothing, not a forecast for an agent holding a conversation.

Hanzo AI Research
AgentsCostIsolationBenchmarks
PDFSource
State at rest, live: hanzo.ai/benchmarks/fleetExecution at rest, live: hanzo.ai/benchmarks/goroutineIsolation, live: hanzo.ai/benchmarks/sandboxHarness: hanzoai/cloud · bench

Hops Do Not Fix Multi-Hop Retrieval

A Negative Result on Long-Horizon Conversational Memory, and What the Failure Actually Is

On LoCoMo, a dense retriever answering single-hop questions finds every required turn 78.2% of the time at k=20. On multi-hop questions the same retriever over the same index finds every required turn 22.7% of the time — but finds at least one of them 80.1% of the time, essentially the single-hop rate. It reliably finds a relevant turn and reliably misses the rest. The textbook fix is to search more than once and let the first result inform the second; we implemented four retrieval policies over one index — a single search, pseudo-relevance feedback, a union over neighbours of the top hits, and a two-deep chain — so that the only variable is the policy. None of them fixes it: the best gained 0.7 percentage points on multi-hop at double the latency, two were worse than a single search, and the one that gained on multi-hop lost 3.1 points on single-hop. The diagnostic says why. Multi-hop questions need 3.1 gold turns on average (up to 19), and 95.4% of that evidence spans more than one conversation session; the median gold turn sits at rank 24 and the median worst gold turn at rank 67, beyond k=20 for 77.2% of questions and beyond k=100 for 43.1%. Additional hops re-search the same similarity space, and a turn that is far away in that space stays far however many times it is queried. The missing signal is linkage, not similarity — a structure to be built, not a search to be repeated. We publish this because the negative result is the useful part: it rules out the cheap fix and names the expensive one.

Hanzo AI Research
MemoryRetrievalLoCoMoBenchmarks
PDFSource
The numbers, live: hanzo.ai/benchmarks/locomoHarness: hanzoai/cloud · bench/brain

Retrieval Scoped by Subject

A factorial measurement of conversational memory on LoCoMo, and the negative controls that locate the effect

We measure a graph-structured conversational memory against a flat nearest-neighbour index on LoCoMo — 10 conversations, 272 sessions, 5,882 turns, 1,986 questions — using the benchmark's own scorer, ported to Go and verified against the original Python on all 1,986 questions and every system reported. Three results follow, and the negative controls are the substance of the paper. First, the aggregate over all five question categories is degenerate: a system that replies “No information available” to every question scores 0.228, above every system here that has to find its own evidence, and only an oracle handed the dataset's own annotated evidence scores higher, at 0.252. A quarter of LoCoMo's questions are adversarial and the official metric awards a point for declining them, so any mean over the full set measures the abstention rate before it measures the memory; the answerable and adversarial halves have to be reported apart. Second, the effect that survives is subject scope. Restricting retrieval to the person a question names scores 0.502 against 0.200 on the 446 adversarial questions (95% CI over conversations [+0.247, +0.347]) at no measurable cost on the 1,540 answerable ones (0.142 against 0.144, CI [-0.010, +0.006]). The mechanism is reader-independent: with the result limit lifted, the graph memory reaches 76% of the annotated evidence for answerable questions and 27% for adversarial ones, where the flat index reaches 99% and 96%. It is selectively blind, and the blindness falls where the trap turns are. Third, the graph itself is inert — empty it entirely and every number is byte-identical.

Hanzo AI Research
MemoryRetrievalLoCoMoNegative Controls
PDFSource
The numbers, live: hanzo.ai/benchmarks/locomo-subject-scopeHarness: hanzoai/semantic · bench/locomo

Hanzo Network Whitepaper

L1 Blockchain for Decentralized AI Compute

Comprehensive whitepaper describing Hanzo Network's Layer-1 blockchain infrastructure, integrating HMM for compute pricing and supporting PoAI consensus for quality verification.

Hanzo AI Team
BlockchainInfrastructureNetworkArchitecture
PDFSource