Hanzo AI

Research Papers

All papers

Native ROCm Inference on a Consumer RDNA3.5 APU

Reaching llama.cpp Decode Parity via a Unified 1-bit-to-Full Quant Core

Zach Kelling

EngineROCmInferenceBenchmarks

Abstract

A measurement-driven engineering campaign that took native-ROCm LLM inference on a consumer AMD Ryzen AI Max+ 395 "Strix Halo" (Radeon 8060S, gfx1151) from the slowest of five backends to decode parity (0.98×) and 0.86× prefill versus llama.cpp on the same GPU. A lane-strided decode kernel reaches 245.8 GB/s — 97% of the memory-bandwidth wall. Documents negative results in detail and derives the physical ~25 tok/s ceiling that bounds both engines.

Preview unavailable in this browser. Download the PDF.