Native ROCm Inference on a Consumer RDNA3.5 APU
Reaching llama.cpp Decode Parity via a Unified 1-bit-to-Full Quant Core
Zach Kelling
EngineROCmInferenceBenchmarks
Abstract
A measurement-driven engineering campaign that took native-ROCm LLM inference on a consumer AMD Ryzen AI Max+ 395 "Strix Halo" (Radeon 8060S, gfx1151) from the slowest of five backends to decode parity (0.98×) and 0.86× prefill versus llama.cpp on the same GPU. A lane-strided decode kernel reaches 245.8 GB/s — 97% of the memory-bandwidth wall. Documents negative results in detail and derives the physical ~25 tok/s ceiling that bounds both engines.