The energy-efficiency bottleneck of large-model inference is spawning an alternative hardware approach.
Traditionally, LLM inference relies on GPUs' arithmetic operations to perform core operations like matrix multiplication. But FPGA's strength isn't raw compute — it's a large amount of distributed on-chip storage and flexible datapath control, capability that is wasted under the traditional arithmetic paradigm.
LUT-LLM, presented at the FCCM 2026 conference, is the first to implement memory-based computation on FPGA to replace arithmetic operations for running billion-parameter language models. The core idea: quantize the LLM's matrix-operation results into discrete encoding tables, and look up results by index at inference, sharply reducing multiply-accumulate operations. The scheme adopts activation-weight joint quantization to minimize quantization error.
On the engineering side there are three key optimizations: bandwidth-aware parallel centroid search reduces decoding latency; efficient 2D table lookup cuts table-access overhead; a space-time hybrid architecture reduces data cache passes and improves throughput.
Benchmarks on AMD V80 FPGA with Qwen 3 1.7B show 4× fewer arithmetic operations, 1.1-3.3× faster generation, and 3-6.6× the energy efficiency of GPU. For scenarios pursuing low-power, low-latency inference, this direction is worth attention. The current scheme, however, is deeply adapted to a specific quantization strategy, and generalization to other model families still requires further research.
The hardware competition for LLM inference is shifting from "compete on compute" to "compete on architecture." Lookup tables replacing computation may be just the beginning.