MBU (Model Bandwidth Utilization)

Also: model bandwidth utilization

MBU is the share of an accelerator’s peak memory bandwidth actually achieved during inference. Autoregressive decode is memory-bandwidth-bound, so MBU — not raw FLOPS — usually sets real token throughput: a chip running at 40% MBU delivers roughly 40% of its theoretical token rate. It is the practical ceiling most inference deployments hit first.

← All terms · Definitions maintained by Mintok. Methodology.