compute-bound·2.0Mtok/s·8d

Memory-bound workloads scale with HBM bandwidth; compute-bound ones don't notice. At your current AI of ~7389.2 vs ridge ~591, doubling HBM recomputes decode at 42.7 tok/s — a +100% change vs the current 21.3. HBM stacks add ~40% to GPU cost.

Widening NVLink from 8 → 16 GPUs per domain keeps more all-reduce traffic on-fabric and lets TP up to 16 avoid the 1.4× TP-over-IB penalty. Re-running step time gives a +0% change on this config — biggest wins when TP currently exceeds the NVLink domain.

Doubling to 2,048 GPUs (DP 16→32) costs another $72.9M. The sim re-runs step time at the new scale and yields 3.77× training throughput — the 0% shortfall vs an ideal 2× is extra all-reduce time eating into compute.