Experimental Open Model DiffusionGemma Reports 1,500 Tokens per Second on a Single NVIDIA H100 GPU

An experimental open-weight language model called DiffusionGemma, described in an arXiv preprint dated 31 July 2026, generates roughly 1,500 output tokens per second on a single NVIDIA H100 GPU, its authors report.

The model refines 256-token blocks in parallel rather than decoding one word at a time, the approach most production chat systems use. The reported speed holds up even against autoregressive (AR) models with state-of-the-art speculative decoding, the team says.

DiffusionGemma is not built from scratch. It is a fine-tune of Google's Gemma 4 mixture-of-experts model, 3.8B parameters activated and 25.2B total, using a two-stage pipeline: supervised fine-tuning for bidirectional denoising, then reinforcement learning combined with sampler distillation. The fine-tune consumed fewer than 10% of the starting model's training tokens.

The authors say the model retains Gemma 4's thinking mode, multimodal input, and long-context support, and can still produce AR output with only minor degradation, pointing toward hybrid diffusion-AR decoding. A Hacker News technical thread flags an entropy-based adaptive diffusion-step schedule that could be reused on other open MoE checkpoints.

The caveats are concrete: this is an arXiv preprint, not peer-reviewed; the speed and capability numbers come from the authors' own evaluation suite; no independent benchmark is cited; and the "Pareto frontier" framing is the authors' own, not an established result.