Google’s New AI Just Broke The AI Speed Limit: DiffusionGemma

Ashwani Kumar — Sat, 13 Jun 2026 11:20:41 +0000

Google’s New AI Just Broke The AI Speed Limit: DiffusionGemma

Google just revealed DiffusionGemma, a new open AI model that generates text in a totally different way and can run up to four times faster on dedicated GPUs. Then Gemini 3.5 Live Translate pushed real-time voice translation into more than 70 languages, while Xiaomi launched MiMo Code to challenge Claude Code with long-term memory, and OpenAI moved closer to a possible one trillion dollar IPO.

The new Gemini 3.5 Live Translate tool is designed to make cross-language communication more fluid and natural compared to traditional systems (4:42). Here are the primary reasons to use it:

Real-Time Translation: Unlike older systems that require you to wait for a person to finish speaking before translating, this tool listens and generates translated speech with only a few seconds of delay while the speaker is still talking (4:19–4:26).
Maintains Natural Quality: It strives to preserve the original speaker’s tone, pacing, pitch, and rhythm, avoiding the robotic sound common in earlier translation software (4:44–4:53).
Broad Language Support: The system supports over 70 languages and more than 2,000 language combinations in a single meeting, removing the need to funnel everything through English (5:52–6:01).
Automatic Detection: You don’t need to manually configure language pairs; the model detects the spoken language automatically (4:54–4:58).
Versatility: It is built for noisy environments—such as airports, meetings, or busy streets—and is being integrated into tools like the Google Translate app, Google Meet, and third-party platforms via API (5:01–6:19).

Google has officially released DiffusionGemma, an experimental open-weight model that completely shifts how AI processes text. While traditional large language models (LLMs) generate words sequentially from left to right like a typewriter, DiffusionGemma applies the mechanics of image generators to text, acting like a printing press that stamps down large blocks of content simultaneously. [1, 2, 3]

By rewriting the rules of text generation, the model achieves blazing speeds of over 1,000 tokens per second on enterprise hardware. [1]

Breaking the Speed Limit

Traditional autoregressive models waste a significant amount of local GPU processing power because they bottleneck on memory bandwidth while predicting text one single word at a time. DiffusionGemma reverses this layout entirely. [1]

4x Faster Generation: It delivers up to a 4x speedup on dedicated local GPUs compared to standard autoregressive baselines.
Blazing Throughput: It pumps out over 1,000+ tokens per second on a single NVIDIA H100.
Consumer Hardware Friendly: It clocks in at 700+ tokens per second on consumer-tier NVIDIA GeForce RTX 5090 GPUs. [1, 2]

How Text Diffusion Works

Instead of picking the next word in a straight line, DiffusionGemma starts with a canvas of 256 random placeholder tokens (essentially text “noise”). [3, 4]

Parallel Decoding: It evaluates and refines all 256 tokens at the exact same time. [1, 3]
Bidirectional Attention: Because it doesn’t process left-to-right, every single token can “attend” to and look at every other token in the canvas. [1]
Intelligent Self-Correction: The model makes multiple passes over the canvas. It locks in the tokens it is confident about, using them as context to fix and refine the surrounding text in real-time. [1, 3, 5]
Adaptive Inference: It features an adaptive stopping mechanism. Simple or highly structured prompts require fewer denoising steps, allowing the model to finish even faster. [6]

Architecture and Hardware Footprint

Developed by Google DeepMind using Gemma 4 architecture and Gemini Diffusion research, the model is highly optimized for local machine deployment: [1, 7]

Mixture of Experts (MoE): Built as a 26-billion total parameter model.
Active Parameters: It only activates 3.8 billion parameters during inference.
VRAM Limits: When quantized, the model fits comfortably within 18GB of VRAM, letting it run on high-end local consumer rigs.
Open Source: Released under a highly permissive Apache 2.0 license for both commercial and research use. [1, 8]

The Critical Catch: Speed vs. Quality [5]

DiffusionGemma is an efficiency and speed play, not a reasoning upgrade. Google explicitly notes that overall output quality is lower than the standard autoregressive Gemma 4 family. [1, 9, 10]

Because of this, it is not meant to replace production LLMs for general chat or complex reasoning. Its bidirectional nature, however, makes it uniquely powerful for “non-linear” tasks where the end of the text constraints the beginning: [2, 9, 11]

In-line text editing and rapid iteration
Code infilling and autocomplete tools
Mathematical graphs and biological/amino acid sequencing
Structured data constraints: For example, when fine-tuned on Sudoku puzzles (a nightmare task for normal LLMs), its accuracy shot up from 0% to 80% because it could evaluate the entire puzzle grid simultaneously. [1, 2, 9]

Ecosystem Support

The model is already available to download on Hugging Face with day-zero support integrated across major inference frameworks, including vLLM, Hugging Face Transformers, MLX, and NVIDIA NeMo. [1, 2, 12]

Google’s New AI Just Broke The AI Speed Limit