Multi-Token Prediction (MTP)

Gemma 4 12B features an encoder-free multimodal architecture that runs locally on 16GB VRAM devices with native audio support. By eliminating separate vision and audio encoders, it reduces latency and pairs with a dedicated MTP model for faster inference, marking the first mid-sized multimodal model with a macOS desktop app for fully offline interaction.

Google AI invites developers to showcase projects on Gemma 4 MTP.

The article briefly mentions that Gemma 4 uses multi-token prediction to accelerate inference but provides no technical details, experimental data, or implementation methods, making it a promotional lightweight announcement with little engineering value.