MoE

A Chinese AI company has broken the bottleneck of running a 60 billion parameter model on mobile devices using ternary quantization, saving 6x memory with minimal performance loss.

DeepSeek builds a low-cost, high-efficiency model system through multiple foundational innovations to drive China's $10 trillion AI hardware ecosystem and achieve its own $1 trillion valuation.

DeepSeek reduces KV cache requirements through innovations, driving China's AI hardware ecosystem toward a $10 trillion industry.

Thinking Machines released TML-Interaction-Small 276B-A12B, a 276-billion-parameter MoE model with only 12 billion active parameters, achieving sub-200ms end-to-end latency, surpassing GPT-4o and Gemini 3.1-Flash in real-time voice interaction, time-aligned microturns, and visual proactivity, effectively eliminating standard VAD.

Hugging Face发布Olmo-core 3，通过分布式训练优化将MoE模型参数容量提升至47B，吞吐量提升2.7倍。

DeepSeek推出新型非对称架构，通过参数优化和训练方法提升模型性能，参数规模达552B。

开源AI堆栈包含模型、推理、网关、Harness和工具五层，开发者可灵活切换模型并优化成本与性能。

Qwen 3.8 Flash Next通过稀疏注意力和混合专家模型优化了大模型的效率和成本，可在普通硬件上运行。

Qwen3.8-Flash以更低价格和开放权重超越DeepSeek V4 Flash与Claude Opus 4.6 Max，采用Qwen4架构并提供125B参数。

通义千问推出Qwen3.8-Flash模型，开放权重并提供API服务，参数达125B，输入输出价格分别为0.16和0.47美元/百万令牌。

llamafile v0.10.5支持本地运行6GB的Ternary Bonsai 27B和118B的Laguna-S-2.1模型，优化了量化和同步机制，提升性能与实用性。

开源AI模型在性能和多样性上取得显著进展，DeepSeek-V4-Flash-0731等模型推动行业创新。

NVIDIA推出30B参数MoE模型Nemotron 3.5 Lightning，3B活跃参数实现4倍输出速度，支持本地硬件高效运行。

Kimi K3模型已在Ollama云平台上线，但需Pro/Max订阅及额外积分，且未包含在订阅计划中。

Kimi K3模型在Ollama云上发布，具备2.8T MoE架构和1M token上下文窗口，但需付费使用。

MLPerf v6.0新增稀疏计算基准，推动AI系统多样性，DeepSeek V3参数达6710亿。

xAI团队通过拆分Prefill和Decode、MoE专家分片及通信计算重叠，实现Grok在10万张GPU上的高效部署。

NVIDIA NeMo AutoModel 提升了 MoE 模型的微调效率，训练吞吐量提高 3.4-3.7 倍，GPU 内存减少 29-32%。

NVIDIA 推出 NeMo AutoModel，基于 Hugging Face Transformers v5 优化 MoE 模型训练，提升 3.4 到 3.7 倍吞吐量。

Meta-Harness架构正在成为AI基础设施的核心，Omnigent等开源项目推动标准化、安全、可扩展的系统集成。

Mellum2 is an open-source 12B parameter AI model from JetBrains, using MoE architecture to activate only 2.5B parameters per token, reducing inference time by over 50% compared to similar-sized models, specifically designed for software engineering environments with applications in routing, RAG pipelines, and private AI deployment.

Step 3.7 Flash by Yujue Star is a new-generation Flash model for production-grade AI Agents, featuring native multimodal understanding, high throughput with low latency, and enhanced web search. It achieves 97% of Claude Opus 4.6's coding performance at only 1/9 the cost per task, ideal for high-frequency, complex real-world workflows.

Step 3.7 Flash 是一个支持多模态的 MoE 模型，仅激活 11B 参数即可处理 196B 参数规模的任务，适用于编码、代理工作流和结构化输出。

This article proposes a new sequence-level load balancing method called Moving Quantile Balancing (MQB), which achieves fine-grained balance in MoE models without relying on auxiliary losses.

Perplexity published new research on deploying the Qwen3 235B model on NVIDIA GB200 NVL72 Blackwell racks, showing that GB200 outperforms Hopper in high-throughput inference for large MoE models.

NVLS all-reduce latency significantly improves from 586.1µs on H200 to 313.3µs on GB200, with notable performance gains in MoE prefill and decode throughput.

NVIDIA released Nemotron 3 Ultra, a 550B MoE open model for long-running agents, offering 5x faster inference and 30% lower costs.

ZEDA is a novel MoE technique that uses self-distillation to dynamically skip experts, improving inference efficiency and giving models compute budget awareness.

通义千问模型Qwen3.8-Flash-Next在vLLM框架下实现NVIDIA和AMD GPU的原生支持，但技术细节披露有限。

Noam Shazeer 加入 OpenAI，曾参与 transformer 和 MoE 架构研究，谷歌曾以 27 亿美元收购 Character AI 以争取他加入。