Introducing Olmo-core 3: Open, scalable training infrastructure for large MoEs
Hugging Face Blog1481 字 (约 6 分钟)
85
Hugging Face发布Olmo-core 3，通过分布式训练优化将MoE模型参数容量提升至47B，吞吐量提升2.7倍。
入选理由：Olmo-core 3采用DDP架构，使47B参数MoE模型吞吐量达52,000 tokens/s/GPU
精选文章#MoE#分布式训练#Hugging Face#大模型英文