MoE Travelogue: 8. Forced Sequence-Level Load Balancing
科学空间4785 字 (约 20 分钟)
85
This article proposes a new sequence-level load balancing method called Moving Quantile Balancing (MQB), which achieves fine-grained balance in MoE models without relying on auxiliary losses.
入选理由：MQB方法基于Quantile Balancing演化而来，适用于序列级负载均衡。
FeaturedArticle#MoE#Load Balancing#Deep Learning#Routing Mechanism#MQB中文