Kaiming He's First Language Model: 105M Parameters, Not Following the GPT Autoregressive Path
量子位2865 字 (约 12 分钟)
85
Kaiming He's team has introduced the first continuous diffusion language model ELF, which achieves a perplexity of 24 on OpenWebText with only 105M parameters and 45B training tokens, outperforming mainstream diffusion models.
入选理由：ELF采用连续扩散方法，仅需32步采样，生成困惑度达到24，优于主流扩散模型。
FeaturedArticle#diffusion language model#continuous diffusion#Kaiming He#ELF#NLP中文