🚀 Introducing FlashQLA: high-performance linear attention kernels built on TileLang. ⚡ 2–3× forwar...
TL;DR · AI 摘要
阿里云推出FlashQLA，基于TileLang的高性能线性注意力内核，实现2-3倍前向加速和2倍后向加速，专为个人设备上的代理AI设计。
核心要点
- FlashQLA实现了2-3倍前向加速和2倍后向加速。
- 采用门控驱动的自动片内CP和硬件友好的代数重构。
- 特别适用于TP设置、小型模型和长上下文工作负载。
结构提纲
按章节快速跳转。
思维导图
用一张图看清主题之间的关系。
查看大纲文本（无障碍 / 无 JS 友好）
- FlashQLA: 高性能线性注意力内核
金句 / Highlights
值得收藏与分享的关键句。
🚀 Introducing FlashQLA: high-performance linear attention kernels built on TileLang.
⚡ 2–3× forward speedup. 2× backward speedup.
💻 Purpose-built for agentic AI on your personal devices.
💡Key insights: 1. Gate-driven automatic intra-card CP. 2. Hardware-friendly algebraic reformulation. 3. TileLang fused warp-specialized kernels.
⚡ 2–3× forward speedup. 2× backward speedup. 💻 Purpose-built for agentic AI on your personal devices.
💡Key insights:
- Gate-driven automatic intra-card CP.
- Hardware-friendly algebraic https://t.co/pA9HCHwFZw" / X
Introducing FlashQLA: high-performance linear attention kernels built on TileLang. 
2–3× forward speedup. 2× backward speedup. 
Purpose-built for agentic AI on your personal devices. 
Key insights: 1. Gate-driven automatic intra-card CP. 2. Hardware-friendly algebraic reformulation. 3. TileLang fused warp-specialized kernels. FlashQLA boosts SM utilization via automatic intra-device CP. The gains are especially pronounced for TP setups, small models, and long-context workloads. Instead of fusing the entire GDN flow into a single kernel, we split it into two kernels optimized for CP and backward efficiency. At large batch sizes this incurs extra memory I/O overhead vs. a fully fused approach, but it delivers better real-world performance on edge devices and long-context workloads. The backward pass was the hardest part: we built a 16-stage warp-specialized pipeline under extremely tight on-chip memory constraints, ultimately achieving 2×+ kernel-level speedups. We hope this is useful to the community!
Learn more: 
Blog: qwen.ai/blog?id=flashq Code: github.com/QwenLM/FlashQLA
Made with AI