量化技术

even your phone can now run a 27 billion parameter model. fully offline, no API, no bill, in under 4GB. it is called Bonsai 27B, and it is the first model of its size to fit in your pocket. here is how they pulled it off, and where it quietly breaks 👇

Robert Youssef(@rryssf) · 8.5 分

Bonsai 27B是首个可在手机上运行的270亿参数模型，通过量化与架构优化实现4GB内存部署。