even your phone can now run a 27 billion parameter model. fully offline, no API, no bill, in under 4GB. it is called Bonsai 27B, and it is the first model of its size to fit in your pocket. here is how they pulled it off, and where it quietly breaks 👇
Bonsai 27B是首个可在手机上运行的270亿参数模型,通过量化与架构优化实现4GB内存部署。
入选理由:采用8位整型量化技术,模型体积压缩至4GB
