even your phone can now run a 27 billion parameter model. fully offline, no API, no bill, in under 4GB. it is called Bonsai 27B, and it is the first model of its size to fit in your pocket. here is how they pulled it off, and where it quietly breaks 👇
Robert Youssef(@rryssf)129 字 (约 1 分钟)
85
Bonsai 27B是首个可在手机上运行的270亿参数模型，通过量化与架构优化实现4GB内存部署。
入选理由：采用8位整型量化技术，模型体积压缩至4GB
精选推文#AI模型#移动端#模型压缩#量化技术英文