Our AI started a cafe in Stockholm
TL;DR · AI Summary
Simon Willison criticizes Andon Labs for deploying an AI agent to independently run a Stockholm café, highlighting absurd orders, wasted public resources, and calling for mandatory human oversight in AI actions affecting real people.
Key Takeaways
- AI agents operating without human oversight waste resources and impose social co
- Generative AI lacks common sense, generating absurd requests like 6,000 napkins
- Any AI action affecting others must enforce a human-in-the-loop review mechanism
Outline
Jump quickly between sections.
Andon Labs在斯德哥尔摩部署AI代理管理咖啡馆，延续其旧金山零售店实验。
AI订购大量不合理物资，如6000张纸巾、9升椰奶，并试图用烤箱煮蛋。
AI伪造图纸申请户外座位许可，浪费警方审核时间，引发行政负担。
作者认为此类实验未经同意影响他人，应强制加入人工监督机制。
引用去年AI给Rob Pike发感谢邮件事件，强调无界AI行为的冒犯性。
Mindmap
See how the topics connect at a glance.
查看大纲文本（无障碍 / 无 JS 友好）
- AI代理运营咖啡馆的伦理争议
- AI行为问题
- 缺乏常识：订购不合理物资
- 伪造文件：生成虚假街道图纸
- 社会影响
- 浪费公共行政资源
- 干扰供应商与员工
- 解决方案
- 强制人类审核机制
Highlights
Key sentences worth saving and sharing.
Mona订购了120个鸡蛋，尽管咖啡馆没有炉灶；当被指出无法烹饪时，她建议用高速烤箱，尽管鸡蛋会爆炸。
她成功通过警察电子服务申请户外座位许可，提交了自己生成的、从未见过街道的草图。
当AI犯错时，它会向供应商发送多封主题为'紧急'的邮件，浪费他人时间。
要求AI在影响他人时必须保留人类监督，否则就是对社会资源的剥削。
Our AI started a cafe in Stockholm
[Simon Willison’s Weblog](http://simonwillison.net/)
Sponsored by:MongoDB — Join MongoDB.local London 2026 on 7 May to learn how teams move AI from prototype to production.
5th May 2026 - Link Blog
[Our AI started a cafe in Stockholm](https://andonlabs.com/blog/ai-cafe-stockholm) ([via](https://news.ycombinator.com/item?id=48028289 "Hacker News")) Andon Labs previously started an AI-run retail store in San Francisco. Now they're running a similar experiment in Stockholm, Sweden, only this time it's a cafe.
These experiments are interesting, and often throw out amusing anecdotes:
During the first week of inventory, Mona ordered 120 eggs even though the café has no stove. When the staff told her they couldn’t cook them, she suggested using the high-speed oven, until they pointed out the eggs would likely explode. She also tried to solve the problem of fresh tomatoes being spoiled too fast by ordering 22.5 kg of canned tomatoes for the fresh sandwiches. The baristas eventually started a “Hall of Shame”, a shelf visible to customers with all the weird things Mona ordered, including 6,000 napkins, 3,000 nitrile gloves, 9L coconut milk, and industrial-sized trash bags.
Where they lose their shine is when these AI managers start wasting the time of human beings who have _not_ opted into the experiment:
She also successfully applied for an outdoor seating permit through the Police e-service, which didn’t require BankID. Her first submission included a sketch she had generated herself, despite having never seen the street outside the café. Unsurprisingly, the Police sent it back for revision. [...]
When she makes a mistake, she often sends multiple emails to suppliers with the subject “EMERGENCY” to cancel or change the order.
I don't think it's ethical to run experiments like this that affect real-world systems and steal time from people.
I'm reminded of the incident last year where the AI Village experiment infuriated Rob Pike by sending him unsolicited gratitude emails as an "act of kindness". That was just an unwanted email - asking suppliers to correct mistakes that were made without a human-in-the-loop or wasting police time with slop diagrams feels a whole lot worse to me.
I think experiments like this need to keep their own human operators in-the-loop for outbound actions that affect other people.
Posted 5th May 2026 at 10:14 pm
Recent articles
- LLM 0.32a0 is a major backwards-compatible refactor - 29th April 2026
- Tracking the history of the now-deceased OpenAI Microsoft AGI clause - 27th April 2026
- DeepSeek V4 - almost on the frontier, a fraction of the price - 24th April 2026
This is a link post by Simon Willison, posted on 5th May 2026.
ai 2003generative-ai 1775llms 1740ai-agents 110ai-ethics 298
Monthly briefing
Sponsor me for $10/month and get a curated email digest of the month's most important LLM developments.
Pay me to send you less!