What people call "distillation" is a super common practice (you use other models to benchmark your m...
TL;DR · AI Summary
Clement Delangue discusses the common AI practice of 'distillation,' suggesting it should be considered fair use when models are open-source, fostering innovation and reducing monopolies.
Key Takeaways
- Distillation, using other models for benchmarking and dataset augmentation, is a widespread AI practice.
- Clement argues for fair use coverage for distillation, akin to public data usage, especially for open-source models.
- The discussion hints at ongoing legal debates in AI around model development and intellectual property.
Outline
Jump quickly between sections.
- §引言
Clement Delangue 对‘模型蒸馏’的普遍性及其对公平使用的看法。
介绍模型蒸馏作为评估和数据增强手段的常见做法。
提出应将开放源代码模型的蒸馏视为公平使用，以促进竞争。
- ·引用案例
提及Musk在法庭上的回答，侧面反映行业内的蒸馏实践现状。
Mindmap
See how the topics connect at a glance.
查看大纲文本（无障碍 / 无 JS 友好）
- 模型蒸馏与公平使用
- 蒸馏实践
- 基准测试、输入评估、数据集增强
- 公平使用论点
- 开放源码模型支持竞争
- 行业影响
- 打破AI垄断趋势
Highlights
Key sentences worth saving and sharing.
Distillation...should be covered by fair use...especially when the resulting models are open-source...
Generally AI companies distill other AI companies.
It benefits all and helps break out monopolies that are strongly forming in AI.
clem 🤗 on X: "What people call "distillation" is a super common practice (you use other models to benchmark your model, to evaluate your inputs or to add a little bit to your datasets) that in my opinion should be covered by fair use (just like using public data is), especially when the" / X
Don’t miss what’s happening
clem 
What people call "distillation" is a super common practice (you use other models to benchmark your model, to evaluate your inputs or to add a little bit to your datasets) that in my opinion should be covered by fair use (just like using public data is), especially when the resulting models are open-source as it benefits all and helps break out monopolies that are strongly forming in AI
Quote
MTS
@MTSlive
·
9h
LIVE TRIAL UPDATE: OpenAI's counsel asked Musk whether xAI has ever "distilled" technology from OpenAI. Musk: "Generally AI companies distill other AI companies." "Is that a yes?" Savitt asked. Musk: "Partly."
·
4
11
91
15