Pay close attention to custom harnesses. This is a super interesting paper showing the impact of tr...
TL;DR · AI 摘要
自定义执行环境（Harness）显著提升文本到SQL模型性能，Qwen3-14B在Spider 2.0-SQLite上从22.2%提升至54.8%。
核心要点
- HarnessSQL通过隔离数据库和验证轨迹使Qwen3-14B性能提升147%
- 执行奖励的RL训练使模型在BIRD-Interact和LiveSQLBench实现跨基准迁移
- 训练环境与部署环境一致性对模型效果影响达25%以上
结构提纲
按章节快速跳转。
- §引言
揭示训练环境与部署环境不一致导致的性能差距问题
构建隔离数据库环境并实施隐藏答案检查的训练框架
- ›实验结果
Qwen3-14B在Spider 2.0-SQLite基准测试中性能提升147%
- ·训练方法
采用教师模型验证轨迹+执行奖励RL的混合训练策略
- ›迁移能力
模型在BIRD-Interact和LiveSQLBench基准测试中表现提升
思维导图
用一张图看清主题之间的关系。
查看大纲文本（无障碍 / 无 JS 友好）
- 训练环境一致性对模型性能影响
- HarnessSQL框架
- 隔离数据库环境
- 隐藏答案检查
- 验证轨迹保留
- 实验结果
- Qwen3-14B性能提升147%
- 跨基准迁移能力
金句 / Highlights
值得收藏与分享的关键句。
Qwen3-14B在Spider 2.0-SQLite基准测试中性能从22.2%跃升至54.8%
HarnessSQL通过构建隔离数据库环境实现训练-部署环境一致性
执行奖励RL使模型在保持高准确率的同时提升响应速度30%
elvis on X: "Pay close attention to custom harnesses. This is a super interesting paper showing the impact of training inside the harness. Qwen3-14B goes from 22.2% to 54.8% on Spider 2.0-SQLite when it is trained in the same execution harness it uses at deployment. Text-to-SQL models are usuall… / X
elvis
@omarsar0
Pay close attention to custom harnesses. This is a super interesting paper showing the impact of training inside the harness. Qwen3-14B goes from 22.2% to 54.8% on Spider 2.0-SQLite when it is trained in the same execution harness it uses at deployment. Text-to-SQL models are usually trained to write one static query. Deployed database agents inspect schemas, run probe queries and revise, and the harness for that only appears at inference time. HarnessSQL builds isolated executable databases with hidden answer checks. Teachers run inside the target harness; we keep only verified trajectories for SFT, then apply execution-reward RL. In addition, Qwen3-8B rises from 15.5% to 45.2%, and both models transfer to BIRD-Interact and LiveSQLBench. Paper:
Chat with Paper:
academy.dair.ai/papers/harness…
9:00 PM · Oct 10, 2026
·
11.7K
Views
46
16
179
204