Microsoft ThinkingBox 在 Hugging Face 上发布，以数据库终态和 20 次重复评测智能体

AI HOT 精选

AI HOT 精选2026年10月3日

Microsoft ThinkingBox 在 Hugging Face 上发布，以数据库终态和 20 次重复评测智能体

6.9内容质量

TL;DR · AI 摘要

Microsoft ThinkingBox å¨ Hugging Face ä¸åå¸ï¼ä»¥æ°æ®åºç»æå 20 æ¬¡éå¤è¯æµæºè½ä½ Â· AIHOT Hugging Fa...

核心要点

主题聚焦：Microsoft ThinkingBox 在 Hugging Face 上发布，以数据库终态和
来源：AI HOT 精选，建议结合原文判断细节。
AI 分析暂不可用，本条为保底评分与摘要。

#AI#后端#云计算#开源

打开原文

Microsoft ThinkingBox å¨ Hugging Face ä¸åå¸ï¼ä»¥æ°æ®åºç»æå 20 æ¬¡éå¤è¯æµæºè½ä½ Â· AIHOT

Hugging Faceï¼Blog

Â·

2026-10-04 06:56

3 å°æ¶å

ç²¾é

AI è¯å

65

Microsoft ThinkingBox å¨ Hugging Face ä¸åå¸ï¼ä»¥æ°æ®åºç»æå 20 æ¬¡éå¤è¯æµæºè½ä½

The Agent Said It Was Done. The Database Disagreed.

AI å¯¼è¯»

Microsoft ä¸ Hugging Face åå¸ ThinkingBox æºè½ä½æ²ç®±ä¸ ThinkingBox-Bench åºåï¼è¦ç 507 ä¸ªæç¶æä¸å¡å·¥ä½æµãæ¯ä»»å¡è¿è¡ 20 æ¬¡ï¼ä»¥ç»å±æ°æ®åºç¶æåå¯ä½ç¨ä½å¯æ§è¡å¤å®ï¼ç°å¯éè¿ OpenEnv å¨ Hugging Face ä¸è¿è¡ã

æ¨èçç±

æ£æ Â· AI ç¿»è¯

Microsoft ThinkingBox æ ¹æ® AI æºè½ä½çä¸çè®°å½èéå ¶çæçè¯å¥æ¥è¯åï¼ç¶åèå¯å®ä»¬è½å¦è¿ç»äºåæ¬¡åå°è¿ä¸ç¹ãç°å·²éè¿ Hugging Face æä¾ã

å¾ 1ï¼ThinkingBox å¨éç¦»ç MCP å·¥å ·ä¼è¯ä¸è¿è¡æºè½ä½ï¼ç¶åå¯¹å ¶çä¸çç»ç«¯åç«¯ç¶æåå¯ä½ç¨è¿è¡è¯åãæèªæä»¬ç ThinkingBox è®ºæ ã

#### è¿æ¯ Microsoft ä¸ Hugging Face çèååå®¢ï¼ç¹å«æè°¢ Tommy Guyï¼Enderis AI åå§äººï¼æ¤åå¨ Microsoftï¼ãHugging Face ç Sergio Paniegoï¼ä»¥åæä»¬æ¤åçå®ä¹ ç Zhuochun Liï¼å¹å ¹å ¡å¤§å¦ï¼ãAli Keramatiï¼å å·å¤§å¦æ¬§æåæ ¡ï¼ãYoungmin Koï¼è¥¿åå¤§å¦ï¼çå ±åæ°å/å®¡é å·¥ä½ã

ç®å½

å·¥å ·è°ç¨ä¸çäºç»æ

ä¸æ¬¡æåä¸çäºå¯é

ä½ è½ä¾èµæºè½ä½èåçæ¨¡ååï¼

ä¸è´æ§çä»£ä»·

å¤±è´¥ç¹å¾

å·¥ä½åç

èªè¡è¿è¡

ä¸ä¸æ¥èµ°å

æ³å¨é è¯»ç»æä¹åå è¯ä¸è¯ï¼è·³å° èªè¡è¿è¡ é¨åã

å·¥å ·è°ç¨ä¸çäºç»æ

è½¨è¿¹æ¯ä¸ç§å£°æãæ°æ®åºç¶ææ¯è¯æ®ãéå¤æ¯ä¿¡ä»»æµè¯ã

ä¸æ¬¡æåä¸çäºå¯é

è¡¨ 1ï¼æä»¬æ¥åçä¸ä¸ªæ°åï¼ä»¥åæ¯ä¸ªæ°åæåççé®é¢ã

ææ

å®è¡¡éä»ä¹

å®åçä»ä¹

pass@1

ææå°è¯ä¸æåçæ¯ä¾

å®éå¸¸è¡¨ç°å¦ä½ï¼

pass@20

å¨ 20 æ¬¡å°è¯ä¸

è³å°æåä¸æ¬¡

çä»»å¡æ¯ä¾

å®

æ¯å¦æå¯è½

åå°ï¼å¹¿åº¦ã

è§æµå°ç 20/20

å®é éè¿äº

å ¨é¨

20 æ¬¡è®°å½å°è¯çä»»å¡

æ¯å¦æ»è½

æ£ç¡®ï¼

è¡¨ 2ï¼ThinkingBox-Bench æé¢åååç pass@1ï¼%ï¼ãæ¯ä¸ªæ¨¡åé½å¨æ¯ä¸ªä»»å¡ä¸è¿è¡äº 20 æ¬¡éå¤è¯éªè¯ä¼°ãç²ä½æ è®°ç»å é¢å è ï¼ä¸åçº¿æ è®°ç¬¬äºåãåæ¬¡å°è¯å¾åä¼°è®¡çæ åè¯¯è§ æä»¬ ThinkingBox è®ºæä¸çè¡¨ 4 ã

æ¨¡å

é¶å® (98)

æ±½è½¦ä¿é© (100)

æ è¡ (104)

æ°åé¶è¡ (104)

å¨è¯¢ (101)

æ»ä½ï¼æä»»å¡å æ (507)

ä¸ææ¨¡å

Claude Opus 5.5

80.97

68.40

54.28

71.25

61.58

67.16

Claude Opus 5

80.71

65.80

49.95

70.62

66.19

66.50

GPT-5.4

76.33

62.65

68.12

65.34

54.60

65.36

GPT-5.6 Sol

67.65

65.30

60.34

59.09

57.52

61.91

Claude Sonnet 4.6

72.35

54.40

58.94

56.39

54.31

59.19

GPT-6 Astra

71.73

46.55

55.87

60.87

56.83

58.31

GPT-5.2

70.20

22.40

53.70

51.15

34.06

46.28

Claude Opus 4.6

68.62

8.30

21.11

35.67

27.82

32.09

o3-pro

37.70

2.95

17.31

24.28

14.60

19.31

Grok-4.3

43.93

2.60

15.14

1.78

9.55

14.38

å¼æ¾æéæ¨¡å

Kimi-K3

82.24

50.80

61.83

41.35

51.63

57.37

Qwen3.8-27B

64.03

47.85

53.41

47.88

45.69

51.70

DeepSeek-V4-Pro

68.21

29.65

43.13

44.86

31.04

43.26

Kimi-K2.6

53.72

24.50

39.52

33.65

37.33

37.66

GLM-5.1

58.67

25.70

35.43

13.27

33.19

Qwen3.6-27B

43.11

29.00

46.39

27.84

18.37

32.94

Qwen3.5-9B

19.90

0.70

4.71

1.15

2.33

5.65

Mistral-Large-3

11.28

1.30

8.99

0.74

4.66

Claude Opus 5.5 ä»¥ 67.16% æ»ä½ é¢å ï¼æ¯ Claude Opus 5 é«åºä¸åä¹äºä¸ªç¾åç¹ãKimi-K3 æ¯æå¼ºç å¼æ¾æéæ¨¡å ï¼ä¸ GPT-6-Astra ç¸å·®ä¸å°ä¸ä¸ªç¾åç¹ã é¢ååæ ·éè¦ ï¼Claude Opus 4.6 å¨é¶å®ä¸å¾å 68.62%ï¼ä½å¨æ±½è½¦ä¿é©ä¸åªæ 8.30%ã

ä¸æ¬¡å¥½çè¿è¡åè¯ä½ æ¨¡åè½åè¿é¡¹å·¥ä½ãå®ä¸ä¼åè¯ä½ å®æ¯å¦ä¼åæ¬¡åå°ãæä»¥æ¯ä¸ªä»»å¡è¿è¡ 20 æ¬¡ï¼ç¶åé®è¿ä¸ªåæ°æå¤å°è½ä¿çä¸æ¥ã

åªæä¸ä¸ª æ¨¡åä¿çäºå ¶ pass@1 å¾åçå¤§é¨åï¼GPT-6 Astra ä¿çäºå ¶åæ¬¡å°è¯çç 78%ï¼Claude Opus 5.5 å Claude Opus 5 åä¿ç 71%ãå¨å¦ä¸ç«¯ï¼GLM-5.1ãKimi-K2.6 å DeepSeek-V4-Pro åä¿ççº¦ 8%ã

ä½ è½ä¾èµæºè½ä½èåçæ¨¡ååï¼

Kimi-K3 è³å°æåè§£å³ä¸æ¬¡çä»»å¡æ¯ Opus 5 å¤ 75 ä¸ª ã

Opus 5 ç¨³å®è§£å³çä»»å¡æ¯ Kimi-K3 å¤ 173 ä¸ª ã

ä¸è´æ§çä»£ä»·

ç¶åæä»¬å°ä¸æ¬¡è¿è¡çææ¬é¤ä»¥ æå çå°è¯æ¬¡æ°ï¼

æ¯æ¬¡æåä»»å¡å°è¯çææ¬ = 507 æ¬¡å°è¯ï¼æ¯ä¸ªä»»å¡ä¸æ¬¡ï¼çä¼°ç®ææ¬ Ã· (507 Ã pass@1)

ç¤ºä¾ï¼ GPT-5.4 è¿è¡ 507 æ¬¡å°è¯ï¼æ¯ä¸ªä»»å¡ä¸æ¬¡å°è¯ï¼çææ¬ä¸º $43.49ï¼pass@1 ä¸º 65.36%ï¼å æ¤ $43.49 Ã· (507 Ã 0.6536) = æ¯æ¬¡æåä»»å¡å°è¯ $0.131ã

å¸ç´¯æææ¬åæ²¿

åæ²¿æä¸ä¸ªå°é¶ã GPT-5.6 Sol ç æ¯æ¬¡æåææ¬æä½ ï¼ä¸º $0.127ï¼GPT-5.4 ä»¥æ¯æ¬¡æåå¤ $0.004 çä»£ä»·å° pass@1 æé« 3.45 ä¸ªç¾åç¹ï¼Claude Opus 5.5 ä»¥æ¯æ¬¡æå $0.276 åå¢å 1.80 ä¸ªç¾åç¹ãè¿ä¸è é½ä»ä½äºææ¬åæ²¿çº¿ä¸ï¼å ä¸ºæ²¡ææ´ä¾¿å®çæ¨¡åè½å¹é å ¶ pass@1ã

Claude Opus 5 æ¯æææ¾çä¾åï¼æ¯æ¬¡æåå°è¯ $0.475ãpass@1 ä¸º 66.50%ï¼å®æ¢æ¯ Claude Opus 5.5 ç $0.276 å 67.16% æ´è´µï¼åæ´ä¸åç¡®ã

ç°å¨ä¸ºä¸è´æ§å®ä»·

æ¯æ¬¡å¯é ä»»å¡çææ¬ = 507 æ¬¡å°è¯è¿è¡ 20 æ¬¡çä¼°ç®ææ¬ Ã· 20/20 å ¨é¨éè¿çä»»å¡æ°

ç¤ºä¾ï¼ GPT-6-Astra æ´ä¸ªè¿è¡çææ¬ä¸º 20 Ã $86.03 = $1,720.60ï¼å¹¶ä¸å¨æ¯æ¬¡å°è¯ä¸é½éè¿äº 231 ä¸ªä»»å¡ï¼å æ¤ $1,720.60 Ã· 231 = æ¯ä¸ªå¯é ä»»å¡ $7.45ã

20/20 å ¨é¨éè¿çä»»å¡æ°

ä¼°ç®ææ¬ï¼20 æ¬¡è¿è¡

æ¯æ¬¡å¯é ä»»å¡çææ¬

128 (25.25%)

$869.80

$6.80

231 (45.56%)

$1,720.60

$7.45

241 (47.53%)

$1,880.77

$7.80

82 (16.17%)

$800.00

$9.76

$3,206.00

$13.30

102 (20.12%)

$1,587.60

$15.56

44 (8.68%)

$878.00

$19.95

68 (13.41%)

$1,406.40

$20.68

38 (7.50%)

$925.80

$24.36

ç°å¨æä¸è´æ§æåã GPT-5.4 ä»¥ $6.80 æä¸º æä¾¿å® çï¼å°½ç®¡åªæ 128 ä¸ªä»»å¡è¾¾æ ãGPT-6 Astra ä»¥ $7.45 è¾¾å° 231 ä¸ªï¼è Claude Opus 5.5 ä»¥ $7.80 è¾¾å°å¹¶åæé«ç 241 ä¸ªã

å¤±è´¥ç¹å¾

å¤±è´¥ç¹å¾

å¤±è´¥å æ¯

å·¥å ·ä½¿ç¨

79.9%

éè¯¯çç¶ææ´æ°

10.3%

ä¸å®æ´çç¨æ·è§£å³æ¹æ¡

7.0%

æ²¡ææ¹åç¶æçæä½

2.9%

å·¥ä½åç

èªå·±è¿è¡

å¼å§ä¹å

å®è£

code

# 1. OpenEnv + the ThinkingBox environment
git
clone
https://github.com/huggingface/OpenEnv
cd
OpenEnv
uv
sync
--project envs/thinkingbox_env --frozen
# 2. The executable benchmark, at the pinned release
git
clone
https://github.com/microsoft/thinkingbox-data
git -C thinkingbox-data checkout thinkingbox-bench-v1.0
# 3. The ThinkingBox CLI, which provides `tb`
uv tool install
"thinkingbox @ git+https://github.com/microsoft/thinkingbox"

å¯å¨ Typesense

code

mkdir
-p .typesense-data
docker run --
rm
-d --name thinkingbox-typesense \
  -p 8108:8108 \
  -v
"
$PWD
/.typesense-data:/data"
\
  typesense/typesense:30.1 \
  --data-dir /data --api-key=Fake --enable-cors
until
curl -fsS http://127.0.0.1:8108/health;
do
sleep
1;
done

å¯å¨ MCP æå¡å¨

å¨ ç¬¬ä¸ä¸ªç»ç«¯ ä¸ï¼å¯å¨ Session Proxy å MCP æå¡å¨ã

code

cd
OpenEnv
tb mcp-start --host 127.0.0.1 --port 7111 \
  --servers
"
$PWD
/thinkingbox-data/servers/servers.yaml"
curl -fsS http://127.0.0.1:7111/health

å¯å¨ OpenEnv æå¡å¨

åå°ç¬¬ä¸ä¸ªç»ç«¯ï¼æ ¹æ®æå®ä¸ä¸ªæ¨¡åç ThinkingBox YAML é ç½®å¯å¨ OpenEnv æå¡å¨ï¼ é ç½®æå ï¼ï¼

code

OPENENV_TB_CONFIG=
"
$PWD
/thinkingbox.yaml"
\
uv run --project envs/thinkingbox_env --frozen server

æ£æ¥å°±ç»ªç¶æ

code

curl -sS http://127.0.0.1:8000/ready

å¯¹ååè¯å

code

echo
"- sandbox_external_retail_group1.py:test_case_ST002_001"
> one_task.yaml
uv run --project envs/thinkingbox_env thinkingbox-eval \
  one_task.yaml \
  --config
"
$PWD
/thinkingbox.yaml"
\
  --output results.jsonl \
  --errors-output errors.jsonl \
  --repeat 1 --message-timeout 1800

ä¸ä¸æ¥èµ°å

å¦æä½ æ£å¨è¯ä¼°ä¸ä¸ªä¼æ¥è§¦çå®è®°å½çä»£çï¼

æ£æ¥ä¸æ¬¡å¤±è´¥ã æ¾ä¸ä¸ªå¹²åç»æ¢ä½ä»ç¶å¤±è´¥çè¿è¡ï¼ççæ°æ®åºä¸å®é åçäºä»ä¹ååãè¿ä¼éæ°å®ä¹ä½ èªå·±çè¯ä¼°æè¡¡éçå å®¹ã

å¤ç°ä¸ä¸ªä»»å¡ ï¼éè¿ OpenEnv ä½¿ç¨ä½ èªå·±çæ¨¡åã

æ¥åä¸ä¸ªéå¤ææ ï¼å¹¶å®ä¹å®ã æ è®ºä½ çç¨ä¾è¯æ k æ¯å¤å°ï¼è¯´æä½ æ¥åçæ¯ best-of-k è¿æ¯ every-of-kï¼ä»¥åä½ æ¯å¦ä½è®¡ç®çã

æ´å¤ç»èå¯ä»¥å¨ä»¥ä¸é¾æ¥ä¸æ¾å°ï¼

ç¯å¢ï¼ envs/thinkingbox_env

OpenEnvï¼ https://huggingface.co/docs/openenv/environments/thinkingbox

æ¡æ¶ï¼ microsoft/thinkingbox Â· æç¨

åºåï¼ microsoft/thinkingbox-data Â· v1.0 çæ¬

æ°æ®éæ¥çå¨ï¼ microsoft/ThinkingBox-Bench

è®ºæï¼ arXiv:2608.19741 æ HF

RL è®ç»ï¼å³å°æ¨åºï¼ï¼ microsoft/thinkingbox-training

ThinkingBox ä»£ç éç¨ MIT è®¸å¯ï¼åºåæ°æ®éç¨ CDLA-Permissive-2.0 ï¼OpenEnv ç¯å¢ä»¥ OpenEnv ç BSD-3-Clause åå¸ã

ThinkingBox å ThinkingBox-Bench ç± Microsoft Copilot Studio å¢éä¸ Toloka åä½æå»ºï¼åä½è æ¥èªå¹å ¹å ¡å¤§å¦ãè¥¿åå¤§å¦ãå¥ä¼¦æ¯äºå¤§å¦åå å·å¤§å¦æ¬§æåæ ¡ï¼ä»ä»¬æ¾å¨ Microsoft å®ä¹ ãæ¬¢è¿å¨ä¸æ¹è¯è®ºåºæ github ä¸æé®

+ OpenRouter ææ¬å¿«ç §ææäº 2026 å¹´ 9 æ 20 æ¥ï¼Opus 5.5 å®ä»·ä¾æ® Anthropic ç½ç«ã

æ¥æºï¼ Hugging Faceï¼Blog Â· huggingface.co

Microsoft / Copilot

Hugging Face

Agent æºè½ä½

è®ºæç ç©¶

è¯æµåºå

#

è®ºæ/ç ç©¶

Agent

è¯æµ/åºå

Microsoft

Microsoft ThinkingBox 在 Hugging Face 上发布，以数据库终态和 20 次重复评测智能体

TL;DR · AI 摘要

核心要点

Microsoft ThinkingBox å¨ Hugging Face ä¸åå¸ï¼ä»¥æ°æ®åºç»æå 20 æ¬¡éå¤è¯æµæºè½ä½ Â· AIHOT

Microsoft ThinkingBox å¨ Hugging Face ä¸åå¸ï¼ä»¥æ°æ®åºç»æå 20 æ¬¡éå¤è¯æµæºè½ä½

å·¥å ·è°ç¨ä¸ç­äºç»æ

ä¸æ¬¡æåä¸ç­äºå¯é

ä½ è½ä¾èµæºè½ä½èåçæ¨¡ååï¼

ä¸è´æ§çä»£ä»·

å¸ç´¯æææ¬åæ²¿

ç°å¨ä¸ºä¸è´æ§å®ä»·

å¤±è´¥ç¹å¾

å·¥ä½åç

èªå·±è¿è¡

å¼å§ä¹å

å®è£

å¯å¨ Typesense

å¯å¨ MCP æå¡å¨

å¯å¨ OpenEnv æå¡å¨

æ£æ¥å°±ç»ªç¶æ

å¯¹ååè¯å

ä¸ä¸æ­¥èµ°å

Microsoft ThinkingBox å¨ Hugging Face ä¸åå¸ï¼ä»¥æ°æ®åºç»æå 20 æ¬¡éå¤è¯æµæºè½ä½ Â· AIHOT

Microsoft ThinkingBox å¨ Hugging Face ä¸åå¸ï¼ä»¥æ°æ®åºç»æå 20 æ¬¡éå¤è¯æµæºè½ä½

å·¥å ·è°ç¨ä¸çäºç»æ

ä¸æ¬¡æåä¸çäºå¯é

ä½ è½ä¾èµæºè½ä½èåçæ¨¡ååï¼

ä¸è´æ§çä»£ä»·

å¸ç´¯æææ¬åæ²¿

ç°å¨ä¸ºä¸è´æ§å®ä»·

å¤±è´¥ç¹å¾

å·¥ä½åç

èªå·±è¿è¡

å¼å§ä¹å

å®è£

å¯å¨ Typesense

å¯å¨ MCP æå¡å¨

å¯å¨ OpenEnv æå¡å¨

æ£æ¥å°±ç»ªç¶æ

å¯¹ååè¯å

ä¸ä¸æ¥èµ°å