ITBench-AA: Frontier Models Score Below 50% on the First Benchmark for Agentic Enterprise IT Tasks — by Artificial Analysis and IBM
Hugging Face Blog861 字 (约 4 分钟)
85
ITBench-AA is a new benchmark series evaluating models on agentic enterprise IT tasks, starting with Site Reliability Engineering tasks where frontier models score below 50% on ITBench-AA's SRE tasks benchmark model performance on Kubernetes incident response, where models and agents must diagnose live systems by reading logs, tracing dependencies, and identifying root-cause entities across complex infrastructure.
入选理由：Claude Opus 4.7 在 ITBench-AA 中表现最佳，得分为 47%
FeaturedArticle#ITBench-AA#Site Reliability Engineering#Frontier Models#IBM#Kubernetes中文