英伟达推出 AI 框架 Polar，让 Codex 跑分暴涨 594.74%

IT Home May 28, 2023. NVIDIA's research team has released the open-source framework Polar this week. It allows existing intelligent agent frameworks such as Codex, Claude Code, and Qwen Code to access GRPO (Generalized Relative Policy Optimization) training without disrupting the original tool calls, context organization, or patch submission methods.

IT Home Note: GRPO is an optimization method for reinforcement learning training that adjusts model strategies based on reward signals, enabling the model to learn better actions in multi-step decision tasks.

In this article, GRPO is primarily used for code agent training, allowing the model to continuously improve its performance in real tool call and patch submission processes.

The paper notes that intelligent agent reinforcement learning is transitioning from single-step tasks to long process tasks, such as code repository modifications, browser operations, and operating system interactions. These tasks often rely on existing execution frameworks that include multiple rounds of calls, tool usage, context compression, and sub-agent collaboration.

The current challenge is that these frameworks are difficult to directly rewrite into traditional reinforcement learning environment interfaces. Forcing them to connect may also result in the loss of key training signals.

NVIDIA Polar does not rewrite intelligent agent frameworks; instead, it focuses on placing agents at the model API boundary with minimal changes to the original harness.

A harness refers to intelligent agent runtimes like Codex CLI, Claude Code, Qwen Code, and Pi. Traditional reinforcement learning infrastructure typically requires rewriting this logic into environment interfaces similar to env.init(), env.step(), and env.reset(). This can be costly and may result in the loss of native execution details.

The core design of Polar treats the interface between the agent and the model as the training boundary rather than modifying the execution framework itself into an environment.

It places the model agent between the execution framework and the inference server, compatible with request styles from Anthropic, OpenAI, and Google, records prompts, sampled tokens, log probabilities, and response content during request forwarding, and then reconstructs this information into trajectories consumable by the trainer.

In terms of system structure, Polar consists of a rollout server and a gateway node. The former is responsible for task submission, session scheduling, state persistence, and callback reception; the latter handles the full lifecycle of session execution, including runtime startup, execution framework preparation, trajectory building, result evaluation, and resource recycling.

The paper also splits initialization, running, and post-processing into independent work pools and sets a READY buffer, allowing runtime warming and evaluation warming to run in parallel in the background, reducing the blocking of long-tail tasks on GPU training.

The experimental part focuses on software engineering tasks. Based on the same Qwen3.5-4B base model, after training with Polar and GRPO (Group Relative Policy Optimization) on four code execution frameworks—Codex, Claude Code, Qwen Code, and Pi—the pass@1 scores in SWE-Bench Verified improved from 3.8% to 26.4% (an increase of 594.74%), 29.8% to 34.6%, 34.6% to 35.2%, and 34.2% to 40.4%.

In terms of efficiency, prefix_merging reduces the number of updates in three training steps from 1185 to 218, wall clock time from 189.5 minutes to 35.2 minutes, approximately 5.39 times faster; the average GPU utilization of rollout also increased from 20.4% to 87.7%.

IT Home provides reference links:

• Polar: Agentic RL on Any Harness at Scale
• Polar

Advertising Disclosure: The external jump links (including but not limited to hyperlinks, QR codes, passwords, etc.) within the text are used to convey more information and save selection time. The results are for reference only. All articles on IT Home contain this disclaimer.