Training AI agents across tools stops interface lock-in

An open method applies reinforcement learning across platforms like Claude Code and OpenCode without modifying the tools themselves.

On October 5, 2026, Hugging Face and FineEnvs released a technical guide titled The ultimate guide to multi-harness RL, led by researcher Adithya S. Kolavi. The project addresses a persistent bottleneck in training AI agents for coding: when a model learns within a single developer tool, it quickly memorizes that interface’s idiosyncrasies and struggles when moved to a different client.

Their solution uses reinforcement learning, where models earn rewards for taking valid actions, distributed across several software environments simultaneously. Crucially, this setup requires no code changes to tools like Claude Code, OpenCode, or Mini-SWE-Agent.

The single-harness problem

In modern software development, agent performance does not depend solely on model weights. It relies on the harness, the external wrapper that orchestrates system prompts, handles tool calls, manages context windows, and retries failed steps.

Benchmark experiments revealed a sharp drop across different harnesses. Liquid AI’s compact LFM2.5-2.6B model solved 62% of benchmark tasks when evaluated inside Mini-SWE-Agent. Running those exact same model weights inside Claude Code dropped the success rate to 33%.

Fine-tuning inside a single harness teaches the model to overfit to that specific tool’s interface quirks. Moving the weights to another environment often breaks the workflow.

How training AI agents with OpenEnv works

To connect disparate tools without rewriting their codebases, the team built an open modular stack around three components:

1. Capture proxy (OpenEnv): sits between the agent interface and the inference engine (such as vLLM or SGLang). It intercepts four common API formats (OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Gemini), logging generated tokens and probabilities without altering tool behavior.
2. Task sandboxing (Harbor v0.22.0): spins up isolated containers to safely execute commands and code modifications.
3. Reinforcement learning (TRL): consumes recorded trajectories using Async GRPO to update model parameters.

With this pipeline, LFM2.5-2.6B boosted its average task completion rate on test sets from 42.2% to 54.2%, with balanced gains across all tested interfaces. In contrast, training restricted to OpenCode lifted its internal score from 34% to 58%, but produced almost no improvement on other platforms.

Real practice beats copying larger models

The study also compared interactive reinforcement learning against supervised fine-tuning (SFT). The authors trained LFM2.5-2.6B on 3,189 successful problem trajectories distilled from Qwen3.8-27B. Supervised imitation stalled at 47.5% task resolution, trailing direct reinforcement learning.

Directly interacting with the environment let the smaller model discover its own error recovery strategies, proving more effective than passive imitation.

The team also implemented an efficiency penalty for redundant commands. This reward structure reduced tool calls by 31.1% on completed tasks, saving execution time and context budget.

Available code and known bottlenecks

Hugging Face and FineEnvs released the full pipeline: proxy code in OpenEnv, training configurations in TRL, the SmolDataEnvs benchmark (1,000 training and 250 test tasks based on data science notebooks), and seven trained checkpoints.

The authors also noted technical boundaries. The tests focused on small architectures (2B to 2.6B parameters) solving Python data tasks. How well the approach scales to models above 30B parameters or complex terminal workflows remains to be tested. On the infrastructure side, the single-process proxy faced contention near 200 concurrent sessions and stalled at 320, showing that concurrency tuning is needed before running on larger clusters.

Sources

• Technical guide on Hugging Face Spaces
• Official Hugging Face announcement on X
• Adithya S. Kolavi post on X
• LFM2.5-2.6B model repository on Hugging Face
• OpenEnv code repository on GitHub
• TRL library on GitHub
• Sina Tech report