Raven: The Harness of Harnesses for Composable Agentic Intelligence
arXiv:2609.33439 · Paper published 2026-09-27 · Original paper ↗
Raven packages language models with domain tools into reusable worker units, using a central host agent to break user requests into workflow graphs, validate file-based handoffs before execution, and run subtasks in parallel.
Previous multi-agent frameworks rely on single-domain tool setups or unstructured text chat between language models. Single-domain setups encode assumptions that fail across heterogeneous tool interfaces. Unstructured messaging introduces inter-agent misalignment, context truncation, and unverified handoffs where errors propagate to downstream workers.
How it works
Decompose request into task graph A user request enters the Host Agent, which loads orchestration rules on demand and breaks the goal into a directed acyclic graph where nodes represent subtasks and directed arrows represent artifact dependencies. Each node assigns a registered specialist agent and specifies input file bindings rather than raw context text, structuring how data flows between workers.
Check graph rules before running The composed graph enters the Graph Runtime, which sequentially validates JSON formatting, graph acyclicity, agent file capabilities, registration status, and dispatch budgets. It outputs an admitted graph or halts execution with a targeted error pointer. Pre-execution validation catches specified structural and capability defects before worker dispatch.
Execute subtasks and record outputs Admitted subtasks with satisfied dependencies enter parallel execution, where assigned specialist agents run their local tool policies and record deliverables to an artifact ledger, which acts as a recorded list of output files. Workers receive file path references instead of duplicated text, which reduces prompt bloat across long workflows.
Evaluate deliverables and resolve errors Completed subtask artifacts enter an independent language model judge, which evaluates outputs against prompt requirements to mark nodes as accomplished or suspended. Suspended nodes trigger exception reports for host continuation or replanning. While the judge can misjudge outputs, this check screens for missing deliverables before downstream nodes consume them.
Illustrative example
To build a multi-part project requiring an algorithm survey, benchmark comparison, code reproduction, and website, a user submits a single request. In an illustrative comparison with unstructured chat, pasting long reports into a shared context causes text duplication and missed file paths. Under Raven, the host creates a graph where two research agents produce survey files on disk, feeding output file paths into a coding agent, which then passes its code path to a design agent. The runtime binds outputs using file references like {{ research_alg.output_path }} in the artifact ledger, allowing the coding agent to read verified research files directly. File references reduce text duplication, though they do not guarantee every agent stays within context limits.
Evidence
Reported evidence and its scope
Reported result
Test conditions
What it supports
Achieved a Node F1 of 0.923 (+14.7 percentage points over best baseline) and Exact Match rate of 0.711 (+10.4 pp) with Qwen3.8-27B, and Node F1 of 0.963 (+10.9 pp) and Exact Match rate of 0.867 (+10.5 pp) with DeepSeek-V4-Flash-0731.
Multi-Agent Orchestration Benchmark (MAOB) across 140 occupational tasks, comparing host-predicted DAGs against reference DAGs before worker dispatch.
Demonstrates improved specialist selection and graph prediction over baseline orchestrators on identical backbones, but does not measure downstream worker execution quality.
Raven-Oncall solved 82.35% (14/17) of AI4S scientific automation tasks versus 64.71% (11/17) for Claude Code under Claude Opus 5, while reducing average monetary cost by 62.5% ($5.10 vs $13.60 per case).
17 long-running scientific automation tasks on registered machines under fixed execution resource limits.
Validates that campaign monitoring, wake-up scheduling, and budget clamping reduce monetary overhead on unattended jobs, though performance varies across different backbones.
Harness self-evolution improved held-out task success across 7 benchmarks with a frozen Qwen3.6-27B backbone, including gains of +15.4 percentage points on AppWorld (41.3% to 56.7%) and +13.9 pp on BrowseComp+ (16.9% to 30.8%).
HarnessBank evolution searching strategy interfaces and screening candidates on training tasks before testing on sealed test splits.
Shows that mutating software execution policies around a frozen model improves task completion without weight updates, though gains transfer unevenly across task domains.
Limits
Theoretical capability expansion requires typed contracts, compatible handoffs, and bounded host planning and worker error probabilities. If host planning errors or cumulative worker errors exceed budget thresholds, theoretical coverage guarantees fail.
Admission validation checks specified structural and capability rules, but does not certify semantic correctness. Likewise, the completion judge uses a language model fallback that can misjudge outputs or experience grading variance.
Harness self-evolution performance is bounded by the underlying frozen language model. Modifying software policy interfaces cannot fix fundamental reasoning limitations or missing pretraining knowledge.
Shared group memory feedback mechanisms are host-level observational judgments that are disabled by default and remain unverified in reported benchmark experiments.
Terms used here
Harness
The executable software wrapper—including tool interfaces, context rules, skills, and control policies—surrounding a language model.
Host Agent
The central language model controller that decomposes user goals into subtasks, matches subtasks to registered specialist agents, builds workflow graphs, and handles exceptions.
Directed Acyclic Graph (DAG)
A workflow diagram of subtasks where nodes represent agent invocations and directed arrows represent data dependencies that flow without looping back.
Artifact Ledger
An append-only recorded list of output files and input references that allows agents to share data by referencing stored files rather than copying raw text.
AgentGarten couples physics engines to a shared neural renderer through untextured 3D geometry maps. This separation lets AI agents interact with photorealistic virtual worlds in real time while keeping physical rules and object coordinates explicitly controllable in code.
2026-10-08 · AI agents · Computer vision · Efficient AI
RealCompanion introduces a longitudinal benchmark derived from 27,218 real-world human-companion messages across up to 120 days, enabling systematic evaluation of an AI companion's selective memory retrieval, profile fact extraction, trait inferencing, and restraint when history is unnecessary.
2026-10-01 · Evaluation and benchmarks · AI agents · Language models
RIDE pushes a student model beyond its reinforcement-learning teacher by training student hidden representations to overshoot teacher representations along the layerwise residual vector between the teacher and its pre-learning base model, bypassing noisy output-token sampling.
2026-09-29 · Reinforcement learning · Language models
papers.luk.media