Harness-Driven Capability Amplification in Fixed-Weight Models

AX Labs · AgencyXcelerate
Research note · living document · claims versioned as evidence changes

Abstract

We investigate whether a substantial portion of effective model capability is inaccessible at the raw-model interface and can be unlocked by automatically synthesizing a fitted cognitive architecture around fixed model weights. We describe a Model-to-Agent Compiler that characterizes a model, generates a model-specific harness spanning state representation, memory, planning, verification, tool use, exploration and context management, and evaluates candidate architectures against immutable external tests. The system recursively proposes and rejects harness mutations while keeping the evaluator outside the candidate. ARC-AGI-3 is used as a primary interactive benchmark, but the research objective is benchmark-agnostic: capability amplification that transfers across unseen interactive domains.

1. Research claim

The narrow claim under test is not that prompting makes a weak model intrinsically equal to a frontier model. The claim is that agent capability is a property of the model-plus-architecture system, and that the architecture can be synthesized from measured properties of the model.

We call the ratio between system performance and raw-model performance capability amplification. We report system-level gains separately from gains obtained through external tools or learned adapters.

2. Model-to-Agent Compiler

The compiler operates as a closed experimental loop:

1. Phenotype: benchmark the raw model across reasoning, coding, memory, planning, uncertainty, tool use, persistence and interactive action efficiency.
2. Compile: generate a model-specific harness containing only mechanisms justified by measured strengths or weaknesses.
3. Execute: run identical held-out tasks through raw and harnessed variants.
4. Verify: use deterministic tests, executable runtimes, external scorecards or other evaluators that the candidate cannot rewrite.
5. Diagnose: classify failures such as perception, mechanic induction, planning, memory, exploration waste, thrashing or serialization.
6. Mutate: alter only the relevant harness modules and create a new candidate.
7. Select: retain candidates only when reproducible evidence improves the primary score without unacceptable regressions.

The outer optimization loop has no arbitrary iteration ceiling. Resource ceilings are explicit experimental constraints, not assumptions about how many harness generations should be sufficient.

2.1 Separation of capability layers

3. ARC-AGI-3 as an interactive test bed

ARC-AGI-3 measures intelligence through interactive environments in which the objective and mechanics are initially unknown. Its methodology emphasizes action efficiency: an agent that requires fewer environment actions to learn and solve a task is treated as more intelligent than one that brute-forces the same result.

This makes ARC-AGI-3 unusually useful for harness research. Internal reasoning, simulation and state management can be improved without making environment actions free. The resulting research objective is therefore not “take unlimited moves,” but “spend unlimited research effort learning how to spend environment moves intelligently.”

3.1 Provider-state evidence

ARC Prize currently reports a striking same-model harness effect for GPT-6 Astra. At High reasoning, the verified ARC-AGI-3 score is reported as 54.82% with the Standard harness and 99.95% with the Provider Adapter harness. The Provider Adapter preserves model-native state and uses provider-specific context management. This is independent external evidence that the surrounding harness can materially change measured agent performance without changing the model family.

This result belongs to ARC Prize/OpenAI, not AX Labs. We cite it as motivation for studying harness architecture as a first-class variable.

4. Current AX evidence

AX Labs maintains a paired harness arena in which the same model receives the same task under raw baseline and compiled-harness conditions.

The important observation is not that four synthetic cases prove general intelligence—they do not. It is that the same evaluation system can detect both large uplift and harmful over-harnessing, and automatically reject the latter.

5. Recursive harness improvement

The current mutation library targets failure signatures rather than rewriting the entire system blindly. Candidate modules include deterministic perception, persistent object tracking, causal transition ledgers, hypothesis competition, information-gain exploration, anti-thrashing state-action memory, coordinate proposal, receding-horizon planning and adaptive reasoning depth.

We use the term recursive harness self-improvement for this process. It is deliberately not unrestricted production self-modification: candidate code may generate new candidates, but only an evaluator external to the candidate can promote them.

6. Offline prize architecture

The ARC Prize competition requires notebook submissions with internet disabled and a runtime limit. For that reason, the AX cloud fabric is a research foundry rather than a competition-time dependency. The final artifact must export into a self-contained runtime containing model weights, optional adapter, harness code, state/memory modules and evaluation-compatible execution.

A successful research pipeline therefore looks like:

Cloud research foundry → phenotype → recursive harness evolution → held-out validation → freeze → offline Linux/Kaggle agent

7. Limitations and falsification criteria

• A public ARC score of 100% is not treated as proof of generalization or as completion of the research program.
• ARC-specific heuristics that do not transfer to unseen interactive domains are considered overfitting, not capability amplification.
• Tool-assisted results are reported as system performance rather than attributed to the base model.
• Learned adapters are reported separately from fixed-weight harness gains.
• Small internal synthetic suites are useful for regression detection but cannot support broad intelligence claims.
• The central hypothesis would be weakened if uplift consistently disappears on held-out domains, if generated harnesses only memorize benchmark-specific structure, or if gains require larger external models that dominate the base model.

8. Research objective

The long-run question is whether automatically synthesized cognitive architecture can consistently unlock large latent capability across fixed models and unrelated interactive domains. A strong result would be a reproducible pattern in which multiple modest base models show substantial raw-to-harness improvement on unseen tasks, with the harness generated from measured model phenotype rather than hand-authored benchmark solutions.

References

Working Paper 0.1. This document is intentionally versioned. New benchmark results will be added with model, harness, evaluator, date and reproducibility status.