Validation

The core acceptance gate passed on 2026-11-05 for runtime commit fa3450c5675c71b2e92e8dcc14dfe47c5cbf696d. Documentation changes after this commit do not change the validated runtime.

Fixed ProgramBench inputs

Five inputs were frozen before generation from the first five lexicographically ordered historical task archives in shard 0000 of Lego-X/programgen-candidates-200k-v1. Their repository commits and executable targets were preserved. The historical model and 33 core execution settings were compared with the new run.

The four successes cover Rust and Go. Every published active test passed both gold and rerun; empty programs and copied reference submissions were rejected. The colm-suite build failed to link WIFSIGNALED, WTERMSIG and WEXITSTATUS. The old and new CMake script/Dockerfile renderings matched. Its rejection remains in the result set. This five-input result is not a general success-rate estimate.

Generation used claude-opus-5, two task workers, 5,400-second agent sessions, 80 turns, four test attempts, and two mode repair attempts, following the frozen successful-task settings. Generated tests were not copied from the historical archives. Subsequent verifier and metadata fixes reused the same generated material and reran the ordinary producer's complete checks. Previous failures, source transitions and publication backups were retained.

NL2Repo and commands

Five distinct NL2Repo inputs were fully certified on the final runtime: benchopt/benchopt, pallets/click, oauthlib/oauthlib, BurntSushi/toml and more-itertools/more-itertools, covering Python and Go. A fresh TOML task was also generated during mixed-mode interruption acceptance; the repeated input was counted once in the cohort.

• A real search request produced a candidate with an immutable commit.
• create exercised the full reference build after Cargo workspace discovery was corrected. Its resume path verified both Harbor and native layouts, each with 433 tests. The final metadata-only revision reused that exact reference binary and independently verified both layouts again.
• produce performed fresh generation on the five fixed PB inputs and complete revalidation on both modes. The combined cohort has 4 PB + 5 NL certified identities, with at least two languages per mode.

Recovery and isolation

The final mixed batch contained a certified PB input, an NL input, a duplicate row and a deliberately invalid binary target. A real producer was killed with SIGKILL (exit -9) while consumer tools were running. Two actual resumers exited successfully, restored valid publications and rejected the negative input. Measured producer concurrency peaked at 2. Owned resources were cleaned before reuse; an unrelated process and unrelated publications remained intact.

Additional runs deleted and corrupted a PB test archive and an NL golden patch, then removed certification evidence. Ordinary produce restored the original files, reverified the tasks and skipped intact results. Both task payloads were byte-identical after recovery. No owned processes, interrupted containers or owner files remained at final inspection.

Regression and distribution checks

GitHub CI passed 1,209 checks on each of Python 3.11 and 3.12, plus 8 Docker checks. The migration audit retained 50 construction-core files and found no runtime import errors. All 29 old/new boundary comparisons matched. Wheel, source archive, clean-clone and out-of-checkout installation checks passed.

Nine older frozen compatibility tasks also passed the corrected replay. The previously intermittent 335-test example passed three complete verifications, including gold and rerun, with its original denominator unchanged.

Corrections found during acceptance

• The default test-count gate was restored to the old pipeline's active validated count. Extra semantic diversity review is optional.
• Gold replay uses the actual reference build. The restricted oracle intentionally filters caller environment variables, so an oracle proxy was not an equivalent positive control. Oracle isolation and the original copied-binary guard remain checked. Historical proxy failures remain recorded separately.
• Cargo workspace and declared binary targets are discovered without bypassing candidate screening.
• Operational stage timestamps stay in private work state, keeping the published stage summary stable across recovery.

Raw logs, exact input/archive hashes, failed attempts and machine-readable reports are preserved outside the source checkout. The evidence distinguishes fresh model generation, retained-material revalidation and historical compatibility.