Integration boundary

ProgramGen owns generation, quality evidence and atomic publication. External Harbor/ProgramBench runners own task execution for training or evaluation. The task's shipped consumer remains the scoring authority.

ProgramBench — Harbor

TASK/
  task.toml                       runner/oracle metadata
  instruction.md                  reconstruction prompt
  task_manifest.json              source, reference, tests and quality
  environment/Dockerfile          agent image recipe
  environment/reference/          black-box client and public hashes
  environment/oracle/             hidden executable, optional runtime, daemon
  environment/docs/               sanitized usage documentation
  workspace/                      public agent starting files
  tests/tests.json                active/ignored catalog with original IDs
  tests/branches/*.tar.gz          hidden test harnesses and fixtures
  tests/Dockerfile
  tests/grader.py
  tests/test.sh                    original verifier entrypoint
  solution/compile.sh              hint, not a reference implementation
  .programgen-verification/        generation evidence and optional certificate

The agent calls /reference/executable, a client for the separately mounted Unix-socket reference service. The executable and runtime under environment/oracle/ remain private to that service. The agent image contains the client and sanitized docs. Hidden tests run in the verifier environment. Use a runner that implements the task's environment.reference_oracle metadata and these separate mounts. ProgramGen's full verification executes the shipped consumer in Docker; downstream training and agent-runner integration are external.

Submissions provide compile.sh, which creates /workspace/executable, and an archive at /logs/artifacts/submission.tar.gz. The original grader emits /logs/verifier/reward.txt and summary.json. Missing IDs, ignored IDs, timeouts and partial reward preserve the historical rules. The optional semantic effective-test lower bound does not remove catalog IDs or alter the score denominator.

ProgramBench — native

NATIVE_ROOT/
  tasks/INSTANCE/
    task.yaml
    tests.json
    task_manifest.json
    Dockerfile.task
    Dockerfile.task_cleanroom
    build_images.sh
    inference.md
    cleanroom/workspace/           native reference executable, hashes, docs/assets
    .programgen-verification/
  blobs/INSTANCE/tests/*.tar.gz
  validate/INSTANCE/
    quality_report.json
    validation_summary.json
    dummy_validation.json
    assertion_lint.json

Native verification also hashes sibling blobs and validate payloads. Its consumer adapter materializes the unchanged native-Harbor scoring contract, checking catalog, archive and grader hashes. The original native input is kept intact. output_format="both" verifies and receipts both layouts.

NL2Repo — Harbor

TASK/
  task.toml
  instruction.md                  complete target-commit specification
  quality_report.json
  workspace/                      source-stripped public project skeleton
  environment/Dockerfile
  environment/Dockerfile.inference
  environment/repo/               build context for the fixed target commit
  environment/pre_cleanup.sh      strips source and hidden test payloads
  environment/build_env.sh
  tests/tests.json                measured denominator and test/import contracts
  tests/test_patch.diff            hidden repository tests
  tests/test_assets/               optional binary fixtures
  tests/python_install_contract.json
  tests/grader.py
  tests/test.sh
  solution/solve.sh
  solution/golden.patch            full target implementation restoration
  solution/golden_assets/          optional restored binary fixtures
  solution/verify.sh
  .programgen-verification/

The inference image is built from the target commit and cleaned before agent use. This is a full-implementation reconstruction task; no parent commit is required. Source-bearing build context, hidden tests and the golden solution belong to build/verifier roles. External runners must preserve these mount boundaries rather than exposing the complete task directory to the agent.

The verifier injects the hidden patch, applies the installation contract, and emits reward.txt, metrics.json and test-output.log. The denominator is max(generation-time expected count, observed passed + failed + errors). The document includes direct/core-indirect API, import and signature contracts. Over-budget documents/tests retain full contracts and receive pool="hard"; a maximum of 0 means unlimited. See verification.

Publication and resume

Producer output is MODE/IDENTITY/{harbor,native}, with .publication.json at the identity root. It binds effective configuration, implementation, every payload/evidence file, and the full certificate. The JSONL journal records outcomes and paths; receipt and file checks authorize resume, not a journal entry alone. Failed candidates keep their work state and raw verification logs.