Engineering case study · 2026

Give it a task.
Get back a verified patch.

I built CodePilot around one rule: the model can propose code, but it cannot declare its own work correct.

It works inside a managed Git workspace, makes bounded changes, runs human-configured checks in a sandbox, repairs real failures, and only completes when the evidence still matches the code being returned.

Open the demo →Read the engineering notes

ENGINEERED BYMalek El Badawi El Najjar

DEMO · 001demo/customer-analytics

TaskAdd CSV export to the customer analytics page and cover edge cases.

Inspect repositoryrepo.listFiles · repo.readFile

READ ONLY

Edit export pathworkspace.writeFile

2 FILES

Validation failedtests · CSV escaping

EXIT 1

Repair patchbounded repair loop

1 / 3

Validation passedtests + typecheck

EXIT 0

Verifiedrevision + config rebound

HASH MATCH

✓ VERIFIED AGAINST CURRENT REVISIONworkspace + config bound

00 / DESIGN CONSTRAINT

Most coding-agent demos end at “done.” I was more interested in everything that has to happen before that word can be trusted.

CodePilot separates model output from execution authority. The model can inspect, reason, request bounded writes, and react to failures. It cannot choose the repository root, validation command, sandbox trust level, repair budget, or what qualifies as verified completion.

That constraint shaped the project more than the UI did. The interesting work is in the boundaries: path confinement, hash-gated mutation, sandbox execution, failure classification, concurrency, evidence persistence, and the final revision check.

System architecture

The model proposes. Runtime code owns authority.

The important line in CodePilot is not between one model and another. It is between untrusted model intent and the trusted code that decides what can actually happen.

TASK → MODEL → POLICY → WORKSPACE → VALIDATION → EVIDENCE

TaskNatural-language goal

Model providerOpenAI / Anthropic

Tool gatewaySchema + policy boundary

Git workspaceConfined, hash-gated writes

SandboxTrusted validation commands

EvidenceRevision-bound completion

Untrusted: model text, tool requests, repository content, validation outputTrusted: workspace identity, policy checks, configured commands, budgets, completion transition

What the runtime enforces

Correctness is a state transition, not a sentence from the model.

01 / AUTHORITY

Validation stays outside model control.

The model can request tests, typecheck, lint, or build. It cannot provide the command. The exact success criteria come from trusted workspace configuration.

02 / MUTATION

Writes are bounded before they touch disk.

Sensitive paths are denied, stale-file overwrites are rejected, patch impact is measured before mutation, and repair work shares one cumulative change budget.

03 / COMPLETION

Completion requires persisted evidence.

Checks must execute and pass, the workspace and validation configuration must still match, and VerificationEvidence is committed atomically with the COMPLETED state.

Engineering notes

Problems that changed the design.

These are the parts of CodePilot I would actually want to discuss in a technical interview—the places where “just call the model” stopped being enough.

A stale file cannot be overwritten silently.

src/tools/workspace/write-file.ts

ProblemA model may read a file and propose an edit after something else has already changed it.

DecisionExisting files require the SHA-256 content hash returned by the read tool. A mismatch rejects the write before mutation.

ResultOptimistic concurrency became part of the tool contract, not a prompt instruction.

The model cannot move the goalposts after it fails.

src/validation · src/agent/verification.ts

ProblemA coding agent should not be allowed to choose an easier test command after seeing a failure.

DecisionValidation tools accept zero model-controlled command arguments. Commands come from trusted workspace configuration and historical records store only their hash identity.

ResultSuccess criteria stay outside model authority throughout repair.

Passing checks are not enough if the code moved afterward.

src/agent/workspace-revision.ts

ProblemA run could pass validation, then the workspace or validation configuration could change before completion.

DecisionCodePilot hashes the workspace revision and validation configuration and re-checks both before atomically persisting COMPLETED with VerificationEvidence.

ResultThe evidence is bound to the code that is actually returned.

Infrastructure failure is not a code failure.

src/sandbox · src/agent/run-coding-task.ts

ProblemA missing Docker daemon or failed sandbox launch should not consume the model's repair budget.

DecisionInfrastructure outcomes are classified separately, retried independently, and never enter the REPAIRING state.

ResultRepair attempts represent code repair, not platform noise.

IMPLEMENTATION SCARS

Two bugs that only showed up once the abstractions touched real Git behavior.

core.autocrlf

Host Git settings were changing workspace bytes.

On Windows, a global core.autocrlf=true setting made git applysilently rewrite LF to CRLF. The patch path now pins core.autocrlf=false on both the dry run and the real apply, so the same patch produces the same bytes everywhere.

git diff <baseline>

Brand-new files were invisible in the review diff.

Untracked files do not appear in the baseline diff CodePilot uses for review. New paths are staged—not committed—after creation so the diff remains complete while the working change stays fully reviewable.

Built with

One application, one runtime boundary, no pretend startup metrics.

TypeScriptNext.jsPostgreSQLPrismaDockerVitestOpenAIAnthropicGitHub Actions

MODEL-AGNOSTIC PROVIDERS · DOCKER-FIRST SANDBOX · POSTGRES PERSISTENCE · CI-VERIFIED RELEASES

DELIBERATE SCOPE

I did not turn the public site into an anonymous code-execution service. Arbitrary public repositories, GitHub OAuth/PR automation, billing, and multi-agent orchestration are outside this release—the project stops at the trust boundary it can defend.

Public demo

Watch a patch fail before it earns “verified.”

The browser demo is deliberately deterministic. It shows the same failure, bounded repair, validation, and evidence vocabulary as the real runtime without exposing an anonymous shell, arbitrary repositories, or credentials on the public site.

Run the fixture →Review the architecture

Give it a task.Get back a verified patch.