Skip to the stories
AI & ML

TDD With AI Agents: Keep the Test Contract in Your Hands

  • William Martinez
  • 437 reads

Write the contract and first test yourself, then let an agent expand coverage and implement against tests you’ve reviewed.

TDD With AI Agents: Keep the Test Contract in Your Hands

When an AI agent writes both the implementation and its tests, a passing test suite can offer less assurance than it appears to. The code may simply match tests shaped around the same assumptions. Test-driven development gives the engineer a firmer role: define the behavior first, check that the tests capture it, and let the agent draft an implementation against that contract.

A person holding checklists beside a cabinet of locked tests labeled TESTS, with a door marked BLOCKED separating it from a robotic arm and machinery.

The division of work matters. You decide what callers should be able to rely on and write an initial test that makes the requirement concrete. The agent can help expand test coverage and write production code, but you review each contribution. That keeps the key design decisions visible while still handing off repetitive drafting.

Loop diagram with five steps showing your steps (define the contract, harden the tests, review the diff) and the agent's steps (draft tests, implement), looping back for the next feature.
TDD with AI agents: define, test, implement, review, and repeat.

A workflow with clear handoffs

The process has five steps, with the engineer responsible for defining and checking the contract and the agent helping with test drafting and implementation:

  1. Define the contract. Describe the observable behavior the feature must provide, then write a first test that captures one important case.
  2. Ask the agent to draft tests. Give it the contract and your test as examples. Have it propose additional cases in the same style.
  3. Review and strengthen the test set. Check every generated case against the intended behavior. Add missing edge cases and correct tests that encode the wrong expectation.
  4. Delegate implementation. Let the agent write the production code to satisfy the reviewed tests. Keep it from changing the tests during this step.
  5. Inspect the diff. Review the implementation and its fit with the contract before beginning the next feature.

The ordering is the safeguard. If the agent sees a finished implementation before it proposes tests, it may write tests that describe what the code already does, including its mistakes. Providing the contract and a hand-written example first gives test generation an independent reference point. It doesn’t guarantee that the tests are complete, so the engineer still has to check them.

Make the contract concrete

A small Python version comparator provides a useful example because its behavior has meaningful ordering rules. Python’s version specifiers define, for instance, that 1.0.dev1 sorts before 1.0a1, which sorts before 1.0. A class that compares version strings therefore needs more than a test that checks ordinary numeric ordering: the tests need to capture the distinctions the feature promises.

The example uses a Version class whose instances can be compared with operators such as < and >. A first test should pin down a clear behavior at that public interface. From there, the agent can draft cases that explore the contract. The engineer’s task is to decide whether those cases cover the intended behavior, especially where version components or release stages change the ordering.

That distinction between a contract and an implementation is useful well beyond version parsing. A contract states what a caller can observe; it doesn’t dictate how the code must achieve it. Tests written at that boundary give the agent room to choose an implementation while still providing a check against the agreed behavior.

Keep test changes under human control

Letting an agent edit tests during implementation weakens the main check in this workflow. If a failing test can be changed to fit the implementation, a green run no longer shows that the code satisfies the original contract. Blocking writes to tests/ while the agent implements makes the handoff easier to reason about: the implementation has to adapt to the reviewed tests.

That control still requires judgment. A test can be wrong, incomplete, or too closely tied to an implementation detail. Review it before delegating code, and look at the implementation diff afterward. A passing run tells you that the code meets the tests you have; it can’t tell you whether those tests captured every requirement.

This approach also makes changes easier to review. When the expected behavior is explicit, a diff can be assessed against a known contract rather than accepted because it looks plausible or because an agent reports success. If a generated test surprises you, resolve what the feature should do before asking for an implementation.

Practical setup and tradeoffs

The sample project uses Python 3.11 or newer and pytest. Its code imports only from Python’s standard library, so pytest is the only additional package needed for the exercise. The implementation belongs in version_check/core.py, the tests in tests/test_core.py, and the package exposes Version from version_check/__init__.py. Keeping tests separate from production code also makes it straightforward to restrict the agent’s writes to the test directory.

The demo uses Claude Code, but the workflow doesn’t depend on that specific tool. The relevant capability is access to project files: the agent needs to read the contract and tests, draft additional cases, and edit the implementation. The same handoffs can be applied with other repository-aware coding agents.

There is a cost to the extra review. Engineers must spend time writing the initial test, checking generated cases, and inspecting the resulting code. For a small, low-impact change, that process may be heavier than the feature warrants. For behavior where edge cases matter, the clearer contract can make agent-written code easier to assess and maintain.

A production version comparator would typically use Python’s packaging library. Rebuilding the behavior in a small exercise is useful for practicing the workflow, but it doesn’t establish that the implementation covers the full versioning rules. The practical lesson is the handoff: specify behavior before code exists, treat agent-generated tests as proposals, and judge the implementation against tests you’ve reviewed.

As agents take on more of the typing, the valuable engineering work shifts toward deciding what should be true and checking that the code makes it true. This TDD loop makes that responsibility explicit—and gives teams a repeatable way to keep it.

===
Source: William Martinez · realpython.com

From the same set