When Green Tests Lie: Building Verification for Agent-Generated Software

Software teams have spent decades treating a green test suite as reassurance.

Agentic development complicates that relationship.

Tests remain essential. But a passing test tells us something narrower than we often pretend:

The implementation behaved as expected under the conditions the test examined.

It does not prove that we expected the right thing.

Consider a simple chain:

wrong requirement
→ correct implementation of wrong requirement
→ tests generated from the same misunderstanding
→ all green
→ wrong product ships

Nothing in that sequence requires incompetence.

In fact, a capable agent may make the problem worse by expressing the original misunderstanding consistently across code, tests, documentation, and review artifacts.

Consistency is not independence

Suppose an implementation agent produces a feature.

A second agent reviews it.

A third generates tests.

A fourth checks the tests.

A fifth summarizes the result.

Five agents agreeing can feel like strong evidence.

But if every agent receives the same flawed requirement, reads the same context, uses similar models, and evaluates against the same interpretation, the system has not produced five independent judgments.

It may have reproduced one mistake five times.

Verification therefore needs more than repetition.

It needs independence.

Build disagreement into assurance

A stronger agentic verification system can deliberately separate how implementation and evaluation are created.

That can include:

Independent test generation. Generate some verification artifacts directly from requirements rather than from the implementation.

Hidden tests. Preserve cases the implementing agent cannot optimize against directly.

Property-based testing. Test invariants across broad input spaces instead of validating only hand-selected examples.

Metamorphic testing. Check whether known transformations of an input produce logically consistent changes in output.

Adversarial review. Give a reviewer the explicit job of finding assumptions, boundary failures, and counterexamples rather than confirming quality.

Requirement-to-test traceability. Make it possible to identify which evidence supports which acceptance condition—and which requirements have no evidence at all.

The aim is not simply more testing.

It is to create different routes to discovering error.

The evaluator needs some independence from the evaluated

There is a deeper principle here.

If the system being evaluated can freely change the evaluator, the assurance boundary becomes weak.

An agent that can rewrite a failing test, modify the acceptance criteria, change the benchmark fixture, and then declare itself successful has been given control over the definition of success.

Sometimes changing a test is legitimate.

Requirements evolve. Tests can be wrong.

But the decision to change the evidence should be distinguishable from the act of satisfying it.

That may require immutable acceptance criteria for a run, protected test suites, approval for evaluator changes, or independent generation of critical checks.

Evidence, not ceremony

Agentic engineering can dramatically increase software throughput.

That makes assurance more important, not less.

If generation becomes ten times faster while verification remains superficial, the organization has not created ten times more engineering capacity.

It has created ten times more opportunities for undetected assumptions to propagate.

The right goal is not:

make every generated artifact pass tests.

It is:

build enough independent evidence that the organization has justified confidence in the resulting behavior.

Green tests still matter.

But as agents become increasingly capable of producing both the answer and the exam, engineering leaders need to remember what those green lights actually mean.

Tests are evidence of correctness.

They are not correctness itself.