Test-First Development
The test is the brief. Write it first.
The test is the brief. Write it before the code, watch it fail for the right reason, then make it pass with the smallest change you can. Beck called it red, green, refactor. The broader frame is that the test is the artefact you commit to first because it is the artefact that proves the behaviour ever existed.
Opinion
I have lost the “tests first or tests after” argument exactly once and I lost it because I gave up. Every other time the “I'll add the tests later” commit has ended the same way: the tests never appear, the regression surfaces a quarter later and the person who would have written them has moved teams. Beck's 2003 book put a name on the cycle and changed who I hired;1 twenty-three years later, the case is stronger, not weaker, because the same argument now binds the agent in the loop as well as the human.
I want to plant a flag on this: the test is the brief rather than a check on the brief. Pair it with T1 Domain-Driven Types and the function signature plus the failing assertion together encode what design notes used to. The common counter, “we don't do TDD because the design isn't clear yet”, names exactly the moment when the discipline pays most. A test that cannot be written cheaply is the design saying the unit is doing too much, which is F1 Single Responsibility arriving in test-shaped clothes.
Agents made this cheaper. Karpathy argues that anything resettable, efficient, and rewardable can be optimised by a model;4 the failing test is the cheapest reward signal a developer ever produced. With tests written first, the agent loop has a target to hit. Without them, the agent generates plausible code at machine speed and the gap surfaces only when something explodes in production. P1 is the floor under AI4 Verifiable Specs: the AC is the brief at the work scale, the test is the brief at the code scale, and both must be machine-checkable.
Copy a note and link
Grab this short comment and drop it into a PR comment or an LLM chat to prompt the right change.
The test is the brief. Write the failing test before the implementation, run it, watch it fail for the reason you predicted, then write the smallest code that makes it green. In typed languages, the type is the first test: define the signature, let the compiler reject the obvious failures, then write the test for the residue. /tenet/test-first-development/P1
AI eyes only
Rule: the test is the brief. Write it first. Run it failing. Then implement.
Reject: writing implementation before a failing test exists. Reject: writing the test after the implementation. Reject: relying on introspection as the design signal.
Generate: write a test that names the requirement, run it, paste the failure into the plan, then implement until the test passes. Each new behaviour earns a new failing test first.
Diagnostic: the test must fail before the implementation exists. If it passes vacuously, the test is wrong. Rewrite it before continuing.
Why?
- The test is the working specification, readable, runnable, falsifiable. The function signature plus the failing test together carry what the prose docstring used to.
- Every shipped behaviour leaves a regression test behind it. The bug that reaches production once does not go out again silently: the suite catches the second occurrence before the diff is merged.
- Writing the test first produces a callable design before any implementation choices have been sunk. A unit you cannot test cheaply is a unit doing too much and the friction shows up before the cost is locked in.
- In typed languages the type is the first test, see T1 Domain-Driven Types. A signature that compiles has already ruled out one whole class of failure.
- Agents iterate on the signal you give them. A failing test is a signal an agent can read, re-run, and grade itself against. A description in chat is not. P1 puts AI4 Verifiable Specs to work at the function scale.
- Tests-first PRs review faster. The reviewer reads the test to learn the intent, reads the code to verify the test, and is done. Tests-after PRs ask the reviewer to reverse-engineer the intent and check the test is honest in the same pass.
- A green suite is a refactoring licence. Without tests, every cleanup is a leap of faith; with them, the cleanups in P6 Leave it Better stop being leaps and become small, safe steps.
Origins
The discipline has a date and an author. Kent Beck's Test-Driven Development by Example (Addison-Wesley, 2003) names red, green, refactor and treats the failing test as the working specification.1 Beck's case rests on a small economic claim: every minute spent writing the test before the code is paid back many times over by the regressions never re-shipped and the design pressure pushes units toward a size that can be tested at all. The 2003 book framed it as a discipline; Extreme Programming Explained (Beck, 1999) had already placed it inside a wider practice.6
The typed-language flavour belongs to Edwin Brady. Type-Driven Development with Idris (Manning, 2017) extends the cycle by adding a “types first” preface: define the type, leave the body as a hole, let the type-checker reject the inconsistencies the test would have caught, then write the test for the residue, then the body.7 Scott Wlaschin's Domain Modeling Made Functional (Pragmatic Bookshelf, 2018) and the broader F# community make the same case for an industrial typed stack;8 Yaron Minsky's “Make Illegal States Unrepresentable” pushes the claim into OCaml.9 The merger of the two traditions is what new P1 names: in dynamically typed languages the test is the first spec. In typed languages the type is and the test follows.
Steve Freeman and Nat Pryce's Growing Object-Oriented Software, Guided by Tests(Addison-Wesley, 2009) is the bridge between the unit-scale and the work-scale.10 Their outside-in TDD starts with an end-to-end acceptance test, an AC-shaped artefact, and works inward through unit tests until the implementation arrives at the bottom. That is the cleanest in-codebase example of how P1 (test-first at the code scale) and P2 Spec-First Execution (AC-first at the work scale) compose into one discipline.
The agent loop is the recent addition. Karpathy's “Verifiability” (2025) names the precondition that makes any of this work for an LLM: the task must be resettable, efficient, and rewardable.4 A failing test satisfies all three. Anthropic's evaluator-optimizer pattern, the Reflexion paper, Self-Refine, and CRITIC are the published versions of the same observation: agents grounded in machine-checkable feedback improve. Agents that grade themselves do not.2 The 2024 Huang et al. paper is the counter-evidence (without external grounding, asking GPT-4 to self-review decreases accuracy) and is the load-bearing reason the test must be runnable, not aspirational.
Quotes
Write a failing automated test before you write any code. Remove duplication.
As an acceptance test, it must run end-to-end to give us the feedback we need about the system’s external interfaces, which means we must have implemented a whole automated build, deploy, and test cycle.
We put the type first, treating it as a plan for a program, and use the compiler and type checker as our assistant, guiding us to a complete and working program that satisfies the type.
If a task/job is verifiable, then it is optimizable directly or via reinforcement learning, and a neural net can be trained to work extremely well.
Evidence
19 external sources, ranked by author authority. The first five are the canon; expand to see the rest, including the qualifiers and the named opposers. Each links out to its primary source.
- 01The book that named red-green-refactor and put a working example next to every claim. The case for the failing test as the working specification is built page by page; the Preface names where TDD falls short (security software, concurrency), and Chapter 32, “Mastering TDD”, takes up what you don’t have to test.
- 02TDD lives inside XP. Pair programming, ten-minute build, continuous integration, and small releases are the surrounding practices that make test-first economic for the original cohort.
- 03The early companion volume to Beck. Walks through TDD in Java with a level of operational detail Beck’s book deliberately leaves out: the canonical second source for the discipline.
- 04Outside-in TDD: start with the failing acceptance test, work inward through unit tests, end at the implementation. The bridge book between code-scale TDD and work-scale Spec-First.
- 05Beck for typed languages. Define the type first, leave the body as a hole, let the type-checker reject inconsistencies before any test runs. The typed-language flavour the merged P1 carries.
- 06Domain Modeling Made FunctionalSupportsIndustrial F# treatment of type-first design. Models the domain in the type system before any test fires. The test then covers the residue the type cannot capture.
- 07Effective TypeScriptSupportsTypeScript-flavoured type-test-implement: items 14, 22, and 38 build the case that types are tests the compiler runs for free and the unit test suite covers the behaviour the type cannot.
- 08The book-length answer to Coplien. Test the public contract rather than the internals. The unit’s contract is the test’s contract. Saves TDD from its over-extraction failure mode without abandoning the discipline.
- 09“Test Coverage”QualifiesSharper than the headline reads. Coverage targets are a smell. The goal is confidence and the metric is whether the team feels safe releasing. Pairs with P3 Definition of Done as the realistic-coverage tenet.
- 10Beck’s own reply to the over-extraction critique two decades later. Tidyings and behaviour changes belong in different commits. The test drives the behaviour change rather than the tidying.
- 11“Mocks Aren't Stubs”QualifiesThe classicist vs mockist distinction. Names the failure mode TDD critics complain about, interaction-test cascades, and presents the discipline as an option, not the only option.
- 12The sharpest published counter to TDD. High test mass on low-value internals is real cost. The reply is to test the public contract, not to abandon the discipline.
- 13DHH’s essay that triggered the Beck/Fowler/DHH debates. Argues TDD pushes architecture toward over-extracted hexagons. Useful steelman for the discipline-as-religion failure mode.
- 14“Verifiability”SupportsSoftware 2.0 automates what you can verify: tasks must be resettable, efficient, and rewardable. Names the AI-era reason a failing test is the cheapest, most-portable reward signal a developer ever produced.
- 15Empirical case for AI-grounded TDD. HumanEval pass@1 lifts from ~80% to 91% when the agent grounds self-feedback in test runs. The signal that justifies the “tests-first for the agent” claim.
- 16Generalises Reflexion. Iterative refinement against an external signal outperforms one-shot generation across many tasks. The test suite is the canonical external signal in code work.
- 17Tool-interactive critiquing. The verification signal must come from running tools (tests, linters, type-checkers) not from the model’s own introspection. The operational version of Karpathy’s thesis.
- 18Counter-evidence used positively. Without external grounding, asking GPT-4 to self-review decreases accuracy. The load-bearing reason the test must be runnable, not aspirational.
- 19Building Effective AgentsSupportsNames the evaluator-optimizer pattern: one LLM generates, another (or an external check) evaluates. The test runner is the canonical evaluator; P1 is what gives the loop something to grade.
Nineteen sources, three stances. The supporters are Beck, Astels, Freeman & Pryce and Brady, the canon for dynamic and typed languages alike. The qualifiers further down hold the “TDD is a discipline rather than a religion” line. The opposers argue against the brittleness of the suite as a design medium: the steelman the reply has to address.
Examples
Before implementation first; the test confirms what already shipped.
function rescueHedgehog(hedgehog: Hedgehog, sanctuary: Sanctuary): RescueResult { if (sanctuary.intake.length >= sanctuary.capacity) { return { ok: false, reason: "full" }; } sanctuary.intake.push(hedgehog); return { ok: true }; } it("rescues a hedgehog", () => { expect(rescueHedgehog(hg, sanctuary).ok).toBe(true); });
After failing test first; the smallest code that turns it green.
it("rejects rescue when the sanctuary is at capacity", () => { const full: Sanctuary = { capacity: 2, intake: [hg1, hg2] }; const result = rescueHedgehog(arrival, full); expect(result).toEqual({ ok: false, reason: "full" }); }); function rescueHedgehog(arrival: Hedgehog, s: Sanctuary): RescueResult { if (s.intake.length >= s.capacity) return { ok: false, reason: "full" }; s.intake.push(arrival); return { ok: true }; }
Before implementation first; the test confirms what already shipped.
function rescueHedgehog(hedgehog: Hedgehog, sanctuary: Sanctuary): RescueResult { if (sanctuary.intake.length >= sanctuary.capacity) { return { ok: false, reason: "full" }; } sanctuary.intake.push(hedgehog); return { ok: true }; } it("rescues a hedgehog", () => { expect(rescueHedgehog(hg, sanctuary).ok).toBe(true); });
After failing test first; the smallest code that turns it green.
it("rejects rescue when the sanctuary is at capacity", () => { const full: Sanctuary = { capacity: 2, intake: [hg1, hg2] }; const result = rescueHedgehog(arrival, full); expect(result).toEqual({ ok: false, reason: "full" }); }); function rescueHedgehog(arrival: Hedgehog, s: Sanctuary): RescueResult { if (s.intake.length >= s.capacity) return { ok: false, reason: "full" }; s.intake.push(arrival); return { ok: true }; }
Before implementation first; the test confirms what already shipped.
function rescueHedgehog(hedgehog, sanctuary) { if (sanctuary.intake.length >= sanctuary.capacity) { return { ok: false, reason: "full" }; } sanctuary.intake.push(hedgehog); return { ok: true }; } it("rescues a hedgehog", () => { expect(rescueHedgehog(hg, sanctuary).ok).toBe(true); });
After failing test first; the smallest code that turns it green.
it("rejects rescue when the sanctuary is at capacity", () => { const full = { capacity: 2, intake: [hg1, hg2] }; const result = rescueHedgehog(arrival, full); expect(result).toEqual({ ok: false, reason: "full" }); }); function rescueHedgehog(arrival, s) { if (s.intake.length >= s.capacity) return { ok: false, reason: "full" }; s.intake.push(arrival); return { ok: true }; }
Before implementation first; the test confirms what already shipped.
def rescue_hedgehog(hedgehog: Hedgehog, sanctuary: Sanctuary) -> RescueResult: if len(sanctuary.intake) >= sanctuary.capacity: return RescueResult(ok=False, reason="full") sanctuary.intake.append(hedgehog) return RescueResult(ok=True) def test_rescues_a_hedgehog() -> None: assert rescue_hedgehog(hg, sanctuary).ok is True
After failing test first; the smallest code that turns it green.
def test_rejects_rescue_when_sanctuary_is_at_capacity() -> None: full = Sanctuary(capacity=2, intake=[hg1, hg2]) result = rescue_hedgehog(arrival, full) assert result == RescueResult(ok=False, reason="full") def rescue_hedgehog(arrival: Hedgehog, s: Sanctuary) -> RescueResult: if len(s.intake) >= s.capacity: return RescueResult(ok=False, reason="full") s.intake.append(arrival) return RescueResult(ok=True)
Before implementation first; the test confirms what already shipped.
RescueResult rescueHedgehog(Hedgehog hedgehog, Sanctuary sanctuary) { if (sanctuary.intake().size() >= sanctuary.capacity()) { return new RescueResult(false, "full"); } sanctuary.intake().add(hedgehog); return new RescueResult(true, null); } @Test void rescuesAHedgehog() { assertTrue(rescueHedgehog(hg, sanctuary).ok()); }
After failing test first; the smallest code that turns it green.
@Test void rejectsRescueWhenSanctuaryIsAtCapacity() { Sanctuary full = new Sanctuary(2, List.of(hg1, hg2)); RescueResult result = rescueHedgehog(arrival, full); assertEquals(new RescueResult(false, "full"), result); } RescueResult rescueHedgehog(Hedgehog arrival, Sanctuary s) { if (s.intake().size() >= s.capacity()) return new RescueResult(false, "full"); s.intake().add(arrival); return new RescueResult(true, null); }
Before implementation first; the test confirms what already shipped.
function rescueHedgehog(Hedgehog $hedgehog, Sanctuary $sanctuary): RescueResult { if (count($sanctuary->intake) >= $sanctuary->capacity) { return new RescueResult(ok: false, reason: "full"); } $sanctuary->intake[] = $hedgehog; return new RescueResult(ok: true); } function testRescuesAHedgehog(): void { $this->assertTrue(rescueHedgehog($hg, $sanctuary)->ok); }
After failing test first; the smallest code that turns it green.
function testRejectsRescueWhenSanctuaryIsAtCapacity(): void { $full = new Sanctuary(capacity: 2, intake: [$hg1, $hg2]); $result = rescueHedgehog($arrival, $full); $this->assertEquals(new RescueResult(ok: false, reason: "full"), $result); } function rescueHedgehog(Hedgehog $arrival, Sanctuary $s): RescueResult { if (count($s->intake) >= $s->capacity) return new RescueResult(ok: false, reason: "full"); $s->intake[] = $arrival; return new RescueResult(ok: true); }
Enforcement
Apply these rules in eslint.config.mjs. The full enforcement across every tenet lives on the implementation page.
| Rule | Tool | Catches |
|---|---|---|
| vitest/expect-expect | @vitest/eslint-plugin | tests with no assertion: passes silently because nothing is checked. |
| vitest/no-disabled-tests | @vitest/eslint-plugin | tests left as it.skip / xit on the main branch. |
| vitest/no-focused-tests | @vitest/eslint-plugin | it.only / fit committed accidentally: silently shrinks the suite. |
| vitest/no-conditional-tests | @vitest/eslint-plugin | tests inside a runtime conditional: the assertion may never run depending on environment. |
| vitest/require-top-level-describe | @vitest/eslint-plugin | loose tests with no describe block: makes the spec output unreadable and breaks behaviour-name conventions. |
| vitest/valid-title | @vitest/eslint-plugin | test titles that aren’t strings or are duplicates within a file. |
| tsc --noEmit | TypeScript compiler | the type-as-first-test phase. Run on save and pre-push; with strict on, a compile failure is a failed test. |
eslint.config.mjsconfiguration snippet
import { defineConfig } from 'eslint/config';
import tseslint from 'typescript-eslint';
import vitest from '@vitest/eslint-plugin';
export default defineConfig({
files: ['**/*.{ts,tsx}'],
plugins: { vitest },
rules: {
'vitest/expect-expect': 'error',
'vitest/no-disabled-tests': 'error',
'vitest/no-focused-tests': 'error',
'vitest/no-conditional-tests': 'error',
'vitest/require-top-level-describe': 'error',
'vitest/valid-title': 'error',
}
});Apply these rules in eslint.config.mjs. The full enforcement across every tenet lives on the implementation page.
| Rule | Tool | Catches |
|---|---|---|
| vitest/expect-expect | @vitest/eslint-plugin | tests with no assertion: passes silently because nothing is checked. |
| vitest/no-disabled-tests | @vitest/eslint-plugin | tests left as it.skip / xit on the main branch. |
| vitest/no-focused-tests | @vitest/eslint-plugin | it.only / fit committed accidentally: silently shrinks the suite. |
| vitest/no-conditional-tests | @vitest/eslint-plugin | tests inside a runtime conditional: the assertion may never run depending on environment. |
| vitest/require-top-level-describe | @vitest/eslint-plugin | loose tests with no describe block: makes the spec output unreadable and breaks behaviour-name conventions. |
| vitest/valid-title | @vitest/eslint-plugin | test titles that aren’t strings or are duplicates within a file. |
| tsc --noEmit | TypeScript compiler | the type-as-first-test phase. Run on save and pre-push; with strict on, a compile failure is a failed test. |
eslint.config.mjsconfiguration snippet
import { defineConfig } from 'eslint/config';
import tseslint from 'typescript-eslint';
import vitest from '@vitest/eslint-plugin';
export default defineConfig({
files: ['**/*.{ts,tsx}'],
plugins: { vitest },
rules: {
'vitest/expect-expect': 'error',
'vitest/no-disabled-tests': 'error',
'vitest/no-focused-tests': 'error',
'vitest/no-conditional-tests': 'error',
'vitest/require-top-level-describe': 'error',
'vitest/valid-title': 'error',
}
});Apply these rules in eslint.config.mjs. The full enforcement across every tenet lives on the implementation page.
| Rule | Tool | Catches |
|---|---|---|
| jest/expect-expect | eslint-plugin-jest | tests with no assertion. |
| jest/no-disabled-tests | eslint-plugin-jest | skipped tests in main. |
| jest/no-focused-tests | eslint-plugin-jest | test.only / fit accidentally committed. |
| jest/no-conditional-expect | eslint-plugin-jest | expect() inside a runtime conditional: the assertion may never fire. |
| jest/valid-expect | eslint-plugin-jest | malformed expect() calls: async expects without await, expect() without a matcher. |
| jest/prefer-strict-equal | eslint-plugin-jest | toEqual where toStrictEqual would catch shape drift the looser matcher misses. |
eslint.config.mjsconfiguration snippet
import jest from 'eslint-plugin-jest';
export default [
{
files: ['**/*.{spec,test}.{js,mjs,cjs,jsx}'],
plugins: { jest },
rules: {
'jest/expect-expect': 'error',
'jest/no-disabled-tests': 'error',
'jest/no-focused-tests': 'error',
'jest/no-conditional-expect': 'error',
'jest/valid-expect': 'error',
'jest/prefer-strict-equal': 'warn',
},
},
];Apply these rules in pyproject.toml. The full enforcement across every tenet lives on the implementation page.
| Rule | Tool | Catches |
|---|---|---|
| PT001-PT030 | Ruff (flake8-pytest-style) | pytest anti-patterns: parametrize misuse, fixture misuse, raises without a match, and similar. |
| --strict-markers | pytest | typos in @pytest.mark.*: protects the test taxonomy that maps tests to acceptance criteria. |
| --strict-config | pytest | unknown options in pytest.ini or pyproject.toml: silent typos in test config. |
| branch = true | coverage.py | branch coverage gaps: both arms of every if; required for honest coverage. |
| fail_under | coverage.py | coverage drift below the team floor; turns coverage into a gate, not a metric to admire. |
| pytest --collect-only -q | pytest | the “does the test exist before the implementation” check: list every test the agent claims to have written, then run them. |
pyproject.tomlconfiguration snippet
[tool.ruff.lint]
select = [
"PT",
]
[tool.pytest.ini_options]
addopts = "--strict-markers --strict-config -ra"
markers = [
"spec: failing-first scenario tied to an acceptance criterion.",
]
[tool.coverage.run]
branch = true
[tool.coverage.report]
fail_under = 90Apply these rules in pmd-ruleset.xml. The full enforcement across every tenet lives on the implementation page.
| Rule | Tool | Catches |
|---|---|---|
| UnitTestShouldIncludeAssert | PMD | test methods with no assertion: pass silently because nothing is checked. |
| UnitTestContainsTooManyAsserts | PMD | tests with more than one logical assertion: splits the “one concept per test” line. |
| UnitTestAssertionsShouldIncludeMessage | PMD | assertions without a failure message: failed test output gives no clue what behaviour was expected. |
| S2699 (assertion missing) | SonarQube Java | Sonar’s heuristic version of the same: picks up cases PMD’s pattern misses. |
| S2925 (Thread.sleep in tests) | SonarQube Java | tests that paper over async behaviour with Thread.sleep: flakes that hide a real failure. |
| S5778 (one method invocation when testing runtime exceptions) | SonarQube Java | tests where the assertion is buried inside a helper such that failures don’t name the test target. |
| javac -Werror | javac | the type-as-first-test phase: compile cleanly with warnings as errors before any test runs. |
pmd-ruleset.xmlconfiguration snippet
<?xml version="1.0"?>
<ruleset name="P1 test-first">
<rule ref="category/java/bestpractices.xml/UnitTestShouldIncludeAssert"/>
<rule ref="category/java/bestpractices.xml/UnitTestAssertionsShouldIncludeMessage"/>
<rule ref="category/java/bestpractices.xml/UnitTestContainsTooManyAsserts"/>
<rule ref="category/java/codestyle.xml/EmptyMethodInAbstractClassShouldBeAbstract"/>
<rule ref="category/java/errorprone.xml/UnnecessaryBooleanAssertion"/>
<rule ref="category/java/codestyle.xml/IdenticalCatchBranches"/>
</ruleset>Apply these rules in .php-cs-fixer.dist.php. The full enforcement across every tenet lives on the implementation page.
| Rule | Tool | Catches |
|---|---|---|
| php_unit_test_class_requires_covers | PHP-CS-Fixer | test classes without an @covers attribute: coverage attribution becomes guesswork. |
| php_unit_method_casing | PHP-CS-Fixer | inconsistent test method casing: readable test names are part of the suite-as-spec promise. |
| php_unit_strict | PHP-CS-Fixer | loose-equality assertions (assertEquals on identity): accepts the wrong type without a word. |
| --enforce-time-limit | PHPUnit | tests that run for longer than the configured limit: flake bait and a sign of missing isolation. |
| --fail-on-risky | PHPUnit | tests PHPUnit flags as risky: assertions missing, output produced, or test isolation broken. |
| --fail-on-warning | PHPUnit | deprecation or framework warnings during a test run: silent test rot. |
.php-cs-fixer.dist.phpconfiguration snippet
<?php
return (new PhpCsFixer\Config())
->setRiskyAllowed(true)
->setRules([
'php_unit_test_class_requires_covers' => true,
'php_unit_method_casing' => ['case' => 'snake_case'],
'php_unit_strict' => true,
'php_unit_construct' => true,
'php_unit_dedicate_assert' => true,
])
->setFinder(
(new PhpCsFixer\Finder())->in(__DIR__ . '/tests')
);AI rules
.cursor/rules/p1-test-first.mdc--- alwaysApply: true --- ## Prickles P1: Test-First Development Write the failing test before the implementation. Watch it fail for the reason you predicted. Then write the smallest code that makes it green. In typed languages, the type is the first test. Define the signature, leave the body as a hole, let the compiler reject the inconsistencies the unit test would have caught, then write the test for the residue. Test the public contract rather than the internals. The unit's contract is the test's contract. Mock cascades through private collaborators are the smell, not the rule. Beck's exceptions still apply: spikes, throwaway demos, declarative config, and generated code are tested via the surrounding code, not as units in their own right.
Repo layout, CI, and ESLint wiring for these paths live on /implementation, not repeated on every tenet.
Counter-argument
The honest steelman is Coplien's, sharpened by DHH: the test suite written first becomes a cage. The team optimises for the granular, the unit-shaped, the easy-to-mock. The suite calcifies around an early architecture and refactoring becomes a rewrite of the tests rather than the code.3 Hillel Wayne extends the point: not all behaviour reduces to a runnable assertion;5 some properties (concurrency, distributed timing, performance under load) are cheaper to model-check than to test. James Coplien's “Why Most Unit Testing Is Waste” is the spiciest version of the argument and has not been fairly answered by anyone who skips its central claim: that high test mass on low-value internals is real cost, not virtue.
Counter-argument retort
Coplien aims well but hits the wrong target. The brittleness he describes is real and it is the brittleness of too many low-value unit tests on internals, not of writing tests first. The fix is not to abandon test-first. It is to write the test against the public surface that F6 Encapsulation tells you to expose, and to let coupling pull the rest along. Khorikov's Unit Testing Principles (2020) makes the case in detail: the unit's contract is the test's contract;11 internal-mock cascades are the smell, not the rule.
Hillel Wayne's extension is the genuine residue. Not everything reduces to a runnable assertion: concurrency, distributed timing, and load-shaped behaviour are cheaper to model-check than to test, which is why TLA+ exists.5 The reply is to keep both in the toolkit and not pretend tests cover everything. Property-based testing covers the algebraic residue. Model-checking covers the temporal residue. The test-first discipline still applies to the bulk of behaviour-shaped code where it lives.
DHH's critique, that test-driven design is a religion producing over-extracted hexagons, is right when the practitioner mistakes the discipline for an architectural style. Beck's own phrase in Tidy First? (2024) is the calm reply:12 tidyings and behaviour changes are different commits. The test drives the behaviour change rather than the tidying. The hexagon is one option among many. The red-green-refactor cycle is agnostic to the architecture it operates inside.
The discipline survives the agent. In fact the agent loop hardens it: an LLM with a failing test in front of it is grounded. An LLM without one is generating plausible prose at machine speed. The literature on this dates from 2023 and 2024 and is unanimous: Reflexion, Self-Refine, CRITIC, evaluator-optimizer all show the same shape.2 The exceptions Beck himself names (spikes, throwaway demos, declarative config, generated code) remain exceptions. They are footnotes on the rule, not replacements for it.
Notes
- [1]Kent Beck: Test-Driven Development by Example (Addison-Wesley, 2003). The canonical statement of red-green-refactor and the case for the failing test as a working specification. Test-Driven Development by Example (Addison-Wesley, 2003). The canonical statement of red-green-refactor and the case for the failing test as a working specification. The Preface names work that tests alone cannot drive, security software and concurrency among them, and Chapter 32, “Mastering TDD”, takes up what you don’t have to test.
- [2]Noah Shinn et al.: “Reflexion: Language Agents with Verbal Reinforcement Learning”, NeurIPS 2023. HumanEval pass@1 lifts from ~80% to 91% when the agent grounds self-feedback in test runs rather than pure introspection: empirical evidence that machine-checkable rewards drive agent improvement. Generalised by Madaan et al. (Self-Refine, 2023) and Gou et al. (CRITIC, 2024).
- [3]James Coplien: “Why Most Unit Testing Is Waste” (RBCS, 2014). The sharpest published counter to TDD: high test mass on low-value internals is real cost. The reply is not that Coplien is wrong about the failure mode but that the fix is to test the public contract, not to abandon the discipline.
- [4]Andrej Karpathy: “Verifiability” (karpathy.bearblog.dev, 2025). Software 2.0 automates what you can verify: tasks must be resettable, efficient, and rewardable. The failing test is the cheapest reward signal in the developer’s toolkit. It satisfies all three conditions exactly.
- [5]Hillel Wayne: “Why Don’t People Use Formal Methods?” (hillelwayne.com, 2019). Argues that not all properties reduce to a runnable assertion: concurrency, distributed timing, and load-shaped behaviour are cheaper to model-check than to test. A complement to TDD rather than a replacement.
- [6]Kent Beck: Extreme Programming Explained: Embrace Change (Addison-Wesley, 1999; 2nd ed. with Cynthia Andres, 2004). Places TDD inside the wider XP practice. Pair programming as continuous review and the ten-minute build are the surrounding context that made test-first economic for the original XP teams.
- [7]Edwin Brady: Type-Driven Development with Idris (Manning, 2017). Extends Beck’s red-green-refactor for dependently-typed languages: define the type first, leave the body as a hole, let the type-checker reject inconsistencies before the test runs. The typed-language flavour the merged P1 names alongside the dynamic-language flavour.
- [8]Scott Wlaschin: Domain Modeling Made Functional (Pragmatic Bookshelf, 2018). Industrial F# treatment of type-driven design: model the domain in the type system first, then let the compiler narrow the space the test has to cover. Pairs with T1 Domain-Driven Types in the Prickles canon.
- [9]Yaron Minsky: “Effective ML” / “Make Illegal States Unrepresentable” (Jane Street tech talks, 2010 onwards). Pushes the typed-first claim into OCaml: a type that rules out an entire class of failure is one whole branch of the test tree the developer never has to write.
- [10]Steve Freeman & Nat Pryce: Growing Object-Oriented Software, Guided by Tests (Addison-Wesley, 2009). Outside-in TDD: start with an end-to-end acceptance test, work inward through unit tests until the implementation arrives. The cleanest in-codebase example of how P1 (code-scale) and P2 (work-scale) compose into one discipline.
- [11]Vladimir Khorikov: Unit Testing Principles, Practices, and Patterns (Manning, 2020). The unit’s contract is the test’s contract, and internal-mock cascades are the smell rather than the rule. The book-length reply to Coplien’s critique that doesn’t require abandoning TDD.
- [12]Kent Beck: Tidy First? A Personal Exercise in Empirical Software Design (O’Reilly, 2024). Beck’s reply to the over-extraction critique: tidyings and behaviour changes belong in different commits. The test drives the behaviour change rather than the tidying: the architectural caging only happens when the practitioner conflates them.