Evidence
Controlled baseline — everything held constant except explicit domain types
The most tightly controlled comparison in this evidence base, and the one that contradicts the easy conclusion. It is published because it did.
What this was
- Status
- Complete — and it refuted the hypothesis it was built to test
- Repository
- kemiller2002/time-entry-state-machine
- Branch
- main
- Ending commit
- 73a63f1ac319
- Executed by
- One session, which had already designed and implemented the change once in F# before the trial began.
Objective
For the same conceptual change, compare a codebase with explicit domain types against an otherwise-competent conventional implementation — holding feature scope, code quality, the change itself, and the agent constant.
Hypothesis
Explicit domain types reduce the cost of propagating a change. Falsification was preregistered: if the conventional baseline needed equal or fewer edits and surfaced equal or fewer stale-call-site defects, the hypothesis would not be supported.
Method
A conventional TypeScript baseline was written to reflect competent practice — shared validation, immutable updates, a discriminated result type, real tests — with the single removed ingredient being opaque domain types for the value that changes meaning. The identical conceptual change was then applied to it and compared against the same change already made to the explicitly-typed F# domain.
Findings
- On field renames there was no measurable difference. Both the structural and the nominal type system caught every call site automatically and completely.
- On arithmetic business rules embedded in expressions, neither type system helped at all. A wrong divisor compiles cleanly in both, because the value is a bare number until it crosses a smart constructor.
- The one real asymmetry was a discovery failure, not a type failure: a grep for the conversion pattern missed a seventh site where the same rounding was written as a division by 360 rather than by 6. It was found only by a second, suspicion-driven search.
- That asymmetry traces to architecture rather than language — the explicitly-typed domain routed duration arithmetic through named accessors instead of inlining the conversion, so there were fewer raw sites to miss.
What this refuted
- This experiment contradicts the stronger form of its own hypothesis. Explicit domain types do not reduce ripple cost as a blanket claim: for arithmetic-embedded business rules they do not reduce it at all.
- The hypothesis record remains open rather than supported, with this trial recorded as contradicting evidence.
- The advantage that was observed is achievable without changing the type system — centralising a conversion is a discipline, not a language feature, and the report says it arguably should have been applied to the baseline too.
Anomalies, failures and confounders
- The change had already been designed and implemented once before the trial, so both writing the baseline and applying the change benefited from knowing the target precisely.
- The same agent that values explicit domain types wrote the competently-conventional baseline it then measured against.
- No blinding.
Limitations
- One change, one baseline, one agent, one language pairing. A single trial is a data point, not a finding.
- It measures the cost of propagating a known, well-understood change through each architecture, not the cost of designing the change.
- It compares F# against TypeScript rather than a typed against a primitive-typed version of one language, so language and type-system variables are not fully separated.
- An independently written baseline would be more convincing than one written by the same author.
Interpretation
This is the most carefully controlled experiment in the evidence base and it went against the
hypothesis. That combination is why it matters more than the trials that agreed.
The design was honest before the result was known. Falsification criteria were written in
advance: if the conventional baseline needed equal or fewer edits and surfaced equal or fewer
stale call sites, the hypothesis would not be supported. The baseline was deliberately competent
— shared validation, immutable updates, a discriminated result type, real tests — with the single
removed ingredient being opaque types for the value that changes meaning.
The result splits cleanly in two, and the split is the finding. Field renames are a structural
change, and both type systems caught every affected site automatically and completely; there was
nothing to choose between them. Arithmetic embedded in an expression is not a structural change,
and neither type system helped at all, because the value is an ordinary number right up until it
crosses a smart constructor — and the arithmetic sits upstream of that boundary in both designs.
The one asymmetry that did appear was almost mistaken for a type-system result. A pattern search
for the conversion missed a site where the same rounding had been written as a division by 360
rather than by 6, and it was recovered only by a second search prompted by suspicion. It would
have been easy to file that under "explicit types win." The report does not: it traces the
difference to the explicitly-typed domain having routed its arithmetic through named accessors
instead of inlining the conversion, and states that the same discipline was achievable in the
baseline and arguably should have been applied there.
So the conclusion is narrower than the hypothesis wanted, and more useful than it would have
been: centralising a conversion is what avoided the missed site, not nominal
typing. Those are different interventions with different costs, and this experiment is the
reason we can tell them apart.
It also leaves a real gap it names itself. Comparing F# against TypeScript does not separate
language from type system. The trial that would — a primitive-typed version of the same language
against an explicitly-typed one — has not been run.