Key takeaways

Test in proportion to failure cost, not to lines of code. A tax calculation line and an admin tooltip are different risks. Equal attention to both is either waste or negligence.

“More tests” without a risk model inflates cost without cutting the most expensive failures. A green pipeline with a thousand brittle checks can still miss the scenario that eats margin or breaks a regulatory report.

Full feature cost is wider than the sprint. Build + verification + support + failure price. If the fourth term is huge, “speeding up” by cutting verification is false economy.

Early detection is cheap relative to late discovery — but not infinitely. There is a point where extra checks barely reduce residual risk and start slowing product learning.

AI makes writing tests cheaper; it does not cancel failure economics. A model quickly drafts cases for generated code. It rarely knows which failure costs the company a week and which costs fifteen minutes on a Friday.

The full cost of a feature

Engineers often price a feature as “how long will implementation take.” That is capital accounting for construction only. In reality there are four layers.

First — creation: analysis, design, code, review, contract alignment. Everyone sees it on the board.

Second — verification: manual scenarios, automated tests, staging, data checks, near-prod runs. This is usually cut first because it does not ship a visible UI “feature.”

Third — support: support docs, flags, migrations, observability, runbooks. Without it the feature becomes a surprise for whoever is on call.

Fourth — failure price: what it costs if production behavior is wrong. That includes direct damage (money, downtime, fines), indirect damage (support load, reputation, churn), and organizational cost (release freezes, heroic night fixes, lost team focus).

The fourth layer is what makes cheap testing expensive. If failure price is low, you can consciously accept risk and ship earlier. If failure price is high, “saving” on verification moves cost into the future with interest.

Example. A team adds a tariff discount. Build takes two days. Happy-path automation takes half a day. In production they miss an edge: the discount applies twice on a retry charge. Weekend direct loss reaches tens of thousands. Investigation, compensation, postmortem take more days of key people. The “simple feature” full cost dwarfs the ticket estimate.

The same code volume elsewhere — changing an email signature. Failure price is near zero. A heavy scenario matrix there is a tax on speed without insurance.

Architecture connects directly: higher coupling and closer proximity to money, permissions, and irreversible actions raise failure price. You cannot stretch one coverage policy across the whole repository. That is like insuring a garage and a nuclear plant with one policy.

Why “more tests” is the wrong answer

Coverage culture grew from a true observation: without automated insurance, refactoring is scary and regressions are expensive. The observation became dogma: more red/green dots mean a better product.

Dogma breaks in three places.

First — false safety. Line coverage shows code ran. It does not show that the right behavior ran with the right data and the right permissions. You can cover 90% of a module and miss the only branch that rounds currency.

Second — attention shift. Teams optimize what you measure. If the KPI is coverage percent, people write tests for getters, pure helpers, and internals that will be rewritten tomorrow. Expensive integration scenarios stay “for later” because they are slow and awkward in the report.

Third — ownership cost of tests. Every test is also code: you repair it when APIs change, when it flakes, when staging updates. A cheap-to-write test can be expensive to live with — especially if a model generated it against current implementation rather than product intent.

So the question is not “how many tests do we have?” It is “which failures do we consciously insure, and which do we accept?” That is risk language familiar to anyone who has thought about error cost in model routing or change boundaries in legacy systems with AI.

The anti-pattern “coverage first, risks later” is cousin to “ship first, policies later.” While the system is small, it looks like speed. At product volume you do not pay for having no tests at all — you pay for having no tests where failure is expensive.

Detection cost versus failure cost

The classic claim “a requirements bug is cheaper than a production bug” is useful as direction and harmful as religion with fixed multipliers. Slides from the 1990s are not a law for your product. Layer logic still holds.

The later a defect is found, the more has already stuck around it: dependent code, data, user habits, sales promises, caches, integrations. A production fix often includes not only a patch but migration, communication, and checking that compensation did not create a new defect.

Practical consequence: invest in early detection where late failure is expensive. Not “shift everything left to the maximum.” Left also has cost: environments, data, people, time to market feedback.

It helps to keep two axes:

High price × high uncertainty is the maximum-insurance zone. Low price × low uncertainty is conscious risk.

Detection cost falls when you have good seams, contracts, and reproducible data. Sometimes “cheap testing” does not mean “few tests” — it means “testing is expensive because of architecture”: a monolith without boundaries, hidden state, environments that take an hour to stand up. Then the right lever is not another overnight E2E, but a seam that makes verification cheaper. That intersects readability-under-change ideas from the Clean Code digest and legacy-without-insurance discipline in the programming books hub.

Not all bugs cost the same

This is the central thesis, and it deserves plain language.

A UI tooltip bug and a bonus accrual bug are both “bugs.” Their economic nature differs. The first burns nerves and a support ticket. The second can burn customer money and legal risk.

If a team allocates test attention proportional to changed lines, it optimizes report convenience, not business resilience. Risk does not live in the diff as text; it lives in the consequences of wrong behavior.

Practical translation:

• Look at irreversibility: money, deletion, external publication, permission changes, legally meaningful actions.
• Look at blast radius: one user or all tenants; one region or the whole contour.
• Look at detectability: will monitoring fail fast, or will data drift quietly for months?
• Look at rollback cost: can you flip a flag, or do you need a reverse migration over dirty data?

Hence the effort rule: test risk, not code. Code is the carrier. Risk is the management object.

In a sprint that looks like this. Payment module change: contract tests on calculation, idempotency scenarios, rounding checks, a canary, a dashboard for sum anomalies. Button copy change: visual check and a quick glance — enough. The same “Definition of Done: +unit +e2e” for both is either paranoia or ritual.

Risk-based testing is long established in quality literature. In 2026 engineering practice a new noise appears: AI offers to “cover everything” because generation is cheap. Cheap generation without ranking by failure price simply accelerates false confidence.

Diminishing returns on checks

Even on a critical path, diminishing returns apply. Early checks remove the most probable and expensive error classes. Later ones catch rarer combinations. At some point the cost of one more scenario exceeds the expected reduction in damage.

This is not an excuse for laziness. It is a call to count marginal benefit. If you already insure calculation, permissions, and payment idempotency, a twentieth variant E2E for a rarely used promo code may cost more than leaving it under a manual smoke and a solid alert.

Signs you are past the payback point:

• Tests fail more often from staging and data than from real regressions.
• Pipeline feedback is measured in hours, and the team bypasses it with local skips.
• New cases copy old ones with micro-changes and add no new risk class.
• “Red” stops being a signal and becomes background noise.

A cluster satellite will unpack diminishing returns in depth (planned: diminishing-returns-testing-2026). Here the principle is enough: the goal is to reduce expected damage, not to maximize check count. When a marginal check barely moves damage, invest the hour in observability, a feature flag, or design simplification instead.

Design simplification is underrated as a “test.” Fewer branches, less hidden state, fewer ways to trigger irreversible action “by accident” — that lowers failure price and verification cost at once.

Speed and reliability as one economy

The “faster versus more reliable” debate is often framed as culture: startup versus enterprise. Economically it is the same balance with different failure prices and different costs of delayed market learning.

If the product is hunting product-market fit, the cost of being late on a hypothesis can exceed the cost of a minor bug on a secondary screen. Then it is rational to accept risk and insure only what kills learning: data loss, broken signup, broken pilot payment.

If the product is regulated or holds customer money, failure price dominates. Then “speed” without insurance is acceleration toward an expensive incident. Real speed there is the ability to change without a long defect tail: good seams, flags, canaries, fast rollback.

A useful lead formulation: not “quality versus speed,” but “what speed is sustainable at our failure price.” A team that “flies” for two weeks and then spends a week fighting production is not fast — it took a loan.

The second cluster satellite covers bug price and this balance (planned: bug-cost-speed-vs-reliability-2026). In the pillar, fix this: choose the balance through failure price and delay cost, not slogans.

The link to production reliability is obvious: pre-release tests are one layer. After release, monitoring, limits, degradation, and playbooks work. Sometimes it is cheaper to strengthen production detection and fast rollback than to prove correctness of every combination before ship — especially when the state space is huge and the price of a single quiet bug is moderate.

How to allocate effort in practice

Here is a working frame you can adopt without a year-long “methodology.”

1. Classify changes by failure price

At PR or ticket level a coarse scale is enough: low / medium / high / critical. Criteria in advance: money, permissions, data, irreversibility, blast radius. Do not argue taste — argue consequences.

2. Choose insurance level by class

Example policy (calibrate for yourself):

Keep the policy short. If you cannot apply it to a ticket in a minute, people will not use it.

3. Separate intent tests from implementation tests

An intent test breaks when the product promise changes. An implementation test breaks when internals change. The latter are cheap to generate and expensive to maintain. The former are what you should write by hand (or carefully edit after a model) on expensive risks.

4. Account for ownership cost of automation

Once a quarter ask: which tests go red most often without payoff? Which never caught a real defect? What blocks fast feedback? Deleting a dead test is also an investment in signal quality.

5. Use AI as a draft accelerator, not as risk owner

Let the model propose cases. A human ranks them by failure price and deletes noise. Ask for scenarios that break when intent changes, not when a private method is renamed. Same principle as AI-assisted code: generation speed is not risk reduction.

6. Make verification cheaper with architecture

Seams, clean module boundaries, reproducible fixtures, test doubles only at edges — all of that lowers detection cost. Sometimes an hour on a seam saves a week of brittle E2E.

A one-page “risk × uncertainty” matrix

A simple 2×2 on the team wall is enough:

• Expensive and unclear — maximum insurance + exploration + canary.
• Expensive and clear — hard contracts and regression on invariants, less exploratory chaos.
• Cheap and unclear — prototype, manual pass, fast user feedback.
• Cheap and clear — minimal formalism; do not spend ritual.

Revisit classes after incidents. A postmortem without re-evaluating failure price is theater.

What to do this week

Do not roll out a brand-new “quality culture” from scratch. Take four concrete steps.

1. Take the last three production incidents. For each estimate: failure price, where the defect could have been caught cheaper, which check was missing not “in general,” but right here.
2. Add two fields to the PR template: “failure-price class” and “how we roll back.” One line each — no essay.
3. Pick one critical flow (payment, permissions, data import). List five invariants you must not break. Hang contract or integration checks on them if they are missing.
4. Delete or disable one noisy automated test that does not protect an invariant. Free the signal.

If after that you still want to “raise coverage by 10%,” ask first which expected damage that reduces. If there is no answer, it is not investment — it is reporting.

FAQ

Further reading

• LLM gateway and FinOps — parallel: error price decides where you need an expensive model and tight control.
• Clean Code digest — tests as insurance for behavior that is expensive to lose.
• Best programming books — including legacy, refactoring, and release reliability.
• Legacy after 15 years and AI — how to change a system when insurance is thin and error cost is high.
• AI systems observability — the post-release layer: see degradation before the customer does.

Quality economics is not a call to test less out of laziness, nor to test more out of fear. It is a call to pay insurance where a fire is expensive, and not to spend a sprint on a policy for a matchbox. Next time you hear “we need more tests,” ask back: more — relative to which failure price?