operating specimen
A successful checkout must result in the access and terms the buyer was promised.
A payment can succeed, the screen can say so, and the buyer can still not have what they paid for. This is one decision about where I drew that line, and the evidence I required before shipping it.
The promise being made
When someone buys, they are promised two things: access to what they bought, and the terms they agreed to. A charge going through is not either of those. It is the moment the promise is made, not the moment it is kept.
So the release question is not "did payment succeed". It is "did this buyer end up with the access and the terms we said they would". Those can come apart quietly, and when they do, every dashboard still reads green.
The decision
Returned the work to draft
The release gate was running but had not been observed passing. Running is not passing. I moved it back rather than let a green-looking pipeline stand in for evidence.
Narrowed the proof to two buyer journeys
Rather than wait on unrelated test debt, I cut the release question down to the two journeys that actually carry revenue. A smaller claim I could verify beat a broader one I could not.
Held the release until those journeys were observed passing
Not asserted, not inferred from a config change -- observed, in a run I could point at.
I made these calls. Other people wrote code alongside them, and the implementation was not mine alone; what I am claiming here is the judgment about what had to be true before this shipped.
Why the coverage we already had was not enough
There were already tests over checkout, and they passed. The problem was what they were confirming.
A stale mapping between the test purchase and the thing it was supposed to grant meant the grant step could quietly do nothing. The purchase completed, the test asserted the parts it knew to assert, and the durable access the buyer was owed was never checked. Coverage existed; it was pointed slightly to the left of the promise.
That is the failure worth naming, because it is not a bug that announces itself. Nothing errors. The suite is green. The gap only becomes visible when you ask a test to assert the thing the customer actually receives.
How it went
- 01
Coverage looked healthy
Checkout tests existed and passed. Nothing on the board suggested a promise could be missed.
- 02
A grant that could silently do nothing
A stale mapping meant the step that hands the buyer their access could no-op without failing anything.
- 03
Release held
Work went back to draft. The gate was executing but had not been observed green, and that difference decided it.
- 04
Scope narrowed to two journeys
The release question was reduced to the two revenue-critical buyer paths, so it could be answered rather than deferred.
- 05
Observed passing, then shipped
Both journeys ran and passed in a recorded run before the work was allowed out.
What changed about the release boundary
Before
- Payment or success-looking state
- Coverage appears successful
- Promised access or terms may be absent
Mapping mismatch, silently doing nothing
After
- Payment outcome
- Named buyer journey observed
- Promised access and tested terms verified before release
What has to be true before release
Two journeys, each stated as a condition the buyer would recognise rather than a test name.
Guest checkout
Someone who buys without an account still ends up with an account and access.
- The purchase completes in the test environment
- The buyer lands in an authenticated state, not a sign-up dead end
- The access they bought is durably recorded, not just displayed
Accepted recurring upsell
Someone who accepts the recurring add-on gets the terms and the wider access they agreed to.
- The recurring terms match what was tested
- The expanded access is the access that was offered
- Both grants are durably recorded, not implied by the purchase
The evidence, and what it does not cover
What was observed
Both journeys passed in one recorded run.
- Journeys
- The two named above, run end to end against the test environment.
- Result
- 2 passed, in 33.4 seconds.
- Required check
- For that same run, the check that gates merging reported no failed, cancelled or skipped jobs among the ones it requires.
What it does not show
This is test-environment evidence. It is not a business outcome.
- Environment
- Everything above ran in a test environment against test payments. None of it is a production measurement.
- Not shown
- Production conversion
- Revenue lift
- Complete billing coverage
- Live provider timing
- Sole authorship
- An entirely green pull request
$ ./book
Where in your customer journey does a success-looking state still stand in for proof that the buyer received what they were promised?
If that question landed somewhere specific, that is usually worth half an hour.