A check goes red in the pipeline and green on your machine, on the same commit. Somebody reruns it. It stays red. Somebody says the pipeline is flaky, somebody else says it is stricter, and the change ships once the branch is rebased onto something that happens to work.
Neither explanation is right. The check is deterministic and it read something nobody passed to it.
The two that got me, a fortnight apart
The first was a test for a gate rather than for code. A hook refuses edits to production source when the branch has no test change, and its test ran the hook directly, against the repository it was living in. That test went red in the middle of a refactor, not because the hook was wrong: I had correctly added tests, so the hook allowed the edit and the assertion expecting a refusal failed.
The gate was right. The test was reading my working tree.
The second was coverage. A persistence adapter is only exercised by tests that
need a container, and those are excluded from the default build. My target/
still held execution data from an earlier run with them enabled, so the default
build counted lines it had never executed, and a clean checkout counted the truth.
| declared input | what actually decided it | |
|---|---|---|
| the hook’s test | a file path | whether my branch had test changes |
| the coverage gate | the compiled classes | which commands I had run before it |
The list, which is longer than a clock
Everybody has been bitten by time and injects a clock. These get injected far less often and are the same defect:
| what it reads | why it differs |
|---|---|
| the git working tree | your uncommitted work is not anybody else’s |
target/, build/, node_modules
|
accumulated from every command you have run |
| environment variables | your shell, not the pipeline’s |
| the filesystem outside the repo | your home directory has things in it |
| network reachability | your VPN, your DNS, your cache |
| the current directory | whatever you last cded into |
A check that reads any of those is answering a question about the reader. The answer is stable per machine, which is what makes it convincing and hard to see.
Two fixes, and they are different
When the ambient thing is the subject, inject it. The test-first hook is about a working tree, so it takes one:
def repo_root() -> Path | None:
override = os.environ.get("HARNESS_REPO_ROOT")
if override:
return Path(override)
...
Each test then builds a scratch repository with exactly the files that case is about. It is the same move as taking a clock as a dependency, applied to a directory instead of to a moment, and it belongs to the same argument as keeping the decidable half in a script: name the thing you depend on, or it will name itself.
When the ambient thing is accumulated output, delete it. Coverage reads
whatever exec data is lying around, and no amount of care makes a stale file
correct, so the entry point runs clean first. It costs a minute and it buys the
only property a gate needs: that it means the same thing everywhere.
The part I want to be honest about
I wrote the second one up as a lesson, and then made the same mistake again two hours later, on the same gate, with new classes.
The write-up did not help, and that is the useful finding.
A note stops you paying for a diagnosis twice. It does not stop you repeating a mistake, because it still depends on somebody remembering at the moment it matters.
So the entry point cleans, and a second check refuses a coverage exclusion for a class the default suite can actually reach. The note now records why those exist, which is the job a note is good at.
If a lesson is about something a script could check, the lesson is not the deliverable. The script is.
If your team has stopped believing the pipeline, that’s the work I do.