> Nothing was wrong, or nothing was examined — from The Handover, the-handover.org/docs/nothing-was-wrong-or-nothing-was-examined
> Authors: Leon Mallett (captivated.online) with Claude Code · Last confirmed working: 2026-08-29
> © Captivated Ltd — free to use in your own work, not to redistribute as a collection. the-handover.org/licence

There is a shape of check that almost every codebase has:

```rust
let violations = collect_violations();
assert!(violations.is_empty(), "found: {violations:?}");
```

It passes in two situations that look identical from outside. Nothing was wrong. Or nothing
was examined. **Only the first is evidence, and the check reports them the same way.**

This is not [a check that cannot fail](/docs/verification-that-cannot-fail) — a separate and
better-known problem, where the assertion is malformed and no input could ever turn it red. These checks are correctly
written. They run, the predicate is sound, the collection really is empty. They have no
subjects. That distinction matters because every remedy for the first problem leaves the
second one untouched: the assertion is fine, so reviewing the assertion finds nothing.

Three instances, from a tool that scores about forty software projects against roughly
seventy statements.

## A criterion that agreed with everything

One statement asked whether a project's dependency lockfile is committed.

The implementation checked whether a lockfile **exists.** Every project that declares
dependencies has one; the package manager writes it. So the check's predicate was true of
every subject it would ever see, and it returned the same answer to all of them.

A full denominator, and no discriminating power whatsoever. It was not measuring the
portfolio, it was counting it.

The gap between *exists* and *committed* is exactly the case the statement was written to
catch — a lockfile sitting in the working tree, unstaged, so a clean clone resolves a
different dependency tree. Nothing in the portfolio had one. The bug was therefore invisible
to every scan, permanently, until someone read the check next to the sentence it was
implementing.

It was fixed anyway, with the reasoning written down: **the absence of a case today is not
the absence of the class.** A class of defect is closed by fixing it, not by observing that
it is currently quiet.

## A failure count that included things it could not answer

Another statement asked whether real crashes get reported back from users' machines. The
portfolio banner showed **0 out of 19** — nineteen projects, none passing.

Three of those nineteen were iOS applications. Apple's Xcode Organizer delivers crash
reports for any App Store build, with no library, no dependency and no configuration. There
is nothing in the repository to detect, because a correctly built project leaves nothing to
observe.

For those three the check could only ever return one answer. They were in the denominator
of a measurement that had no power over them, and their contribution to "0 out of 19" was
not a finding — it was the check counting its own blind spot as a failure.

The fix was not to exclude them quietly, which would have moved the number without saying
why. The floor became **0 out of 16, with `[+3 unverifiable here]` printed alongside**, and
the drill-down lists those three under "not answerable from the repository". Not a gap and
not a pass. A reader can see the question was asked and where the answer actually lives.

**A silently changed denominator is a worse outcome than the wrong one**, because the wrong
one is at least arguable.

## A test that passed five claims nothing could contradict

The rubric declares prerequisite edges: statement B is unsatisfiable unless statement A
holds. A test asserts the portfolio never contradicts one — no project satisfies B while
failing A.

It passed. Every run, green.

There are five such edges, and **not one of them could have been refuted.** They all depend
on a statement currently at 0 out of 44 — no project has a tagged release, so no project can
have a release that builds reproducibly *from* a tag, so nothing can be in the state that
would contradict the edge. The test was asserting something about an empty set.

That is the purest form of it. The assertion is correct. The logic is right. It will start
doing real work the moment a project tags a release. Until then a green result means
**"nothing could have gone wrong here"**, and it was being read as "nothing did."

## The fix: make the check declare its candidates

A module with one function. A check of this shape now reports how many things it actually
examined, and zero is announced:

```
UNARMED: `no_criterion_edge_is_contradicted_by_the_portfolio` examined 0 candidates,
so it could not have failed. Green here means untested, not clean.
```

Three decisions in that, each of which took a wrong turn first.

**It prints; it does not fail.** The obvious move is to assert that candidates exist. That
is wrong, and expensively so: a check with nothing to examine is frequently legitimate — a
lint over a rule no statement uses yet, a tripwire waiting for the portfolio to reach a
state. Failing forces someone to invent a fake candidate to get the build green, and a
fabricated subject is worse than the silence it replaces. **A rule that can only be
satisfied by lying will be satisfied by lying.**

**The wording is load-bearing, and it is tested.** An earlier version reported the count and
nothing else. "0 candidates" reads as good news to anyone skimming — zero problems. So the
message has to say *could not have failed* and *untested*, and there is a test asserting
those exact words survive editing. Testing the phrasing of a warning feels excessive until
you notice the entire defect being addressed is one where the reader draws the wrong
conclusion from technically accurate output.

**Return a value that cannot be dropped.** The function returning the message is marked
`#[must_use]`, so a call site that computes the arming and ignores it fails to compile. A
diagnostic about checks that quietly do nothing should not itself be quietly ignorable.

## What this looks like elsewhere

The shape is not specific to a scoring tool. Anywhere a report says *no problems found*,
ask what the denominator was:

- A test suite run with a filter that matches **zero tests** exits 0.
- A security scan whose path configuration is wrong scans an empty directory and reports
  clean.
- A linter with a glob that stopped matching after a directory rename passes every commit.
- A migration that iterated over the wrong table updated every row it found.
- A CI job whose changed-files detection returns nothing skips the work and goes green.

Every one produces the same output as genuine success, and every one degrades **silently and
in the safe-looking direction.** A check that starts failing gets attention within a day. A
check that stops examining anything can run green for a year, and its greenness is precisely
what stops anyone looking.

## The single question

For any check that reports the absence of a problem: **how many things did it look at, and
could any of them have come out differently?**

If the answer to the first is zero, the result carries no information. If the answer to the
second is no, it carries no information either — a check whose predicate is true of every
subject it can see, like the lockfile one, has a full denominator and no power at all.

The reason this class survives so well is that both failures present as success, and success
is the state nobody investigates.