Five failures from building a tool that scores code quality: what the proxies actually measured, what the checks never examined, why validation had no oracle, and what an authoritative interface costs when it is wrong.
Assumes:You are building, or relying on, a tool that scores work its author cannot be trusted to score. Read in order: three kinds of measurement failure, then how to validate any of it, then what happens when it is wrong. The last two are two halves of the same argument about authority.
Download 5 documents + MANIFEST
Six ways a code scanner mistook writing about a practice for following it, and a seventh found inside the check built to prevent the other six.
Four ratios used to score code quality, and four different reasons each one measured something other than what it claimed.
Three checks that passed because they had no subjects, and why the fix is to make each one declare how many things it looked at.
Three attempts to validate a scoring tool against human judgement, and why all three failing was evidence for the tool's premise rather than against it.
What to do when a tool built to outrank your judgement is wrong, given that overruling it destroys the only property it had.
Shipped inside the archive. Reproduced here so you can read it before downloading.
# Measuring your own work
Five failures from building a tool that scores code quality: what the proxies actually measured, what the checks never examined, why validation had no oracle, and what an authoritative interface costs when it is wrong.
Source: https://the-handover.org/packs/measuring-your-own-work
Built: 2026-08-29
Documents: 5
Licence: Handover-1.0 (© Captivated Ltd) — https://the-handover.org/licence
## What this pack assumes
You are building, or relying on, a tool that scores work its author cannot be trusted to score. Read in order: three kinds of measurement failure, then how to validate any of it, then what happens when it is wrong. The last two are two halves of the same argument about authority.
## Contents
### docs/writing-about-a-thing-is-not-doing-it.md
Title: Writing about a thing is not doing it
Summary: Six ways a code scanner mistook writing about a practice for following it, and a seventh found inside the check built to prevent the other six.
Authors: Leon Mallett, Founder of Captivated Ltd (captivated.online)
Source: https://the-handover.org/docs/writing-about-a-thing-is-not-doing-it
Last confirmed working: 2026-08-28
Marks: Hard-Won
SHA-256: 9742b4c5e60645474347c4b2a07da28d99d772f786e260ae0c03a90ed1467852
### docs/four-ratios-four-different-failures.md
Title: Four ratios, four different failures
Summary: Four ratios used to score code quality, and four different reasons each one measured something other than what it claimed.
Authors: Leon Mallett, Founder of Captivated Ltd (captivated.online)
Source: https://the-handover.org/docs/four-ratios-four-different-failures
Last confirmed working: 2026-08-28
Marks: Hard-Won
SHA-256: d8a859506c167d7fe7b22013b38b95482da57047143860e958f1d3876f88b018
### docs/nothing-was-wrong-or-nothing-was-examined.md
Title: Nothing was wrong, or nothing was examined
Summary: Three checks that passed because they had no subjects, and why the fix is to make each one declare how many things it looked at.
Authors: Leon Mallett, Founder of Captivated Ltd (captivated.online)
Source: https://the-handover.org/docs/nothing-was-wrong-or-nothing-was-examined
Last confirmed working: 2026-08-29
Marks: Hard-Won
SHA-256: 2ab9cd2194ff17ffc7fc50caf266f9ebd9ebd6ee70839c67039d3d113f9bfd73
### docs/we-could-not-get-ground-truth.md
Title: We could not get ground truth, and it took three attempts to accept it
Summary: Three attempts to validate a scoring tool against human judgement, and why all three failing was evidence for the tool's premise rather than against it.
Authors: Leon Mallett, Founder of Captivated Ltd (captivated.online)
Source: https://the-handover.org/docs/we-could-not-get-ground-truth
Last confirmed working: 2026-08-28
Marks: Hard-Won
SHA-256: 5c7616cb4f9d3dd1fc32d5bb976a55fe77f73f5a4e633c846801c164762bbe84
### docs/when-the-tool-disagrees-with-you.md
Title: When the tool disagrees with you, neither of you gets to win
Summary: What to do when a tool built to outrank your judgement is wrong, given that overruling it destroys the only property it had.
Authors: Leon Mallett, Founder of Captivated Ltd (captivated.online)
Source: https://the-handover.org/docs/when-the-tool-disagrees-with-you
Last confirmed working: 2026-08-28
Marks: Hard-Won
SHA-256: 109e60a2641af641469b70819645502b7a8d30e179b518db6e4ae56450d1a3c7
## Licence and attribution
Copyright is retained by the authors and Captivated Ltd. You may use these
documents in your own projects — private, public, commercial, client work —
and adapt them to fit. What is reserved is republishing the collection as a
collection: mirrors, or catalogues assembled out of these documents.
Each file carries an attribution header naming its authors, its source and the
date it was last confirmed. Keep it: once a file has been separated from this
pack, that header is the only thing that still credits the person who wrote it.
## Verifying this pack
Each SHA-256 above is of that file exactly as it appears in this archive. Verify one
with `shasum -a 256 <file>`; a file whose hash does not match was modified after
packaging.
Document files are byte-reproducible: the same source document, at the same
confirmation state, always produces the same bytes. The archive as a whole is not,
because this manifest records the build date above — two builds on different days
differ in that line and therefore in the archive, while every document file inside
stays identical. The per-file hashes are the thing to check.