verification evidence
why
green CI and a successful build prove the edit compiles, not that the changed boundary works. a type guard fix that passes typecheck may still miss one branch at runtime; a migration that succeeds locally may behave differently on the production schema.
rule
completion is evidence that the changed boundary works, not that an edit was made. choose the narrowest meaningful checks, then run broader checks before a repository-wide claim.
evidence order
- parse or generate the changed contract.
- typecheck the owner and direct consumer.
- build user-facing or shared outputs.
- exercise a narrow runtime smoke path when behavior changed.
- inspect the focused diff and
git diff --check.
select the smallest proof that can falsify the claim
| claim | direct evidence | evidence that is too weak alone |
|---|---|---|
| MDX/document contract is valid | parser/index rebuild and link/frontmatter inspection | a file existing on disk. |
| generated client matches its source | generator run plus generated-diff inspection | editing generated files until typecheck passes. |
| package contract remains consumable | owner and direct-consumer typecheck/build | typechecking only the producer. |
| user flow works after release | live route/flow probe and authoritative side effect | successful deploy command or health endpoint. |
| a fix solved a bug | reproducer fails before and passes after the root-cause change | an unrelated broad test suite. |
decision process
- start with the claim, ask what observation could prove it false, run that check first.
- broaden only when the boundary has consumers, generated artifacts, shared runtime behavior, or a deployment effect the narrow check cannot cover.
- never let a long green command list substitute for the one missing proof.
evidence record
- record together: command or interaction, environment, observed output, scope, remaining unverified boundary.
- a static check supports a source claim, not a live claim.
- a live happy path supports one outcome, not every authorization or failure branch.
good and bad
bad
claim: the fix worksevidence: pnpm typecheck && pnpm build && pnpm test # all greenmissing: the reproducer for the reported failure was never runresult: the bug ships; green checks substituted for the one proof
green CI never reproduces the reported failure; the one falsifying observation was never run, so a passing suite stands in for a proof it cannot provide, and nobody reopens the investigation until the next customer report for the same defect.
good
before: repro script fails on the pre-change buildchange: root-cause fix applied at its ownerafter: repro script passes on the rebuilt boundaryrecord: auth branches unverified by this proof — named, not hidden
the falsifying observation runs on both sides of the change, so the claim is grounded in behavior, not build status. naming the unverified boundary keeps the next reviewer from inheriting a completeness claim the evidence does not support.
do not
- claim a deployment succeeded without live UAT.
- add tests that only freeze incidental file structure.