The read-and-verify rule
The single discipline that separates useful AI testing from dangerous AI testing.
There is one rule that separates useful AI testing from dangerous AI testing: read every generated test, and confirm it would fail for the right reason. A test you did not read is not an asset. It is a liability with a green checkmark.
Why this matters: when a model writes the code and the test in the same pass, a mistake in the logic often becomes a matching mistake in the test. The test hard-codes whatever the buggy code produces, both halves agree, and the suite passes while the feature is broken.
The verification move is simple and it never lies. Break the code on purpose and rerun the test. Change a plus to a minus, return the wrong value, or comment out the check. If the test still passes, it was never protecting anything.
A quick checklist for every generated test:
- Does the assertion state the expected value, not just that a value exists?
- Would it fail if the behavior were wrong?
- Is it testing the requirement, or merely repeating what the code happens to do?
- Does the test name describe the behavior under test?
Read-and-verify takes seconds per test and turns a pile of plausible code into a suite you can actually rely on.
Take one AI-generated test that passes. Deliberately introduce a bug in the code it covers and rerun. If the test still passes, rewrite the assertion until the broken code makes it fail.
Enjoying the free lessons? Get an email when we publish new courses and updates — no spam, unsubscribe anytime.
Discussion (0)
Ask a question or share what worked for you. Comments are reviewed before they appear.
No comments yet. Be the first to start the discussion!