The real danger: confident inaccuracy

The failure mode that should keep a technical writer awake is not obvious nonsense. It is a fluent, well-formatted paragraph that is simply wrong.

A language model predicts plausible text. When it does not know the real name of a flag, it does not stop — it produces a name that looks like the ones it has seen. So you get:

• A --verbose flag that was never implemented
• An /api/v2/users endpoint that does not exist
• A timeout parameter with a default of 30 seconds that you invented together
• A config key spelled the common way rather than the way your product actually spells it

Every one of these reads as confident, correct documentation. That is exactly what makes it dangerous. An obviously broken doc gets ignored; a plausible wrong doc gets trusted, copied, and shipped, and then a reader files a bug against instructions that never worked.

The rule that follows is blunt: a fluent wrong doc is worse than no doc. No doc sends a reader to the source; a wrong doc sends them off a cliff with confidence.

This does not mean avoid AI. It means treat every specific, checkable detail it produces — names, numbers, endpoints, defaults, version strings — as unverified until you confirm it against the actual product or code. Numbers, dates, and version strings are among the most commonly invented items, so watch those hardest.