# Review log — 2026-09-26

AI-assisted source inspection, not independent human coding.

All detected blanket cases were checked against saved source user-agent and rule lines. This caught Cadence's inline `User-agent: OAI-SearchBot Allow: /` syntax: the parser initially treated the whole value as an unknown token and fell back to wildcard Disallow. It is now ambiguous, with a regression test, and removed from confirmed blanket counts. Original observations remain unchanged; `results.json` is the reviewed/recomputed classification.

Confirmed examples: Garmin explicitly includes PerplexityBot in a disallowed agent group; Simon Property Group separately lists PerplexityBot and Perplexity-User in its disallowed group. Airbnb and Northrop Grumman explicitly disallow GPTBot and ClaudeBot, while their search-agent groups carry different rules. PG&E and Prudential explicitly disallow GPTBot. Fortinet's ClaudeBot and Perplexity-User outcomes derive from wildcard fallback, not explicit named targeting; this is visible in the data and is not described as targeted blocking.

Reviewed the offending lines for every one of the 64 syntax-ambiguous files. Most use paths without a leading slash (including wildcard-prefixed patterns), full URLs, orphan directives or inline directives. This strict inventory declines to resolve vendor-specific tolerance. One malformed line conservatively makes the file ambiguous for all eight agents, even when another group's meaning looks clear. Consequently the counts are conservative confirmed observations, not estimates of prevalence. A 0 means zero confirmed blanket policies in the classifiable subset, not that every company permits access.

19 HTML/content responses and one oversized response remain ambiguous. 95 inaccessible responses and 19 missing files are separate categories. No recovery attempt impersonated an AI crawler. The first sandbox DNS-only run is retained separately and excluded. Unit tests cover specific-vs-wildcard priority, repeated groups, training/search separation, comments, empty rules, exceptions, root-only rules and inline malformed syntax.

Reproduce: node --test scripts/ai-crawler-rules.test.mjs; node scripts/analyze-ai-crawler-audit.mjs. Collection is resumable with node scripts/run-ai-crawler-audit.mjs. Always rerun the analyzer after collection.