The Robots Are Taking Exams in My Search Console

Over a recent 28-day window, my site collected 3.78 million impressions in Google Search Console, and 91% of them came from queries no human typed.¹ The tell was a blog post about Claude Code hooks that looked like a breakout: 30,000 impressions in its first two weeks, at a click-through rate of 0.14%. Every SEO playbook says a page with big impressions and a dead CTR needs a title rescue. Then I pulled the actual queries, and the top ones were multiple-choice exam questions, pasted into Google verbatim, six slightly different ways: “a developer wants to prevent claude from reading sensitive .env files. which type of hook…” Nobody clicks, because nobody is there. The searcher is a certification quiz checking its own answer key.

TL;DR

• A 13-day-old post drew 30,105 impressions at 0.14% CTR, the classic profile of a broken title. The queries behind it were verbatim exam questions and SDK jargon strings; genuine human queries numbered in the single digits. The title was fine. The audience was synthetic.⁴
• Site-wide, 3.78M impressions in 28 days shrink to 340K (9%) once you exclude queries containing a double quote or an underscore, a cheap but effective machine-query signature. Filtered CTR nearly triples, from 0.42% to 1.10%, and average position improves from 11.2 to 7.7.¹
• The machine queries split into three species with completely different value: exam takers (natural-language test questions, zero clicks ever), error pasters (verbatim error strings, 3-10% CTR, the highest-intent visitors I have), and retrieval sweeps (quoted phrases and snake_case identifiers at bulk scale, ~0.35% CTR).
• No regex separates humans from machines cleanly. The exam questions read as human and are not; the error pasters look robotic and are people in trouble. Query shape lies; click behavior tells the truth.
• The practical shift: never optimize a title, or panic about a “declining” page, before splitting its queries into human and machine. Impressions stopped being a vanity metric and became a mixed one.

The post that almost got a title rescue

The post in question, an explainer on Claude Code hooks, went live on July 1. By July 13 it had accumulated 24,377 impressions in its first week and held that pace in its second. My dashboard flagged it the way dashboards do: enormous exposure, 43 total clicks, 0.14% CTR at an average position of 6.6.⁴

I have a proven playbook for exactly that profile. A guide on this site once sat at 1.4 million impressions with a 0.28% CTR until a title rewrite, so the reflex was strong: rewrite the title, watch the CTR recover, count the wins.

The reflex survived until the query report loaded. The top query, 191 impressions on its biggest variant and 254 across the family, was a complete exam question, options and all, in the flat register of a certification bank. Six near-duplicate phrasings of the same question hit three different hooks posts on my site, plus the German, French, and Polish translations.⁴ Zero clicks across all of them. Second place was “anthropic agent sdk hitl lifecycle hooks pretooluse,” which no human has ever typed with their fingers. The genuinely human queries, the kind with typos and hope in them, added up to single digits.

A title rewrite optimizes the one element of a search result that only humans read. The audience for the impression mountain does not read titles. It reads answer snippets, or nothing at all. I would have spent the change on readers who were never there, and then drawn exactly the wrong conclusion when the CTR failed to move.

Three species of machine query

Once I knew what to look for, I pulled the machine-signature queries for the whole site. They are not one phenomenon. They are three, and confusing them costs real money.

The exam takers are the strangest cohort. The queries are grammatical English, so they pass every naive human filter, yet they behave like no human searcher: identical question families, repeated across weeks, distributed across locales, never a single click. My best guess at provenance, and I want to be plain that it is a guess: AI-generated practice tests and certification-prep tools that validate their own answer keys against live search results, plus some students pasting quiz questions whole. Google counts each one as an impression on my page, because my page genuinely appeared in the results.² The impression is real. The reader is not.

The error pasters are the opposite, and they are the reason a crude “delete all machine-looking queries” filter destroys value. A pasted error like xcrun: error: unable to find utility "mcpbridge" trips the filter on its quoted utility name, and it clicks through at 6.25% from position 2.7. “failure in void _uiapplicationevaluateruntimeissuefornoscenelifecycleadoption” carries its underscore into a 9.68% CTR. Config identifiers behave the same way: “codex trust_level” at 8.70%, “hermes custom_providers” at 4.55%.¹ These beat my site-wide human CTR by a factor of three to nine. The query text is machine-authored, because it was written by a compiler or a config schema, but the person pasting it has a problem right now and will read anything that matches. The highest-intent visitors on my site arrive wearing a robot costume.

The retrieval sweeps are the bulk of the 3.4 million. Quoted exact strings, API surface names, documentation fragments. My earlier post, The Assistants Are the Audience Now, traced the delivery side of the same phenomenon at the network edge: roughly 17,000 AI-agent requests a day against about 300 human visitors, most of it live user-directed fetching rather than training crawls.⁵ The Search Console data completes the loop from the demand side. The assistants do not only fetch my pages; they also issue Google searches on the way, and Google dutifully logs each one as an impression, position and all.

How much of Search Console is still human

The filter I use is one regular expression. In the Search Analytics API, exclude queries matching ["_], any query containing a double quote or an underscore:

{
  "dimensionFilterGroups": [{
    "filters": [{
      "dimension": "query",
      "operator": "excludingRegex",
      "expression": "[\"_]"
    }]
  }]
}

Quoted strings signal exact-match verification, which humans rarely bother with. Underscores come from identifiers, and identifiers come from code, not from thumbs.³ Running my last 28 days through it:

The filter is a heuristic, and it fails in both directions. The exam questions pass it, since they contain neither quotes nor underscores, so the 91% machine share is a floor, not a ceiling. And the error pasters fail it while being the most human traffic I have. I treat the filtered slice as “mostly human, understated” and the remainder as “mostly machine, plus my best debugging visitors,” and I check the query lists by eye before any decision that costs effort.

The distortion goes beyond CTR. The machine queries drag my apparent average position from 7.7 to 11.2, because retrieval sweeps probe long-tail phrasings where my pages rank anywhere. A dashboard that mixes the slices reports a site slipping in the rankings while the human experience holds steady.

The guide that looked like it was dying

The same split rescued my read on a bigger page. My Hermes guide shows 763,439 impressions over the last 28 days with 569 clicks: a 0.075% CTR at a blended average position of 13.5. Taken at face value, the page has collapsed. Its tracked human-only baseline is a 1.53% CTR at position 8.5, and the latest human-filtered week still holds position 8.4.¹

Both readings describe the same page in the same month. The blended number says the guide is failing; the filtered number says humans find it exactly where they always have, while an ocean of machine retrieval rose around it. Composition shift, not ranking loss. If I had trusted the blended view, the natural moves would have been a rewrite, a new title, maybe a URL migration: expensive surgery on a healthy patient.

I now assume any established page can show the same symptom, and I would bet most Search Console “performance declines” in AI-heavy niches decompose the same way. The denominator changed species.

What I actually changed

Split before deciding. Every CTR judgment, title experiment, and decline investigation on my site now starts with the human-filter pass and a manual scan of the top 20 queries. Five minutes of reading query text has repeatedly reversed the conclusion the aggregate suggested.

Feed the error pasters deliberately. Exact error strings are the best keywords I never chose. The pages that match them sit at positions 2 through 4 and convert at debugging-emergency rates. Error messages, config keys, and CLI output now get verbatim headings in my technical posts, because someone pasting that string into Google is the single most rescuable visitor on the internet.

Stop courting the exam takers. Zero clicks means zero, forever. A question bank checking its answer key cannot be converted by any title, snippet, or schema markup. The only rational response is to ignore that volume when judging a page, and maybe to enjoy that a certification exam somewhere apparently keys its answers against my blog.

Report human numbers or report nothing. My own dashboards now chart the filtered slice, annotate the blended one, and keep the daily snapshots in git so composition changes stay visible over time. Impressions still matter, but only after you know who is impressing whom.

The deeper shift matches where the last post ended. The assistants read my pages 17,000 times a day at the edge, and now I can watch them queue up at Google’s front door on the way in. Search Console has quietly become a log of machine curiosity with a thin human stream running through it. The stream is the part that pays attention. Measure the stream.