The Assistants Are the Audience Now

For most of a year I read my own traffic wrong. I watched human visitors, search clicks, and session depth, and I treated the bot rows in the logs as noise to filter out. Then I actually added them up. On a representative day my site served about 301 human visitors and roughly 19,800 requests from AI agents. The agents were not scraping me for training. The largest single source, by a wide margin, was ChatGPT fetching my pages live to answer someone’s question in that moment. My real audience had quietly stopped visiting. It was reading me through an assistant, and the assistant was reading me about sixty-six times more often than the humans it was answering.

TL;DR

• On a representative recent day my site drew ~301 human visitors and ~19,800 AI-agent requests. The 7-day average agent volume is ~19,850/day against a 28-day human average of ~301/day, a ratio near 66 to 1.¹
• Most of that agent traffic is not training crawlers. It is user-directed live retrieval: ChatGPT-User at ~13,100 requests/day and Claude-User at ~2,300, agents that fetch a page only because a human just asked their assistant something.¹²³
• The vendors’ own docs draw the line explicitly. OpenAI: “ChatGPT-User is not used for crawling the web in an automatic fashion.” Anthropic: Claude-User fetches “when individuals ask questions to Claude.” Cloudflare formalizes it as a distinct Agent behavior, separate from Training and Search.²³⁴
• The training crawlers everyone worries about were a rounding error by comparison: GPTBot at 55 requests and ClaudeBot at 212 on the same day.¹²³
• The practical shift: your content is increasingly consumed as answer-substrate. The reader is a human who never lands on your page, and the thing that actually reads you is an assistant retrieving you at question time. That changes what you measure, what you write, and what “traffic” even means.

The Number That Reframed Everything

The site is a personal one: technical guides and essays, ten locales, a few hundred human visitors a day. Respectable, not viral. For months my dashboard told a tidy story about search clicks and session length, and the AI-agent rows sat in a corner I mentally labeled “crawlers, ignore.”

The labeling was the mistake. When I finally pulled the Cloudflare edge logs into the same view as the human analytics, the proportion was not close. Humans: about 301 a day on a 28-day average. AI agents: about 19,850 requests a day on a 7-day average, and 19,785 on the single representative day I broke down by source.¹ Both windows are stable, so the ratio does not depend on which I pick: 19,785 against 301 is 65.7 to one, and the smoother 7-day average lands at 65.9. Call it sixty-six to one.

A fair objection lands immediately, and I want to concede it before it undermines anything else: those units are not identical. The 301 is unique human visitors. The 19,800 is agent requests, and a single human visit also spans several page requests. So this is not “sixty-six times more readers.” It is closer to “the assistants request my pages about sixty-six times more often than humans arrive to read them.” The honest comparison still points the same direction, because the interesting part is not the exact multiple. It is what the agent requests turned out to be.

Not the Crawlers You Were Worried About

The default assumption, mine included, is that a wall of AI-agent traffic means training scrapers hoovering your content into the next foundation model. That is the fear that launched a hundred robots.txt edits. It is not what my logs show.

Here is the same representative day, broken down by user-agent:¹

Read the top two rows again. ChatGPT-User and Claude-User together account for more than fifteen thousand of the day’s roughly twenty thousand agent requests. Neither is a training crawler, and the distinction is not my interpretation. It is documented by the vendors in plain language.

OpenAI’s bot documentation says OpenAI uses ChatGPT-User “for certain user actions in ChatGPT and Custom GPTs,” that “when users ask ChatGPT or a CustomGPT a question, it may visit a web page,” and then states flatly that “ChatGPT-User is not used for crawling the web in an automatic fashion.”² The bulk training crawler is a separate agent, GPTBot, described as crawling “content that may be used in training our generative AI foundation models.”² On my representative day GPTBot made fifty-five requests. ChatGPT-User made thirteen thousand.

Anthropic draws the identical line. Claude-User “supports Claude AI users. When individuals ask questions to Claude, it may access websites using a Claude-User agent.” ClaudeBot, the training crawler, “helps enhance the utility and safety of our generative AI models by collecting web content that could potentially contribute to their training.”³ Same shape: the user-directed agent dwarfs the training crawler, 2,274 to 212.

Cloudflare, which sits in front of a large slice of the web and has no incentive to flatter any single vendor, formalizes exactly this split. Its verified-bot taxonomy classifies AI bots by behavior into distinct categories: Agent, “user-directed agents visiting a page on behalf of a human”; Search, “crawling to build search indexes or RAG databases”; and Training, “crawling to train or fine-tune models.”⁴ The operative discriminator is human direction. A training or search bot crawls autonomously to build a persistent dataset. An Agent fetch is fired per request, by a human interaction, to answer one specific question live. My traffic is overwhelmingly the third thing.

What a Live Fetch Actually Means

Sit with the mechanics, because they change the mental model completely. When ChatGPT-User hits my page, the sequence that produced it looks like this: a person opened an assistant, typed a question, the assistant decided my page was worth reading to answer, it fetched the page in real time, extracted what it needed, and composed a reply. The person got their answer. They may never have seen my name, my layout, my other posts, or the small note at the bottom recommending one of my apps.

That is a reader. It is simply a reader I never meet. The assistant is a courier, and my analytics only ever saw the courier’s van, thirteen thousand trips a day, and called it noise.

That inversion is the quiet story under the noise about AI and content. The training-crawler debate is about whether models ingest your work once, at some point, into weights. That debate matters, but it is the wrong thing to watch if you want to understand your live audience. The live audience arrives through Agent fetches, continuously, each one fired by a human interaction happening right now. Every ChatGPT-User fetch in my logs traces back to a person who asked their assistant something my page could answer, even if a single question triggers more than one fetch and some fetches never surface to anyone. The scale, fifteen thousand fetches a day against three hundred human visits, says the people my content actually reaches are mostly on the far side of an assistant.

What This Changes About Measuring a Site

Once you accept that the assistants are a real audience, your instrument panel is suddenly missing its most important gauge. Standard analytics is built on the assumption that reading happens on your page: sessions, scroll depth, time on site, the conversion element below the fold. None of that fires when a human reads you through Claude. The assistant scrolls nothing, converts nothing, and bounces on every visit by definition. If you judge your content by on-page engagement alone, your fastest-growing audience is statistically invisible.

Three adjustments follow, and I have started making all three.

First, treat AI-agent request logs as an audience metric, not a security metric. I now track ChatGPT-User and Claude-User volume per page the same way I track human uniques, because that count is the closest proxy I have for “how often is an assistant using this page to answer someone.” The pages agents fetch most are not always the pages humans click most, and the gap is a content signal I was throwing away.

Second, stop optimizing solely for the on-page moment. A page written to be skimmed by a human who landed via search is not the same as a page written to be extracted cleanly by an assistant answering a question. The second job rewards a clear, self-contained answer near the top, unambiguous claims, and structure a retriever can lift without the surrounding chrome. It is why I put a direct answer block at the head of every post. That block is for the courier as much as the reader.

Third, accept that attribution gets harder and measure the shadow it casts instead. I cannot see the human behind a Claude-User fetch. What I can see is a second-order signal: humans who arrive on my site already referred by an assistant, the small stream of people who read the answer, wanted the source, and clicked through. It is a trickle next to the fetch volume, nine a day against fifteen thousand, but it is the visible tip of the invisible audience, and its trend is the honest scoreboard for whether being good answer-substrate eventually sends humans back to the source.

The Strategy Underneath

There is a temptation to read all this as doom for independent sites: the assistants strip-mine your content, answer for you, and keep the human. Sometimes that is exactly what happens. But the framing is too flat, because it ignores what the fetch is actually selecting for.

An assistant fetches your page because, at question time, it judged your page the best available source for that specific query. That is not the old search game of ranking for a keyword. It is being the thing an answer engine reaches for when it needs to be correct. The currency is not backlinks or keyword density; it is being demonstrably, retrievably right about something enough people ask. A page that is accurate, specific, and current gets fetched. A page of thin restated commodity content does not, because the model already knows the commodity part and only reaches out when it needs something it does not reliably contain.

So the incentive the Agent traffic creates is, for once, aligned with making genuinely good pages. Not pages engineered for a ranking algorithm, but pages worth retrieving to answer a real question. The measurable reward has moved from “did a human land and scroll” to “did an assistant judge this the best source and read it to a human.” I would rather compete on the second.

The Position

Treat the assistants as your primary audience, because on the numbers they already are, and build for the fetch as much as the visit. The human on your page is now the minority reader, and often the one who cared enough to come find the source after the assistant answered. The majority reader is the assistant itself, retrieving you live whenever a human question calls for it, and never touching your analytics. That is not a crawler problem to block. It is a distribution channel to understand, and right now it is the largest one most content sites have.

The site of the next few years is measured in two numbers, not one. There is the traffic you can see, humans on the page, and there is the traffic you have to infer, assistants reading you to answer humans you will never meet. Mine run about sixty-six to one in favor of the readers I cannot see. I suspect yours are closer to that than your dashboard is telling you, because the rows are sitting in the same corner where I left mine, labeled noise.

Key Takeaways

• Count AI-agent requests as audience, not noise. On my site they outnumber human page-arrivals roughly 66 to 1, and the composition matters more than the multiple.¹
• Most agent traffic is live, user-directed retrieval, not training. ChatGPT-User and Claude-User fetch pages because a human just asked their assistant something; the vendors document this and Cloudflare classifies it as a distinct Agent behavior.²³⁴
• Training crawlers are a small fraction. GPTBot and ClaudeBot combined were under 300 requests on a day when user-directed agents made over 15,000.¹²³
• On-page analytics misses your fastest-growing audience. Assistants scroll nothing and convert nothing, so engagement metrics render the Agent audience invisible; track per-page agent fetch volume as a proxy instead.
• Write for retrieval. Lead with a clear, self-contained answer; be specific, accurate, and current. An answer engine fetches the page it judges the best source at question time, which rewards being genuinely right over being keyword-optimized.

FAQ

What is the difference between ChatGPT-User and GPTBot?

GPTBot is OpenAI’s training crawler, which collects content that “may be used in training” foundation models. ChatGPT-User is a user-directed agent that fetches a page when a person asks ChatGPT a question; OpenAI states it “is not used for crawling the web in an automatic fashion.”² In my logs the two differ by orders of magnitude: ChatGPT-User made 13,128 requests on a day GPTBot made 55.¹

Are AI assistants reading my website in real time?

If you serve content that answers common questions, almost certainly yes. Agents like ChatGPT-User, Claude-User, and Perplexity-User fetch pages live when a human asks the assistant something the page can answer.²³ These are distinct from training crawlers and from search indexers, and on my site they are the dominant form of agent traffic.

How is this different from SEO?

Classic SEO optimizes to rank in a results page a human then clicks. Answer-engine retrieval optimizes to be the source an assistant fetches and reads to compose an answer, often without the human visiting at all. The reward shifts from ranking signals toward being accurate, specific, and current enough that a model reaches for your page when it needs to be correct.

Should I block AI agents in robots.txt?

That is a real choice, but decide it per behavior, not in a lump. Blocking training crawlers (GPTBot, ClaudeBot) affects whether your content trains future models. Blocking user-directed agents (ChatGPT-User, Claude-User) affects whether assistants can answer humans using your page live, which for many sites is now the largest audience. One caveat worth knowing: user-directed agents have a weaker record of honoring robots.txt than the training crawlers do, so a robots.txt block is a clearer lever on training than on live fetches, and an edge rule may be the more reliable control there. Cloudflare’s taxonomy separates these categories precisely so you can treat them differently.⁴

How do I measure an audience I cannot see on my own site?

You cannot directly attribute the human behind a live fetch, so measure two things instead: per-page AI-agent request volume from your edge logs as a proxy for how often assistants use each page, and the smaller stream of human visitors who arrive already referred by an assistant, as the visible shadow of the invisible audience.

Sources