How to Separate Human Visitors From AI Agent Traffic in Website Analytics

Can You Really Tell Humans and AI Agents Apart?

Partly. Declared crawlers and fetchers can be identified with good accuracy from server logs, because they announce themselves and mostly skip analytics scripts. Agentic browsers, which run inside real browser engines on a person's behalf, cannot be separated cleanly in standard analytics and must be treated as a probabilistic segment.

That difference is why the answer to "how much of my traffic is AI?" depends on which kind you mean. Treat the three groups below as separate problems with separate tools.

The Three Kinds of AI Traffic

Operators describe these bots differently, and their behavior changes, so check the current documentation from OpenAI, Anthropic, Google and Perplexity for exact names. As one example of a shift, ChatGPT Atlas was shut down on August 9, 2026, and OpenAI moved its browsing agent capabilities into other ChatGPT surfaces, which is a reminder to review your lists regularly. We explain that change in what happened to ChatGPT Atlas.

Where to Find Declared Bots

Declared bots live in server logs and CDN analytics, not in GA4. Two habits help.

1. Classify by user agent, then verify. Match the user agent against a maintained list of known AI bots. For anything that matters, confirm the request really came from the operator: OpenAI publishes IP ranges in JSON files, Google's documentation describes reverse DNS checks and lists user-triggered agent IPs, and Anthropic's help pages note it does not publish IP ranges, so verification there relies on other signals.
2. Use your CDN's dashboard. Services such as Cloudflare label verified bots and provide AI-specific crawl reporting, which saves you writing your own log parser. If you already use one, start there.

Also remember that GA4 excludes traffic from known bots and spiders automatically. If your analytics total looks lower than your server request count, the gap is largely bots and non-browser requests, which is expected.

Note the robots.txt caveat. Crawlers such as GPTBot and ClaudeBot state that they respect robots.txt, but user-triggered agents behave differently: Google's documentation says its user-triggered fetchers generally ignore robots.txt because a person requested the fetch. Do not assume a robots rule will stop every kind of agent. Our guides on AI crawler rate limiting and agent permissions for websites cover enforcement.

The Hard Part: Agents That Look Like People

Agentic browsers are the reason this topic is difficult. They load your page in a normal browser engine, run your analytics script and can carry a real user's cookies and login. HUMAN Security data reported by PPC Land put Perplexity's Comet at roughly 47 percent of measured agentic traffic in May 2026, ahead of ChatGPT Atlas at about 20 percent before Atlas closed. Figures like these come from a security vendor's view of its own customers, so treat them as directional.

What can you do when the browser itself gives no label? Look for behavior and provenance.

• Speed and precision. Forms completed in a couple of seconds, perfect field order and no mouse or scroll activity before a click.
• Automation flags. Browsers driven by automation frameworks may expose signals such as navigator.webdriver, though a well-built agent can hide it.
• Identity signals. Some agents are starting to sign requests so sites can verify them. Cloudflare and Google have both experimented with this approach, but adoption is uneven.
• Session shape. Visits with many pages in a tight sequence and no idle time behave unlike readers.

None of these is proof. Treat sessions that trip several at once as "likely automated," not "confirmed."

A Practical Segmentation Plan

Aim for three labels in your reporting, not a perfect binary.

To implement it:

1. Log requests at the edge with the bot classification attached.
2. Add a lightweight client-side check that sends a custom GA4 event or dimension when automation signals fire, so you can filter conversion reports.
3. Compare conversion rate, form completion time and bounce behavior across segments.
4. Review the rules quarterly. Agent products change fast, and rules that worked in the spring can age out.

For the marketing side of the same problem, see AI search referral analytics, which covers the human visitors who arrive from answer engines.

What Site Owners Often Get Wrong

Assuming robots.txt is a gate. It is a request that well-behaved crawlers honor. It does not authenticate anyone, and user-triggered fetchers are treated differently by some operators. If a resource must be protected, use authentication or server-side rules.

Trusting the user agent alone. Anyone can send a header that says "GPTBot." Real enforcement checks IP ranges, reverse DNS or signatures, and falls back to rate limits when identity is unclear.

Blocking everything that looks automated. Agents that shop, book or fill forms for a customer can be a growing source of legitimate orders. Blanket blocks turn away paying users. A more careful approach is to challenge suspicious behavior, not a category of software.

Reporting one blended conversion rate. If likely-automated sessions are mixed into human numbers, both the human conversion rate and your test results become unreliable. Segment first, then calculate.

Forgetting that logs and analytics disagree by design. A gap between raw requests and GA4 sessions is normal. Treat it as information about your bot share, not as an error to fix.

When an Agent Is Acting for a Real Customer

Some automated visits deserve a warm welcome. A shopper who asks an assistant to compare products and add one to a cart is a genuine customer whose session happens to be driven by software. Sites that treat every automated visit as hostile will lose those sales.

The practical answer is to make legitimate agent behavior easy and abuse expensive. Keep essential flows accessible and predictable, as discussed in how to design a website for both humans and autonomous browsers. Require login or confirmation for sensitive actions. Apply sensible rate limits per account or per IP. And log agent-assisted orders separately when you can, so you can measure whether they convert, refund or complain differently from human-only orders.

What Classification Looks Like in Practice

A raw access log line has a timestamp, an IP address, the requested path, a status code and a user agent. Suppose you see many requests whose user agent contains a known AI bot name. The steps to classify them are simple and repeatable.

First, match the user agent against your list and label the request "claims to be" that bot. Second, check the source IP against the operator's published ranges, or run a reverse DNS lookup where the operator documents one, and upgrade the label to "verified" if it passes. Third, group by path to see what the bot is reading. A search bot reading article pages is expected. A bot hammering an internal search endpoint or a login page deserves a closer look.

Anything that claims to be a bot but fails verification goes into a "suspicious" bucket. That bucket often contains scrapers borrowing a respectable name, and it is the group most worth limiting. Keep the counts by label over time. A weekly chart of verified, claimed-but-unverified and unlabeled requests shows you whether machine traffic is growing and whether the unverified share is rising, which is a stronger signal than any single day.

Decide What You Want, Not Just What You Detect

Detection is only useful if you know the goal. An agent booking a table for a customer is a customer. A scraper hammering your search endpoint is a cost. Write down which behaviors you welcome, which you limit and which you stop, then apply rate limits, authentication and verification where needed. Our guide to designing for both humans and autonomous browsers covers the design side, and the guide to AI browser agents explains how the products work.

Frequently Asked Questions