How to Design a Website for Both Humans and Autonomous Browsers

What Does It Take to Serve Both Humans and Autonomous Browsers?

Build one site with the content in server-rendered HTML, semantic and accessible markup, predictable interactions, clear feedback and safe handling of sensitive actions, then test it with people, with a keyboard and with an agent. The two audiences want mostly the same things: clarity, stability and honesty.

An autonomous browser is a browser, or an agent inside one, that navigates, reads and acts for a person. Examples include the agentic features of AI browsers and assistant extensions. Product names churn quickly. ChatGPT Atlas, for instance, was shut down on August 9, 2026, with its agent features moving elsewhere, so design to principles, not to any single product. For the current product picture see our guide to AI browser agents.

The Design Checklist

1. Put the content in the HTML. Main text, prices, links and structured data should be in the first response. Many AI crawlers do not run scripts, as explained in how AI search handles JavaScript websites.

2. Use semantic, accessible markup. Real buttons, links, headings and landmarks give both assistive technology and agents something to work with. See why accessibility matters for AI agents.

3. Label everything. Every control needs a visible and accessible name. Avoid "click here" and icon-only buttons with no text.

4. Keep interactions predictable. Standard patterns, native controls and consistent placement reduce guesswork.

5. Give explicit feedback. Announce loading, success and errors in text, near the action, with roles that assistive tools can read.

6. Avoid interruptions. Pop-ups, auto-advancing carousels and sticky overlays block agents and irritate people. Make any overlay easy to dismiss.

7. Stabilize layouts. Elements that shift as the page loads make click targets move under a cursor.

8. Handle sensitive actions safely. Require confirmation and verification for payments and irreversible changes. See AI agent permissions for websites.

9. Offer actions where pages are awkward. For repeated tasks, an API or tool layer is more reliable than click-throughs. See agent-friendly APIs.

10. Be honest. Show the same content to every visitor. Do not hide text for agents or cloak content by user agent.

Why These Principles Matter

Research and practitioner reports support the direction. A CHI 2026 study from UC Berkeley and the University of Michigan, summarized by Search Engine Journal, tested Anthropic's Claude Sonnet 4.5 on 60 web tasks. Success was 78.33 percent on standard pages, 41.67 percent with keyboard-only interaction and 28.33 percent with a magnified viewport. The study used one model and a limited task set, so do not treat the figures as universal, but the pattern is clear: the conditions that trouble people with disabilities also trouble agents.

Traffic trends point the same way. HUMAN Security data reported by PPC Land put Perplexity's Comet at roughly 47 percent of measured agentic traffic in May 2026, a reminder that browsing agents already account for real sessions on real sites. Vendors keep changing their products, so treat any single figure as a snapshot.

Make Pages Easy to Read in Two Ways

People scan. Agents parse. Both like the same page shape.

• Answer first. Put the direct answer or the main fact near the top of the page.
• Use descriptive headings. Headings that say what the section contains, not clever teasers.
• Prefer text over images of text. Prices, specs and policies belong in text a reader can select and a parser can read.
• Use tables for comparisons. Real <table> markup with header cells communicates relationships that images cannot.
• Add structured data. Product, Article, Organization and policy markup that matches the visible page.
• Write clear link text. "Compare the three plans" beats "Learn more."

Make Actions Easy to Complete

Most failures happen at forms, menus and checkouts. Apply the same care everywhere.

1. Use native form controls with labels, types and autocomplete values. See AI-friendly forms explained.
2. Provide a keyboard path for every interaction, with visible focus.
3. Keep multi-step flows stable with progress shown, saved state and a way back.
4. Preserve input on errors and explain how to fix each problem.
5. Allow guest actions where you safely can, so a task does not stall at a login wall.
6. Design the handoff. If a task needs a person, for example approving a payment, make that step obvious and easy, and let the agent stop there.

For shopping in particular, our guide to building an agent-friendly checkout page walks through the details.

Keep the Site Safe From Misuse

An open, friendly site still needs protection. Combine welcome with control.

• Rate limit by client and by action, and respond with 429 and Retry-After. See AI crawler rate limiting explained.
• Use risk-based bot checks instead of challenging every visitor.
• Verify identity where possible, with published IP ranges or signed requests, and relax limits for verified clients.
• Segment your analytics so likely-automated sessions do not distort human metrics. See separating human visitors from AI agent traffic.
• Sanitize user-generated content. Reviews and comments can carry hidden instructions aimed at agents.

A Three-Way Test Routine

Test three ways and compare.

Do all three on your top journeys: finding information, contacting you, signing up and buying. Log every failure and fix the ones that appear in more than one test first, because they hurt everyone.

Plan for the Times an Agent Gets Stuck

Even a well-built site will lose some agent runs. Design the failure path as carefully as the success path. Make it possible to save progress, such as a cart, a draft form or a selection, so a person can pick up where the agent stopped. Offer a clear "continue on your phone" or "send me a link" option when a task needs human input, such as identity verification or payment approval. Show plain messages when something cannot be done, with a way to reach support.

Also think about what the agent reports back. A confirmation page with an order number and a summary in text lets an assistant tell its user exactly what happened, and a clear failure message lets it say so honestly. Those small details reduce the chance that a person believes a task was completed when it was not, which is the failure with the highest cost to trust.

A Thirty-Day Plan

Week one: baseline. Pick five key journeys. Run the three-way test. Check raw HTML for missing content. Review server logs for AI crawlers and agents.

Week two: structure. Fix headings, landmarks, link text and button names. Move critical content into the initial HTML. Add or repair structured data.

Week three: interaction. Repair forms, errors and overlays. Stabilize layouts. Test checkout and contact flows again.

Week four: control and measurement. Add rate limits and risk-based checks. Publish an agent policy page. Set up analytics segments. Decide whether one read-only API action is worth exposing.

Then repeat quarterly. Browsers, agents and standards change, and a short recurring check catches regressions before they cost you customers.

Make It Someone's Job

Agent-friendly design tends to fall between teams. Designers own layout, developers own markup, marketing owns content and security owns bot rules, and nobody owns the whole picture. Give one person or a small group responsibility for the combined result. Add a short agent-readiness check to your definition of done for new pages and features: does it work with the keyboard, is the content in the HTML, are controls labeled, and does the risky action need confirmation? A five-minute check at release is far cheaper than a redesign later.

What Not to Promise

Design cannot guarantee that agents choose your site, complete every task or behave well. Agents make mistakes, products change without notice and some visitors will misuse open features. The point is to remove avoidable obstacles and put sensible controls around risky actions. That is good practice whether or not a single agent ever visits, which is why the investment is low-regret.

Also keep an eye on emerging proposals such as WebMCP, which would let pages describe actions to in-browser agents. It is a draft and may change, so build good pages first and revisit it later. For a broader view of where the web is heading, read what the agentic web could look like by 2030 and how websites may change for AI agents.

Frequently Asked Questions