How AI Search Handles Dynamic JavaScript Websites

Can AI Search Read a JavaScript-Heavy Website?

Sometimes. Google's crawler executes JavaScript, and systems built on its infrastructure benefit. But a widely cited analysis found major AI crawlers fetching HTML without executing scripts, so content that only appears after JavaScript runs may be invisible to them. The safe approach is to put your important content in the initial HTML.

The analysis in question, from Vercel and the analytics firm MERJ, examined crawler traffic on Vercel's network using data from late 2024. It found that crawlers from OpenAI, Anthropic and others did not render JavaScript, while Google's Gemini, which relies on Googlebot's infrastructure, and Apple's crawler did. Products and crawlers change, and that data is now old, so use it as a reason to test, not as a permanent rule.

Raw HTML Versus the Rendered Page

When a browser loads a page, it downloads HTML, then fetches scripts, runs them and updates the page. The final result is called the rendered DOM. A crawler that only reads the first response sees the raw HTML before any of that happens.

For a traditional server-rendered page, the raw HTML already contains the article text, headings, links and product details. For a client-side rendered single-page app, the raw HTML may be little more than an empty container and a script tag. The visible content is assembled in the browser from data fetched by the script.

Google handles this in two waves. It fetches and indexes the HTML first, and queues the page for rendering with an up-to-date Chromium-based service later. That works, but rendering can be delayed, and pages that depend on it are more fragile. Systems that skip rendering entirely have no second wave at all.

What Different Visitors Do

Read the table with care. It summarizes one public analysis plus general knowledge, the products change, and user-triggered fetchers can behave differently from indexing crawlers. The reliable way to know is to test your site, as described below.

Where JavaScript Sites Lose Content

Some patterns make content invisible to crawlers that do not render.

• Client-side rendered apps. Pages built entirely in the browser from API data.
• Content loaded after user actions. Tabs, accordions and "load more" buttons that fetch text only when clicked.
• Lazy-loaded sections. Content that appears only on scroll and never exists in the HTML otherwise.
• Prices and stock injected by script. Product data added after load, so a crawler sees a page with no price.
• Client-side routing without real links. Navigation built on click handlers rather than anchor tags with href.
• Metadata set in the browser. Titles, canonical tags and structured data added by scripts instead of the initial response.
• Wrong status codes. Single-page apps that return 200 for pages that do not exist, so crawlers cannot tell a missing page from a real one.
• Content behind interstitials. Cookie walls or overlays that block content until dismissed.

None of these is unusual. Modern frameworks make them easy to create, which is why the fix belongs in your rendering strategy rather than in patching individual pages.

Put the Important Content in the First Response

You have four main options, listed from sturdiest to least.

1. Static generation. Build pages to HTML at deploy time. Frameworks such as Astro, Next.js, Nuxt and SvelteKit support this. It is fast, cheap to serve and ideal for content that does not change every minute.
2. Server-side rendering. Generate the HTML on the server for each request, then let the browser take over. It suits pages that change often, at the cost of more server work.
3. Incremental or on-demand regeneration. Build static pages and refresh them when data changes, a middle ground offered by several frameworks.
4. Pre-rendering for bots. Serve pre-rendered HTML to crawlers and the app to users. Google describes this dynamic rendering as a workaround, not a long-term solution, and it adds maintenance and risk. Prefer options one to three.

Whichever you choose, aim for a simple rule: the main text, headings, prices, specifications, internal links, title, canonical tag and structured data should be in the HTML the server sends. JavaScript should enhance the page, not create it. That principle, often called progressive enhancement, also helps accessibility, because assistive technology and agents both benefit from content that exists without waiting for scripts.

How to See What a Crawler Sees

You can check in a few minutes.

1. Fetch the raw HTML. Run curl -s https://www.example.com/page and search the output for a distinctive sentence from the page. If it is missing, non-rendering crawlers will not see it.
2. Disable JavaScript. In Chrome DevTools, open the command menu, choose "Disable JavaScript" and reload. Whatever disappears is script-dependent.
3. View source, not inspect. "View page source" shows the raw response. "Inspect" shows the rendered DOM. Compare them.
4. Use Search Console. The URL Inspection tool shows Googlebot's rendered HTML and screenshot, and highlights blocked resources.
5. Check your logs. See whether AI crawlers are fetching your pages, what status codes they receive and whether they request your JavaScript files.
6. Test the crawlers' user agents. Fetch the page with a bot user agent to confirm nothing serves them an error.

curl -s -A "Mozilla/5.0 (compatible; test)" https://www.example.com/page | grep -i "your key sentence"

If the grep finds nothing, that content depends on scripts.

Smaller Rendering Details

Do not block your own resources. Google needs access to your JavaScript and CSS to render pages. Blocking them in robots.txt can leave Googlebot with a broken view.

Use real links. Navigation should use <a href> elements, not click handlers, so crawlers can discover pages.

Give each view a real URL. Avoid fragment-only routing for content pages. Use paths that return unique HTML.

Return correct status codes. Missing pages should return 404 from the server, not a 200 with an error message in the page.

Keep structured data server-side. JSON-LD added by scripts may be missed. Put it in the initial HTML.

Mind performance. Heavy hydration slows pages, and slow pages get crawled less effectively. A speed-testing tool such as the one in the video above shows where time goes.

What Common Setups Mean in Practice

The framework name matters less than the rendering mode you use with it.

The middle rows show why a partial approach can work: article text and prices in the HTML, interactive extras loaded later. The trap is putting something important in a widget. If a review count, a stock status or an FAQ only exists inside a script-loaded component, treat it as invisible to some visitors.

Agents That Browse

A separate group of visitors executes your JavaScript happily: agentic browsers that act for a user. They see the rendered page, run your scripts and interact with your controls. For them the concerns are different, and they overlap with usability. Stable layouts, labeled controls and predictable flows help them. We describe how to serve both groups in designing a website for humans and autonomous browsers, and the analytics side in separating human visitors from AI agent traffic.

A Quick Audit for a JavaScript Site

• The main content of each page appears in the raw HTML.
• Titles, meta descriptions, canonical tags and structured data are in the initial response.
• Internal navigation uses real anchor links.
• Missing pages return proper 404 status codes.
• Prices, stock and key specifications are present without scripts.
• JavaScript and CSS files are not blocked in robots.txt.
• Search Console's URL Inspection shows the content you expect.
• Logs show AI and search crawlers receiving successful responses.

For the broader question of what these systems can understand about a whole site, see can AI search engines understand entire websites.

Frequently Asked Questions