‹ all guides

When AI clients cannot read rendered pages

Sources checked 2026-09-15

Some clients read the initial HTTP response without running a browser. Compare that response with the rendered page before choosing server rendering or a Markdown representation.

Many sites render their content with JavaScript in the browser. A person's browser waits a moment and the page fills in. A client that reads only the raw HTTP response sees the page before that script runs, so it reads a loading shell instead of the content.

Client behaviour varies. Some AI clients run a full browser and execute JavaScript before reading a page, so they see the same content a person sees. Others fetch the URL directly and read only the response body, without executing any script. A site cannot assume which behaviour applies without checking, and the practical fix is the same either way: make sure the finished content is present in the first response for clients that need it.

On this page

What the first response contains

For a JavaScript-rendered page, the first HTTP response usually contains an HTML shell: a document structure, script tags, and little visible text. The content a person eventually sees is built in the browser after those scripts run. A client that stops at the raw response reads the shell, not the content, and has no way to know that more is coming.

Options

Prerendering renders the page on the server or at the edge and returns finished HTML in the first response, so any client reads the content immediately, without running scripts. A separate option is to serve a Markdown representation of the page on request, which skips the rendering question for clients that ask for it and costs a fraction of the tokens a full HTML page would. These options are not mutually exclusive. A site can keep its interactive build for people and serve prerendered or Markdown content by request, deciding on the incoming request rather than rebuilding the whole site. On turva.dev that decision lives in a Cloudflare Worker that reads the request's Accept header and returns the matching form.

Content parity

Whichever form a client is served, technical scan and manual review both check the same requirement: the content in that response must match what a person sees on the rendered page. A prerendered response that leaves out sections, or a Markdown twin that drifts from the live page, fails that check even if the response itself loads instantly. Content parity is a maintenance commitment, not a one time build step.

How to verify

Fetch the URL with a plain HTTP request and read the response body directly, then load the same URL in a browser and compare what each one contains. A gap between the two is the failure this guide describes. Checking both forms, rather than only the rendered page, is the only way to know what a non browser client actually receives.

Frequently asked

Why do AI agents see empty pages?

Some AI clients read the initial HTTP response without running a browser, so if a page relies on JavaScript to fill in its content, that client reads the loading shell rather than the finished content. Other AI clients run a full browser and render the page the way a person's browser does, so the outcome depends on the client.

How do you fix empty pages for AI clients?

Serve the real content in the first response for clients that need it, either through prerendering at the server or edge, or by serving a Markdown representation of the page on request. A Markdown response skips the rendering question and costs a fraction of the tokens.

Is prerendering the only fix for empty pages?

No. Serving a Markdown version of the page on request also works, and it skips rendering and costs a fraction of the tokens. Either approach only helps once the finished content is actually present in the response the client receives.

Sources