New researchTechnical SEO & AI Search · 2027Explore the framework

How to make your website work for AI agents

How AI agents read and use web pages, and what web.dev recommends so they can complete tasks: semantic HTML, labels, visible controls and stable layouts.

Part of the AI Search series. For the full picture, start with Technical SEO and AI search readiness: the complete audit checklist.

AI agents don’t just read pages; they try to use them: compare products, fill in forms, book appointments. When a site’s controls are unlabelled divs, or the layout jumps while the agent clicks, the task fails. Google’s guide to generative AI search points site owners to web.dev’s guide to agent-friendly websites for this.

How do AI agents see a page?

web.dev describes three channels:

Channel What the agent does
Screenshots “Takes a snapshot of the rendered page and uses a vision model to identify elements”
HTML “Analyzes the DOM and reads the HTML”, understanding “the logical hierarchy of the DOM tree”
Accessibility tree Reads “roles, names, and states of interactive elements”

What does web.dev recommend?

  • Real elements. “Use <button> and <a> tags over modified <div> and <span> elements.”
  • Labelled fields. “Add the for attribute on <label> tags to link them to inputs. This helps the AI agent understand the purpose of a field.”
  • Visible, sizeable controls. Interactive elements needed to complete an action should be “visible and larger than 8 square pixels.”
  • Clear affordances. “Set cursor: pointer in CSS, which is a strong signal for actionability.”
  • No hidden controls. “Avoid transparent overlays that hide interactive elements.”
  • A stable layout. “Agents that take screenshots will likely be confused if your website layout is constantly shifting.”

Most of this is accessibility best practice, which is why it helps people using screen readers just as much.

What can you check with a crawl?

A crawler can’t run an agent, but it can find the patterns agents struggle with. In Crawlens:

Agents also need to reach the site at all: user-triggered agents such as ChatGPT-User and Claude-User come from AI companies’ infrastructure, and bot protection can block them. See how to block AI crawlers for which agents do what. If people use your docs or API through AI agents, a short llms.txt file can point them to the most useful pages.

For the rest of AI search readiness, see the AI search readiness checklist.

Checklist

  • Buttons and links use <button> and <a>, not clickable divs
  • Every form field has a linked <label> or accessible name
  • Interactive elements visible, reasonably sized and not hidden behind overlays
  • Layout stable while loading and interacting (CLS 0.1 or less)
  • Key links and content present without JavaScript
  • Images have alt text
  • Bot protection doesn’t block the agents you want to serve

Frequently asked questions

How do AI agents read websites?

According to web.dev, through three channels: screenshots analysed by a vision model, the HTML and DOM structure, and the accessibility tree, which lists the roles, names and states of interactive elements.

Is making a site agent-friendly the same as making it accessible?

Largely. Semantic elements, labelled form fields, visible controls and stable layouts help screen readers and AI agents in the same way, because both rely on the accessibility tree and clear structure.

Does Google say agent-friendliness affects rankings?

No. Google's guide to generative AI search mentions agentic experiences as something to explore if relevant to your business, and points to web.dev's guide; it doesn't describe it as a ranking factor.

· Founder, Crawlens

Dien builds Crawlens, a desktop crawler for technical SEO audits, and writes about the checks it runs: crawling, indexing, rendering and Search Console data.

Audit your own site with Crawlens

Crawl, run 70 checks, and see them next to Search Console and Core Web Vitals data.

Download free for Windows