In 2026, the most common reason brands are not cited by AI assistants is not a lack of content, but a technical eligibility gate. If an AI crawler cannot access your site, the model cannot quote you.

The Eligibility Gate

Google's official Generative AI Optimization guidance (May 2026) identifies crawlability as the #1 technical priority. This is binary: you are either eligible for citation or you are invisible.

The Major AI Crawlers

To be cited, your server must allow requests from the following user-agents:

  • GPTBot / OAI-SearchBot (OpenAI/ChatGPT)
  • ClaudeBot (Anthropic/Claude)
  • PerplexityBot (Perplexity)
  • Google-Extended (Google Gemini)

Common Blocking Points

1. WAF & Bot Protection

Many modern hosting platforms (Vercel, Cloudflare) have "Bot Protection" enabled by default. While this stops malicious scrapers, it often inadvertently blocks AI crawlers with a 403 Forbidden error. A 403 response is a signal to the AI that the source is unreliable or restricted.

2. Robots.txt Restrictions

Legacy SEO practices often involved blocking certain crawlers to save crawl budget. In the GEO era, a Disallow: / for GPTBot is a direct instruction to OpenAI to never recommend your brand.

3. Soft-404s and JavaScript Rendering

AI crawlers prefer clean, semantic HTML. Sites that rely heavily on client-side JavaScript to render core facts often return "empty" shells to crawlers. If the facts aren't in the initial HTML payload, they don't exist to the AI retriever.

The CiteWorks Standard

We audit your site's bot-readiness by simulating requests from every major AI user-agent, and make sure your key facts are served in clean, server-rendered HTML that crawlers capture on the first pass.