Anthropic’s Claude is one of the four AI assistants most people query for answers. Unlike ChatGPT, Claude grounds its responses using content fetched directly by Anthropic — through a crawler called ClaudeBot. If your site is missing from Claude’s answers, the cause is almost always one of five things specific to how ClaudeBot sees the web.

The short answer

Claude cites pages that ClaudeBot can fetch, Anthropic’s grounding can extract, and Claude judges trustworthy. Each step is its own failure mode.

1. ClaudeBot can’t reach your site

The single most common reason a site doesn’t show up on Claude is that ClaudeBot is blocked. Check two places:

• robots.txt — look for User-agent: ClaudeBot followed by Disallow: /, or a sweeping Disallow under User-agent: *.
• Edge layer — Cloudflare, Fastly, Imperva, and others ship default "block AI bots" rules that 403 ClaudeBot even when robots.txt allows it.

Allow ClaudeBot explicitly. The standard rule:

User-agent: ClaudeBot
Allow: /

2. Your content is rendered client-side

ClaudeBot fetches HTML the way a curl request does — it doesn’t run a full headless browser the way Google’s rendering pipeline does. If your page only paints its content after a client-side JS bundle executes, Claude sees an empty shell.

The fix is server-rendering (or static generation) anything you want Claude to read. View the page source — not DevTools — to confirm the answer text is in the raw HTML.

3. Your page doesn’t look like an answer

Anthropic’s grounding favours pages that already shape themselves as answers. Practical rules of thumb that work well for Claude specifically:

• H1 = the question, in the user’s words.
• Lead paragraph = the direct answer, in 2-3 sentences, before any throat-clearing.
• Sub-questions as H2s with their own direct answers underneath.
• Plain prose beats keyword-stuffed copy. Claude outputs naturally clean writing and tends to cite sources that match that voice.

4. You lack the trust signals Claude weighs

Across the AI engines we’ve tested, Claude is the most sensitive to authorship and citation patterns. If your page has none of these, it’s heavily demoted:

• A real author byline linking to an author page.
• Visible publish and update dates, mirrored in JSON-LD as datePublished and dateModified.
• Outbound citations to primary sources (papers, official docs, named experts).
• An Organization JSON-LD block with sameAs links to your social profiles, so Claude has an entity to anchor citations to.

5. There’s no structured data to extract

JSON-LD isn’t just for Google. Claude’s grounding step can use it to confirm what a page is about, when it was written, and who wrote it. At minimum, add:

• Article schema on every post.
• Organization schema sitewide (usually in a layout template).
• FAQPage schema on any Q&A content.

Validate with Google’s Rich Results Test before shipping — if Google’s parser rejects your JSON-LD, other parsers will too.

What about llms.txt?

llms.txt is an emerging convention from llmstxt.org — a plain-text index of your most important content, designed for AI crawlers. Adoption is still uneven, but adding one costs minutes and is a strong intent signal. You can see ours at /llms.txt.

How to check if your site is the problem

The fastest way is to scan it. The form below checks ClaudeBot access, content structure, JSON-LD, and trust signals — same rules described above.

Related guides

• Why isn’t my website showing up on ChatGPT?
• Why isn’t my website showing up on Google AI Overviews?

Frequently asked questions

Ready to fix what’s blocking you? Create a free account or see pricing.