Can ChatGPT Read Your Website? Four Checks That Decide It

"Reading your site" means four different things

When someone asks whether ChatGPT can read their website, they usually mean one thing: is the bot blocked. That is the first check, not the whole question.

There are four separate gates, and a page has to pass all of them before an assistant can name you in an answer. The crawler has to fetch the page. The content has to exist in what arrives. A model has to be able to tell what the page is about. And there has to be something to ground on, so it can describe your business accurately.

The reason this matters is that all four failures look identical from your side. Nothing errors, nothing appears in your analytics, and no ranking drops. Traffic from AI assistants simply never starts. So you have to go and check.

Check 1: can the crawler fetch you at all

Open yourdomain.com/robots.txt and read it. You are looking for groups that name the OpenAI agents, which are GPTBot, OAI-SearchBot and ChatGPT-User. They do different jobs: one grounds and trains, one serves search results inside ChatGPT, and one fetches a page because a user asked about it right now.

Here is the part that catches people. Crawlers follow a precedence rule where a bot obeys the group that names it specifically, and only falls back to the wildcard User-agent: * group when no named group exists. A generous wildcard does not override a restrictive named group. This is exactly how a site ends up looking wide open while blocking OpenAI completely.

Reading that correctly across a dozen user-agents is tedious, which is why we built a tool that does it. The AI crawler access checker fetches your robots.txt, applies the real precedence rules, and reports allowed or blocked per bot.

Check 2: does your content exist without JavaScript

Googlebot renders JavaScript. Most AI crawlers fetch the HTML and work with what they get.

That difference is invisible until it costs you. If your prices, product descriptions or service copy are injected by a script after the page loads, a crawler may receive a shell with a navigation bar and nothing else. The page looks perfect to you and nearly empty to the model.

The two-second test: right-click and choose view page source, not inspect element. Inspect element shows you the page after JavaScript has run. View source shows you roughly what the crawler received. Search that source for a sentence you know is on the page. If it is not there, an assistant is not reading it either.

Check 3: can a model tell what the page is about

Assistants quote passages, not pages. So a page can be fetchable and fully rendered and still be useless to one.

What makes a passage quotable is that it stands alone. If a paragraph only makes sense after the two above it, a model cannot lift it into an answer without dragging context along, so it reaches for a source where somebody wrote a complete answer in one block. Your title and meta description do similar work at page level: they should say plainly who you are and what this page covers, not tease.

This is also where a lot of otherwise strong sites lose. The writing is good, the structure is a mess of headings that do not correspond to questions anyone asks.

Check 4: is there anything to ground on

The last gate is whether the model can describe your business accurately once it has decided to mention you.

Two things carry this. Schema markup in JSON-LD names your entities explicitly: this is an Organization, this is a Service, these are FAQs with real answers. Without it the model infers your business from prose, and inference is where it gets your name, your location or your offering wrong. An llms.txt file is a plain text summary at your domain root, in your own words, that a crawler can fetch at query time.

Neither one makes an assistant visit you. Both make it get you right once it does. If you do not have an llms.txt yet, our generator builds one in your browser in a few minutes.

Fix them in this order

If several checks fail, the order is not negotiable, because the earlier gates make the later ones irrelevant.

Rewriting your service pages while GPTBot is disallowed is the most common wasted effort we see. The pages get better and absolutely nothing changes, because the crawler still never arrives.

The fastest way to check all four

Doing this by hand takes an afternoon. Our AI visibility checker scores a site from 0 to 100 across nine concrete signals covering all four gates, including crawler access, structured data, llms.txt, title and meta, heading structure, canonical and HTTPS, sitemap, and entity and contact signals.

It is free, it needs no login, and it returns the specific gaps rather than a grade. It is also the same audit we run at the start of a Search Dominance engagement, which is the honest reason it exists: it is easier to show you the gaps than to describe them.