What Is llms.txt? Why Your Business Needs One
What Is llms.txt? Why Your Business Needs One (and Where robots.txt Fits)
An llms.txt file hands AI tools a short summary of your business in your own words: what you do, where you work, and which pages to read first. It sits beside robots.txt, the older file that decides which crawlers (the automated software AI tools send out to read websites) may read your site at all. Get both right and AI tools can read your site and know what matters on it. Checking both takes about two minutes.
Key Takeaways
- llms.txt tells AI tools what your business does. It gives them your summary and your most important pages, in your own words.
- OpenAI's crawlers are among the bots most likely to request it. Server-log research across 6,455 websites found GPTBot and OAI-SearchBot were the likeliest to ask for it.
- robots.txt is the file that can genuinely and accidentally lock AI out of your site, usually without anyone deciding to do it.
- Different AI bots do different jobs. Blocking one doesn't affect another, even from the same company.
- Both files take about two minutes to check, starting today.
If you've asked ChatGPT about your own company and got back a vague answer or a competitor's name, or your web developer mentioned llms.txt and you nodded along, this is written for you. Below: what each of the two files does, how to check yours from a browser in about two minutes, and which one to fix first. I've worked in search for more than 15 years, and many businesses lock AI out of their own websites through robots.txt without knowing it.
What is an llms.txt file?
An llms.txt file is a plain text file, written in a simple formatting style called markdown, that sits at the root of a website (the top level, as in yourdomain.com/llms.txt) and lists the site's most important pages in a short, organized format: a heading, a one-line description of the business, then a list of links grouped by what they cover. The idea is to hand an AI tool a clean summary instead of making it dig through every page to figure out what a business does.
In our AI gap report analysis of 17 local businesses across 1,352 AI answers, we found 32% of answers named the maximum of five businesses.
It does a different job from robots.txt: it does not block anything or grant permission for anything. Think of it as the table of contents an AI tool reads first, so it learns what your business is before it reads the rest.
Why it matters: an AI tool reading your website has to piece together what you do from dozens of pages. An llms.txt file gives it that answer up front, with links to the pages that prove it. Server-log research from Complete SEO across 6,455 websites found that OpenAI's GPTBot and OAI-SearchBot were the likeliest crawlers to request llms.txt while reading a site.
I think llms.txt files matter, because AI tools use them to learn about a business. I'd add one to any local service website, even one that already reads clearly to a human, because the file is written for the AI tool doing the reading.
A clear llms.txt is one of the cheapest ways to help AI tools describe your business accurately: what you offer, where you work, and what you do not do. For anyone who'd rather not write one alone, GetLocalLeads.AI's free AI Visibility Audit is the simplest first step.
Do LLMs respect robots.txt?
The major AI crawlers, OpenAI's included, check robots.txt before reading a site, and each one can be allowed or blocked there. robots.txt has been around for thirty years. It's the file search engines, and now AI crawlers, check first to see what parts of a site they're allowed to read. Where llms.txt tells AI tools what to read, robots.txt controls whether they may read at all. Get it wrong, and you can lock out a real chunk of your visibility without ever meaning to.
Based on our internal study of 41 local service businesses, 29 of the 41 (71%) passed our crawl-access check, so access was the part most of them already had right.
When it does go wrong, it's usually by accident. A security plugin added years ago to stop scraper bots (programs that copy content off websites) sometimes ships with a default rule that blocks AI crawlers too. Other times a rule written for a staging site (a private test copy of the website) gets copied to the live one and never revisited. Nobody wakes up and chooses to block ChatGPT. It happens in a file almost nobody looks at twice.
Not every AI bot from the same company does the same job, either. OpenAI's own developer documentation confirms that GPTBot crawls content for model training (teaching the AI), while a separate bot called OAI-SearchBot powers live ChatGPT search results, and the two are controlled independently in robots.txt. Blocking one has no effect on the other.
A lot of owners treat "the AI bots" as one on-off switch. On the surface that makes sense, because GPTBot and OAI-SearchBot come from the same company. But a business that says "we already blocked the AI bots" may have blocked the training crawler while leaving itself invisible in ChatGPT's actual search answers, or the reverse. Other AI companies run their own separately named crawlers, each controlled by its own line in the same file.
Say a contractor installed a security plugin two years ago to keep bad bots off the site, checked the box that looked safest at the time, and never opened that settings page again. The plugin's default happens to disallow every AI crawler along with the scrapers it was built to stop. Nobody chose that outcome, and nobody would notice unless they went looking. Worth checking whether that's your website too, or book a call and let GetLocalLeads.AI check it with you.
Is llms.txt worth it?
Yes, and so is checking the robots.txt beside it. In GetLocalLeads.AI's experience, AI visibility (whether AI tools like ChatGPT can find your business and bring it up when someone asks) can shift within about 24 hours of a change like an updated robots.txt or llms.txt file, though results are never guaranteed. Traditional SEO rankings take much longer to move. That's a small window for something that costs almost nothing to check.
The math is lopsided in a useful way. Adding an llms.txt costs almost nothing and gives every AI tool that reads your site a clear summary of your business. Checking robots.txt costs nothing either, but getting it wrong silently costs something real: a fast-growing share of customers now ask an AI assistant for local recommendations, and the AI visibility basics are the place to start if that shift is new to you. A wrongly blocked robots.txt shuts that whole channel off without a warning light anywhere on the dashboard.
Take an HVAC company that just spent real money on a redesign, new photos, and a clearer services page, all aimed at looking trustworthy to a customer researching repair options. None of that spending reaches the customer whose first stop was an AI assistant if the site's robots.txt quietly tells every AI crawler to stay out, because the AI never got to read the site. Those are the real stakes: a closed door nobody remembered closing. I'd spend those five minutes before spending money on anything else, and GetLocalLeads.AI can walk you through whatever the check turns up.
How to check robots.txt?
Checking your robots.txt for AI blocks takes one step: type yourdomain.com/robots.txt into a browser, on your phone or at your desk, and read what comes back. This is the whole self-diagnostic for "is my website blocking ChatGPT," and it takes less time than reading this sentence.
A clean, healthy file usually looks like a short list of lines starting with "User-agent" (which names the bot a rule applies to) followed by "Disallow" lines (which name the folders that bot must stay out of), aimed at specific spots such as an admin login page, not the whole site. A red flag looks like a bare "Disallow: /" applied to every visitor (the slash on its own means the entire site), or a named AI bot such as GPTBot or OAI-SearchBot singled out and disallowed on its own line. If an AI bot's name shows up next to the word "Disallow," that bot is blocked, full stop.
Checking the other file works the same way: type yourdomain.com/llms.txt. Most small business sites don't have one yet, which makes it an easy edge over competitors who leave AI tools to guess what they do.
When we check a site ourselves, we look at four things in this order. First, robots.txt, and whether any AI crawler is blocked. Then we confirm llms.txt exists and that the address actually serves the file. Next we make sure the sitemap (the list of pages a site hands to search engines) is declared and lists real pages. Last, we check that the business writes its name the same way on its website and on its profiles.
What you do with what you find matters more than the check itself. If you're worried a red flag here means a big bill or a rebuilt website, it usually doesn't. A confirmed robots.txt block is a same-day fix, usually a quick conversation with whoever manages the site, the hosting provider, or the developer who built it, since the fix is often a single line removed from one file. A missing llms.txt is a quick add: one file, written once, and updated whenever your services or service area change.
The two files do different jobs, so check both. If what you find is hard to read, that's a fair thing to bring to a call with GetLocalLeads.AI.
How to make an llms.txt file?
You make an llms.txt by writing a short plain text file and saving it at the root of your site, at yourdomain.com/llms.txt. Before you do, fix any confirmed robots.txt block, because a block there stops AI tools from reading anything else, including your llms.txt. That single fix removes an active barrier. I wouldn't spend an afternoon polishing llms.txt while robots.txt still keeps AI crawlers out. If the site is on WordPress or a similar platform, the robots.txt rule usually lives in a plugin's settings screen or a single editable text file, and a hosting support team can usually point to it if you're not sure where to look.
Then add your llms.txt the same week. Put three things in it: a one-line description of your business, your services and service area, and links to the pages that prove them. Keep it current whenever your services change.
This is exactly what the AI-visibility guarantee from GetLocalLeads.AI, an AI visibility and digital marketing agency for local and multi-location service brands, covers: making sure a client's robots.txt and llms.txt files are configured so AI can find and crawl the site, since many businesses unknowingly lock AI out of their own website. It's a narrow, specific guarantee: it covers whether AI can get in and read the site, not whether an AI tool decides to name the business afterward, which is its own separate mechanism. I'll be straight about why it stops there: AI recommendations change from one run to the next and from one AI tool to another, so a well-configured site still won't appear in every answer. That's why we call faster movement possible, never guaranteed. If checking your own two files turns up something you're not sure how to fix, that's worth a conversation, and GetLocalLeads.AI's AI visibility work is the place to start.
Frequently Asked Questions
Is llms.txt the same as robots.txt?
No, and confusing the two is the most common mistake in this topic. robots.txt controls what crawlers are allowed to access, and getting it wrong can genuinely hide a site from AI tools entirely. llms.txt tells AI tools what your business does and which pages to read first. One controls access; the other explains your business.
Do I need an llms.txt file right now?
Yes. It is the simplest way to give AI tools an accurate summary of your business, and it costs almost nothing to add. Check robots.txt first, because a block there stops AI from reading anything, then add llms.txt the same week.
How do I know if my robots.txt is blocking AI?
Type yourdomain.com/robots.txt into a browser and look for a bare "Disallow: /" or a named AI bot listed as disallowed on its own line. If you want a fuller picture of how AI tools treat your business right now, beyond just this one file, check where you stand next.
Will blocking one AI bot hurt my Google ranking?
Not directly. Google's ranking system and OpenAI's crawlers are separate systems entirely, and even within OpenAI, blocking the training bot doesn't affect the separate bot that powers live ChatGPT search results. Each block only affects what it specifically names, never anything wider than that.
Who usually breaks this by accident?
Most often a security plugin with an overly broad default that was never revisited, or a rule copied from a staging site that shipped to the live one by mistake. Almost nobody blocks AI crawlers on purpose; it's nearly always a setting nobody remembered was there.
Check robots.txt today so nothing is locked out, then add llms.txt so AI tools know exactly what your business does. Neither file will make the phone ring on its own, but together they decide whether AI can read your site and what it learns there. If you haven't already, work through the self-check guide for the fuller picture. Then get your free AI Visibility Audit and let GetLocalLeads.AI take the second look.