robots.txt · Shopify
Shopify robots.txt, rule by rule
Shopify ships one of the most opinionated default robots.txt files on the web — roughly two dozen rules aimed almost entirely at one problem: filtered collection URLs multiplying out of control. Here is what every group does, what you can override through robots.txt.liquid, and what you should leave alone.
User-agent: *
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkouts/
Disallow: /checkout
Disallow: /12345678/checkouts
Disallow: /12345678/orders
Disallow: /carts
Disallow: /account
Disallow: /collections/*sort_by*
Disallow: /*/collections/*sort_by*
Disallow: /collections/*+*
Disallow: /collections/*%2B*
Disallow: /collections/*%2b*
# also blocks blog tag-combo URLs, oseid tokens, policy pages, and search
Disallow: /*preview_theme_id*
Disallow: /*preview_script_id*
Disallow: /apple-app-site-association
Disallow: /.well-known/shopify/monorail
Disallow: /cdn/wpm/*.js
Disallow: /recommendations/products
Disallow: /services/login_with_shop
User-agent: adsbot-google
Disallow: /checkouts/
Disallow: /checkout
Disallow: /carts
Disallow: /orders
User-agent: Nutch
Disallow: /
User-agent: AhrefsBot
Crawl-delay: 10
User-agent: AhrefsSiteAudit
Crawl-delay: 10
User-agent: MJ12bot
Crawl-delay: 10
User-agent: Pinterest
Crawl-delay: 1
Sitemap: https://example.com/sitemap.xmlTaken from a live Shopify storefront. Store-ID paths shown as 12345678 — yours carries your real store ID. Shopify revises these defaults over time.
What every default rule is defending against
Grouped by intent rather than listed in file order, because several rules exist to solve the same problem in different URL shapes.
/admin, /account, /cart, /carts, /orders, /checkout, /checkouts/Blocks every transactional and authenticated URL on the storefront.
None of these are indexable pages, and letting crawlers walk into cart and checkout URLs creates junk sessions and abandoned-checkout noise in your analytics.
/12345678/checkouts and /12345678/ordersThe same checkout and order URLs, prefixed with your numeric store ID.
Shopify serves these routes under both shapes, so it emits both rules with your real store ID substituted. If you see a random number in your robots.txt, that is what it is — not a bug.
/collections/*sort_by* and /*/collections/*sort_by*Blocks every sorted variant of a collection page.
Sorting by price, title or best-selling returns the same products in a different order under a new URL. Left crawlable, one collection becomes half a dozen duplicates. The /*/ variant covers the same URLs under a market or locale prefix.
/collections/*+*, /collections/*%2B*, /collections/*%2b*Blocks tag-filtered collection URLs, in all three encodings of the "+" separator.
This is the most important rule in the file. Shopify joins collection tag filters with a plus sign, and tag combinations multiply — a collection with ten tags can generate hundreds of near-identical URLs. The percent-encoded variants exist because that "+" reaches the server encoded as %2B or %2b depending on the client.
blogs/*+*The same tag-combination block, applied to blog listings.
Shopify blog tag filters have the identical explosion problem as collection tags, so they get the identical treatment.
searchBlocks internal search result pages.
Search result pages are thin, infinite, and generated by whatever anyone types. Google explicitly discourages indexing them.
policies/Blocks the auto-generated refund, privacy, terms and shipping pages.
These are near-identical across millions of Shopify stores because most merchants use the generated templates. There is nothing to rank for and some duplicate-content risk.
/*preview_theme_id* and /*preview_script_id*Blocks theme and script preview URLs.
These render an unpublished theme over your live content. Indexing one means Google holds a copy of a draft design as if it were your store.
*?*oseid=*Blocks order status email links.
The oseid parameter is a per-order token in order-status emails. These are personal URLs that should never enter a search index.
/recommendations/products, /services/login_with_shop, /cdn/wpm/*.js, /.well-known/shopify/monorail, /apple-app-site-associationBlocks internal endpoints that return fragments, scripts or JSON rather than pages.
They exist to serve the storefront, not readers. Crawling them wastes budget and produces indexable garbage.
User-agent: adsbot-googleA narrower group for Google Ads landing-page quality checks.
AdsBot needs to see more of your store than a normal crawler does, so it only gets the transactional URLs blocked. AdsBot also ignores wildcard rule groups, which is exactly why it needs its own.
User-agent: Nutch — Disallow: /A blanket block on the Apache Nutch open-source crawler.
Nutch is the engine behind a lot of homegrown scrapers. Shopify blocks it outright by default.
Crawl-delay: 10 for AhrefsBot, AhrefsSiteAudit, MJ12bot; 1 for PinterestThrottles third-party crawlers rather than blocking them.
Keeps SEO and backlink crawlers from hammering the storefront. Worth knowing that Googlebot and Bingbot ignore Crawl-delay entirely — it only affects the bots that choose to honour it.
What Shopify lets you edit
robots.txt is a theme file on Shopify. That has two consequences most merchants discover the hard way: you need theme code access to change it, and nothing goes live until the theme is published.
Shopify used to lock robots.txt completely
For most of Shopify's life the file was untouchable. Now merchants can override it through a theme template called robots.txt.liquid, which renders the file served at /robots.txt.
Where to create it
Online Store → Themes → ⋯ → Edit code → Add a new template → select robots.txt. Shopify requires it to be a Liquid template; it cannot be a JSON template. Because it lives in the theme, editing robots.txt needs theme code access, and the change only reaches shoppers once that theme is published.
Keep the default groups, add on top
Shopify's docs are explicit that while you can replace all the template content with plain text rules, you should use the provided Liquid objects wherever possible — because the default rules are updated regularly to keep SEO best practices applied. Loop over robots.default_groups and inject your own rules inside it.
{% for group in robots.default_groups %}
{{- group.user_agent }}
{%- for rule in group.rules -%}
{{ rule }}
{%- endfor -%}
{%- if group.user_agent.value == '*' -%}
{{ 'Disallow: /pages/internal-only' }}
{%- endif -%}
{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}
{% endfor %}AI crawler rules for Shopify
Shopify's defaults say nothing about AI crawlers. Append your own groups after the default loop so Shopify's maintained rules stay intact. What each bot does is covered in our AI crawlers reference →
{% for group in robots.default_groups %}
{%- comment -%} keep Shopify's maintained defaults above {%- endcomment -%}
{% endfor %}
User-agent: GPTBot
Allow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Google-Extended
Allow: /To block them instead, swap Allow: / for Disallow: / in each group — but be clear about the cost. Blocking removes your products from the crawls behind ChatGPT, Claude and Perplexity, which is where a growing share of product research now happens. Google-Extended is the narrower case: it governs whether your content is used for Gemini and Vertex AI generative products, and does not affect your inclusion in Google Search.
The Shopify-specific mistakes
Each of these is something the Shopify architecture makes uniquely easy to get wrong.
Replacing the whole template with plain text
The moment you hardcode the rules, you stop receiving Shopify's updates to them. Shopify revises the default rule set as its URL structure and SEO guidance change; a frozen copy slowly drifts out of date and starts leaking URL patterns you never intended to expose.
Deleting the collection filter rules to "get more pages indexed"
Removing the *sort_by* and *+* rules does not get more products indexed. It hands crawlers a combinatorial explosion of duplicate collection URLs and drains the crawl budget that should be going to your product pages.
Editing robots.txt on an unpublished theme
robots.txt.liquid is theme code. Edit it on a duplicate or draft theme and the live /robots.txt is unchanged until that theme is published. This is the most common "I saved it and nothing happened" on Shopify.
Using Disallow to remove a product or page from Google
Disallow stops crawling, not indexing — a blocked URL with inbound links can still surface as a bare title. To remove a Shopify page you need a noindex meta tag rendered conditionally in theme.liquid, which requires the page to stay crawlable.
Blocking /cdn/ wholesale
Merchants sometimes block the CDN path to save bandwidth or hide assets. That is where your product images, CSS and JavaScript are served from. Block it and Google renders your store as a broken page.
Build the file instead of copying one
Our robots.txt generator takes your store URL and produces a file with the right sitemap directive and crawl rules, as part of a free strategy run.
Open the robots.txt generatorOr let the agent ship it
Rankealo's SEO agent can open a pull request with the robots.txt fix already written, so a crawl-blocking rule gets reviewed and merged instead of sitting in a report.
See how the SEO agent worksrobots.txt on another platform?
Shopify robots.txt questions
Can you edit robots.txt on Shopify?
Yes. Shopify exposes the file through a theme template called robots.txt.liquid, which you create under Online Store → Themes → Edit code → Add a new template. It has to be a Liquid template rather than a JSON one. Before that template existed, robots.txt on Shopify was entirely locked, which is why a lot of older advice still says it cannot be changed.
Why does Shopify block collection URLs with sort_by and plus signs?
Because both generate duplicates. Sorting a collection returns the same products under a new URL, and tag filters joined by a plus sign multiply into hundreds of near-identical collection URLs. Shopify blocks all three encodings of that plus sign (+, %2B and %2b) because the separator reaches the server differently depending on the client. Removing these rules is one of the fastest ways to waste a store's crawl budget.
Does the default Shopify robots.txt stop Google from indexing my products?
No. The default rules block checkout, cart, account, internal search, policy pages, previews and filtered or sorted collection variants. Product pages, canonical collection pages and blog articles are all crawlable, and the file includes a Sitemap directive pointing at your sitemap.xml.
Should I block AI crawlers like GPTBot on my Shopify store?
Usually not. Blocking GPTBot, ClaudeBot or PerplexityBot removes your products from the crawls that feed AI shopping and research answers, so you lose the citations and referral traffic that come with them. If you do want to add AI crawler rules, append them after the default_groups loop in robots.txt.liquid so Shopify's maintained defaults stay intact.
