Create a robots.txt for your site, keep private paths out of search, and choose which AI crawlers may read your content.
User-agent: *
Disallow: /admin
Disallow: /cart
Disallow: /checkout
Disallow: /account
Sitemap: https://yourbusiness.com/sitemap.xml
Used for the sitemap line at the end.
Admin, cart and account pages usually don't belong in search. Tick any AI crawlers you want to block.
Save it at your domain's root, so it's at yourbusiness.com/robots.txt.
It depends. Blocking training crawlers keeps your content out of future models. Blocking AI search crawlers can also stop assistants like ChatGPT from finding and recommending you.
It stops crawling, not indexing. To keep a page out of results, use a noindex tag instead.
Shopify, Wix and Squarespace generate their own and let you edit parts of it. Use this as a reference for the lines to add.
MAALI is an AI marketing team for your business. Its MAALI Brain and free SEO check read your whole website, rank what to fix, and then write articles and social posts in your brand voice. You approve everything.
Get my MAALI Brain