Sitemaps & crawling
XML sitemaps, robots.txt, AI crawler rules, llms.txt, Markdown pages and IndexNow.
6 guides
- XML sitemapsRankKit SEO's XML sitemaps build on WordPress core sitemaps and leave out noindexed content. Find your sitemap URL, choose what's included and submit it.Read guide
- News and video sitemapsPROPublish a Google News sitemap of articles from the last 48 hours and a video sitemap built from YouTube, Vimeo and self-hosted videos in your posts.Read guide
- Edit robots.txtAdd your own robots.txt rules, allow or block 15 AI crawlers one by one, and see exactly what RankKit SEO writes into your site's virtual robots.txt.Read guide
- Control and track AI crawlersSee which AI bots crawl your site, which pages they read and why, and allow or block each AI crawler from RankKit SEO's robots.txt settings.Read guide
- llms.txt and Markdown pages for AIRankKit SEO publishes /llms.txt and clean Markdown copies of your pages for AI agents, plus llms-full.txt and an AI usage policy in RankKit SEO Pro.Read guide
- IndexNow: instant indexing for Bing and othersTurn on IndexNow so Bing, Yandex, Naver and Seznam hear about new and updated posts within a minute, and submit any URL by hand from Tools.Read guide