So first, there's a way for you to create several smaller sitemaps with like 1000 URLs each then link them together by a sitemap index file. It's way easier on Google too.
Batch these sitemaps in increments, so don't just submit a shit ton of small sitemaps at once.
I'd highly recommend you start with high-priority pages first and batch those URLs together before moving on to low-priority pages. You get to define what "high-priority" means for you -- i.e. top selling, high search intent, etc.
Even then, you're going to want to ruthlessly optimize every single page for crawlability (metadata, titles, descriptions, internal links, the works).
Are you on a shared server? If so, that'll impact your strategy. (If not, go fucking wild -- but not too wild lol.) Prioritize high-priority pages and use robots.txt.
Monitor your GSC for any errors. You'll want to look out for crawl errors, indexing issues, and warnings. Fix those as soon as they pop up. It's always good practice to monitor server logs too.
Before you submit the batches, test with a smaller batch (like 500 URLs) then watch to see how bots handle it. Pay attention how Google crawls and indexes them BEFORE you submit larger batches. Refine the process if needed.
There's more, but this should get you started.
I haven't worked on a website EXACTLY like yours, but I've handled large websites for clients (tens of thousands of pages), and it was a process getting everything properly indexed. Be patient, monitor the process, adjust where needed.
Thanks! Yep, already have the sitemap-index.xml + sitemap-n.xml thing working so that won't be a problem. I'm just concerned about the total number of pages even if spread across a few sitemaps.
Makes sense to do smaller tests, so I'll go with that approach.
So first, there's a way for you to create several smaller sitemaps with like 1000 URLs each then link them together by a sitemap index file. It's way easier on Google too.
Batch these sitemaps in increments, so don't just submit a shit ton of small sitemaps at once.
I'd highly recommend you start with high-priority pages first and batch those URLs together before moving on to low-priority pages. You get to define what "high-priority" means for you -- i.e. top selling, high search intent, etc.
SSG and SSR >>> CSR.
(A link if you want to know more context: medium.com/@natelapinski/how-…)
Even then, you're going to want to ruthlessly optimize every single page for crawlability (metadata, titles, descriptions, internal links, the works).
Are you on a shared server? If so, that'll impact your strategy. (If not, go fucking wild -- but not too wild lol.) Prioritize high-priority pages and use robots.txt.
Monitor your GSC for any errors. You'll want to look out for crawl errors, indexing issues, and warnings. Fix those as soon as they pop up. It's always good practice to monitor server logs too.
Before you submit the batches, test with a smaller batch (like 500 URLs) then watch to see how bots handle it. Pay attention how Google crawls and indexes them BEFORE you submit larger batches. Refine the process if needed.
There's more, but this should get you started.
I haven't worked on a website EXACTLY like yours, but I've handled large websites for clients (tens of thousands of pages), and it was a process getting everything properly indexed. Be patient, monitor the process, adjust where needed.
Thanks! Yep, already have the sitemap-index.xml + sitemap-n.xml thing working so that won't be a problem. I'm just concerned about the total number of pages even if spread across a few sitemaps.
Makes sense to do smaller tests, so I'll go with that approach.