Crawl budget & URL scheduling – How it may impact website rankings
You think you’ve done everything from the perspective of SEO but sadly enough you still see your website has dropped in ranking? Why is it so? Did you know that Google determines the importance of a webpage and also allots crawl budget at the same time? Gary Illyes of Google was of the opinion that 30x redirects will never lead to a loss or dilution of Google Page Rank. There are many optimizers who have greeted this kind of claim with enough scepticism.
During the procedure of a migration, Googlebot requires collecting huge levels of data for mapping, accumulation of logs and during this time rankings of your website may fluctuate. But along with that when the visit of Googlebot plays a key role in fluctuation of rankings in migration, this can lead to ‘URL scheduling’, which is a vital component of the crawl budget.
Crawl budget is equal to URL scheduling combined with host load
What Googlebot wants to visit URLs and how often, is what constitutes URL scheduling and on the other hand, host load is all that is based around what Googlebot visits from a host/IP based on the server sources and capacity of the server. All these combined makes up crawl budget for a host or IP. If you have a 10-page brochure website, you wouldn’t see any loss of visibility during migration of the site but what would happen if you have an e-commerce site with hundreds and thousands of URLs?
Everything has to start as bare minimum with a complete site crawl by Googlebot. You may even require a few total site crawls as Googlebot will understand more regarding URLs and it also knows how everything will fit in. If you have larger sites, this may not happen as soon as you predict it to occur.
Googlebot’s schedule isn’t your migration schedule!
You have your own personal migration schedule. That doesn’t actually mean that Googlebot’s schedule will also fall into place. They have their own working schedule and crawl frequency of URLs is done on a per-URL basis. As per Google, there are too many factors which affect the crawl frequency of URLs.
It’s important to take into account ‘importance’
Crawl scheduling is always driven by the extreme importance of URLs. Gary Illyes also states that we shouldn’t always keep focusing on PageRank as the only driver of ranking or crawling. Hence Page Rank and Page Importance aren’t the same but yes one of them may include another. Page Importance may include Page Rank.
