Ranking in the search engines requires a website with flawless technical SEO.
Luckily, the Yoast SEO plugin takes care (of almost) everything on your WordPress site.
Still, if you really want to get most out of your website and keep on outranking the competition, some basic knowledge of technical SEO is a must.
In this post, I’ll explain one of the most important concepts of technical SEO: crawlability.
What is the crawler again?
A search engine like Google consists of a crawler, an index and an algorithm.
The crawler follows the links.
When Google’s crawler finds your website, it’ll read it and its content is saved in the index.
A crawler follows the links on the web.
A crawler is also called a robot, a bot, or a spider. It goes around the internet 24/7.
Once it comes to a website, it saves the HTML version of a page in a gigantic database, called the index.
This index is updated every time the crawler comes around your website and finds a new or revised version of it.
Depending on how important Google deems your site and the amount of changes you make on your website, the crawler comes around more or less often.
And what is crawlability?
Crawlability has to do with the possibilities Google has to crawl your website.
Crawlers can be blocked from your site.
There are a few ways to block a crawler from your website.
If your website or a page on your website is blocked, you’re saying to Google’s crawler: “do not come here”.
Your site or the respective page won’t turn up in the search results in most of these cases.