Indexing

A page is indexed by Google if it has been visited by the Google crawler (“Googlebot”), analyzed for content and meaning, and stored in the Google index. Indexed pages can be shown in Google Search results (if they follow Google’s webmaster guidelines). While most pages are crawled before indexing, Google may also index pages without access to their content (for example, if a page is blocked by a robots.txt directive).

Crawling (Scanning)

Crawling is the process of finding new or updated pages to add to Google (Google crawled my website). One of the Google crawling engines crawls (requests) the page. The terms “crawl” and “index” are often used interchangeably, although they are different (but closely related) actions.

Crawling/Indexing features Google Search Console 

Ask Google to crawl (or recrawl) your page

If you’ve recently added or made changes to a page on your site, you can request that Google re-index your page using any of the methods listed here.

Temporarily block pages or images from Google

To temporarily (6 months) remove pages or images on sites that your own from Google, use the Removals tool.

Request a crawl or recrawl

The simplest way to request a crawl for a single page is using the URL Inspection tool:

1. Open the tool
2. Enter the full URL of the page and click Enter
3. Click Request indexing. The request will fail immediately if there is an access problem with the page. The request will fail silently if you have exceeded your quota of indexing requests.

To request a crawl for multiple pages, use a sitemap

 

Submit a sitemap, monitor submitted sitemaps, monitor index coverage of URLs in a sitemap

To submit a sitemap, or monitor whether Google could read your submitted sitemap, use the Sitemaps report.

Note that the Sitemaps report shows information only about sitemaps submitted using that report. Sitemaps found independently by Google won’t be shown there. However, you can submit a sitemap through the report even if Google already knows about it, in order to be able to monitor Google’s crawl attempts.

The only ways to block a sitemap from being read are to delete the sitemap or block it using robots.txt.

To monitor indexed URLs by sitemap, choose a sitemap in the dropdown selector on the Index Coverage report.