Googlebot to Start Crawling Sites Over HTTP/2 in Nov 2020

Starting November 2020, Googlebot will support crawling over the more modern version of the HTTP protocol HTTP/2 (or h2 for short) for some select sites. Hypertext Transfer Protocol (HTTP) is the protocol the internet primarily uses for transferring data. HTTP is an application-layer protocol for transmitting hypermedia documents, such as HTML. It was designed for communication between web browsers and web servers, but it can also be used for other purposes.

HTTP/2 (originally named HTTP/2.0) or h2 is a major revision of the HTTP network protocol used by the World Wide Web. HTTP/2 was developed by the HTTP Working Group of the Internet Engineering Task Force (IETF). Due to the architecture and the features implemented for clients (such as web browsers) and web servers, HTTP/2 is much more robust, efficient, and faster than its predecessor HTTP protocol.

By making this change, Google expects to enable crawling more efficient in terms of server resource usage. Using the architecture of HTTP/2, Googlebot is able to open a single TCP connection to the server and efficiently transfer multiple files over it in parallel, instead of requiring multiple connections. The server and Googlebot will need fewer connections, as a result the fewer resources on crawling which will increase the page speed of a webpage and ultimately increase the user experience of the web visitors.

Some of the most prominent benefits of HTTP/2 are discussed below –

• Multiplexing and Concurrency - Multiple requests over a single TCP (transmission control protocol) connection. Fewer TCP connections open means fewer resources spent.
• Data Compression of HTTP Headers - Drastically reduced HTTP header sizes will save resources.
• Pipelining of Requests - HTTP pipelining is a technique in which multiple HTTP requests are sent on a single TCP connection without waiting for the corresponding responses.
• HTTP/2 Server Push – It allows an HTTP/2-compliant server to send resources to a HTTP/2-compliant client before the client requests them. Though this feature is not yet enabled; it's still in the evaluation phase.

Google is continuously trying to give the best possible user experience to the web visitors. The technology of HTTP/2 will help Google to accomplish its mission of providing best possible user experience by saving resources both on the server side, and on Googlebot side. Read Google's announcement on crawling some sites over HTTP/2 in the Google Webmaster Central Blog.