How Google Search organizes information
When you search, Google looks through hundreds of billions of webpages and other digital content organized in our Search index to find helpful information - more information than all of the libraries of the world.
Finding information by crawling
Most of our Search index is built through the work of software known as crawlers. These automatically visit publicly accessible webpages and follow links on those pages, much like you would if you were browsing content on the web. They go from page to page and organize information about what they find on these pages and other publicly accessible content in Google’s Search index.
Sorting and organizing content
When crawlers find a webpage, our systems render the content of the page, just as a browser does. We take note of key signals - from keywords to website freshness - and we keep track of it all in the Search index.
The Google Search index covers hundreds of billions of webpages and is well over 100,000,000 gigabytes in size. It’s like the index in the back of a book - with an entry for every word seen on every webpage we index.
Google never accepts payment to crawl a site more frequently - we provide the same tools to all websites to ensure the best possible results for our users.
Constantly processing new info
Because the web and other content is constantly changing, our crawling processes are always running to keep up. They learn how often content they’ve seen before seems to change and revisit as needed. They also discover new content as new links to those pages or information appear.
Google also provides a free toolset called Search Console that creators can use to help us better crawl their content. Creators can also make use of established standards like sitemaps or robots.txt.
Google never accepts payment to crawl a site more frequently - we provide the same tools to all websites to ensure the best possible results for our users.
Indexing more than webpages
Our Search index covers more than just what’s on the web, because helpful information can be located in other sources.
In fact, we have multiple indexes of different types of information, which is gathered through crawling, through partnerships, through data feeds being sent to us and through our own encyclopedia of facts, the Knowledge Graph.
These many indexes mean that you can search within millions of books from major libraries, find travel times from your local public transit agency, or find data from public sources like the World Bank.
