Google index hits trillion page milestone

 

Gallery

By Nicole Kobie

Posted on 28 Jul 2008 at 17:11

Google claims to have detected a trillion unique URLs, as the web hits a new milestone.

In its blog, the web search giant claims its engineers "stopped in awe" when they realised how big the web had become, after the index hit the trillion mark, with the web growing by several billion every day.

The first index by Google in 1998 found 26 million pages, with the billion page mark passed in 2000.

"This graph of one trillion URLs is similar to a map made up of one trillion intersections. So multiple times every day, we do the computational equivalent of fully exploring every intersection of every road in the United States," Google software engineers Jesse Alpert and Nissan Hajaj wrote in the blog.

However, Google noted that while it's found more than one trillion pages, many are simply auto-generated copies.

"So how many unique pages does the web really contain? We don't know; we don't have time to look at them all," the blog reads. "The size of the web really depends on your definition of what's a useful page, and there is no exact answer."