James Frey Official Website
Join the JAMES FREY mailing list

The Scawy Web

from CNN Money

The Deep Web you don’t know about

By Jose Pagliery



What we commonly call the Web is really just the surface. Beneath that is a vast, mostly uncharted ocean called the Deep Web.

By its very nature, the size of the Deep Web is difficult to calculate. But top university researchers say the Web you know — Facebook (FBFortune 500), Wikipedia, news — makes up less than 1% of the entire World Wide Web.

When you surf the Web, you really are just floating at the surface. Dive below and there are tens of trillions of pages — an unfathomable number — that most people have never seen. They include everything from boring statistics to human body parts for sale (illegally).

Related story: Shodan, the scariest search engine on the Internet

Though the Deep Web is little understood, the concept is quite simple. Think about it in terms of search engines. To give you results, Google (GOOGFortune 500), Yahoo (YHOOFortune 500) and Microsoft’s (MSFTFortune 500) Bing constantly index pages. They do that by following the links between sites, crawling the Web’s threads like a spider. But that only lets them gather static pages, like the one you’re on right now.

What they don’t capture are dynamic pages, like the ones that get generated when you ask an online database a question. Consider the results from a query on the Census Bureau site.

[ click to continue reading at CNN Money ]

Posted on March 12, 2014 by Editor

Filed under Weirdness | | No Comments »