How Search Engines Work

Search engines are the key to finding specific information on the World Wide Web. Without the use of sophisticated search engines, it would be virtually impossible, particularly in light of how fast the internet grows everday, to locate anything on the Web without knowing a specific URL.

There are basically three types of search engines: Those that are powered by crawlers, or spiders; those that are powered by human submissions; and those that are a combination of the two.

Crawler-based engines send crawlers, or spiders, out into cyberspace. These crawlers visit a Web site, read the information on the actual site, read the site's meta tags and also follow the to which the site connects. The crawler returns all that information back to a central depository where the data is indexed.

In contrast to the latter, human-based ones employ individuals to submit information that is subsequently indexed and catalogued.

Sources:
http://www.webopedia.com/DidYouKnow/Internet/2003/HowWebSearchEnginesWork.asp