by Melissa J Luther

Created on: January 09, 2008   Last Updated: December 22, 2008

A search engine is a computer program that retrieves documents containing specified keywords from a database. In the case of the internet search engines, the documents are web pages and the database is the search engine's internal index of the internet.

Notice that the database is not the internet itself, but an index of the internet. Many people believe that typing in keyword(s) and hitting "search" results in a "live search" of the internet. It does not.

The search results page (SERP) is actually the result of a search of the search engine's index. The live search of the web has already taken place. Actually the live search of the web is always taking place, but it is completely independent of individual users' searches.

In order to create its searchable index, the computer program "bot" crawls around the web reading all the information it can on each page. At the same time, it is looking for links from the current page to other pages. It follows these links and reads all the information on these pages, as well as following more links. It continues in this fashion, cataloguing whatever it can find. All the information it finds is stored in the search engine index. Remember, the index is what users actually search.

When a searcher enters keywords, the search engine goes about searching the index for relevant pages. Then it ranks those pages, with the pages it considers most relevant on top. If a site has not been indexed yet, it will not show up in the SERP.

Despite what it may seem like, the entire internet has not been "crawled" by the search engines. For one thing, a new page or website without links leading to it is impossible for the bot to find. That is why webmasters of new sites manually submit the site to the major search engines. Even if there are links to a site, it can take months before the search engines get around to finding those links. In addition, the bots have problems with things that are not straight text or html. So Frames, Flash and lots of graphics can confuse the bot.

So why do certain sites show up at the top of the SERP for particular keywords? Well, aside from paying for top listings, webmasters have optimized their sites to the search engines' preferences. This can be a tricky proposition, since the algorithms the search engines use are proprietary and changing all the time. However, there are certain things that result in higher listings.

Age of the site is an important factor. An older, more established site will get preference over

• 1
• 2
• Next Page >>