How do they work? How do they manage to generate such relevant results in a matter of seconds? How the generated results are so up to date? These are some of the many questions that often crops up in your mind when looking for any information online through the search engine. Curious enough to find an answer to your questions, you resort to various online resources and technical manuals that look difficult to decipher and hence confuse you a lot more.
Understanding the working of a search engine is not indeed that complex. Having said that, a search technology in itself is very sophisticated and yes, a lot complicated to understand for the ones with limited technical knowledge. However, this should not stop us from gaining a basic overview on a search engine"s working.
The working of a search engine can be compared to that of a huge library with books on diverse subjects. Both of them have one thing in common i.e. they both deal in information. A huge resource of information is a prerequisite for both of them to work. To build on this resource, a well stocked library collects and stores books on various subjects from the publishers; a search engine, in a similar way, crawls and indexes WebPages from as much of the World Wide Web as it can. This information is then kept under specific heads. So, a book on human anatomy is essentially stacked in the biology segment of a library. A member just needs to look into the biology section to retrieve that book. In case of a search engine, information retrieval is through search query and hence when a visitor types in "human anatomy" into the search box, the search engine returns those results that have the word "human anatomy" or related references to it. The number of occurrences of such words in a web page is often referred to as keyword density.
Now once this vast information is collected, the question that comes to our mind is how to ensure that the information delivered is of highest quality. One way of validating quality in a library is to bank on the publishers" credentials, author"s past works and also the popularity of a book which can be measured by the number of times it has written about in various other sources and media. However, the quality and relevancy of the media is important. Reference of a well known book in a news channel like BBC provides more authenticity than a reference in some channel that people hardly know of. In the context of a search engine, the quality of a web page can be measured by the domain it belongs to and the information it deals with. The popularity of a web page is measured by the number of references it has from other web pages across the World Wide Web. So a web page on human anatomy finding a mention on the website of a BBC underlines its authenticity in bold. The keyword density coupled with such relevant references is a prime determinant for the relevancy of the results.
Now let"s have a look at how this vast information is managed effectively. While a library keeps track of all the recent book releases, a search engine too keeps track of the recent website updates. So, while a librarian needs to get in touch with the publishers to get the latest books, a search engine revisits its database of websites from time to time to find out the updates and index anything new based on certain parameters. Books that are least subscribed, damaged and banned are kept separately and then finally discarded by the librarian. In a similar way, web pages that do not function properly or deal in information on banned subjects are done away with. This helps both the entities to manage space and information in the optimal manner.
The popularity of a library or a search engine is based on the quantity and quality of information resources it deals with and the ease of its accessibility. A search engine is an integration of a well equipped library with world class automated data management and information retrieval technologies, powerful enough to give us what we are looking for within seconds. No wonder the buzz around search engine optimization is reaching phenomenal levels each day!