3
Apr

We’ve all heard the old adage, “Don’t believe everything you read.” The Internet is full of stuff to read; how do we know what to believe? While there are numerous search engines that present us with documents in response to our queries, how do we know if the information presented in these documents is accurate? Granted, much of what’s in the Internet is personal opinion and sometimes all we want is someone’s viewpoint. There are times, however, when we need to know that the information we are reading is of high quality. We may be researching product features to make a purchase decision, company information to form competitive intelligence strategy, or medical information to address a medical concern.

A major part of the answer to the question of whether information is accurate or not is to examine its source. This is where federated search engines really shine. By their nature, federated search applications usually query deep web database sources. The databases can’t be crawled. There are no links for Google to follow to extract all documents in such a database. Now, let’s consider the type of content that lives in these non-crawlable databases. Publishers who specialize in scientific, technical, and business research articles are most likely to store their documents in databases and to make their content searchable by federated search engines. Geological, geographic, demographic data lives in databases. Much political data lives in databases as well.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

28
Mar

The first thing that most people notice when they use a federated search application is that it’s not nearly as fast as Google. We’ve all gotten spoiled. This is not only the information age, it’s the age of quick information; we all want every search to be as fast as a Google search. However, by its very nature, federated search can’t be as fast as Google. Federated search is at the mercy of the sources it federates. If a source is slow to return results to the federated search application, then there’s nothing the federated search application can do, or is there?

Deep Web Technologies has been displaying incremental results for some time now. The idea is simple: display results in chunks as they are received from the sources being searched. Science.gov, WorldWideScience.org, and Scitopia.org are three applications that display incremental results. While there are challenges to this approach, there are some significant benefits as well. The aim of displaying incremental results is to minimize the time the user has to wait to see some results. In the show-something-quick department, incremental results works well. The major challenge arises when you try to figure out what to do with the rest of the results as they come in.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!