Feb
[ Editor's note: While I (Sol) am the primary author of this blog, Abe does occasionally post. As founder, President, and CTO of Deep Web Technologies, and a pioneer in developing web-based information retrieval, Abe's posts are very insightful, coming from a great depth of understanding of the federated search industry. Deep Web Technologies, as many of you know, sponsors this blog. You can read more about that in the About page. As editor of the blog I have to walk the fine line of publishing posts that serve my sponsor while not having the blog become a soapbox for Deep Web. What seems fair to me, and I invite discussion, is to permit "soapbox" posts from any vendor of federated search (commercial or otherwise) as long as the post is not primarily self-serving, is of interest and value to the entire federated search community, and is respectful of others. I also welcome guest posts from librarians, bloggers, and others who have a viewpoint to share. Guest posts can question vendor actions, they can raise concerns, they can express opinions, and they don't have to be particularly nice, but they don't get to be insulting.
Abe's post below is an example of a soapbox post that provides value to readers and promotes a couple of presentations that Abe will be giving in May. Contact me at the email address in the About page to discuss your idea for a post. ]
There has been a lot of discussion in recent posts on this blog, especially in Saturday’s CILIP post, and in the earlier post on Is federated search as bad as librarians think? on whether federated search should be used by students and researchers or whether these students would be better off searching information sources directly. I agree with the comments that if there is a researcher who knows which two or three information sources to search then federated search doesn’t provide value to this researcher and he/she should just search these sources directly. However, for the large number of students and researchers who have access to dozens, perhaps hundreds of information sources, and don’t know where the “gem” of an article might be found then federated search is an indispensable tool.
As one who has been involved in federated search for a long time - I built one of the first federated search engines (we called it distributed search then) in early 1999 - I have seen a lot of improvement in federated search in the last eight years. However, there are a lot of poor federated search implementations out there, particularly ones implemented early, that are responsible for the negative perceptions of federated search.
I strongly believe that there are a lot of differences among the capabilities of federated search products/services in the market today, differences that are not easy to quantify, and that cannot be distilled by simply comparing lists of features. This is why I am very supportive of the effort in this blog to create one place from where different federated search products can be compared.
I believe that that single most important factor that can affect the success of a federated search implementation is the quality of the search results that are returned by the federated search engine. Sol does a great job discussing this subject in his blog post – What determines the quality of search results. As Sol points out, quality of search results is not so easy to quantify, but if a library evaluating multiple federated search products is able to compare products that each search the same sources, significant differences in the quality of search results should be apparent.
In May I will be giving talks on this subject at two library group conferences. First, on May 7th, I will be speaking at a one day conference on Next Generation Library Technologies put on by the Southeastern NY Library Resources Council, then on May 9th, I will be speaking at a Conference put on by the Atlantic Provinces Library Association. Both of these conferences are being held at great venues, the FDR Presidential Library and Prince Edward Island. I encourage you if you are in the area to register for one of these conferences and attend one of my talks. I will be sure to provide links to my talks on this blog after the conferences.
Abe
If you enjoyed this post, make sure you subscribe to the RSS feed!
Tags: federated search
4 Responses so far to "Not all Federated Search Engines are Created Equal"
February 25th, 2008 at 6:11 pm
“which two or three information sources to search then federated search doesn’t provide value to this researcher”
A librarian who works with researchers (like, non-teaching researching faculty) tells me that even if they DO know which two or three sources to search, they don’t in fact want to search two or three sources separately, or learn two or three separate interfaces.
February 26th, 2008 at 2:17 pm
Jonathan,
That’s a good piece of data to have. Thank you. It further reinforces the image that it doesn’t matter that native searching brings back better results if the resistance to native searching is so high.
February 26th, 2008 at 6:24 pm
What do you think about opportunities for hosted federated searching in the school library market? We are up to about 50 databases! Follet has their onesearch which will simply cross-search d.b.’s, and Gale now has their powersearch plus which will search other vendor products, but not many. I like the looks of webfeat express too, though at $8,000 it is probably too costly. Do you know if I’m missing anything in between?
February 27th, 2008 at 4:58 pm
Sarah,
I will respond to your comment in the blog. In brief, no, I don’t know if you’re missing something or not in the price range you’re looking for. But, I will respond with a blog post in the next few days.