Federated Search: The year in review

As 2007 comes to an end I’d like to reflect on some of the major happenings in federated search this past year, hopefully setting a precedent for a yearly post.

In early November Microsoft announced that it was giving away Microsoft Search Server 2008 Express, an Enterprise-level search product based on SharePoint technology. Microsoft Search Server 2008, already available as a Beta, incorporates federated search capabilities based on the Open Search standard. I believe that this announcement will not have any impact on federated search in libraries but will have some impact on vendors trying to sell federated search products into corporate IT departments. Federated search capabilities in the Search Server product are limited to sources that can be accessed via Open Search. Merging/relevance ranking of results are not supported because Microsoft doesn’t believe it can be done well. All in all, I believe that this announcement by Microsoft will help to bring federated search more into the mainstream and is good for our industry.

With Microsoft Search Server 2008, Microsoft is clearly firing some salvos at the Google Search Appliance, in particular its OneBox offering which also supports some federated search capabilities.

In August, CARE Affiliates, a company that provides products and services to libraries based on open source software announced it was licensing WebFeat connectors for use with MasterKey’s federated search service at an annual cost that is lower than WebFeat’s. I don’t really understand why WebFeat licensed its connector technology to a competitor entering the same federated search market as WebFeat. Maybe Todd Miller, CEO of WebFeat can explain this to us.

2007 saw significant downward pressure in the cost of federated search services being provided to libraries with SerialsSolutions and WebFeat leading the way. It seems that both of these vendors are now making a push into the part of the library market with very limited budgets. We are seeing services from these two vendors starting at under $8000 a year.

This year also saw the emergence (at least on my radar) of several open source federated search projects. LibraryFind, developed by the Oregon State University Libraries, is built with Ruby on Rails and possesses what seems to be an interesting caching system designed to improve the speed of search results. A cursory look through the LibraryFind website only shows connector definitions for 20 information sources, all accessible only through Marc Record or Z39.50 interfaces. dbWiz is a federated search engine developed by Simon Frazer University as part of a larger suite of software tools called reSearcher used to locate and manage electronic library resources. reSearcher and dbWiz seem mostly to have been adopted at a number of universities in Western Canada. MasterKey mentioned earlier in this post is another open source federated search engine.

I am not hearing of any significant adoption of any of these federated search tools, someone please correct me if I’m wrong. In federated search, a majority of effort by providers of federated search products and services goes towards development, monitoring and upgrade of thousands of connectors to information sources. Unless a large open source effort were to get started to create and maintain connectors for one of these open source federated search tools libraries don’t have the resources or the inclination to go the open source route.

We also saw a number of acquisitions this year. Most interesting to me personally is Inxight’s acquisition (actually in 2005) of IntelliSeek’s federated search business. This was followed this summer by Business Objects acquiring Inxight for about $75 million, which was followed within a few months by an announcement that SAP was acquiring Business Objects for about $6 billion. This year also saw the completion of the acquisition of ProQuest by the Cambridge Information Group (CIG) in a transaction valued at $222 million. In 2005 CSA (part of CIG) launched MultiSearch, a federated search product in partnership with MuseGlobal. Will MultiSearch be replaced by SerialsSolutions’ 360 Search (acquired a few years ago by ProQuest) federated search product? That would be my guess.

Finally 2007 saw an increase in content providers improving access to their content for the benefit of federated search product and service providers and their users. We are seeing more XML gateways and some Web Services-based interfaces to content which is a really good thing as “screen scraping” connectors are becoming harder to build and maintain. Content providers are starting to use technologies such as AJAX in their search interfaces which make it impossible to federate these sources.

2008 promises to be an interesting year for federated search. We’ll keep you all up to date on new and significant events as they occur so please check the blog often or better yet, subscribe through our RSS feed or sign up to receive posts via email.

Best wishes for a happy and prosperous New Year to all of our readers.

Tags: federated search