OpenTranslators announcement raises questions

Yesterday, CARE Affiliates issued a press release announcing a new product offering. A snippet of the announcement is this:

CARE Affiliates announced a new product, in conjunction with its strategic partners, Index Data and WebFeat, called OpenTranslators. OpenTranslators is intended to reshape the way libraries select and use federated search and metasearch technology. OpenTranslators will allow libraries to use the federated search interface of their choice to access over 10,000 databases using SRU/SRW/Z39.50.

I have lots of questions about the offering but here is what is clear. CARE is a company in the business of providing products and services to libraries and information centers based on open source software. Index Data develops, customizes and supports open source information retrieval software, including federated search software. WebFeat is a developer of proprietary federated search software. WebFeat has a large number of translators (connectors to SRU/SRW/Z39.50 databases). CARE, in collaboration with Index Data, has developed a gateway to provide access to these WebFeat translators.

There are a few things I don’t know for sure but strongly believe. This announcement is pretty significant. This increased ease of access to databases will give libraries more options for access to content. Enterprise search users in corporate America will be enticed to consider incorporating federated search into their search environments.

This announcement by CARE follows a trend. Just a few days ago Microsoft announced that it was acquiring Fast Search, a leading provider of enterprise search solutions. That acquisition will escalate the popularization of enterprise search and will indirectly drive the increased awareness of federated search as I believe the distinction between federated search and enterprise search is an artificial one and that customers mostly want a “search everything” solution. Before that, Microsoft announced in November that it was giving away its enterprise search software, which has federated search features. Again, popularization of enterprise search and in this case perhaps federated search as well. This CARE announcement, timed to coincide with the start of the ALA Midwinter Meeting, should create some buzz among the library community at ALA and may just make federated search enticing to many more organizations, in particular libraries, if the price is right.

If you haven’t read Abe’s year in review post you might want to. Abe wrote about Microsoft competing with Google’s search appliance and about it giving away its Enterprise-level search product with federated search features based on Open Search. Abe wrote specifically about CARE licensing WebFeat connectors for use with Index Data’s Masterkey federated search service. Abe noted the downward price pressure for federated search services to libraries. The trend is clear: federated search will become more ubiquitous and cheaper.

Now for the questions, and this press release raises many. It will take time and research and a number of posts to address them all.

1. How “open” are these OpenTranslators? When I think of open standards and open source I think of having access to modify, maintain, debug or customize the connectors and gateway. It sounds like this offering by CARE will be a subscription service. Plus, I’m sure I won’t have access to the underlying WebFeat connectors behind the gateway. So, I don’t see anything open to me about this offering.
2. How does access to content work when authentication is involved if the connectors to the SRU/SWR/Z39.50 database are WebFeat connectors and there’s a CARE/Index Data gateway in between? I’m sure WebFeat has this worked out; I’d just like to know what the mechanism is.
3. What will access cost? What is the licensing model?
4. Are there really 10,000 databases accessible from this service as advertised in the press release? I’m well aware that one publisher can provide access to a whole slew of journals. Is a database a journal or a publisher? The figure is much less impressive if it refers to accessing the offerings of 100 publishers each providing access to 100 of their journals.
5. Who is hosting access? Is there a plan to handle scaling of the service if it becomes real popular? It will become a real access problem if this service becomes immensely popular. Granted, this will be a good problem to have but nevertheless it will be a problem.

In summary, I think this announcement is important and follows a trend of growth in the federated search industry, I think the new offering will be of particular value to libraries as it will give them more options for providing federated search to their patrons, and by increasing the downward price pressure that being able to produce connectors cheaply supports it will further the popularization of the industry. I have questions. I’m interested to hear from CARE and the others in this partnership. I will do research and continue writing about this announcement as I learn more.

Tags: CARE Affiliates, federated search, Index Data, OpenTranslators, WebFeat