Ouch! That feedback must hurt!

Check out this article at the Concordia University Library News Blog: Goodbye, BearHunt. BearHunt is (soon to be ‘was’) their federated search system. They didn’t like it and they’re not shy about telling you how much they didn’t like it. I’ve never seen so many negative comments about a federated search solution. I counted ten complaints. I’ve broken them out in my reproduction of Concordia’s article here:

Goodbye, BearHunt

QuickSearch via BearHunt will no longer be available beginning March 1st. There are a number of reasons that the library chose to discontinue this federated search service. Most important, as everyone is tightening their belts,

BearHunt came at a substantial cost while offering no new content.

Also important,

usage was low compared to cost.

When it first became available, we were excited to try a federated search, but along with other CLIC libraries and the rest of the library world,

we’ve realized that the technology needed to support the idea isn’t quite there yet.

The relevancy rankings don’t seem to work correctly or consistently,

links were frequently broken,

searches executed slowly, and

the interface left a lot to be desired.

We also felt

there was a possibility that it was misleading searchers to think they were searching ‘everything’, while in fact there are a number of both purchased and freely available resources that aren’t included in a BearHunt search.

Also,

the search is not complete in that it stops at a predetermined time interval instead of continuing until all searches have had a chance to complete.

In the end,

BearHunt and federated search as it now exists seems at odds with our mission to teach academic research skills tailored to specific situations and inquiries, using the full breadth of our resources.

We will look forward to new federated search products with anticipation, especially if they meet our criteria of completeness and have added-value in their presentation.

It’s too bad that blog articles like this one paint the picture that all federated systems suck, and that the technology is too immature to be of value. While there are some universal truths in the article (e.g. federated search times out and doesn’t get every document from every source, and federated search is slow (compared to Google, I assume)), a number of Concordia’s problems were specific to their vendor and their implementation.

Broken links shouldn’t happen. Federated search returns links to documents that the underlying search engines provide and I can’t understand why those result links would be bad. Maybe there’s an issue with a link resolver but, whatever the problem is, it’s not a general problem with federated search. Likewise, relevance ranking being crappy is not universal. Some vendors build better connectors, get better results, and rank them better. Read this article to learn what determines quality of search results. Similarly, poor user interface design is vendor specific and, depending on the implementation, customers can have influence over the application’s look and feel.

I’ve never been shy about saying that federated search is a necessary evil and not a panacea. Yes, the technology has lots of warts. But, when the only alternative is to search a few dozen sources one at a time, federated search looks really good.

If you enjoyed this post, make sure you subscribe to the RSS feed!

Tags: federated search