Jun
In March I reported on an article that Barbara Quint, editor-in-chief of Information Today’s Searcher Magazine, published for DCLnews: Federated Searching
Good Ideas Never Die, They Just Change Their Names. DCLnews is one of the publications of Iris Hanney‘s business support services company, Unlimited Priorities.
Abe Lederman, Deep Web Technologies founder and president and sponsor of this blog, was quoted in this article regarding his experience with one particularly thorny aspect of federated search:
So how do federated search services handle [author searching] problems? In an article written by Miriam Drake that appeared in the July-August 2008 issue of Searcher entitled “Federated Search: One Simple Query or Simply Wishful Thinking,” a leading executive of a federated service selling to library vendors was quoted as saying, “We simply search for a text string in the metadata that is provided by the content providers - if the patron’s entry doesn’t match that of the content provider, they may not find that result.” Ah, the tough luck approach! In contrast, Abe Lederman, founder and president of Deep Web Technologies (www.deepwebtech.com), a leading supplier of federated search technology, responded about his companies work with Scitopia, a federated service for scientific scholarly society publishers, “We spend a significant amount of effort to get it as close to being right as possible for Scitopia where we had much better access to the scientific societies that are content providers. It is not perfect and is still a challenge. The best we can do is transformation.”
I reviewed Miriam Drake’s article last July.
A few days ago Barbara Quint published another article at the DCLnews Blog: Interview with Deep Web Technologies’ Abe Lederman. Barbara Quint discusses a number of topics with Abe. You’ll notice that these aren’t your run-of-the-mill superficial questions. Many of them are tough. How do you retain the human intelligence that went into building a search interface? How important is it for data to be structured? What do you think about discovery services?
One particularly interesting question was about how to handle incomplete coverage, especially when some sources let you search some fields but not others:
A user may want to search 2000-2010 and some databases may display the date, but not let you search on it; some won’t do either. Where the database doesn’t let you search on a date range but displays it, you may get results outside of the date and display them with the unranked results. How to make it clear to the user what is going on is a big thing for the future.
This is a good reminder that the devil is in the details in federated search and that quality of search results is very variable from vendor to vendor.
Here are the questions covered in the interview:
- What is your background with federated search and Deep Web Technologies?
- How do you make sure that you retain all the human intelligence that has gone into building the original data source when you design your federated searching?
- How important is it for the content to be well structured? To have more tags and more handles?
- How do you work with clients? Describe the perfect client or partner.
- How are you handling new types of data, like multimedia or video?
- How do you gauge user expectations and build that into your work to keep it user-friendly?
- How do you warn (or educate) users that they need to do something better than they have, that they may have made a mistake? Or that you don’t have all the needed coverage in your databases?
- What about new techniques for reaching “legacy” databases, like the Sitemap Protocol used by Google and other search engines?
- Throughout the history of federated search – with all its different names, there have been some questions and complaints about the speed of retrieving results and the completeness of those results from lack of rationalizing or normalizing alternative data sources. Comments?
- If the quality of the search experience differs so much among different federated search systems, when should a client change systems?
- In my last article about federated searching, I mentioned the new discovery services in passing. I got objections from some people about my descriptions or, indeed, equating them with federated search at all. People from ProQuest’s Serials Solutions told me that their Summon was different because they build a single giant index. Comments?
Read the full interview at the DCLnews Blog.
If you enjoyed this post, make sure you subscribe to the RSS feed!
Tags: federated search
