Today is the third of four parts of this interview. Today’s discussion includes what has made WebFeat so successful, managing a huge connector base, and the selling of WebFeat to ProQuest.
Todd is the second luminary that I recognize in this blog. You can find future and past luminary interviews in the luminary category. I invite you to nominate people who deserve to hold the federated search luminary distinction.
This is the second installment of my interview with federated search luminary, Todd Miller. Part I is here. Today, Todd answers questions about the early days of WebFeat, about WebFeat and libraries, and about WebFeat’s contributions to federated search.
Todd is the second luminary that I recognize in this blog. You can find future and past luminary interviews in the luminary category. I invite you to nominate people who deserve to hold the federated search luminary distinction.
I was fortunate to have had the opportunity to interview Todd Miller, founder of WebFeat, for this federated search luminary series. I recently published a preview of the interview. In this first installment, Todd shares wisdom that he gained early in his career; these early experiences would prepare him to start and grow WebFeat.
Todd is the second luminary that I recognize in this blog. You can find future and past luminary interviews in the luminary category. I invite you to nominate people who deserve to hold the federated search luminary distinction.
Todd Miller founded WebFeat in 1998. Ten years later, in February of this year, Todd sold the company to ProQuest. During his tenure with WebFeat, Todd made major contributions to federated search that earn him the distinction of being a luminary in the industry, along with Kate Noerr, who I interviewed recently.
I emailed Todd a set of questions. I got back seven single-spaced pages worth of responses and it was all outstanding. So, I’m going to serialize the interview into a number of parts, publishing the first on Monday. In this article I want to list the interview questions and provide excerpts of a few of Todd’s responses; overall I find Todd’s writing to be intelligent, insightful, and at times quite witty.
In reviewing the past few months worth of comments I’m realizing that there’s a goldmine of wisdom in your experiences with federated search that is not being noticed by readers who don’t follow comments. So, I’d like to bring attention to a number of your insightful comments and to respond where a response is warranted. Note: You cam subscribe to the comments feed by clicking on the smaller of the two RSS icons near the top of the rightmost sidebar on any blog page or by clicking here.
Beyond shining the spotlight on your excellent comments, a good blogger attends to his comments in a timely manner, right? I last responded to comments in a big batch post on March 31st. Gulp! Well, I’m turning over a new leaf; moving forward I’ll respond to comments as they come in and I’ll do that in the form of a response comment, or in a blog article. Until the next comment arrives - go ahead and test me - I’m responding to those lonely comments that are waiting for a response. Here are my responses to the first batch of comments.
On March 21st I wrote Is federated search ranking impaired, commenting on a blog post that commented on a presentation by Tom Wilson of the University of Alabama. My post led to several comments including a very elaborate response by Mr. Wilson himself highlighting some real problems with federated search and relevance ranking. Peter pointed out another impairment of federated search relevance ranking, one that is often overlooked or forgotten. Abe chimed in with his own detailed response, further emphasizing the challenges of relevance ranking and hinting at how they can be addressed. And, Dave explained how the University of Arizona performs relevance ranking in one of their in-house databases, and that that way of relevance ranking is well received by its audience. This was a great discussion. Thanks, everybody for your comments. I encourage everyone to read the comment thread.
There are lots of terms used interchangeably with “federated search.” I was interested to know how frequently each of these synonyms was used since I maintain a bunch of alerts related to various of these terms to keep an eye out for interesting web pages, blog posts, and events. So I set out to do some exploration with Google. Here are the eight terms (quoted phrases) I googled for (in alphabetical order) and noted the document counts for:
Six years ago I moved from the Bay Area to New Mexico to be closer to my brother Abe and to his family. For five of those years I was an employee of his at Deep Web Technologies. Now, I just write for this blog and do some project work for him. Ever since starting to work for Deep Web, and to this day, I’ve supported DOE OSTI (The US Department of Energy Office of Scientific and Technical Information) in a number of capacities. OSTI is chartered to disseminate scientific and technical information to the public, especially as it pertains to DOE’s interests. OSTI has built a number of highly visible applications for this purpose, and some of these perform federated search and use technology developed by Deep Web.
[ Editor’s note: Scott Rice, E-Learning Librarian at Appalachian State University, reviews an essay in Christopher Cox’s book about federated search. The article looks at the age old question of information literacy and whether federated search helps or hurts library patrons in their journey towards this literacy.
Given the quality of the essays in Mr. Cox’s book plus the severe lack of any books related to federated search, I highly recommend the book. You can purchase a copy of Mr. Cox’s book of essays from the publisher, Taylor & Francis, who donated the review copies, by calling their Customer Service department, Monday-Friday 9 A.M. – 5 P.M. EDT, at (800) 634-7064.
[ Editor's note: This post is from Abe. Yes, Abe and I sometimes talk about federated search over dinner. Sad, isn't it? ]
Friday night I was having dinner with Sol and somehow the conversation turned to federated search. Specifically, we got into a discussion on content providers who return results that are not very useful to a federated search engine. This is a special problem for federated search engines such as our Explorit engine that puts significant emphasis on relevance ranking of search results. If a content provider’s search engine returns results that are unranked or poorly ranked or returns results which provide little information to rank on (e.g. only a short title is returned and no snippet is returned) then the results returned by that content provider will be relegated to the back of the result list and users will not see these results.
I am honored to have had the opportunity to interview Kate Noerr for the federated search luminary series. Kate is co-founder, Chairman, and CEO of MuseGlobal, a leading supplier of content integration software. Kate, through MuseGlobal as well as in prior businesses, has been developing innovative solutions to what are now termed “federated search” problems since the 1980s. Kate is not satisfied to address only the challenges of federation. Her company considers these interrelated areas to be critically important as well: harvesting, transformation, enhancement, security, source maintenance, and multiple delivery mechanisms.
