I’m new to the term “data federation.” How about you?

Michael Bergman, federated search luminary, just wrote on the subject, preferring the term “data mixing.” He explains the concept:

What is Data Mixing and Why is it So Hard?

As a new term there is no “official” definition of data mixing. However, I think we can consider it as generally equivalent to the older data federation concept.

Data federation is the bringing together of data from heterogeneous and often physically distributed data sources into a single, coherent view. Sometimes this is the result of searching across multiple sources, in which case it is called federated search. But it is not limited to search. Data federation is a key concept in business intelligence and data warehousing and a driver behind master data management (MDM).

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

My fur was raised when I saw Serials Solutions’ claim that their discovery service was an evolutionary step beyond federated search. I raised my concerns a couple of times: here and here. My beef isn’t with Serials Solutions as a business, it’s with their position that it’s fine to not search content that they don’t provide access to. There’s no room (yet) in their discovery service model to include access to quality content that can only be searched live, i.e. via federated search. Carl Grant joined the conversation and various people commented, making the topic a very lively one.

My concern was, and is, that libraries and research organizations would consider giving away their responsibility to select quality sources for their patrons for what I imagine to be two primary reasons: (1) library patrons don’t like to wait 30 seconds for federated search results, and (2) (possibly) cost savings. I don’t have a lot of sympathy for the Google generation. Even though I’m an American and my culture has taught me that immediate gratification is a good thing I think 30 seconds is a small price to pay to see better results. Cost I can’t speak to as I don’t have any figures.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

When I reviewed Going Beyond Google I made a mental note to try to find an inexpensive consumer-oriented guide to performing research in the deep Web. While Going Beyond Google is a great book that I highly recommend for use in LIS programs, the book is a class text and at $65 it’s not a book that is aimed at the masses.

When I learned about About.com’s $18 guide to Online Research I became very curious to see if I had found a complement to Going Beyond Google. I got a review copy from the publisher and what follows are my impressions of the book.

The Online Research book is authored by Wendy Boswell, About.com’s guide to Web Search. The book is 276 pages long and has 15 chapters plus several appendices. The book was published in 2007. While this may seem pretty current, depending on what month the book was published it might be two and a half years old. That’s getting old given the numerous references to web resources.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

Michelle Manafy, Chair of the upcoming Enterprise Summit West Conference in San Jose in November, came up with the catchy “Wonder or Waste?” title for the conference’s panel discussion about federated search.

Michelle asked me to moderate the panel, an honor I was grateful to accept. Michelle and I worked together to identify and recruit panelists and to define the topic. Here’s what we came up with:

Federated Search: A Wonder or a Waste?

Thursday, November 19, 2009
10:45 am – 11:30 am

Moderator: Sol Lederman, Federated Search Blog
Dr Peter Noerr, CTO, MuseGlobal, Inc.
Carl Grant, President, Ex Libris North America
Christopher Cox, Dean of Libraries, Western Washington University

Opinions about the value of federated search vary widely. Some view it as the optimal way to discover unified content. Others believe it to be a slow and poor substitute for searching the underlying sources. Some see it as a necessary evil and learn to tolerate it within their organizations. Come listen to our three panelists discuss their experience with numerous federated search deployments. Learn about the benefits federated search can offer as well as its shortcomings and common pitfalls. Walk away knowing if federated search is right for your organization and how to get from a blank search screen to one that delivers the results your users need.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

Daniel Tunkelang, Endeca co-founder and Chief Scientist, wrote a guest article articulating a particular problem with federated search. In the article, Daniel wrote:

But federation is no panacea, at least as it is implemented today. A federated search application brokers a query, sending it to multiple search providers (i.e., the search interfaces to a variety of content repositories), whose results it then attempts to assemble into a coherent whole. Unfortunately, since most search providers provide little more than the top-ranked result pages, federated search applications are largely reduced to assembling a unified ranking of those disparate result pages.

This functionality is significant, and I do not mean to dismiss it. But it is not enough. In particular, this approach to federation necessarily assumes a lowest common denominator of search functionality–a consequence of the requirement to evenhandedly broker among a variety of search applications that vary in the richness of their APIs.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

Search Marketing Sage published an article yesterday entitled Federated Search: Did You Know? I read the article plus one of the articles it referenced, Understanding Federated Link Building: A Primer With Examples, at Search Engine Land. I found them interesting enough to tweet them but I didn’t think about responding to them because both publishing blogs are SEO (search engine optimization) related and I didn’t think there was much of a connection between what I blog about and SEO. So, I forgot about the two articles.

To be honest, warning lights went on in my head when I noticed articles in the SEO community about federated search. My knee jerk reaction was that the articles were going to tell people how to exploit federated search engines to get spam into the search engine indexes. The articles turned out not to be spammy at all but I still didn’t have any thoughts on what I might be able to contribute about SEO and federated search. The federated search applications I write about all provide scholarly content and getting your site aggregated by such an application is next to impossible, unless, of course, you’re a university or other major institution that provides top notch content. So, I mentally filed the two articles away.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

In the past few days a number of noteworthy happenings have hit my radar. Here are three:

ScienceResearch.com debuts as the most comprehensive Deep Web science search engine

Blog sponsor Deep Web Technologies announced the official launch of ScienceResearch.com. The science research portal itself is not new. What’s new is that it now searches over 400 sources in real-time. It runs inside an Amazon Cloud to quickly address changes in demand. Amazon’s Web Services Blog reported on the launch. Paula Hane at Information Today also covered the relaunch in a good amount of detail. All of this press coincides with Deep Web Technologies’ founder Abe Lederman’s presentation today at SLA: Journey to 10,000 sources. ScienceResearch.com is a key accomplishment on the road to searching 10,000 sources at once.
Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

[ Editor's Note: This is a guest article by Daniel Tunkelang. (See his bio below.) Daniel is passionate about designing search systems that improve users' experience with information retrieval. This passion comes across very strongly in his book about faceted search, which I recently reviewed.

This article addresses a limitation with federated search that could be removed if content sources provided specific metadata to federated search engines to improve relevance ranking. Good food for thought. ]

Daniel Tunkelang is the Chief Scientist and a co-founder of Endeca, a leading vendor of search technology. Before joining Endeca’s founding team, he worked at the IBM T. J. Watson Research Center and AT&T Bell Labs. Daniel pioneered the annual workshops on human-computer information retrieval and recently published a book on faceted search. He blogs at The Noisy Channel.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

Product manager for Blogs.com and lead for their blogger training, Andy Wibbels, wrote an outstanding blog article, “What is an API?” As a programmer I know what an API is but I have a hard time explaining the concept to non-programmers. Now, Andy has done the explaining for me.

Andy’s article does a nice job of explaining without overwhelming, and his short introduction skillfully avoids going into more detail than most people want. If you’ve ever wanted to explain (or understand) the connection between mashups and APIs, or how Twitter’s massive and rapid success can be attributed to embracing APIs, then this is the article for you.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

The SLA 2009 Annual Conference is fast approaching. Blog sponsor Deep Web Technologies will have a booth at the conference. If you’re going to be attending they’d welcome your visit. Abe Lederman, founder and President of Deep Web Technologies, will be a presenter. His talk and contributed paper are titled “Science Research: Journey to Ten Thousand Sources.” The talk will be on Monday (June 15) from 3:30 to 5:00 as part of the “Adapt, Leverage and Communicate (Part I)” contributed paper session.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!