In May, search consultant Avi Rappoport delivered a presentation at the Enterprise Search Summit: Federated vs. Aggregated Search Architectures.

Avi Rappoport is an enterprise search consultant, helping companies improve search engine functionality for websites and intranets. She has a degree from UC Berkeley’s (then) School of Library and Information Science and spent 10 years in software development before becoming a search consultant. She is the editor of SearchTools.com and a frequent speaker and author, providing a strong focus on search usability in the broadest sense and sharing her conviction that search engines can always be better.

Avi created a web page with a summary of and links to a couple of versions of her presentation.

I greatly appreciate Avi’s consideration of the pluses and minuses of federation aggregation (i.e. discovery service) in a world that is often polarized about one approach being better in all cases.

My research for this presentation indicated that each is useful in specific circumstances (I know, no surprise there). Many data sources are obviously best accessed by one or the other, but it’s the corner cases that are tricky. Aspects to consider include:

• size of the content in the source
• how often your users need that content
• content change rate
• importance of real-time access control permissions changes
• content licensing rules
• available tools for indexing / querying
• difficulty of extracting and indexing
• quality of the internal search engine
• difficulty of sending queries and receiving results

The final slide has some sage advice:

Be open-minded, analyze the benefits of each approach for each data source.

One size does NOT fit all.

[ Editor's Note: This is a very touching article by Nena Moss first published in the OSTI Blog. My dad suffered with Alzheimer's for a number of years before he died so I can relate to Nena's experience. Disclaimer: I have been paid to support OSTI in a number of capacities for the past eight years. ]

My mother died in March 2010 after a 15-year battle with Alzheimer’s, so I pay particular attention to news about this dreadful disease. A recent New York Times article caught my eye: “Sharing of Data Leads to Progress on Alzheimer’s.”

How did sharing data lead to progress on Alzheimer’s? A collaborative effort, the Alzheimer’s Disease Neuroimaging Initiative, was formed to find the biological markers that show the progression of Alzheimer’s disease in the human brain. The key was to share all the data, making every finding public immediately – “available to anyone with a computer anywhere in the world.”

Read the rest of this entry »

“Federated fetching” is a new term to me. I discovered it at Srinivas Reddy’s Weblog, referencing the O’Reilly book, Beautiful Data:

When we deal with web scale data ‘discoverability’ of information is key. While ‘web search’ provides a lot of value today what we really need is to enable ‘data find data’. I like the differentiation in the book between ‘federated search’ and ‘federated fetch’. The latter needs adaptive systems that can discover new data correlations based on user context and new data collected.

This reference got me curious. Was the Web buzzing with discussion of federated search vs. federated fetch? Not exactly, according to Google, although there are 740 references to the phrase but only 24 of them are considered unique enough for Google to display. Interestingly enough, the first reference is to Jeff Jonas “When Federated Search Bites” article which I wrote about a month ago.

Once a directory reveals a pointer, you can go fetch it. Federated fetch does scale.

Google Books provides the term in the context of the Beautiful Data book:

So, federated fetch is the “end game,” if I understand the concept correctly. It’s what you get when, for example, a link resolver gets you to the full text copy of a book you can actually read.

There you have it, a new phrase I learned today.

Early this year O’Reilly published Search Patterns, by Peter Morville and Jeffery Callender. This is Morville’s fourth information/search-related book. Search Patterns addresses the intersection of user interface and search.

Search Patterns is an absolutely outstanding book. I don’t get excited about search-related books very often but this one totally captivated me. O’Reilly sent me a review copy some months ago. It sat in a pile until I started seeing reviews and references to the book on the Web. The press prompted me to open the book.

The first thing I noticed in flipping through the book was the many high-quality color screen shots and illustrations. Plus, Search Patterns is printed on glossy paper to enhance the visual elements of the book.

At 173 pages (plus index) and a nice balance of text and images, Search Patterns is, at the surface, a quick read. But, there are numerous gems throughout the book so allow yourself plenty of time to read (and reread) sections that draw you.

Read the rest of this entry »