When I reviewed Going Beyond Google I made a mental note to try to find an inexpensive consumer-oriented guide to performing research in the deep Web. While Going Beyond Google is a great book that I highly recommend for use in LIS programs, the book is a class text and at $65 it’s not a book that is aimed at the masses.

When I learned about About.com’s $18 guide to Online Research I became very curious to see if I had found a complement to Going Beyond Google. I got a review copy from the publisher and what follows are my impressions of the book.

The Online Research book is authored by Wendy Boswell, About.com’s guide to Web Search. The book is 276 pages long and has 15 chapters plus several appendices. The book was published in 2007. While this may seem pretty current, depending on what month the book was published it might be two and a half years old. That’s getting old given the numerous references to web resources.

Read the rest of this entry »

Michelle Manafy, Chair of the upcoming Enterprise Summit West Conference in San Jose in November, came up with the catchy “Wonder or Waste?” title for the conference’s panel discussion about federated search.

Michelle asked me to moderate the panel, an honor I was grateful to accept. Michelle and I worked together to identify and recruit panelists and to define the topic. Here’s what we came up with:

Federated Search: A Wonder or a Waste?

Thursday, November 19, 2009
10:45 am – 11:30 am

Moderator: Sol Lederman, Federated Search Blog
Dr Peter Noerr, CTO, MuseGlobal, Inc.
Carl Grant, President, Ex Libris North America
Christopher Cox, Dean of Libraries, Western Washington University

Opinions about the value of federated search vary widely. Some view it as the optimal way to discover unified content. Others believe it to be a slow and poor substitute for searching the underlying sources. Some see it as a necessary evil and learn to tolerate it within their organizations. Come listen to our three panelists discuss their experience with numerous federated search deployments. Learn about the benefits federated search can offer as well as its shortcomings and common pitfalls. Walk away knowing if federated search is right for your organization and how to get from a blank search screen to one that delivers the results your users need.

Read the rest of this entry »

Daniel Tunkelang, Endeca co-founder and Chief Scientist, wrote a guest article articulating a particular problem with federated search. In the article, Daniel wrote:

But federation is no panacea, at least as it is implemented today. A federated search application brokers a query, sending it to multiple search providers (i.e., the search interfaces to a variety of content repositories), whose results it then attempts to assemble into a coherent whole. Unfortunately, since most search providers provide little more than the top-ranked result pages, federated search applications are largely reduced to assembling a unified ranking of those disparate result pages.

This functionality is significant, and I do not mean to dismiss it. But it is not enough. In particular, this approach to federation necessarily assumes a lowest common denominator of search functionality–a consequence of the requirement to evenhandedly broker among a variety of search applications that vary in the richness of their APIs.

Read the rest of this entry »

Search Marketing Sage published an article yesterday entitled Federated Search: Did You Know? I read the article plus one of the articles it referenced, Understanding Federated Link Building: A Primer With Examples, at Search Engine Land. I found them interesting enough to tweet them but I didn’t think about responding to them because both publishing blogs are SEO (search engine optimization) related and I didn’t think there was much of a connection between what I blog about and SEO. So, I forgot about the two articles.

To be honest, warning lights went on in my head when I noticed articles in the SEO community about federated search. My knee jerk reaction was that the articles were going to tell people how to exploit federated search engines to get spam into the search engine indexes. The articles turned out not to be spammy at all but I still didn’t have any thoughts on what I might be able to contribute about SEO and federated search. The federated search applications I write about all provide scholarly content and getting your site aggregated by such an application is next to impossible, unless, of course, you’re a university or other major institution that provides top notch content. So, I mentally filed the two articles away.

Read the rest of this entry »

In the past few days a number of noteworthy happenings have hit my radar. Here are three:

ScienceResearch.com debuts as the most comprehensive Deep Web science search engine

Blog sponsor Deep Web Technologies announced the official launch of ScienceResearch.com. The science research portal itself is not new. What’s new is that it now searches over 400 sources in real-time. It runs inside an Amazon Cloud to quickly address changes in demand. Amazon’s Web Services Blog reported on the launch. Paula Hane at Information Today also covered the relaunch in a good amount of detail. All of this press coincides with Deep Web Technologies’ founder Abe Lederman’s presentation today at SLA: Journey to 10,000 sources. ScienceResearch.com is a key accomplishment on the road to searching 10,000 sources at once.
Read the rest of this entry »

[ Editor's Note: This is a guest article by Daniel Tunkelang. (See his bio below.) Daniel is passionate about designing search systems that improve users' experience with information retrieval. This passion comes across very strongly in his book about faceted search, which I recently reviewed.

This article addresses a limitation with federated search that could be removed if content sources provided specific metadata to federated search engines to improve relevance ranking. Good food for thought. ]

Daniel Tunkelang is the Chief Scientist and a co-founder of Endeca, a leading vendor of search technology. Before joining Endeca’s founding team, he worked at the IBM T. J. Watson Research Center and AT&T Bell Labs. Daniel pioneered the annual workshops on human-computer information retrieval and recently published a book on faceted search. He blogs at The Noisy Channel.

Read the rest of this entry »

Product manager for Blogs.com and lead for their blogger training, Andy Wibbels, wrote an outstanding blog article, “What is an API?” As a programmer I know what an API is but I have a hard time explaining the concept to non-programmers. Now, Andy has done the explaining for me.

Andy’s article does a nice job of explaining without overwhelming, and his short introduction skillfully avoids going into more detail than most people want. If you’ve ever wanted to explain (or understand) the connection between mashups and APIs, or how Twitter’s massive and rapid success can be attributed to embracing APIs, then this is the article for you.

Read the rest of this entry »

The SLA 2009 Annual Conference is fast approaching. Blog sponsor Deep Web Technologies will have a booth at the conference. If you’re going to be attending they’d welcome your visit. Abe Lederman, founder and President of Deep Web Technologies, will be a presenter. His talk and contributed paper are titled “Science Research: Journey to Ten Thousand Sources.” The talk will be on Monday (June 15) from 3:30 to 5:00 as part of the “Adapt, Leverage and Communicate (Part I)” contributed paper session.

Read the rest of this entry »

OSTI (the US Department of Energy Office of Scientific and Technical Information) recently joined YouTube and released a couple of short videos pertaining to the Deep Web. Disclosure: I consult for OSTI.

Deep Web Video

To make the web work better for science, OSTI has developed state-of-the-art technologies and services including a deep web search capability. The deep web includes content in searchable databases available to web users but not accessible by popular search engines, such as Google. This video provides an introduction to the deep web search engine.

Read the rest of this entry »

I recently discovered a video at the Federal Library & Information Center Committee (FLICC) web-site titled Federated Search Technologies. The video (divided into two pieces) is over two hours long. It includes three presentations from members of organizations who have implemented federated search solutions:

• Betty Day, University of Maryland—MetaLib (ExLibris)
• Karen Huffman, National Geographic Society—AGent (AutoGraphics)
• Tamas Doszkocs, National Library of Medicine—ToxSeek

Read the rest of this entry »