My radar (Google Alerts) pointed me this morning to this article by Barbara Quint at Information Today. My first response to “EBSCO Exclusives Trigger Turmoil” was “What a mess!” Quint shares the saga of EBSCO and Gale lobbing volleys at each other during the ALA Midwinter meeting. EBSCO announces new acquisitions that were ‘exclusive to EBSCO for the library “marketspace.”‘ Major competitor Gale issued a letter to the library community urging “librarians to get involved in opposing publishers granting exclusives, at least to EBSCO.” Read Quint’s article for all the gory details.

If you’re a librarian running or contemplating a discovery service, how do you feel? EBSCO has some new content I assume is going to become available via their EBSCO Discovery Service and some content is going to disappear from Gale’s holding which I assume means it will disappear from Serials Solutions’ Summon discovery service which includes Gale as a major participant.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

Blog sponsor Deep Web Technologies built a federated search tool for Stanford University. I was involved with the first prototype and I’m proud of what the Stanford/Deep Web Technologies partnership has accomplished. Stanford’s Grace Baysinger - head librarian and bibliographer for the Swain Library of Chemistry and Chemical Engineering - provides a nice overview of the search tool in the “Speaking of Computers” newsletter. And, Deep Web Technologies’ account manager, Debbie Taylor, shares a little bit of her involvement with the project as well on the Deep Web Technologies Blog.

My involvement with the Stanford federated search tool was multi-faceted. I worked with Stanford to understand their requirements. I also worked with Deep Web Technologies’ connector team to build the source interfaces Stanford wanted. I even used my limited web design skills to create the HTML that was used in the very first prototype. There were many conversations and many details that were worked out during a number of phone conversations. When I left my full time position of engineer and manager at Deep Web Technologies I lost touch with the folks at Stanford who led the way in introducing federated search to their university. I hear that they’re very happy with their search tool. I’m proud to have had a role in the building of that tool.

If you enjoyed this post, make sure you subscribe to the RSS feed!

I recently purchased a copy of How UK academic libraries choose metasearch systems from Emerald Insight. I was reading the article on my way to writing a review of it when I was struck by this statement:

Advice comes from successful, as well as unsuccessful implementations. One of the most common pieces of advice is that libraries should remember to treat metasearch as a service, not a product, and to ensure that there is adequate post-installation vendor support.

What struck me was that the statement, which seemed so obvious to me, was deemed important enough by the authors to state explicitly. Of course, federated search is a service. Whether or not you host federated search on your own servers with your own staff managing the server hardware, software, and application there’s plenty of service that someone will have to perform, and it’s on an ongoing basis. Off the top of my head, here are some service aspects to federated search:

• Building connectors - the more work that goes into building connectors the better the search experience
• Monitoring/disabling/re-enabling connectors when sources become unavailable and when they come back
• Updating or rewriting connectors when sources change their search interface
• Updating the application when the vendor provides new maintenance or feature releases
• Customizing the application to suit your needs

None of these tasks are easy. Each requires vendor support or time from your staff.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

Abe pointed me to an article, “An info island or the kitchen sink: when you think about federated search engines, think about what you’ll include in the searches.” While the article is a bit dated (2007) it has this air of timelessness to it.

The author, Richard Todd, considers three paths to source selection:

In the course of implementing a federated search engine at my organization, I eventually recognized three fundamental approaches for determining which of the available sources to include in our system. I call these three guiding concepts the kitchen sink approach, the Easter Island approach, and the gatekeeper approach.

Without reading further can you guess what these three approaches are? Abe told me that he was contacted by someone who threw out these terms as if Abe should be familiar with them. He wasn’t. Neither was I.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

Five noted experts in the federated search are serving as volunteer judges for our second annual contest. In the next few weeks they will have selected the winners.

Here are the bios of the judges.

Abe Lederman has 25 years of experience in computer software engineering. He began his career with Hewlett Packard and was then recruited to become a founding member of Verity, a startup pioneer in the field of search engine technology. In 1994, Abe left Verity and later founded Innovative Web Applications (IWA), a software consulting firm primarily serving Los Alamos National Laboratory (LANL). While consulting with LANL and the U.S. Department of Energy’s Office of Scientific and Technical Information (OSTI), Abe came to realize the enormous potential for federated search technology to accelerate the diffusion of knowledge. He founded Deep Web Technologies in 2002. Abe holds Bachelor of Science and Master of Science degrees in Computer Science from the Massachusetts Institute of Technology.

Todd Miller Todd Miller is President and Founder of gwabbit, maker of the award-winning series of gwabbit semantic contact capture and management products. Declared a “Head-slappingly simple solution to a grating problem” by BusinessWeek, gwabbit has won numerous industry awards, including Inc Magazine’s list of “Top Smartphone Apps for 2009,” as well as winner of the 2010 CES Mobile Apps Showdown, and a DEMOgod award — the highest honor bestowed by the nation’s premier technology conference for launching startup companies.

Prior to launching gwabbit, Todd founded the federated search company, WebFeat, which was sold to ProQuest in February 2008. WebFeat was used by thousands of libraries and information centers worldwide. Todd was also president of Knight Ridder Information’s SourceOne subsidiary and also worked at Ziff Davis’ Information Access Company subsidiary, where he developed and managed the highly successful InfoTrac product line.

Miller holds four patents in the field of search technology and authentication and session management. He has received awards from the Gartner Group, Reed Publishing, The DEMO Conference, and others. Miller has been a featured speaker on IBM’s eBusiness Tour and was featured in IBM’s “Success Stories” campaign. In his abundant spare time, Miller is a competitive horseback rider. He resides in Carmel Valley, CA with his wife and their rescue dogs, horses, and other quadrupeds.

Helen Mitchell has enjoyed working in the Enterprise Search arena for 20+ years. She currently has 2 main roles - 1) As Principal for Team Technology, Inc. leading an Enterprise Search Requirements project for the FDA as well as a search implementation for FDIC and 2) As Principal of her own IT Consulting Firm, Enterprising Solutions. Helen works closely with public and private sector clients to develop search strategies and solutions across disparate data sources and the deep web. She’s conducted Federated Search Webinars and will present 2 Continuing Education Federated Search courses at the Special Library Association’s Annual Conference. Being vendor neutral, she utilizes the ‘best’ technologies for clients to reduce costs, improve quality and efficiency, reduce ‘pain points’ and achieve a positive search experience. She retired from FDA in 2008 after 32 years of service where she led one of the largest enterprise search implementations among Civilian Federal Agencies. She facilitated integration of more than 20 million documents and applications into this system utilizing federated search connectors to improve their content findability. Helen is a frequent speaker at conferences and seminars on search technologies.

Richard Tong is the Founder and President of Tarragon Consulting Corporation. Tarragon develops custom solutions to the critical knowledge management and content analysis problems faced by Government and Industry. Tarragon uses its in-depth business and technical expertise to support organizations in the creation of high-value applications for customers, suppliers, and employees.

Prior to starting Tarragon, Dr. Tong co-founded, and was CTO at, Sageware, Inc.. Prior to Sageware, Dr. Tong was Director of Advanced Technology at Verity, Inc.

Dr. Tong has over twenty-five years experience in the design and deployment of advanced information systems for Government and Industry. He has extensive consulting experience with Booz Allen Hamilton and with Advanced Decision Systems, where he directed the R&D team that created the information retrieval capability that led to the forming of Verity.

Dr. Tong received his Ph.D. from the University of Cambridge, England, and was a NATO Research Fellow at the University of California, Berkeley.

Dr. Tong serves as a technical advisor to a number of companies, and provides expert technical assessments to support investment and M&A decisions. Dr. Tong has published extensively and is currently a member of the TREC Program Committee.

Dr. Walt Warnick is Director of the U.S. Department of Energy Office of Scientific and Technical Information (OSTI); he directs the agency’s scientific and technical information operations. Dr. Warnick embraces the opportunities offered by the web to accelerate the spread of knowledge about science and technology. To this end, he has championed aggressive efforts to capitalize on technological advances to develop and provide state-of-the art products and services for sharing knowledge. Dr. Warnick and his colleagues continuously work to further advance web search technology.

Dr. Warnick was elected Fellow of the American Association for the Advancement of Science (AAAS) in 2005 “for leadership in the federal scientific information community and for contributions to the conceptualization, development and implementation of innovative programs that significantly advance access to government information.” He earned his Ph.D. and M.S. in Mechanical Engineering from the University of Maryland and his Bachelor of Engineering Science from The Johns Hopkins University.

If you enjoyed this post, make sure you subscribe to the RSS feed!

RUSQ (The Reference & User Services Quarterly journal) has published an article: Student Feedback on Federated Search Use, Satisfaction, and Web Presence: Qualitative Findings of Focus Groups. Here’s a piece of the introduction:

Illinois State University’s Milner Library conducted focus groups in the summer and fall of 2007 as part of its user-centered approach to implementing a federated search engine. The feedback supplemented the comments from usability testing conducted in the summer of 2006. The purpose of the focus groups was to learn about students’ use of and satisfaction with the federated search engine and to gather their ideas on how to incorporate it into the library website.

I’m always delighted to discover articles about how federated search is used and received so I consider this article, published in 2009, to be a good find. The PDF version of the article is nine pages long, long enough to be substantial yet short enough to be an easy read.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

Multilingual federated search is a big deal for a couple of reasons. First, no one has done it up to now. Yes, Google just added translation into its universal search. And, no doubt Bing will follow suit. But, being able to search the quality scientific and technical information that sometimes is only available via federated search, and doing it in foreign languages, is important.

The second reason that multilingual federated search is so important is because China, Japan, Russia, and other nations produce large volumes of research output. As the world shrinks we can’t afford to ignore the non-English literature. In a blog article the author noted that Thomson Reuters highlighted the importance of China’s research output on the basis of sheer volume :

According to citation analysis based on data from Web of Science, China is ranked second in the world by number of scientific papers published in 2007. Scientific’s World IP Today Report on Global Patent Activity 2007 reported that China almost doubled its volume of patents from 2003 to 2007, and looks set to become a strong rival to Japan and the United States in years to come.

The bottom line: federated search is about research and research is global.

Read the rest of this entry »

If you enjoyed this post, make sure you subscribe to the RSS feed!

Deep Web expert Marcus Zillman makes freely available a tremendous wealth of resources for people who want to understand the Deep Web, particularly how to search it. His recently published article, Deep Web Research 2010, reminds me of how prolific Mr. Zillman is and that the only fault I can find in Mr. Zillman’s work is the overwhelming volume of resources he provides.

Deep Web Research 2010 provides a list of perhaps 200 resources in 10 categories:

1. Articles, Papers, Forums, Audios and Videos
2. Cross Database Articles
3. Cross Database Search Services
4. Cross Database Search Tools
5. Peer to Peer, File Sharing, Grid/Matrix Search Engines
6. Presentations
7. Resources - Deep Web Research
8. Resources - Semantic Web Research
9. Bot Research Resources and Sites
10. Subject Tracer Information Blogs

There are many fascinating links to follow. I did notice that some of the links were either to sites where one would need to subscribe to a service or buy an article and that some of the links were old and the content could be out of date. But, those concerns aside, one could spend days or weeks getting lost in readings about the Deep Web.

If you enjoyed this post, make sure you subscribe to the RSS feed!

The commercial investing and personal finance web-site, The Motley Fool, published a brief article about the deep Web. No, the article does not dive very deep in its assessment of the deep Web. It’s a very cursory introduction. But, it’s great to see such coverage at all in such a well respected investment site.

The article tells two ways that investors can make money from the deep Web: buy stock in companies that mine deep Web data, and use the deep Web in your investment research.

I strongly agree with this paragraph:

If you have a keen interest in [the deep Web], it is also possible to invest in technology that is going to indirectly help open up this ‘invisible web’. Let me cite one area that I think holds promise. I reckon analytics programs that can successfully measure the response of ad campaigns to the deep web community or successfully trace browsing habits in these areas will make their owners/investors a lot of money.

Yes, I believe sentiment mining is in its infancy and will only grow in its importance to marketers. If you have a few minutes you might enjoy the article.

If you enjoyed this post, make sure you subscribe to the RSS feed!

Last reminder. Contest entries are due today.

If you enjoyed this post, make sure you subscribe to the RSS feed!