Measuring return on search investment

I have always found return on investment (ROI) to be a very elusive subject. Business people want to know how much revenue their investments are producing. In the federated search world, vendors want to be able to help their customers to measure their ROI. But, most of what I’ve ever heard related to ROI has been statements like “our technology will help you save time, it’ll save you money, it’ll minimize the chance that you’ll miss important documents, and we won’t fill your results list with junk.” How do you translate those vague statements into something you can measure?

Judy Luther, President of Informed Strategies, and a judge for out federated search writing contest, tackles the question of measuring ROI. She has authored an Elsevier Library Connect white paper, University investment in the library: What’s the return? A case study at the University of Illinois at Urbana-Champaign. The white paper presents an interesting approach to measuring ROI in an academic library. The approach presented may or may not work in other environments.

Here’s the simple question that motivated the ROI exploration:

At the University of Illinois at Urbana-Champaign (UIUC), Paula Kaufman, the University Librarian and Dean of Libraries, sought to identify the library’s contribution by saying “for each dollar invested in the library, the university received x dollars in return.” This statement framed the question of value from an economic perspective and guided the development of this case study.

Luther’s study …

… examined the use of citations drawn from library resources in grant proposals, the success rate for proposals, and the average grant award. The university provided institutional data on the percent of faculty who are principal investigators, their success rate with grant proposals, the amount of university grants, and the library budget. A survey was conducted with UIUC faculty that validated assumptions in the model and provided measures that confirmed the importance and frequency of citations in grant proposals, and the likelihood that the citations used in grant proposals were drawn from library resources.

Luther’s ROI study basically compared grant proposal money received (i.e. revenue) to library expenditure. The model for measuring ROI was inspired by a March, 2003 article in Information Outlook by Roger Strouse, Demonstrating value and return on investment: the ongoing imperative. The article considered three ROI-related factors:

(1) time saved by library users, (2) money saved by library users, and (3) revenue generated when the library was used.

Luther’s study included three important assumptions which were validated through a faculty survey.

• Faculty use citations in grant proposals.
• Citations are important in the grant awards process.
• Citations come from resources provided by the library.

Luther summarizes the findings of the study and provides a concrete ROI number:

The faculty participating in the survey clearly confirmed assumptions in the model, and an analysis of the responses provided three factors that serve as part of the model: 95% of responding faculty state that citations are important in securing grant awards, 94% of responding faculty use citations in grant proposals, and 94% of responding faculty obtained citations via the campus network or Library Gateway. Using these factors in the model produces a return of $4.38 in grant income to the university for every dollar invested in the library in 2006.

The study, and its approach, should perk up the ears of library software vendors and their customers. Federated search vendors would be wise to study this and other methodologies and to use measurable savings as carrots in the selling process.

[ Final note: In conducting research for this article I discovered an ALA web page with a comprehensive list of articles, books, studies and presentations specific to library ROI. ]

If you enjoyed this post, make sure you subscribe to the RSS feed!

Tags: federated search, judy luther