Abstract
Return on investment (ROI) is one method of measuring the value of a library's e-journal collection. In an international study designed to test an ROI formula developed as a case study at the University of Illinois, ROI of the value of e-journals to grants income was found to vary depending on the mission and subject emphasis of the institution. Faculty members report that e-journals have transformed the way they do research, including making them more productive and competitive. Future studies will examine ROI beyond grants income and beyond the value of e-journal collections.
INTRODUCTION
In an era of decreasing resources and increasing calls for accountability, academic libraries all over the world face the challenge of demonstrating and quantifying their value to their funders and to all of their stakeholders. Return on investment (ROI) is one approach to meeting this challenge.
A series of research studies are testing ways to calculate ROI in academic libraries. These studies are international in scope, including participants from North America, Europe, Africa, and Asia and include multiple phases of increasing complexity. A characteristic of these studies is that they involve multiple types of institutions, both as participants and as funders. Public and private research universities, as well as university consortia and professional organizations are participating, with funding from the private sector and public sector grant funding.
The economic crisis that has put pressure on budgets is one reason for the pressure to demonstrate the value of the library. Couple that with a downturn in some perceptions of the value of the library and we have what can be called a “value gap.” The value gap occurs when the cost of library collections and services increases over time, while the perceived value declines. This can be demonstrated by plotting the Association of Research Libraries (ARL) expenditure data over time with the percent of respondents to recent Ithaka studies that say the role of the library as a gateway or starting point for research is essential or important to them (see ).Footnote 1 , Footnote 2 (It should be noted that the perception of other roles is increasing, while the perception of the role as a gateway is decreasing.)
FIGURE 1 Value gap emerges: ARL expenditures versus percentage of survey respondents rating the role of the library as a gateway as essential.
ROI is one approach to meeting the challenge of demonstrating value. The basis of ROI studies is to quantify and demonstrate the library's economic value to the institution. For every euro or dollar or yen spent on the library, the university receives euros or dollars or yen back in the form of additional grants income or donations, or long-term value to the community from an educated workforce, more productive faculty, and more successful students and graduates. ROI must be articulated within the mission and objectives of the specific institution. Any ROI project must be measurable, replicable both in the same institution and in others, and meaningful (and interesting) to funders.
This is not a simple thing. An international team is working on this topic over several phases of increasing complexity. Phase 1 was a case study at the University of Illinois that tested just one aspect of ROI, which was the value of the library in the grants process through proposals, the return in grant funding, and grants reporting. It was funded by Elsevier and led by consultant Judy Luther. Phase 1 focused on grants income because it has a definable source of income, is a priority in research libraries, and has identifiable data that can be collected. Phase 2 is underway now. It takes the same method of looking at grants and grant funding, but expands it internationally to eight more institutions in eight countries to test how replicable the methods and findings of phase 1 are in a wider context. Grants and grant funding are relatively easy, but of course only give a very partial view of the value of the academic library, so phase 3 will go beyond grants. Phase 3 proposes to increase the focus dramatically by looking at multiple values of the university libraries, multiple ways that libraries provide returns on investment, and multiple measures. Most of the phase 3 testing will be done in three universities in the United States, but with the Association of Research Libraries participation to develop ROI tools that can be tested in other universities.
PHASES 1 AND 2
Essential at the beginning of the ROI process were meetings with the top-level administrators who provided insights into what they value to support the mission of the university and where the library might fit into that. The input from provosts, chancellors, or vice presidents helped to direct the type of data collected and the ways to present it that would speak to administration. The interviews were also a source of qualitative data that illustrate the value of the library. Focusing on how the library supports the mission and goals of the broader institution was an essential component of both phase 1 and phase 2 ROI studies.
The University of Illinois case study provides insights into what administrators value, because administrators there are similar to those at all research universities. They want the faculty to make research contributions by focusing on new intellectual directions and they try to help faculty by strengthening interdisciplinary work, finding resources, building connections, and increasing the efficiency and impact of research. Improving the intellectual climate and research reputation of their university in turn helps attract and retain productive faculty, which in turn attracts funding and increases the research reputation. These interviews helped direct phases 1 and 2. In phase 1, the interviews with administrators helped focus the further ROI efforts on faculty research productivity and grants funding.
It is important to stress and get buy-in from the university administrators that the library be viewed as an asset and not as a cost center or as overhead. The aim of ROI is to establish a relationship between the library and its university that could be expressed in quantifiable terms and that would satisfy administrators. To do this, the library needs to be viewed as an asset within the definition of ROI, where income is generated as a proportion of the amount invested in the asset. From that vantage point, the relationship between the library and the income generated through its use might be formulated.
Isolating the relationship between the library and the grant funding process is a good place to start in calculating a formula for ROI. Faculty generate income for the institution through grants and use the library and collections in grant proposals. The first challenge is to determine the proportion of the grants income that can be assigned to the library collections.
To begin, the grants process itself has to be considered. While the researchers are first and foremost in this process, the resources offered through the institutional library are pivotal to their endeavors. Much of the research process depends upon access to information, and most of that information is provided by the library. The library has a role in all aspects of the research grants process, especially providing resources to conduct research, write articles, reports, and proposals. The key is to tie the library directly to obtaining grants.
According to Paula Kaufmann, Dean of Libraries of the University of Illinois at Urbana-Champaign (UIUC), at the beginning of phase 1 it became evident that they needed to clearly articulate the purpose of the study, specifically what it was and was not attempting to do. The ultimate goal was to demonstrate that library collections contribute to income-generating activities, by quantifying a return on university's investments in its library and focusing on the library's role in externally funded research process. According to Kaufmann, the study had to be careful about what it was, but also what it was not. To avoid any unintended and unnecessary misinterpretations of intent, it must be articulated that this study is not trying to claim an allocation back to library, is not solely a budget argument, is not a cost or time savings exercise, and is not creating a predictive model.
Several high profile ROI studies have been conducted over the years in public library settings. Both the returns and the investments are quite different in those cases, because the downstream effects of civic economic growth is critical in public library ROI. How the library contributes to a city's and region's economy due to the resources and contributions of the public library figure greatly into ROI studies of public libraries. Two reports in public libraries in the United States have gotten quite a bit of attention: Worth Their Weight, published by the Americans for Libraries Council,Footnote 3 and Making Cities Stronger, published by the Urban Libraries Council.Footnote 4 In addition, major studies in Ohio and Florida calculated that for every dollar invested in a public library, the return to the community is between $3.81 and $6.54 dollars.Footnote 5 , Footnote 6 Library Research Service has a simplistic ROI calculator for public libraries.Footnote 7
Judy Luther, the consultant for phase 1, was inspired by an article by Roger Strouse of Outsell on special library ROI that could be adapted to academic libraries. The article was the result of a value study in a corporate library setting undertaken by Outsell in the early 2000s. The subsequent article published in 2003 showed three ways of measurement: time saved by library users, money saved by using the library, and revenue generated with the assistance of the library.Footnote 8 It is the third that is the focus of phases 1 and 2. Other measures will be added in phase 3.
In phases 1 and 2 quantitative and qualitative data were gathered directly from tenure-track faculty through a survey. Data on the number and value of grant proposals and grants and library budget figures were obtained from the university research and budget offices. Interviews with administrators both guided what is important and provided additional qualitative input on the value of the library.
The phase 1 study at UIUC, based on faculty applying for grant funding, and factoring in the survey responses, showed that more than 78 percent of faculty with grants use citations in their proposals and just over 50 percent of grants awarded are from proposals with citations obtained from the library. Data from their research office show that the average grant income at UIUC is just about $64,000. Multiplying these three numbers gives an average grant income generated through the use of the library of just over $25,000. Multiplying this number by the number of grants expended in a year at UIUC and dividing that by the total library budget yielded a return on investment of $4.38 for every dollar invested in the library. This is just the value in grants; total ROI will be greater.
The phase 1 case study was just a first step. Phase 2 keeps the focus on ROI of the e-journals collection for grants, but extends the phase 1 coverage to eight more institutions in eight countries to examine similarities and differences across countries and types of institutions. The University of Illinois at Urbana-Champaign is a major research university with a long tradition of grants. Phase 2 tests to see if the method works in other universities and is applicable in other countries. Expanding to other institutions and a variety of countries posed some challenges such as differences in terminology (academic ranks, expenditures, or income); variations in data kept and for how long; differences in fiscal year, academic year, calendar year and even hemisphere; differences in languages; and the availability of data, the biggest problem in most institutions.
Surveys included demographic questions and questions about the number of proposals, grants, and papers written; the importance of the library electronic collections to this process; and the number of citations and readings per citation. In addition, we asked the open-ended question: “How has access to electronic resources available over the university computer network and/or from the university library changed the way you work? Please comment.” Faculty contributed many comments about the role of references in grant proposals. Many addressed the value of electronic resources to their work or the value of citations to research literature in proposals. These comments are typical:
| • | “A sure way to kill a proposal is not to give proper credit or to not update new developments.” (U.S. University) | ||||
| • | “With the current workload, I could not continue with research without the convenience of access from my own computer.” (South African University) | ||||
| • | “You have access to many more articles and although you do not read them completely, you are more aware of what is going on in the field.” (Western European Research Institute) | ||||
| • | “Access has made collecting research resources infinitely more efficient, and facilitated interdisciplinary research.” (U.S. University) | ||||
They also commented on how e-journals have increased their productivity, leading to more grants submitted and changes in the way work is done. For example:
| • | “I could not submit as many grants. With grant funding levels at 4–6% of submitted proposals I would not have achieved my current funding level.” (U.S. University) | ||||
| • | “I guess that on average the online access saves me more than 10 hours per week.” (Western European Research Institute) | ||||
| • | “The convenience of desktop delivery has improved my efficiency and dare I say it my ability to be a better researcher and teacher.” (Hong Kong University) | ||||
| • | “My productivity would drop at least four fold if I had to go to the library for all my needs.” (U.S. University) | ||||
| • | “Completely changed the way I work by increasing my productivity. I … spend more time reading [articles].” (U.S. University) | ||||
| • | “The task of finding the most pertinent articles on a new topic used to take a full afternoon. The same work can now be completed in 15 to 30 minutes.” (U.S. University) | ||||
Overall, the survey respondents described a dependency on the library for their research and grant proposal needs that can be translated into an economic value. Again, these comments are typical:
| • | “Such access has become an essential research tool.” (Japanese University) | ||||
| • | “It would be impossible to be competitive internationally without electronic access to publications.” (U.S. University) | ||||
| • | “It has helped me open or discard lines of research at the very beginning by knowing what other researchers have published or are soon going to publish.” (Western European Research Institute) | ||||
| • | “Electronic access greatly improved and simplified work for publication, preparation of proposals, and research work with students.” (U.S. University) | ||||
This type of qualitative evidence is important in telling the ROI story, in addition to the actual calculation. The comments of the faculty match well with the values and priorities of the administrators, showing that access to a good online library helps retain faculty and helps them be more productive and do better research.
In the first look at phase 2 surveys, references are considered essential in all the institutions surveyed, although how important they are does seem to vary somewhat from institution to institution. References were rated as essential, very important, or important to grants by a range of 71 to 98 percent of faculty respondents.
The average of the reported number of citations per proposal also varies, from twenty to forty-six citations per proposal, as does the average percentage of citations that were accessed through the library's online system or university network. The mode percent of citations from the online library ranged from 50 to 99 percent. For every article cited, faculty members reported reading many more articles, so the value of the library collection is higher than just those selected for citing.
Actual ROI calculations vary. This variation has much to do with the purpose of the institution, with the high being for a pure research institute and lower for teaching/research universities in countries without a high number of competitive grant funds. Caution is recommended when comparing ROI across institutions.
There are also differences based on subject discipline, which may account for the differences between institutions depending on the degrees they offer or relative size of subject disciplines. At one institution, for example, over half of all respondents in the sciences, social sciences, and health fields say that between 75 and 99 percent of the items they cite in grant proposals come from the library's online system, while only 7 percent of arts and humanities faculty have this high of a percentage. We need to look at this subject difference closely and build it into future ROI calculating tools.
PHASE 3 AND CONCLUSION
Of course, grants and grants income are not the only ROI of the academic library or the only value of the library. Phase 3, which we hope to do next year, will broaden the focus to examine other functional areas, including all aspects of research, teaching, and learning, and social/professional areas. Also, ROI is not the only value measurement; ROI and other outcomes will be measured in many ways.
ROI is necessary not only to measure current values of the library to the institution. ROI measures can also help librarians guide change and priorities into the future. New scholarly endeavors, such as the library's role in e-science, collaborative scholarship, and institutional repositories cut across all of the traditional functional areas. Measurements must be of sufficient granularity as to allow flexibility.
The major challenge for phase 3 is to develop downstream measures for the outcomes of access to the library collections and services. Within each functional area of research, teaching, and service, the further downstream the more difficult the measure is. Libraries can and do measure inputs and outputs; outcomes and ROI are the real challenge. The early stages of phase 3 will be to decide which of these to measure and how to measure them. Phases 1 and 2 show that academic library collections help faculty be productive and successful; libraries help generate grant income; and electronic collections are valued by faculty and bring value to the university.
ROI is only one method for measuring the value of a library's collections and services. The benefit of multiple methods is that numbers in and of themselves rarely tell the full story. Interviews and surveys allow the faculty to tell their story of how they use the library in their grant proposal process, in teaching, and in their research and work life. What we hope to show as the studies progress is that the library's products and services help faculty be successful, help students be successful, and generate both immediate and downstream income that provides good return for the investment.
FURTHER READING: ACADEMIC LIBRARIES
Housewright, Ross, and Roger, Schonfeld. Ithaka's 2006 Studies of Key Stakeholders in the Digital Transformation in Higher Education. N.p.: Ithaka, 2008. http://www.ithaka.org/research/Ithakas%202006%20Studies%20of%20Key%20Stakeholders%20in%20the%20Digital%20Transformation%20in%20Higher%20Education.pdf (accessed July 11, 2009). | |||||
Jones, D. Yvonne. “How Much do the ‘Best’ Colleges Spend on Libraries? Using College Rankings to Provide Library Financial Benchmarks.” College & Research Libraries 68, no. 4 (July 2007): 343–351. | |||||
Kaufmann, Paula. “The Library as Strategic Investment: The University of Illinois ‘Return on Investment Study’ from the Director's Perspective.” Paper presented at the LibraryConnect Seminar, Tokyo, Japan, July 16, 2008. http://www.ideals.uiuc.edu/bitstream/handle/2142/8770/LC08_ROILibrary %20Directors%20Tokyo.pdf?sequence (accessed July 11, 2009). | |||||
Luther, Judy. University Investment in the Library: What's the Return? A Case Study at the University of Illinois at Urbana-Champaign. San Diego, CA: Elsevier, 2008. http://libraryconnect.elsevier.com/whitepapers/0108/lcwp010801.html (accessed July 11, 2009). | |||||
Mezick, Elizabeth M. “Return on Investment: Libraries and Student Retention.” Journal of Academic Librarianship 33, no. 5 (September 2007): 561–566. doi:10.1016/j.acalib.2007.05.002. | |||||
Tenopir, Carol. “The Value Gap.” Library Journal 134, no. 12 (July 2009): 20. | |||||
Tenopir, Carol, and Donald W. King. “Perceptions of Value and Value beyond Perceptions: Measuring the Quality and Value of Journal Article Readings.” Serials 20, no. 3 (November 2007): 199–207. | |||||
FURTHER READING: PUBLIC LIBRARIES
Griffiths, José-Marie, Donald W. King, and Thomas Lynch. Taxpayer Return on Investment in Florida Public Libraries: Summary Report. Tallahassee, FL: State Library and Archives of Florida, 2004. http://dlis.dos.state.fl.us/bld/roi/pdfs/ROISummaryReport.pdf (accessed July 11, 2009). | |||||
Levin, Driscoll, and Fleeter. Value for Money: Southwestern Ohio's Return from Investment in Public Libraries. Columbus, OH: Levin, Driscoll & Fleeter, 2006. http://9libraries.info/docs/EconomicBenefitsStudy.pdf (accessed July 11, 2009). | |||||
Library Research Service. “Return on Investment for Public Libraries.” http://www.lrs.org/public/roi/ (accessed July 11, 2009). | |||||
Urban Libraries Council. Making Cities Stronger: Public Library Contributions to Local Economic Development. Evanston, IL: Urban Libraries Council, 2007. http://www.urban.org/UploadedPDF/1001075_stronger_cities.pdf (accessed July 11, 2009). | |||||
FURTHER READING: CORPORATE LIBRARIES
Strouse, Roger. “Demonstrating Value and Return on Investment.” Information Outlook 7, no. 3 (March 2003): 14–19. | |||||
Griffiths, José-Marie, and Donald W. King. Special Libraries: Increasing the Information Edge. Washington, DC: Special Libraries Association, 1993. | |||||
Allen, Kenneth B. Enhancing Competitiveness in the Information Age. Washington, DC: Special Libraries Association, 1997. | |||||
ACKNOWLEDGMENTS
The University of Tennessee, College of Communication and Information, “Return on Investment” study team has helped with all aspects of phase 2. Special thanks to Joseph Park, Amy Love, and Lei Wu who have had a major role in this study. Also thanks to Donald W. King, University of North Carolina, Chapel Hill and Bruce Kingma, Syracuse University, for their expert advice. Dean Paula Kaufmann, University of Illinois Libraries, and Judy Luther, Informed Strategies, conducted the phase 1 study. Phase 1 was funded by Elsevier and phase 2 was supported by Elsevier.
Notes
Association of Research Libraries, “University & Library Expenditures,” http://www.arl.org/stats/annualsurveys/eg/index.shtml (accessed January 12, 2010).
Ross Housewright and Roger Schonfeld, Ithaka's 2006 Studies of Key Stakeholders in the Digital Transformation in Higher Education (n.p.: Ithaka, 2008), http://www.ithaka.org/research/Ithakas%202006%20Studies%20of%20Key%20Stakeholders%20in%20the%20Digital%20Transformation%20in%20Higher%20Education.pdf (accessed July 11, 2009).
Americans for Libraries Council, Worth their Weight: An Assessment of the Evolving Field of Library Valuation (New York: Americans for Libraries Council, 2007).
Urban Libraries Council, Making Cities Stronger: Public Library Contributions to Local Economic Development (Evanston, IL: Urban Libraries Council, 2007), http://www.urban.org/UploadedPDF/1001075_stronger_cities.pdf (accessed July 11, 2009).
Levin, Driscoll, and Fleeter, Value for Money: Southwestern Ohio's Return from Investment in Public Libraries (Columbus, OH: Levin, Driscoll & Fleeter, 2006), http://9libraries.info/docs/Economic BenefitsStudy.pdf (accessed July 11, 2009).
José-Marie Griffiths, Donald W. King, and Thomas Lynch, Taxpayer Return on Investment in Florida Public Libraries: Summary Report (Tallahassee, FL: State Library and Archives of Florida, 2004), http://dlis.dos.state.fl.us/bld/roi/pdfs/ROISummaryReport.pdf (accessed July 11, 2009).
Library Research Service, “Peer-Based Return on Investment Calculator,” http://www.lrs.org/public/roi/calculator.php (accessed July 11, 2009).
Roger Strouse, “Demonstrating Value and Return on Investment,” Information Outlook 7, no. 3 (March 2003): 14–19.