Building a Social Compact for Preserving E-Journals

Abstract

In the shift of scholarly publishing from the print environment to online, the preservation of the scholarly record has become more complex, difficult, and uncertain. Current efforts are not adequate. Core criteria and certification processes have been developed for digital preservation repositories. Broad library support is crucial to the sustainability of third-party preservation archives. Publisher participation in third-party archives has grown, but coverage is incomplete and uneven, leaving some categories of materials well-covered and others vulnerable. Preservation programs need to coordinate their efforts and view themselves as collaborators rather than competitors. The task of preserving the digital scholarly record cannot fall to libraries alone—it requires a social compact among all involved parties.

KEYWORDS:

• CLOCKSS
• digital preservation
• Keepers Registry
• LOCKSS
• Portico
• preservation archives
• Scholars Portal

Kenney began her session with a hat tip to NASIG’s 30th anniversary. Thirty years ago, she noted, there would have been no session on this topic, because thirty years ago there were no real e-journals beyond a few in early experimental stages. In 1991, the first pre-print archive¹ was created by high-energy physicists who wanted faster access to journal articles. Kenney added more perspective with the fact that the first edition of the ARL Directory of Electronic Journals, published in 1991, listed only 27 e-journal titles.² But what began as a rumble in the early 90s is becoming a tectonic shift as the contemporary scholarly record moves from print to electronic format.

WHY WORRY?

This shift, Kenney said, brings with it a number of causes for concern. First, the roles of vendors, publishers, and libraries in the traditional scholarly information chain were well-defined. Publishers and vendors pushed content out, while libraries preserved it and made it accessible. The “digital first ecology” disrupts these traditional roles. In addition, it is much more difficult and complex to protect digital materials than print, and current efforts are not adequate. There is also uncertainty on a whole range of issues, Kenney added, citing technology, policy, how to sustain preservation, how to provide access, and what kind of scope and coverage will be provided. Finally, Kenney noted, the well-defined relationships and roles of the print era continue into the digital era and are inhibiting collaborative approaches to solving these problems today. “Yet today,” she warned, “That’s exactly what we need. No one can do it by themselves. It requires all parties to be at the table.”

To emphasize the scope of the problem, Kenney cited data illustrating the dramatic rise in library expenditures for e-resources in recent years. Association for Research Libraries (ARL) statistics show that between fiscal years 2008 and 2011, the percentage of expenditures for e-resources rose from 48% of budgets to 62%, an increase of 29% in just five years.³ At Kenney’s own institution, Cornell University, 60% of the serials budget went to electronic resources, a figure that has increased, she said, by nearly 68% over the last five years. The 2013 Ithaka S + R survey⁴ showed that in all categories of institutions (baccalaureate, master’s and doctoral), expenditures for electronic journals and databases exceeded all other materials as a percentage of budget spent. The survey also indicated that the trend of greater expenditures for e-resources is expected to continue.

The Ithaka survey also asked library directors about the importance of various functions performed by libraries. Kenney noted that, comparing the 2013 survey with the 2010 survey,⁵ results were “pretty typical across all library types,” with the exception of two functions, the “buyer” role and the “archive” role. Further, she observed:

Among respondents, the percentage who rated the “buyer” role as important increased from 86 to 90%, and the percentage who rated the “archive” role as important increased from 87 to 93%. It’s clearly more important in the research library community than in other communities, who rated it in the 65 to 70% range. What was also interesting was when you drilled down to library directors below the doctoral institutions, they said while they were concerned about preservation, their immediate supervisor (read provost or chancellor) was less concerned. Only 52% of them were supportive of it.

This, she said, leads to two conclusions: “One, the role of preservation that the library plays is disproportionately born by the research library community. And two, there is a smaller base of support for preservation … [which] can lead to the free rider syndrome.”

To illustrate the continuing challenge we face in digital preservation, Kenney quoted a statement issued in 2005 by a group of academic librarians, university administrators, and other interested parties who met at the Andrew W. Mellon Foundation:

Digital preservation represents one of the grand challenges facing higher education. Yet … the responsibility for preservation is diffuse and the responsible parties have been slow to identify and invest in the necessary infrastructure … the digital portion of the scholarly record is increasingly at risk and solutions may require unique arrangements within the academy for sharing preservation responsibility.⁶

Kenney said that the only thing she would add to that quote today is, “… unique arrangements within the academy and beyond [must be developed] for sharing of preservation responsibility.”

LIBRARY ISSUES WITH E-JOURNAL PRESERVATION

In 2006, Kenney and colleagues at Cornell published the “first formal assessment of e-journal archiving programs,” E-Journal Archiving Metes and Bounds: a Survey of the Landscape.⁷ In preparing the report, they interviewed librarians, publishers, and the “burgeoning” number of third-party preservation programs. They examined a range of issues and made recommendations. While the report is now somewhat dated, Kenney noted, much of it still remains true today. Five issues of importance to libraries were identified by library directors interviewed. Kenney explained:

[1] They recognized that there’s a sense of urgency. Digital preservation, they acknowledge, is much different from physical preservation, because we don’t physically own, but rather license the content. [2] And that would lead to the need for collective response. (But it’s more of a great hand-waving in the sky that has sort of characterized their concerns.) [3] They also had a very strong recommendation that there had to be independent third-party trusted archiving programs, independent of the publishers, and, I would dare say, independent of the individual writers themselves. [4] They were really concerned about the kind of content that their scholars need, what kind was covered, and what circumstances would apply for it, not only preservation, but eventual access. [5] And while they were concerned, there were competing financial pressures being placed on them, so preservation didn’t sort as high as it might have, based on their concerns that they represented. You know, when JSTOR was founded, the supposition was that it was going to free up costs in terms of shelf space and handling of print materials, to be able to support a print archive, and in fact, that money was spent in increasing access to electronic journals. It did not translate into a preservation responsiveness.

IDENTIFYING PRESERVATION PROGRAMS

Kenney and her colleagues had several basic criteria for preservation programs to be included in their study. They felt it was important for the programs to be not-for-profit, and independent of publishers in order to be neutral. They focused on programs that archived scholarly, peer-reviewed e-journals. The programs needed to have formal legal mandates in place, and relationships with publishers that would allow them to be sustainable. Finally, the researchers wanted to focus on archiving programs that benefitted multiple academic libraries, not just a single institution.

With these criteria in place, Kenney and colleagues focused initially on a group of twelve archiving programs: Canada Institute for Scientific and Technical Information (CISTI) (Canada), Controlled Lots Of Copies Keep Stuff Safe (CLOCKSS), Online Computer Library Center, Inc. (OCLC) ECO, OhioLINK Electronic Journal Center (OhioLINK EJC), Koninklijke Bibliotheek e-Depot (KB e-Depot) (Netherlands), Kooperativer Aufbau eines Langzeitarchivs Digitaler Informationen (kopal/DDB) (Germany), Los Alamos National Laboratory Research Library (LANL-RL), the Lots of Copies Keep Stuff Safe (LOCKSS) Alliance, Ontario Scholars Portal (Canada), Preserving and Accessing Networked Documentary Resources of Australia (PANDORA), Portico, and PubMed Central. From the perspective of almost ten years later, Kennedy observed, “I went back and did a quick review of those that continue to remain relevant and principal contenders for us today, and three of them really rise to the surface. The LOCKSS Alliance is important, Portico, and the Scholars Portal, which serves Canada.”

CORE PRINCIPLES FOR DIGITAL PRESERVATION REPOSITORIES

In the year following the publication of E-Journal Metes and Bounds, four preservation organizations met to develop a set of core criteria for digital preservation repositories. These included the Center for Research Libraries (CRL), the United Kingdom’s Digital Curation Center, DigitalPreservationEurope, and NESTOR (Germany). They identified ten core principles:

1. The repository commits to continuing maintenance of digital objects for identified community/communities.

2. Demonstrates organizational fitness (including financial, staffing, and processes) to fulfill its commitment.

3. Acquires and maintains requisite contractual and legal rights and fulfills responsibilities.

4. Has an effective and efficient policy framework.

5. Acquires and ingests digital objects based on stated criteria that correspond to its commitments and capabilities.

6. Maintains/ensures the integrity, authenticity, and usability of digital objects it holds over time.

7. Creates and maintains requisite metadata about actions taken on digital objects during preservation as well as about the relevant production, access support, and usage process contexts before preservation.

8. Fulfills requisite dissemination requirements.

9. Has a strategic program for preservation planning and action.

10. Has technical infrastructure adequate to continuing maintenance and security of its digital objects.⁸

REPOSITORY CERTIFICATION AND LIBRARY SUPPORT

Following this statement of principles, the Center for Research Libraries and OCLC published Trusted Repositories Audit & Certification: Criteria and Checklist,⁹ or TRAC, in 2007. This document, Kenney said, provided the basis for CRL to begin formal certification of repositories. CRL has certified five repositories to date, three of which archive e-journals: Portico, Scholars Portal, and CLOCKSS.¹⁰

Library support is crucial to the sustainability of third-party digital archives. Kenney reiterated that research libraries currently bear most of the burden of supporting these programs, and used the case of ARL libraries to illustrate: 21% (27) participate in CLOCKSS, 75% (94) in Portico, and 6.4% (8) in Scholars Portal, though the latter figure represents half of Canadian ARL libraries. Participation in LOCKSS is more difficult to discern, she said, as there are a number of LOCKSS-based programs, though more than 50% are currently participating, or have at some point. When Kenney surveyed ARL libraries about their participation in these programs, 43.2% indicated they participated in only one program, 27% participated in two, and another 27% said they participated in three programs. Some respondents, she said, commented on the reasons for participating in multiple programs:

Rick Anderson [University of Utah] said it’s a matter of good citizenship, and that content is not fully covered by any one of those programs. Another said that it is early enough in the day with e-journal archiving still to take a belt-and-suspenders approach to digital preservation. A third talked about the importance of technical diversity, and the need to develop a “national research safety net.”

CONTENT AND COVERAGE

Kenney then turned to the issues of content and coverage. Publisher participation in archiving programs has grown over the years, she said, and gave the following numbers: CLOCKSS, 202 publishers; Global LOCKSS, 530 publishers; Portico, 255 publishers; and Scholars Portal, 33 publishers, with eight more soon to be included. But she cautioned that the numbers of publishers participating is only part of the picture:

You also have to consider the number of journals. So for instance, Scholars Portal only works with 33 publishers, but they preserve nearly 18,000 full text journals, as compared to Portico, which has 255 publishers, but only 9000 journals, with another [3000] in process. It’s really difficult to get a definitive list of publishers, of journals, of the date spans, of the comprehensiveness of the coverage itself. We do know that major publishers are very well represented in the preservation programs. I really don’t lose a lot of sleep over preservation of Elsevier’s journals. But there is significant duplication across preservation programs. And while redundancy is a critical thing, how much redundancy is necessary, versus expanding the kind of content coverage that is available?

Kenney also noted that publisher overlap in archiving programs is very uneven. The 2006 E-Journal Metes and Bounds study looked at 128 publishers participating in preservation programs. It showed that 71% participated in only one program, 20% in two, and only 13% in three or more. The result, she said, is that some categories of content, such as English language Science, Technology, and Math (STM) literature, are very well preserved, while others are much more vulnerable.

In response to the growing number of preservation programs and the complexity of tracking what is being archived where, EDINA at the University of Edinburgh and the International Standard Serial Number (ISSN) International Centre in Paris developed what Kenney introduced as “that really great thing called the Keepers Registry.”¹¹ Ten archiving programs, or “Keepers,” share their metadata with the Keepers Registry, which compiles and makes it public. Keepers include the Library of Congress, the British Library, HathiTrust, Scholars Portal, the CLOCKSS Archive, the Global LOCKSS Network, Portico, and others. Kenney indicated that the Keepers Registry works with the ISSN Register, and the majority of content covered by it has an ISSN or eISSN assigned. As of May 2015, the number of “serials with volumes ingested and archived” by at least one Keeper was 27,463, and about 36% of that (9,785) was archived by three or more. Kenney cautioned that these numbers do not mean that coverage is comprehensive.

Kenney showed the results for a Keepers Registry search for Journal of Human Genetics, a title that is archived in several repositories, including CLOCKSS, the Global LOCKSS Network, Scholars Portal, and Portico. She pointed out that the publisher column displayed variously Nature Publishing Group, Springer-Verlag, Springer, and Springer Japan; and further, that Springer and Nature merged in 2015. This example, she noted, demonstrates how difficult it can be to track who publishes what, and where rights and responsibilities for titles reside as they change hands.

INADEQUATE PRESERVATION COVERAGE

Kenney next focused on the issue of inadequate preservation coverage. She first cited a 2010 study by Joint Information Systems Committee (JISC) in the United Kingdom.¹² JISC administers a national educational institution e-journal licensing program, and wanted to know which publishers they work with participated in archiving programs. They found that only about half of the approximately 7,000 journals for which they provide online access are preserved in one or more of the programs examined (LOCKSS, CLOCKSS, and Portico). Of course, this means that half are not preserved; moreover, the overlap between programs is unknown.

The following year, 2011, Cornell and Columbia University Libraries (2CUL) collaborated in a study to determine how much of their e-journal content was being preserved.¹³ For the 2CUL (pronounced “too cool”) study, Kenney said, they did a “deep dive” analysis of both libraries’ e-journal holdings and compared them to what was being preserved in LOCKSS and/or Portico, programs in which both participated. The results, Kenney said, were “pretty shocking”:

Only about 13–16 percent of our e-journals were preserved in either LOCKSS and/or Portico, and the vast majority of that content was that which was identified with an ISSN or an eISSN….About 48 percent of what we subscribe to have ISSNs or eISSNs.

Cornell’s e-journal titles with ISSN or eISSN assigned number around 45,000. Of these, she said:

Portico preserves about 10,000 of them, or 22 percent. LOCKSS preserves only about 11.5 percent of them, or 5,000. Between the two of them, preserved by both LOCKSS and Portico, only 7.6 percent were preserved. And of that, if you take out the overlap, LOCKSS uniquely preserves only 3.9 percent of what we’re interested in, while Portico uniquely preserves 14.5 percent of what we’re interested in. So combined, they represent 26 percent of Cornell’s e-journal titles with ISSNs, or only 13 percent of all of our e-journals. Similar findings were found to be true with Columbia as well.

Then in 2012, the Keepers Registry compared holdings of the 2CUL libraries and Duke University against seven preservation programs.¹⁴ They found again that only titles with ISSN assigned had coverage, and that only 23–27% of holdings were preserved by any of the programs they examined. This study, Kenney said, also compared the current titles of Elsevier, Springer, Cambridge, and Sage to the title lists in Portico, JSTOR, LOCKSS, and CrossRef, and found that, even among these major publishers, there were 156 titles that were potentially unpreserved. Moreover, Kenney said, they found considerable overlap between preservation programs. For example, CLOCKSS overlaps 66% with LOCKSS, 65% with Portico, and 72% with e-Depot.

The studies above showed that preservation programs primarily archive content with ISSN/eISSN assigned. “So beyond the ISSN world, which is inadequately covered,” Kenney continued, “there is a whole huge array of material that is particularly vulnerable.” These include third party publications; titles purchased in packages, especially aggregators; small publisher titles; and Open Access titles. Small publishers typically do not have their own preservation programs, but also do not participate with any third-party programs. And Open Access publishers present other problems. Kenney explained:

We know the Open Access journal program is growing, but there is a very long tail of publishers, and this makes any organized and systematic collaboration complicated. So reaching out to stakeholders, such as the Public Knowledge Project, SPARC US and Europe, the Directory of Open Access Journals, and Archive-It, we are hoping to explore promoting best practices for OA journals. Both LOCKSS and Portico indicated a willingness to participate in that program. But as we reached out to them, many of the publishers did not respond, or were confused by what we meant by “preservation.”

COORDINATING REPOSITORY PROGRAMS

Preservation programs have both significant overlaps and enormous gaps in coverage, Kenney reiterated, yet there is little coordination across repositories. But coordination of efforts, she said, is necessary to move forward:

We need to consider how much redundancy is important, and then try to expand the amount of distinctiveness of titles that are preserved. We need common approaches to metadata, to finding aids, and registries for services, common approaches as to how access might be triggered, and a potential to share technologies and strategies that can reduce costs, and equally important to these third-party preservation programs, at the last, is that they need formal succession plans.

However, at this point, archiving programs are competing for publisher participation and library support. “They see themselves as competitors more than as collaborators in this world,” Kenney said, “And that has to change.”

CONCLUSIONS

In conclusion, Kenney said, we must start with the premise that all of the scholarly record must be preserved. At present, only a minor percentage of resources are archived, and that is inadequately documented. “We know it’s small,” she said, “We just don’t know how small.” Enormous amounts of content are highly vulnerable and not part of the “formal processes of preservation.” The task of preserving the digital scholarly record, Kenney maintained, cannot fall to libraries alone. It is also the responsibility of scholars, publishers, preservation programs, vendors, aggregators, and others. “All must own this issue,” Kenney said, “It requires a social compact among parties, no matter what their differences. We may have differences, but [must] look for common approaches that will enable us to move forward.” While there are viable archiving programs, Kenney pointed out that they have concerns about ongoing support and face continuing technical challenges as e-journals become more complex. They need the support of the entire community. Further, no one archiving program can meet all needs, so a range of programs must be supported, and they must work together.

In addition, Kenney cautioned, we need to broaden our focus to the issue of access to preserved content. She advocated shifting from the dark archive model, with content made accessible only in the case of a trigger event, to a moving wall model, where all content would be Open Access after a specified period. Governments that support research, including the United States, are already moving in this direction, she noted. In addition, Kenney expressed concern for the long term financial viability of dark archives. To assure access, we also need stronger national legal mandates for electronic deposit, including in the United States. Finally, we need licenses that grant archiving rights, and standard means to record access rights for archived titles.

Kenney wrapped up with recommendations for libraries, publishers, and archiving programs, as well as specific actions NASIG could take to advance progress toward a more comprehensive preservation program.

Recommendations for Libraries

Libraries, she said, must acknowledge that preservation is a shared responsibility, requiring all parties to be at the table. While we may have areas of disagreement with publishers, for example, preservation is a domain where we have a common need. Libraries should also urge the publishers they work with to have archiving programs. Equally important, Kenney said, “We need to press our colleagues in other libraries to get on the bandwagon, to mobilize, and provide incentives for them to expand the net of preservation and the responsibility and the costs associated with it.” Libraries need to communicate transparently about their commitments and decisions regarding preservation programs. Kenney noted that even within the ARL community, it is difficult to get this information, “which should be manifestly evident.” Participation in at least one preservation program “should be essential,” though participation in more than one could help foster the spirit of cooperation rather than competition among programs. Libraries must also, she urged, “move beyond a sense that there’s a registry of what’s being published and identify those materials most at risk, that are most vulnerable.” Finally, she said, libraries should press existing repositories to meet their needs better, and to participate in networks.

Recommendations for Publishers

Publishers, Kenney said, need to enter into relationships with one or more archiving programs, and added that a growing number are doing so. They should submit content to archiving programs in standard formats “that invite comparison and analysis and content assessment.” They should extend liberal archiving rights in licensing agreements, including to aggregators, and think about access restrictions in new ways.

Recommendations for E-Journal Archiving Programs

For archiving programs, Kenney recommended first that they begin to look at ways to expand into covering more vulnerable content. This could include cooperating with other repositories to share the load, and even selling content. They should be certified, or at minimum open to audit, and able to present evidence that they are meeting standards for long term, well managed collections. They should be clear about what they are preserving, and secure the requisite property rights. They should negotiate better in terms of licensing materials that could already be in the public domain, but that has not yet been released. Ultimately, Kenney urged, they must “form a network of mutual support and interdependence.”

What Can NASIG Do?

Kenney wrapped up with recommendations for NASIG. She noted, “You could use the prestige of NASIG to elevate this issue, to identify the risks and ways to engage in addressing the challenges. You have a really good reputation for bringing all the parties together. I love your code of conduct, which says that you provide a harassment-free experience for all members.” Kenney acknowledged that discussions on e-journal archiving, and the concerns of libraries, publishers, and aggregators “can get very, very heated,” but that NASIG could provide a neutral venue where conveners’ focus could remain on track and on task. NASIG could endorse collective action on this issue, and campaign to mobilize the community to engage and assume responsibility. The organization could also endorse a set of best practices and recommend specific actions. Kenney’s examples included recommending use of the Knowledge Bases and Related Tools (KBART) metadata exchange standard, or a review of Standard Electronic Resource Understanding (SERU). Additionally, NASIG’s ties with similar organizations in other countries could help foster international cooperation on this issue. Finally, Kenney said that NASIG could help identify other parties with a stake in preservation, including funding bodies, to help in solving remaining policy and technical issues.

Kenney closed her formal remarks with a bit of philosophy:

A social compact is what seventeenth and eighteenth century philosophers believed people made when they left the state of nature and entered into civilized society. In that new state, individuals surrendered some autonomy in order to create [a set of] rules by which to live. Thomas Hobbes, [who] was a major proponent of the social compact, or contract, wrote in Leviathan, “Life, in the state of nature, is solitary, poor, nasty, brutish, and short.” We all have a vested interest in avoiding that vision. So thank you very much.

A question and answer period followed. Chris Bulock from California State University, Northridge, began by asking whether, given the issues with Open Access publications noted by Kenney, she could offer any solutions for improving preservation of those titles. Kenney replied that the OA community has increasing awareness of the importance of preservation, but has not yet fully engaged in the responsibility. Growing prestige of OA titles, as well as federal requirements for grant-supported research, will help move OA titles into preservation programs. Bulock also asked about the difficulties of archiving online titles that do not assign volumes and issues, which typically display a date and a list of articles, with the newer articles mixing in with the older. Kenney responded, “I think that it’s probably not all that difficult, but it does require a different approach as to how you categorize the content. … I’m much more concerned about the complexity of the content and the underlying metadata associated with that content.”

Referring back to Kenney’s assertion that the entire scholarly record should be preserved, Wendy Robertson from the University of Iowa next asked whether she referred only to peer-reviewed journals, or if she was thinking more broadly, considering how scholarly communication is evolving. Kenney acknowledged that even preserving only peer-reviewed journals is a huge problem. She suggested approaching the issue incrementally, perhaps archiving ISSN-identified materials as a first step. She noted that the Obama mandate covers data preservation, but also said that the federal government needs to do more in terms of responsibility for long-term preservation of research content.

On a different note, Katy Ginanni from Western Carolina University asked about raising awareness of the importance of preservation among university presidents and chancellors, especially at smaller institutions. Kenney acknowledged that it is “a tough sell” in difficult economic times, but that we need to raise awareness at the highest administrative levels. She cautioned, “It’s one of those things, like spending money on pipes, you don’t see [the need for] until your basement gets flooded.”

Anna Creech from the University of Richmond asked about how to get participation from small, undergraduate-focused, liberal arts institutions, especially when they do not have funds to support preservation programs. Kenney replied that even small institutions depend on e-journals, and have an interest in their preservation. Further, if all types of libraries across the world were to support preservation, the costs for individual institutions would be small. They are disproportionately large now because there are so few participants. She added that moving wall access would provide more incentive for more institutions to become engaged in preservation.

A final questioner asked about preservation rights in e-resource licenses and wanted to know if Kenney had any recommendations on sample licenses. Kenney replied that the ARL model license does not go far enough, and recommended that the ARL community work to develop a license covering preservation issues and set a target date for its implementation. She added that Columbia and Cornell are preparing a report that will include recommendations on model preservation license issues.

Kenney’s session provided a good overview to the critical subject of e-journal preservation, including an introduction to tools, programs, and specific projects undertaken to date. Her liberal use of statistics made for occasional difficult listening, but she successfully conveyed the urgency and scope of the problem. Her call to build a social compact to engage with the complex issues surrounding e-resource preservation and her specific recommendations to stakeholder groups and NASIG, if undertaken, would surely move the world of scholarly communication forward in this area.

Additional information

Notes on contributors

Anne R. Kenney

Anne R. Kenney is University Librarian, Cornell University, Ithaca, New York.

Kathryn Wesley

Kathryn Wesley is Continuing Resources and Government Documents Librarian, Clemson University, Clemson, South Carolina.

Notes

¹ Cornell University Library, “arXiv.org,” http://arxiv.org/ (accessed September 18, 2015).

² Directory of Electronic Journals, Newsletters, and Academic Discussion Lists (Washington, DC: Association of Research Libraries, Office of Scientific and Academic Publishing, 1991).

³ After fiscal year 2011, ARL stopped reporting expenditures for e-resources as a separate category.

⁴ Ithaka S+R, “US Library Survey 2013,” http://sr.ithaka.org/research-publications/ithaka-sr-us-library-survey-2013 (accessed July 28, 2015).

⁵ Ithaka S+R, “US Library Survey 2010,” http://sr.ithaka.org/research-publications/us-library-survey-2010 (accessed July 28, 2015)

⁶ Don Waters et al., “Urgent Action Needed to Preserve Scholarly Electronic Journals,” http://www.arl.org/storage/documents/publications/ejournal-preservation-15oct05.pdf (accessed July 28, 2015).

⁷ Anne R. Kenney et al., E-Journal Archiving Metes and Bounds: A Survey of the Landscape (Washington, DC: Council on Library Resources, 2006).

⁸ Center for Research Libraries. “Ten Principles,” http://www.crl.edu/archiving-preservation/digital-archives/metrics-assessing-and-certifying/core-re (accessed July 29, 2015).

⁹ Trusted Repositories Audit & Certification: Criteria and Checklist (Chicago, IL: Center for Research Libraries, 2007), http://www.crl.edu/sites/default/files/d6/attachments/pages/trac_0.pdf. (accessed July 29, 2015). TRAC became ISO standard 16363:2012.

¹⁰ Center for Research Libraries. “Certification & Assessment of Digital Repositories,” http://www.crl.edu/archiving-preservation/digital-archives/certification-assessment accessed July 29, 2015. A sixth repository was certified on July 1, 2015.

¹¹ “The Keepers Registry,” http://thekeepers.org/ (accessed July 30, 2015).

¹² JISC Collections, “Which NESLi2 and NESLI2 SMP Publishers are Participating in E-Journal Archiving Programmes?” http://www.jisc-collections.ac.uk/archiving/participation (accessed July 30, 2015).

¹³ Cornell University Library and Columbia University Library, “Final Report of the 2CUL LOCKSS Assessment Team,” https://www.2cul.org/sites/www.2cul.org/files/2CULLOCKSSFinalReport.pdf (accessed July 30, 2015).

¹⁴ Peter Burnhill, “Tales from the Keepers Registry: Serial Issues about Archiving & the Web,” Serials Review, 39, no. 1 (2013): 3–20. doi:10.1016/j.serrev.2013.02.003 (accessed July 30, 2015).