Abstract
This article outlines a methodology to generate a list of local core journal titles by doing a citation analysis and details the process for retrieving and downloading data from Scopus. It analyzes correlations among citation count, journal rankings, and journal usage. The results of this study reveal significant correlations between journal rankings and journal usage. No correlation with citation count has been found. Limitations and implications for collection development and outreach are also discussed.
INTRODUCTION
With the challenge of addressing the increasing cost of academic journals, librarians are constantly assessing the library’s subscriptions and evaluating if the collection meets the needs of faculty and students. Different metrics are available to provide a list of core titles for a subject area, such as the journal impact factor (JIF) by the Institute for Scientific Information in Journal Citation Reports, and the SCImago journal rank (SJR) developed from the information in Elsevier’s Scopus database. However, generic metrics cannot represent campus-level data to make informed collection decisions. Conversely, while local usage data shows the overall use by the campus population, it does not reveal the actual use of materials in a researcher’s scholarly publication. Citation analysis is a non-obtrusive way to study researchers’ publications to identify information use patterns. Compared with journal metrics and usage data, it provides a more comprehensive analysis of the campus researchers’ information use behavior. However, conducting a citation analysis is laborious and time consuming. If correlations can be found among journal citation count, journal rankings, and usage, librarians could rely consistently on one combination of them to do more effective and efficient collection development. The purpose of this study is to find if such correlations exist.
LITERATURE REVIEW
The literature review shows that librarians conducted studies to determine the relationship among journal citation count, usage, and impact factor before the advent of electronic journals. Scales tested the relationship between citation count and frequency of use back in 1976 and suggested that ranked lists were not valid guides for journal selection.Footnote1 On the contrary, both Spearman’s rs and Pearson’s r tests indicated a positive correlation between in-house journal use and citation, and a positive correlation between journal circulation and citation in Blecic’s study.Footnote2 Later, librarians started to incorporate electronic journal usage in their studies. McDonald found that print journal use was a significant predictor of journal citation prior to the adoption of online journals, and online use was also a significant predictor of citation.Footnote3 Nevertheless, even though Feyereisen and Spoiden revealed a positive correlation between journal electronic use and citation, the correlation was moderate.Footnote4
Some authors also included the journal impact factor. Ralston’s citation study of psychiatry faculty publications revealed a positive correlation between impact factor and citation.Footnote5 Similarly, Vallmitjana and Sabate found a positive correlation between impact factor and citation in PhD dissertations in the chemistry field.Footnote6 Besides the relationship between impact factor and citation, Schimidt, Davis, and Jahr also included circulation. Their study demonstrated a stronger positive correlation between citation and circulation than impact factor.Footnote7 Other studies showed no relationship between impact factor and usage. In Pan’s study, no relationship existed between impact factor and in-house use.Footnote8 In the case of impact factor and electronic use, no relationship was found in Duy and Vaughan’s study.Footnote9
METHODOLOGY
Instead of manually going through faculty publications and typing citation information into a spreadsheet, Elsevier’s Scopus database was used to identify publications by faculty from the school of communication at the University of Houston. Elsevier’s Scopus was selected over Thomson Reuters’ Web of Science because of the availability of functionality and its comprehensiveness in the social sciences. A combined search of author affiliation and author name was conducted. After data cleaning, a total of fifty-one publications, which included forty-eight journal articles and three book chapters, were identified that met the criteria of publication by faculty from the school of communication between the years of 2006 and 2014. The fifty-one publications were added into “my list” in Scopus. Searches for references in these publications were performed and the results were downloaded into a comma separated values file.
The source data for the study were 1689 citations retrieved from publications by faculty from the school of communication between the years of 2006 and 2014. The format of citations was coded manually. After this process, the pivot table function in Excel was used to calculate the number of citations in each format, generating a list of journal titles by citation count.
JIF was obtained from the 2012 edition of Journal Citation Report and SJR 2012 was from Scopus. Journal usage data was retrieved from JR1 reports in EBSCO Usage Consolidation. Both Hyper Text Markup Language (HTML) and Portable Document Format (PDF) downloads were counted as usage. Since EBSCONet did not provide usage data for all years, only 2013 usage data was collected since it was the most current usage data available for an entire year. The final data set was uploaded into SPSS version 22 and a Spearman’s correlation was performed to determine the relationship between journal citation count, JIF, SJR, and journal usage statistics.
RESULTS AND DISCUSSION
Merged into an Excel spread sheet were: the citation count of 147 journal titles that have been cited at least twice, JIF scores for 118 journals, SJR scores for 131 journals, and usage data for 108 journals with 2013 usage available. illustrates descriptive statistics of the datasets.
Table 1. Descriptive statistics of the datasets
Relationship between Citation Count and Journal Rankings by JIF and SJR
The first set of tests determined if there was a relationship between citation count and journal rankings. One-hundred and forty-seven journal titles that had been cited at least twice were selected. Among those titles, 118 had JIF and 131 had SJR. A Spearman’s correlation was run to determine the relationship between citation count and JIF. There was no significant correlation between citation count and JIF, with a correlation coefficient rs=–.015, p = .872. A negative rs indicated that while citation count increases, JIF decreases, which means the communication faculty tends to cite journals with low impact factor. However, a rs value close to 0 indicates no linear relationship between these two variables. It is a random correlation (See ).
Figure 1. Correlation between citation count and JIF: A random correlation.
The National Communication Association has several concerns about impact factor, one being that they are misused to define the discipline and research.Footnote10 SJR is an alternative metric that is intended more for social sciences. Thus, SJR data was also obtained and a Spearman’s correlation was run to determine the relationship between citation count and SJR. There was no significant correlation between citation count and SJR, with a correlation coefficient rs = .125, p = .154. shows correlations between citation count and journal rankings, both JIF and SJR.
Table 2. Correlations between citation count and journal rankings (JIR and SJR)
Relationship between Citation Count and Usage
This study also examined the relationship between citation count and journal usage. The result showed no significant correlation between the two, with a Spearman’s correlation coefficient rs = .006, p = .951 (see ). It is necessary to note that journal usage data from EBSCONet include faculty, staff, and student use. The purpose of the downloads is not only for faculty publications, but also for teaching and student research. If we could separate the usage data by user demographics, it would lead to more accurate findings.
Table 3. Correlations between citation count and usage
Relationship between Journal Rankings and Usage
This set of tests was performed to find out the relationship between journal usage data and journal rankings. There was a moderate, positive correlation between journal usage and JIF (rs = .415, p < .01). A same correlation was also found between journal usage and SJR (rs = .417, p < .01). In this study, the correlation between usage data and both IF and SJR is significant (see ). shows a positive significant correlation between usage and JIF. This correlation has important meanings for collection development. In the case of the University of Houston, communication faculty are not citing high-impact journals, but the entire campus population is using high-impact journals. Despite the fact that high-impact journals are not cited in communication faculty publications, the library should still consider journal rankings for subscribing to journals because other University of Houston patrons have a need for them either in teaching or research. Since usage is correlated with both JIF and SJR, collection librarians can use either of them. It is also worth noting that SJR covers more journal titles, especially in social sciences. SJR would be a valuable tool for social sciences librarians.
Table 4. Correlations between usage and journal rankings
Figure 2. Correlation between usage and JIF: A positive significant correlation.
LIMITATIONS AND CONCLUSION
There are several limitations to this study. First, the Scopus database was used to retrieve faculty publication and citation data, including publications indexed in Scopus at the time of search. A more valid way of collecting publication and citation data would be by obtaining updated faculty curriculum vitae and accessing the cited references from the actual publication. However, this process is time consuming. Liaison librarians have multiple duties, from teaching to collection development, so the methodology in this study provides a quick way for librarians to understand faculty citation behavior. Second, EBSCONet’s journal usage data includes faculty, staff, and students, but data is not currently available to limit by user demographics. It would be more accurate to compare faculty use with faculty citation. Lastly, usage data included in this study is from the year 2013. Future studies should consider collecting multiple years of usage to correspond with multiple years of citation data.
The results of this study provide some insights for collection development. Citation analysis can provide valuable information about users’ information use behavior, including a list of highly cited local core journal titles. However, at the University of Houston, citation count had no relationship with either journal rankings or journal usage. Besides using journal rankings and journal usage, librarians can use citation analysis to have a better understanding of local faculty’s citation behavior. It provides an alternative way of examining collection development related to journals. At the campus level, both JIF and SJR have signification correlations with journal usage, which is evidence showing patrons’ use of high-impact journals. Journal rankings and journal usage are still valid factors to consider when subscribing to journals. Citation analysis, together with journal rankings and journal usage, provides different aspects for librarians in assessing the collection.
Acknowledgments
The author thanks Jackie Bronicki, Assessment and Statistics Coordinator at University of Houston, for the help of retrieving journal usage data. The author also thanks Alexandra Simons, History and Political Science Librarian at University of Houston, for commenting on this article.
Additional information
Notes on contributors
Wenli Gao
Wenli Gao is the Communication, Sociology, and Anthropology Librarian, University of Houston, Houston, Texas.
Notes
1 Pauline A. Scales, “Citation Analyses as Indicators of the Use of Serials: A Comparison of Ranked Title Lists Produced by Citation Counting and from Use Data,” Journal of Documentation 32, no. 1 (1976): 17–25. doi:10.1108/eb026612.
2 Deborah D. Blecic, “Measurements of Journal Use: An Analysis of the Correlations between Three Methods,” Bulletin of the Medical Library Association 87, no. 1 (1999): 20–25.
3 John D. McDonald, “Understanding Journal Usage: A Statistical Analysis of Citation and Use,” Journal of the American Society for Information Science and Technology 58, no. 1 (2007): 39–50. doi:10.1002/asi.20420.
4 Pierre Feyereisen and Anne Spoiden, “Can Local Citation Analysis of Master’s and Doctoral Theses Help Decision-Making About the Management of the Collection of Periodicals? A Case Study in Psychology and Education Sciences,” Journal of Academic Librarianship 35, no. 6 (2009): 514–522. doi:10.1016/j.acalib.2009.08.018.
5 Rick Ralston, Carole Gall, and Frances A. Brahmi, “Do Local Citation Patterns Support Use of the Impact Factor for Collection Development?” Journal of the Medical Library Association 96, no. 4 (2008): 374. doi:10.3163/1536-5050.96.4.014.
6 Núria Vallmitjana and L. G. Sabate, “Citation Analysis of PhD Dissertation References as a Tool for Collection Management in an Academic Chemistry Library,” College & Research Libraries 69, no. 1 (2008): 72–82. doi:10.5860/crl.69.1.72.
7 Diane Schmidt, Elisabeth B. Davis, and Ruby Jahr, “Biology Journal Use at an Academic Library: A Comparison of Use Studies,” Serials Review 20, no. 2 (1994): 45–64. doi:10.1016/0098-7913(94)90028-0.
8 Elizabeth Pan, “Journal Citation as a Predictor of Journal Usage in Libraries,” Collection Management 2, no. 1 (1978): 29–38. doi:10.1300/J105v02n01_03.
9 Joanna Duy and Liwen Vaughan, “Can Electronic Journal Usage Data Replace Citation Data as a Measure of Journal Use? An Empirical Examination,” Journal of Academic Librarianship 32, no. 5 (2006): 512–517. doi:10.1016/j.acalib.2006.05.005.
10 National Communication Association, “Impact Factors, Journal Quality, and Communication Journals: A Report for the Council of Communication Associations” (National Communication Association, 2013). http://www.natcom.org/uploadedFiles/More_Scholarly_Resources/CCA%20Impact%20Factor%20Report%20Final.pdf (accessed November 5, 2014).