h-Index and m-Quotient Benchmarks of Scholarly Impact in Criminology and Criminal Justice: A Preliminary Note

Abstract

Because Scopus and metrics like the h-index and m-quotient have become increasingly popular for assessing the impact of social science scholarship, criminology and criminal justice (CCJ) departments may be tempted to use those metrics when making important decisions like tenure and promotion. However, since no discipline-wide standards based on those metrics yet exist, CCJ departments have no comparative basis for interpreting the results of citation analyses of a particular faculty member’s scholarship. To identify what a set of disciplinary standards might look like, we used Scopus and calculated mean and median h-index and m-quotient values for faculty members (n = 504) in CCJ Ph.D. granting departments (n = 35) by rank and for editorial board members (n = 91) of Criminology, Justice Quarterly, and the Journal of Research in Crime and Delinquency. Our results illustrate how comparative disciplinary standards could be developed and used by those in CCJ departments to assess the impact of faculty members’ scholarship.

Notes

1. ISI is now known as the Thomson Reuters Web of Knowledge, a commercially available product from Thomson Reuters (nd).

2. Bibliometrics constitute a set of methods used to quantitatively analyze scientific and technological literature and more recently have been expanded to include the social sciences, literature, and the arts. Perhaps, the best known of these methods is citation analysis, but use of these tools has been extended as advances in computing allow for quick analysis of very large databases. For discussion of bibliometrics, see De Bellis (2009) and Kolowich (2009).

3. Such concerns are legitimate, as evidenced by recent growth in both scholarly support for, and critiques of, such tools (e.g. Lacasse, Hodge, & Bean, 2011).

4. Texas Southern was excluded from the analysis as we could not locate a working website from which to determine faculty at the time of data collection. Also, Florida International was omitted from the analysis, because they offer a Ph.D. in Public Affairs instead of CCJ (see Kleck & Barnes, 2011).

5. American University uses traditional titles for faculty in non-research positions. We excluded these individuals from the analysis. We thank faculty at American University for pointing this out to us. For John Jay College, we included only the department of Criminal Justice as it most closely resembles other CCJ departments (Kleck & Barnes, 2011). For the University of North Dakota, only on-campus faculty were included. Additionally, for those departments that housed degrees in multiple disciplines (e.g. sociology and criminology), we included only the faculty designated on the department website as CCJ affiliated which included departments at the University of Delaware, the University of Florida, and Penn State.

6. In two instances, we found individuals listed on more than one department web page. With these two cases, we opted to include them with their most recent affiliation. Additionally, we did not make claims about previous faculty performance or about faculty that may have been hired after the data collection period (16-23 July 2011). We recognize that faculty move across departments and this movement can change citation measures for the departments losing or gaining faculty members. Thus, the findings reflected the faculty listed on departmental websites as of July 2011.

7. Scopus only includes articles published from 1996 onward when calculating h-index scores. Thus, we used 1996 as the date of first publication for faculty members who had pre-1996 publications.

8. Recall that we used years since first-publication rather than years since awarding of the doctorate as the denominator in m-quotient calculations.

9. Correcting for co-authors may be important considering the increase in co-authored manuscripts in CCJ since the late 1960s (Gonzalez-Alcaide, Melero-Fuentes, Aleixandre-Benavent, & Valderrama-Zurian, 2012; Tewksbury & Mustaine, 2011).