Crystal ball or moneyball: does publishing success during graduate school predict career publication productivity?

Abstract

In academic departments, arguably the most important decisions that faculty will face collectively will be the hiring of a new colleague. Questions often emerge, however, concerning how to predict whether an applicant will have a successful career in terms of publishing productivity. Some methods of prediction are clinical (what we term the ‘crystal ball’ method), which involve the weighing of several pieces of information with no set decision rules. Other methods are more actuarial (what we term the ‘Moneyball’ method) and rely heavily on an applicant's publication record while in graduate school. To examine the degree to which publication productivity in graduate school predicts publishing success over the course of one's career, we gathered detailed data from the curriculum vitae of 407 faculty residing in institutions affiliated with the Association of Doctoral Programs in Criminology and Criminal Justice. We also identified four faculty ‘outliers’ to target for qualitative interviews to provide additional nuance and context to our statistical models. Analyses of these vitae indicate that pre-degree publication productivity – measured in multiple ways – significantly predicted career publishing success. Nevertheless, the magnitude of this relationship varied across models and the implications for search committees and hiring decisions are discussed.

Keywords::

• publishing productivity
• hiring
• tenure
• promotion

Notes

^1. As peer-reviewed journal publications also remain the academic currency within the field, other forms of publications such as books, non-peer-reviewed articles, technical reports, and other publications were not included for the purpose of this project.

^2. International universities are not ranked, however, the top universities are identified by country.

^3. Our data indicate that the number of pre-Ph.D. publications increased substantially after 1990 (analyses are available upon request). As such, subsequent analyses were conducted on only those who received their Ph.D. after 1990. These results were similar to those presented with two exceptions. The variable for lead/solo pre-Ph.D. publications in elite journals became statistically significant at predicting career publications in elite journals (b = 0.31, p < 0.05) and lead/solo career publications in elite journals (b = 0.33, p < 0.05).

^4. Additional analyses were considered using the number of unique co-authors that someone published with both in graduate school and over the course of their career. The problem, however, is that this variable was too highly correlated with each of our dependent variables (consistently above 0.80) that they could not be used in the multivariate models. Yet what this does indicate is that productive scholars generally work with a lot of different people (not necessarily the same people over and over again exclusively). This finding is not terribly surprising and is consistent with the recent literature on co-authorship networks in criminology and criminal justice (Rice, Hickman, and Reynolds 2011).

^5. These cases were identified because their productivity residual values were consistently large across all of our indicators of career publication productivity. Accordingly, as is typically the case with qualitative research, these supplemental analyses are not necessarily concerned with representativeness. Instead, the approach taken here places at a premium honesty and insight on the part of respondents (Orbuch 1997) – something we trust these scholars to provide.

^6. ‘Always be closing’ is a reference to the epic speech that Alec Baldwin's character delivered to a room full of real estate agents in the 1992 film Glengarry Glen Ross.

Additional information

Notes on contributors

Kimberly A. Kaiser

Kimberly A. Kaiser is a doctoral student in the School of Criminology and Criminal Justice at Arizona State University. Her research interests include judicial decision-making, reentry policy and practice, and the role of procedural justice on offender reentry success. Her work has appeared in Criminology and Public Policy.

Travis C. Pratt

Travis C. Pratt is a Fellow with the University of Cincinnati Corrections Institute. His work focuses primarily on criminological theory and correctional policy. He is the author of Addicted to Incarceration: Corrections Policy and the Politics of Misinformation in the United States, and his work has appeared in a number of peer-reviewed journals including Criminology, Journal of Research in Crime and Delinquency, Journal of Quantitative Criminology, Justice Quarterly, and Crime and Justice: A Review of Research.