The effect of journal metrics on academic resume assessment

ABSTRACT

It has been widely argued that journal metrics are used in assessing publication records on resumes for academic jobs and assessments. Within that debate, two important considerations emerge. Firstly, academics belonging to different career cohorts may have different experiences which are reflected in their recommendations related to targeting and assessing publication records. Secondly, recruitment or assessment expectations that include publishing in highly rated journals may lead to the targeting of a smaller number of high rated publications, with perceived lower rated journal outlets being discouraged. Using an experimental design, an online survey collected data from 1011 academics across management and psychology disciplines in the UK and USA, exploring the association between journal ratings, the number of publications on a resume, and the length of time spent in academia. Analysis indicated that when assessing an academic resume, the discouraging of lower rated journal publications may be dependent on the length of time spent in academia. Specifically, in the context of exactly the same high rated journal publications on a resume, those who had been in academia for 10–20 years were less favorable towards the inclusion of additional low rated journals. The findings contribute to how we view the socialization of institutional influences on career decision making in higher education. The results add to emergent evidence of behavioral responses to the institutional pressures on academic careers, and how individuals at different career stages may be impacted differently. This has implications for the management of academic career progression and academic recruitment processes.

KEYWORDS:

• Journal metrics
• article impact
• academic careers
• scholarship of teaching and learning
• academic resumes

1. Introduction and literature review

If I don’t write for our top journals, I might as well be writing a letter to my mother.

These words by James Walsh (2011, 218) reflect some of the restrictions upon academia caused by journal ratings. Advice on formulating a resume for job applications in higher education has changed over time, but the use of journal metrics to assess publications on a resume continues to be a highly debated topic. Within that debate, two considerations emerge. Firstly, different career cohorts may have different experiences, which may be reflected in their recommendations for both targeting and assessing publications. Secondly, expectations of publishing in highly rated journals are sometimes associated with recruitment and performance management (Vogel, Hattke, and Petersen 2017), and potentially influence both the targeting and associated assessment of publication records towards a relatively small number of highly rated outlets, with publication in lower rated outlets being discouraged.

There are many journal rating systems available ranging from the ABS list (Association of Business Schools), ERA (Excellence in Research for Australia) to Thomson and Reuters Impact Factors. Higher education, including in the management discipline, has arguably become dominated, and in some cases constrained, by the use of journal ratings metrics in recruitment and promotion (Adler and Harzing 2009; Gulati 2007; Rafols et al. 2012; Walsh 2011). For the purposes of this research, the use of metrics to assess journal quality will be referred to as journal ratings. Obtaining journal ratings can be challenging and changeable for new channels of research (Serenko and Bontis 2013), as well as there being differences in perceptions across disciplines and countries (Alexander, Scherer, and Lecoutre 2007; Morris, Harvey, and Kelly 2009). Concerns are raised that journal rating systems do not always reflect a difference in quality or contribution, favoring the English language, and writing can be constrained through tailoring with a particular journal’s ranking in mind (Adler and Harzing 2009; Ferrara and Bonaccorsi 2016; Mingers and Willmott 2013; Tourish and Willmott 2015). This exacerbates concerns about the fairness of access to knowledge (Harzing and Adler 2016) and there are increasing calls for fairer and more inclusive metrics (Harzing and Alakangas 2016).

This research investigates the effects of presenting low rated journal publications on an academic resume and the length of time spent in academia. It is hypothesized that exposure to these changes in discourse, and age or experience effects, may result in expectations being influenced by how long an individual has been in academia. The next section sets out the background, followed by methods and data, the results and a discussion with conclusions.

2. Background

2.1. Change in the discourse over time

The perception of what was desirable in an academic resume has varied over time, and between disciplines and countries, including changes in the balance between the perceived quality and quantity of a candidate’s papers. In the 1980s, a major metric for research productivity at many institutions was the numerical output of volume of papers, but this led to criticisms that too much attention to the number of publications on a person’s resume and too little attention to the quality of the papers (Long, Allison, and McGinnis 1993; Mooney 1991). It has been argued that one consequence of this was a proliferation of the scientific literature without a proportional increase in knowledge (Reidenberg 1989). Owing to criticisms of using the number of publications as a metric for assessing publication records, the use of journal ratings became a dominant new metric for assessing publication records on an academic resume. However, it is now argued that journal ratings and impact factors are having too great an influence on academic careers hiring decisions.

Since a critique was made of the impact of journal ranking lists (Adler and Harzing 2009; Willmott 2003) as well as the social impact of broader university rankings (Espeland and Sauder 2007), there has been a fierce debate on funneling research into highly rated journals. Such publications in high rated journals are treated as ‘magic numbers’ (Hussain 2011), becoming ‘golden eggs’ in getting jobs and grants (Hitt and Greer 2012; Vale 2012). Fierce institutional competition through journal metrics could be a source of discrimination (Özbilgin 2009; Rosenstreich and Wooliscroft 2012), including a disjuncture between research and practice in management (Keleman and Bansal 2002) and excess influence accruing to the editors of a few high-ranked journals. With the potential for low rated journals to be avoided by young researchers trying to build an impressive promotion file (Segalla 2008), and universities ‘craving’ academics who publish in high-ranking journals in order to improve their university’s status (Nkomo 2009), other research and research outlets could be stifled (Willmott 2011). Recent research reflects on this change in the context of its impact on recruitment and the role of academic disciplines (Paisey and Paisey 2018).

It is also noted that there may be other factors interacting with the change in discourse on what aspects of a publication record in an academic resume should be assessed. In the early 1990s, universities were said to be changing from traditional institutions, increasing responsiveness to ‘customers,’ students, and research councils (Peters 1992), related to broader social changes (Halsey 1992), and mediating pressures toward the commodification of academic work (Willmott 1995). This also coincided with the expansion of academia in the countries such as the UK. Polytechnics and colleges, which were previously funded primarily for teaching, demanded funds to support their research (Elton 1992). The introduction of Research Assessment Exercises for universities in the UK in the late 1980s was heavily dependent on the quality of publications and has influenced the discourse and expectations of academics. The reconfiguration of this assessment in 2008 (renamed Research Excellence Framework (REF) with greater emphasis on research impacts) may have partly been a response to criticisms of the constraints placed on academia by the influence of journal ratings (Geary, Marriott, and Rowlinson 2004). Indeed, there are indications that within the REF process, perceptions of what is meant by impact, and its assessment, might not be entirely uniform (Samuel and Derrick 2015; Smith, Ward, and House 2011), and a possible confusion with impact factors of academic journals, despite open access journals potentially having wider reach (Watermeyer 2016).

2.2. Career stages

Research into the different career stages of management scholars encourages early career scholars to target high rated publications, advising broad topics with wide appeal (Podsakoff et al. 2018). Interdisciplinarity is being increasingly linked with academic success (Biancani et al. 2018), and is often associated, together with innovation, with high rated publications (Vogel, Hattke, and Petersen 2017). The preference for social impact is lowest amongst senior, extrinsically motivated academics. Equally, there are indications of a preference for social impact amongst junior staff members, suggesting a socializing, or perhaps age or experience related, effects of the impact agenda. This is consistent with the finding that those who obtain PhD training in countries with an active impact agenda, value social impact higher than those who obtained training elsewhere (Salter, Salandra, and Walker 2017).

Qualitative research indicates that the pressure to publish in high rated outlets was particularly pronounced in economics. However, while journal metrics were emphasized by senior managers, this was not always over-emphasized in the mentorship of early-career researchers (O’Connell, O’Siochru, and Rao 2021). Meanwhile, other qualitative research indicates that younger academics in tenure-track positions tend to be subject to performance reviews, which can include a narrow focus on high-rated publications (Pietilä 2019), especially in research-intensive institutions. Further analysis of resumes themselves indicates that older academics present most information in chronological order. Mid-career academics tend to emphasize research grants and categorize journal articles by rating. Young researchers emphasize a wider range of qualities, perhaps owing to less experience or the types of posts being applied for (Macfarlane 2020). This recent qualitative evidence indicates the impact journal metrics might have on career advice, and self-presentation, at different career stages, as well as in different disciplines.

Considerable dispute has centered around what is meant by impact. For example, measuring impact on fellow academics and the discipline, or wider impact on society, policy and practice. How the impact of research is measured, and how journal metrics are thus derived, including their impact on the reliability and relevance of research (Antonakis et al. 2014; Kiri, Lacetera, and Zirulia 2018; Mårtensson et al. 2016; Prasad, Segarra, and Villanueva 2019). This has led to research into the changes in how publication records may be compiled and reviewed (Liu, Olivola, and Kovács 2017), including self-citation (Seeber et al. 2019) and wider changes to how publication quality in business and management academia is viewed (Baruch, Point, and Humbert 2020; Kuskova, Podsakoff, and Podsakoff 2011). It is argued that this nexus between rigor, relevance and research service, emphasizes a ‘single’ stakeholder of research peers, having the potential to reinforce associations of quality and institutional control (Renwick, Breslin, and Price 2019; Rindova et al. 2018). There attempts to reduce the use of impact factors as the metric for assessing the quality of academic candidates. In 2022, Utrecht University are moving to judge its scholars by other standards, including their commitment to teamwork and their efforts to promote open science, inspired by the DORA (2012 Declaration on Research and Assessment) (Woolston 2021).

Another nexus is research-teaching. Using the job-demands-resource model, quantitative evidence suggests that perceived pressures for teaching performance are related to exhaustion and disengagement (McCarthy and Dragouni 2020). Interestingly, this was less pronounced for research pressures. However, there was still an impact on disengagement, and a notable quantified relationship with the number of years as an academic (McCarthy and Dragouni 2020). Further to this, it has been highlighted that levels of engagement and vigor are not equally correlated to research productivity, with opportunities to work with graduate students being seen as a supportive mechanism (Christensen, Dyrstad, and Innstrand 2020). There may also be mismatches between academic expectations and the socialization of academics (Adcroft and Taylor 2013).

2.3. Low rated journals and behavioral socialization

It is argued that assessing research output is often limited to the recognition of the journals where they are published, adopting the assumption that the quality of the output itself is indicated by journal quality (Cuellar, Truex, and Takeda 2019). Bias persists in academic recruitment, even in business and management schools (Minefee et al. 2018), including in the way journal metrics have been derived (Mingers and Yang 2017). It is hypothesized that there may be a ‘less is better effect’ causing those with a longer resume that includes low rated journal publications to be viewed negatively. Powdthavee, Riyanto, and Knetsch (2018), studying economists including 52 PhD students, found that when participants examined a resume in isolation, the short resume without lower rated journal papers was preferred. However, during joint evaluation, of long and short resumes, the short resume was not preferred. It is hypothesized that in direct comparison participants could see that the high rated journals appeared on both resumes, leading to no negative impact.

This research investigates this issue further, indicating that there might be differences when comparing with different specific disciplines. In addition, based on the evidence for journal metrics exerting different pressures on academics depending on the length of time in academia, we predict this affects the judgement of lower rated journal papers.

Based on this discussion, the paper’s research hypotheses are:

H1: The rejection of lower rated journals will not be uniform across academic disciplines.

H2: The rejection of lower rated journals will not be uniform across the length of time spend in academia.

3. Materials and method

As pointed out by King et al. (2018), biases in organizational science such as confirmation bias (the tendency to seek information that confirms preconceptions), anchoring bias (initial views or information overly influence later views) and adjustment (reference to an initial value), overconfidence bias, and social dynamics may infect the scholarship process. Research has also suggested that, when confronted with a number of job applications, recruiters follow a strategy of picking applicants with positive characteristics (‘diamonds’) rather than eliminating applicants with negative characteristics (‘lemons’) (Eriksson and Rooth 2014). Building on this, Powdthavee, Riyanto, and Knetsch (2018) used a randomized control trial, displaying different publication records to participants.

The current research also utilized a randomized control trial, displaying one of two different resumes. The first resume showed only the fictitious candidate’s four higher rated publications, plus their name, ethnicity, gender, prior experience, research grants, place and year of degrees obtained (the ‘short’ resume). The second resume contained identical information and publications of the candidate, but in addition had eight lower rated publications (the ‘long’ resume). The randomized control trial was designed to elicit the effect of omitting or retaining lower rated publications on an academic resume. Participants considered only one of the two resumes. Powdthavee, Riyanto, and Knetsch (2018) allowed participants to rate their preference for a given resume, given that their hypotheses relied on a more granular comparison between resumes including joint comparison. The participants in our study only saw one resume, with one of only two possible resumes randomly assigned. This research, therefore, emphasized the impact of lower rated journals on a participant’s willingness to hire a candidate on a binary yes/no basis.

Participants were asked to consider the resume for an outlined job description for an associate professor/senior lecturer, as such a position raises questions of the presentation and recognition of ‘exceptionality’ (Miller and Morgan 1993). Two versions of the job description and the resume and qualifications of the candidate were used, one for UK participants and a second with ‘translations’ of some national differences for USA participants. The defined candidate was from a British educational background. A first class honors degree was translated as comparable to summa cum laude in the USA, albeit that this particular translation is difficult to make. An explanation of their university being in the Russell Group meaning the top 24 research universities in the UK was given. Research grants were also converted into dollars as well as ESRC grants being stated as being the UK equivalent to NSF. The job description remained consistent across countries.

3.1. Participant recruitment

Responses were collected across countries and disciplines from 1011 faculty staff via an online experimental survey design. There were 288 and 131 responses from UK-based and USA-based psychology faculty, respectively, and 426 and 166 from UK-based and USA-based management faculty. To control the differences of hiring focus of different types of institution and faculty (Meizlish and Kaplan 2008), two social science disciplines were sampled from top 40, research-orientated, universities in their respective counties, according to QS world ranking at the time of data collection.

All participants were recruited through emailing 11,324 university faculty and asking them to complete the online survey (the ‘long’ or ‘short’ resumes were randomly assigned, participants were not informed that there were two surveys). 1583 UK psychology faculty, 3851 UK management faculty, 1466 USA psychology faculty and 4424 USA management faculty were emailed. The resultant response rate averaged around 9% across all disciplines and countries; however, response rates were higher from the UK (13%) compared to the USA (5%).

The emails were personalized and addressed to the recipient by title and full name. This information and their contact details were collected from faculty web pages. The aim was to contact those academics with the highest likelihood of sitting on appointment panels, which comprised of emailing all faculty at assistant professor (USA)/lecturer (UK) or higher. Teaching fellows (UK) and lecturers (USA) were excluded as were research assistants, PhD students, adjunct professors and professors of practice. In order to control for any selection bias, all faculty that met these criteria were emailed.

3.2. Compiling of resumes

An important consideration in the design of this research was the makeup of the publication records presented on the resumes. A resume was presented for a set outlined the position of senior lecturer/associate professor, with descriptors (including journal titles) only changing to represent that the applicant was applying for a position in a management or psychology faculty, or a position in the UK or USA. Within the scope of this, in addition to the comparable descriptors in each context, the candidate resume needed to be representative across the two faculties and two countries.

The resume needed to be of sufficient quality, including adequate highly rated publications, to be considered for the position of senior lecturer/associate professor at a university in the top 40 universities in the UK and USA, according to QS world rankings. That is to say that the resume would fit in with the institutional expectations of recruitment at these types of institution. Indeed, the embeddedness of journal rating into university ratings was a justification for targeting these types of institution. After some piloting amongst academic colleagues, fictitious articles in four highly rated publications were added to the resumes. The resume also included a research council grant of £90,215 which was given relevant USA dollar and research council conversions. The value of the grant remained the same across countries and disciplines.

With this in mind, resumes were drawn up that had four high rated publications on them. The longer resume had eight additional low rated publications added but was otherwise identical to the shorter resume. Table 1 demonstrates that in all cases there was a clear separation between the rating of the chosen high and low rated publications across all available journal metrics. While creating a polarity in the candidate’s resume, this was necessary to ensure that there would not be a subjective interpretation of high and low rated journal publications. Publications, grants and their titles were adapted to suit psychology or management applications.

Table 1. Ratings of journal publications shown on candidate resumes.

Journal name	Journal abbreviation	ABS	ERA (2010)	SJR area rank	TR (ISI) IF disciplinary area rank (2014)	TR (ISI) impact factor (2014)	Eigen score		No. of entries on complete resume
Psychology
							(Eigen Factor)	(Article Influence)
High Rated				1042 Total	646 Total (all psychology areas)
Psychological Science	PSYCHOL SCI	n/a	A*	18	28	4.940	0.06739	3.227	2
Cognition	COGNITION	n/a	A*	47	55	3.479	0.02471	1.978	1
Journal of Experimental Psychology: Learning, Memory and Cognition	J EXP PSYCHOL LEARN	n/a	A*	78	99	2.862	0.01560	1.492	1
Low Rated
Psychological Reports	PSYCHOL REP	n/a	C	686	528	0.560	0.00294	0.202	4
Perceptual and Motor Skills	PERCEPT MOTOR SKILL	n/a	C	721	534	0.546	0.00245	0.175	4
Management
High Rated				1106 Total	337 Total (business, finance, management)
Academy of Management Journal	ACAD MANAGE J	4*	A*	6	3	6.448	0.02813	5.738	2
Journal of Management	J MANAGE	4*	A*	9	4	6.071	0.02099	4.548	1
Journal of Management Studies	J MANAGE STUD	4	A*	27	20	3.763	0.01220	2.572	1
Low Rated
European Journal of International Management	EUR J INT MANAG	1	C	572	284	0.457	0.00044	0.172	4
Cross Cultural Management: An International Journal	CROSS CULT MANAG	1	C	482	296	0.396	0.00100	0.300	4

4. Results

Figure 1 shows that across all countries and disciplines the long resume, that included the low rated publications, was preferred to the short resume – as would be expected with a ‘rational’ decision where the longer resume showed greater achievements.

Figure 1. Yes appointable for short and long resume across countries and disciplines. Note: Error bars at 95% CI.

Table 2 shows the binary regression results for the likelihood of rejecting the resume, controlling as covariates for the categorical variables of gender, academic discipline, country, and which resume length was randomly assigned.

H1: The rejection of lower rated journals will not be uniform across academic disciplines.

Table 2. Binary logistic regression; likelihood of rejecting the resume.

Years in academia		B	S.E.	Wald	df	Sig.	Exp(B)
Less than 10 years	Male	−0.372	0.220	2.843	1	0.092	0.69
	Psychology	1.053	0.223	22.387	1	0.000	2.868
	UK	−0.231	0.255	0.815	1	0.367	0.794
	Short Resume	0.702	0.217	10.508	1	0.001	2.019
	Constant	−0.75	0.309	5.888	1	0.015	0.472
10–20 years	Male	−0.319	0.246	1.687	1	0.194	0.727
	Psychology	0.582	0.244	5.681	1	0.017	1.79
	UK	−0.028	0.300	0.009	1	0.925	0.972
	Short Resume	0.061	0.243	0.063	1	0.801	1.063
	Constant	−0.208	0.352	0.347	1	0.556	0.813
More than 20 years	Male	0.176	0.276	0.407	1	0.524	1.193
	Psychology	0.716	0.241	8.794	1	0.003	2.045
	UK	−0.786	0.239	10.814	1	0.001	0.455
	Short resume	0.686	0.237	8.379	1	0.004	1.986
	Constant	−0.427	0.322	1.759	1	0.185	0.653

It is clear in Table 2 that across all amounts of time spent in academia psychology scholars were more likely to reject a resume. This is reflected in Figure 1.

However, Table 3 shows the preference for either the long or short resume between academic disciplines. Academics in management were more favorable to both resumes compared to psychology. Most importantly in relation to hypothesis H1, the results show that both management and psychology scholars prefer the long resume that included the lower rated journal publications.

H2: The rejection of lower rated journals will not be uniform across the length of time spend in academia.

Table 3. Candidate would be recruited descriptive statistics.

		Short resume				Long resume				Sig. between resumes
		Yes	No	Total	Sig.	Yes	No	Total	Sig.
	Total	49.2%	50.8%	508		60.4%	39.6%	503		0.000
Gender	Male	51.0%	49.0%	316	0.371	64.3%	35.7%	305	0.018	0.001
	Female	46.8%	53.2%	190		53.6%	46.4%	194		0.185
Years as an academic	<10	49.8%	50.2%	207		66.7%	33.3%	195		0.001
	10–20	53.2%	46.9%	143	0.356	52.8%	47.2%	144	0.035	0.950
	>20	44.9%	55.1%	158		59.8%	40.2%	164		0.008
Discipline	Psychology	36.7%	63.3%	207	0.000	48.6%	51.4%	212	0.000	0.014
	Management	57.8%	42.2%	301		69.1%	30.9%	291		0.004
Country	UK	53.2%	46.8%	370	0.003	61.9%	38.1%	344	0.318	0.019
	USA	38.4%	61.6%	138		57.2%	42.8%	159		0.001

In Table 2, gender does not significantly predict the preference for rejecting a resume given the length of time spent in academia. However, the results show that while females are more likely to reject a candidate’s resume if they have been in academia for less than 20 years, this is no longer the case for female academics who have been in academia more than 20 years. However, the sample had only 80 females who had been in academia over 20 years compared to 242 males, contrasted to 167 males and 120 females who had been an academia 10–20 years, and 212 males and 184 females who had been in academia less than 10 years. Although only indicative, there may be a mismatch between women’s’ expectations on themselves and others in early career (Madison and Fahlman 2020). Recruitment panels may value a resume overall (as they are likely to include males and may have a majority of males). However, this will depend also on the seniority of those males and females on an appointment panel, as well as younger males being generally more favorable towards resumes. It should be noted here that the candidate resume in this study was male. This is an issue that requires further research.

Importantly to hypothesis H2, the results show that academics who have been in academia for less than 10 years are significantly more likely to reject the short resume without the lower rated publications compared to the long resume that included lower rated publications. Those who had been in academia for more than 20 years were also significantly more likely to reject the short resume compared to the long resume. However, those who had been in academia for 10–20 years were indifferent between the two resumes and were no more likely to reject either the short or long resume. Therefore, although the results do not find evidence for a rejection of lower rated publications overall, there was no additional value given to the long resume with eight additional lower rated publications on it for those who has been in academia 10–20 years.

In Table 2, country had a significant relationship with appointing the given resume amongst those who had been in academia for more than 20 years, but not for those who had been in academia less than this. To investigate this further, Table 4 shows the descriptive statistics for appointing the given resume by years in academia and country. The overall finding that the long resume is preferred is found in both the USA and UK samples of those who have been in academia for more than 20 years. However, this preference is expressed as a stronger rejection of the short resume in the USA sample in this category, compared to a stronger acceptance of the long resume in the UK sample (Figure 2).

Figure 2. Yes appointable for the long and short resume given the number of years in academia. Note: Loess Method, 50% points fit, Epanechikov Kernel.

Table 4. Descriptive statistics for appointing resume by years in academia and country.

Years in academia	Country		Appointable		Total	Sig.
			Yes	No
Less than 10 years	UK	Short resume	82	73	155
		Long resume	95	53	148	0.03
	USA	Short resume	21	31	52
		Long resume	35	12	47	0.001
10–20 years	UK	Short resume	64	54	118
		Long resume	58	52	110	0.462
	USA	Short resume	12	13	25
		Long resume	18	16	34	0.456
More than 20 years	UK	Short resume	51	46	97
		Long resume	60	26	86	0.013
	USA	Short resume	20	41	61
		Long resume	38	40	78	0.043

In summary, the results indicate that when studying across countries and different academic disciplines, a resume which includes additional low rated publications, is still preferred. However, there is no additional benefit for having additional low rated publications on a resume when reviewed by an individual who has been in academia 10–20 years.

5. Discussion and conclusion

5.1. Implications for theory

It has been observed that there could be a socializing effect of the agenda on the impact of academic research (Salter, Salandra, and Walker 2017). There has also been research into how the use of journal metrics might have led to changes in how publication records may be compiled and reviewed (Liu, Olivola, and Kovács 2017), It has been acknowledged that there is a lack of research on the impact of low rated journals (Powdthavee, Riyanto, and Knetsch 2018), hypothesizing that there may be a ‘less is better effect’ causing a longer resume with low rated journal publications to be viewed negatively.

Within the findings of this research, a ‘less is better effect’ is not found, with a longer resume including additional low rated publications being preferred. Investigating further, it was found that the academic discipline, range of academics included in the sample, as well as interaction with other institutional pressures are all important in participants’ responses. Within the data collected here, it is demonstrated that the length of time spent in academia is predictive of the preference for the inclusion or omission of low rated journal publication on an academic resume. Those having less or more than 10–20 years’ experience preferred the longer resume, and those with 10–20 years’ experience were indifferent between them.

This could be evidence of socialization and a cohort effect affecting the assessment of academic resumes. It is acknowledged here that this socialization could have multiple sources. The discourse on how academic resumes should be built has changed over time (Adler and Harzing 2009; Long, Allison, and McGinnis 1993; Mooney 1991; Reidenberg 1989; Willmott 2003) and this has had associated impacts on how academics present themselves, including at different career stages (Macfarlane 2020). However, academic development does not happen in isolation, and the construction of the academic self plays out across cohorts through supervisory arrangements (O’Connell, O’Siochru, and Rao 2021) as well as competing institutional pressures and metrics (McCarthy and Dragouni 2020). Nevertheless, the randomized control trial presented here, presents clear evidence that those who have been in academia for 10–20 years view the construction of an academic resume with lower rated journal publications differently to those who have been in academia more and less time, in both disciplines studied.

The results presented here could also be based around different pressures in academic disciplines. Hayes (1983) find a less is better effect amongst psychology scholars for resumes with three high quality journal publications. However, they find no result for those with two high quality publications, and the effect is reversed for those with only one. The resume in his research had four high rated publications. It might be that the threshold to find this effect is now over four publications. Anderson et al. (2019) point to online studies increasing the expectation of frequent psychology publications including in high rated journals. Frequency of publication may also, therefore, be more salient than in 1983, favoring the long resume. Powdthavee, Riyanto, and Knetsch (2018) used five high quality publication which may have also been enough to signal significant quality, notwithstanding the different publication pressures reported by economists (O’Connell, O’Siochru, and Rao 2021). This study did not include economists in the sample.

It should be noted that the research here assumes that writing articles for low rated journals, does not reduce the production of papers for high rated journals (i.e. ‘opportunity cost’). Also, the low rated papers here were still respected journals and not ‘vanity publishing’ (where the author has to pay for their book to be published, or a modern version of open access journals where the effective acceptance rate is extremely high so long as the fee is paid).

5.2. Implications for policy and practice

Pressures continue for academics to present themselves in the best light, including high rated journals (Podsakoff et al. 2018), especially in tenure-track positions (Pietilä 2019). Meanwhile, this is traded off with different role expectations, including teaching, that can have an impact on engagement. Those with higher institutional support for research are the least impacted (Christensen, Dyrstad, and Innstrand 2020; McCarthy and Dragouni 2020). This may influence the extent of engagement with the discourse around the usefulness of lower rated publications (e.g. Walsh 2011) and affect the presentation the academic self at different career stages. Hopwood (2008) noted that counting publications in high rated journals was particularly prevalent in the USA, marginalizing other activities. It is noted that these pressures were also established in the UK, with processes in the USA and UK being likely to go on to influence Europe. The results of this study based on a UK and USA sample may therefore replicate in Europe at a later date, notwithstanding processes in Europe to move away from impact factors (Woolston 2021).

Expectations on how the academic self should be presented have clear implications on how academic resumes are viewed. The data in this research indicates that the formation of recruitment panels could benefit from considering a range of academics from a wider number of years in academia. An academic recruitment panel consisting of academics who have been in academia 10–20 years are likely to have different perceptions on the presentation of low rated journal publications than those who have been in academia more or less time than this. Further research would be useful into the influence of gender differences, suggested in some results here, and whether other disciplines and educational or socio-demographic characteristics of faculty exhibit similar results.

Our results find that low rated journal publications do still add some value to a candidate’s resume and do still hold some worth in the way academic resumes are compiled and reviewed for recruitment.

Disclosure statement

No potential conflict of interest was reported by the author(s).