How soccer scouts identify talented players

ABSTRACT

Scouts of soccer clubs are often the first to identify talented players. However, there is a lack of research on how these scouts assess and predict overall soccer performance. Therefore, we conducted a large-scaled study to examine the process of talent identification among 125 soccer scouts. Through an online self-report questionnaire, scouts were asked about (1) the players’ age at which they can predict players’ soccer performance, (2) the attributes they consider relevant, and (3) the extent to which they predict performance in a structured manner. The most important results are as follows. First, scouts who observed 12-year-old and younger players perceived they could predict at older ages (13.6 years old, on average) whether a player has the potential to become a professional soccer player. This suggests that scouts are aware of the idea that early indicators of later performance are often lacking, yet do advise on selection of players at younger ages. Second, when identifying talented players, scouts considered more easily observable attributes, such as technical attributes. However, scouts described these often in a broad sense rather than in terms of specific predictors of future performance. Finally, scouts reported that they assess attributes of players in a structured manner. Yet, they ultimately based their prediction (i.e. final score) on an intuitive integration of different performance attributes, which is a suboptimal strategy according to existing literature. Taken together, these outcomes provide specific clues to improve the reliability and validity of the scouting process.

Highlights

• Based on a large sample of soccer scouts, we examine three issues that are important in the process of identifying talented soccer players: The age at which good performance predictions can be made, which attributes are relevant predictors, and how performance predictions are formed.

• Scouts who observe players in young age cohorts believe that the age at which they can predict performance is older than the players they typically scout, suggesting that they are aware that early indicators of performance are often lacking.

• Technical performance attributes are considered as most important performance predictors by scouts, but these are often described in a broad - rather than specific - sense.

• Scouts indicate that they predict performance in a structured manner, but form their overall performance prediction on an intuitive integration of different performance attributes, which can be a suboptimal approach.

KEYWORDS:

• Scouting
• selection psychology
• football
• performance prediction
• talent identification
• decision-making

Introduction

Talent identification in soccer is the complex process of recognizing and selecting players that have the greatest potential to excel in the future (Johnston, Wattie, Schorer, & Baker, 2018). In practice, talented players are often identified by talent scouts (Reeves, Littlewood, McRobert, & Roberts, 2018). These scouts typically play an important role in the initial phases of a club’s talent identification process. Scouts mainly observe and assess players who are not yet recruited by elite soccer academies in soccer trials or games (Reeves, McRobert, Lewis, & Roberts, 2019). Based on observations of current soccer performance, they make predictions of players’ future performance to advise on selection decisions (Larkin, Marchant, Syder, & Farrow, 2020). In this sense, the task of a scout differs from that of a coach, who is typically (also) involved in long-term player development processes (Johansson & Fahlén, 2017).

In order to make valid and reliable performance predictions, and to ultimately decide whether a player has the potential to excel, scouts need to go through a process in which they address different issues. Specifically, they must (1) define the age cohort of talented players for which they can predict performance, (2) consider what soccer-specific attributes are relevant predictors of performance and how to assess them, and (3) form an overall performance prediction based on assessments on these predictors. However, little is known about the way in which scouts address these important issues (Larkin & Reeves, 2018; Reeves et al., 2018). Therefore, we conducted a large-scaled study examining the processes of talent identification among soccer scouts.

Predicting performance

The scout’s task to predict future performance of young players is incredibly difficult (Bergkamp, Niessen, den Hartigh, Frencken, & Meijer, 2019). Across different sports, research has shown that athletes develop in different – often nonlinear – ways, and that reliable indicators of future elite performance are often not yet present or developed in young players (Baker, Schorer, & Wattie, 2018; Den Hartigh, Van Dijk, Steenbeek, & Van Geert, 2016; Güllich, 2014). Still, soccer scouts are mainly assigned by their club to identify young (e.g. 13–15-year-old) to very young (younger than 12-year-old) players (Ford et al., 2020). An interesting first question is then whether scouts’ beliefs align with their scouting practices. Specifically, for which age cohort of players do scouts perceive they can make reliable predictions of future soccer performance in the first place?

A second important question, specifically focused on the act of scouting, concerns the operationalization of soccer-specific performance predictors. Although there is a large body of literature on the predictive value of various attributes (Ivarsson et al., 2020; Murr, Raabe, & Höner, 2018b; O’Connor, Larkin, & Mark Williams, 2016), only a few studies have examined what soccer scouts and coaches consider relevant attributes for future performance (Larkin et al., 2020). Larkin and O’Connor (2017), for instance, found that Australian scouts and coaches (n = 20) perceived technical (e.g. first touch, 1-vs-1), psychological (e.g. positive attitude, personality), and several miscellaneous (e.g. X-factor) attributes as most important when identifying under (U)-13 soccer players. In contrast, they deemed motor skills (e.g. speed), physical attributes (e.g. strength), and defensive ability less important within the talent identification process. These findings are in accordance with a recent study by Roberts, McRobert, Lewis, and Reeves (2019b), who found that scouts and coaches (n = 99) considered decision-making, positioning, and passing accuracy more important for central midfielders than physiological attributes such as stamina. Finally, Jokuschies, Gut, and Conzelmann (2017) found that coaches (n = 5) most often named personality-related attributes as talent criteria, whereas few named motor skills or physical attributes. Still, findings across these studies and their included samples were relatively small and diverse. Hence, studying what a large sample of soccer scouts considers key attributes to predict performance is warranted.

A third major question is how scouts score and combine information on these predictors into an overall performance prediction. Since these predictions are essential in the decision to select a player, it is important that they are valid and reliable. Although scouts and coaches account for multidimensional attributes, research suggests that they generally do not assess these attributes in a structured manner when predicting performance. Qualitative studies showed that coaches primarily predicted performance based on their overall impression, intuition, or “coaches eye” (Roberts et al., 2019a). In other words, coaches did not use explicit criteria and relied on holistic performance predictions (Johansson & Fahlén, 2017). Coaches reported that they were able to recognize patterns that resonated with their ideal performance image based on their impressions (Christensen, 2009), and “knew it when they saw it” (Miller, Cronin, & Baker, 2015). Yet, they had difficulty verbalizing what these patterns of performance looked like exactly and how they weighed the performance attributes (Christensen, 2009).

It is interesting to note that the holistic approach can be sub-optimal, because it typically leads to inconsistent predictions within and between decision-makers (Dawes, Faust, & Meehl, 1989; Den Hartigh, Niessen, Frencken, & Meijer, 2018). Relatedly, there is a large body of evidence that shows that reliability and predictive validity improve when prediction processes increase in structure (Dana & Rick, 2006; Huffcutt & Arthur, 1994). Strategies such as explicitly defining criteria, systematically scoring information, and combining scores according to a decision rule are valid ways to improve predictions (Arkes, Schaffer, & Dawes, 2006; Meijer, Neumann, Hemker, & Niessen, 2020). In sports, few studies have evaluated to what extent scouts apply these strategies to reach their final performance prediction (see MacMahon, Bailey, Croser, & Weissensteiner, 2019 for an exception).

The current study

Based on the questions above, we aimed to explore – through a self-report measure – how soccer scouts identify talented players. In line with the difficulty of predicting future performance of young players, we first examined at what age scouts perceive they can predict a player’s performance. Second, we analysed what attributes scouts consider to be important for future performance. Finally, we examined to what extent scouts report scoring and combining this information in a structured manner. We therefore conducted a large-scaled study among soccer scouts across the Netherlands.

Materials and methods

Participants

Ethical approval was granted by the Ethical Committee of Psychology, University of Groningen (code PSY-1819-S-0024). We recruited professional and part-time scouts from professional soccer clubs and scouts associated with The Royal Dutch Football Association (KNVB). First, heads of scouting of ten different clubs in the Dutch Eredivisie were approached by e-mail, of which four distributed a digital questionnaire to their organization’s scouts. These scouts are responsible for identifying players for the club’s developmental academy or first team. Second, four scouting coordinators of the KNVB were approached and agreed to distribute the questionnaire to their regional scouts. These regional scouts are responsible for identifying players for KNVB’s “Youth Plan Netherlands” (JPN) program. JPN is a platform which targets talented youth players from under U11 to U17 (for girls U16) who have not yet been recruited by a professional soccer club. A total of 125 scouts responded and completed the questionnaire. Almost all scouts (n = 123, 98%) indicated they were male, and most of them (n = 110, 88%) scouted male players. Scouts were on average 58.2 years old (SD = 12.3) and had 11.2 years of experience (SD = 8.39). Furthermore n = 63 (50%) observed players in the U12 and younger age cohort, n = 45 (26%) in the U13-U15 cohort, n = 9 (7%) in the U16-U18 cohort and n = 8 (6%) observed adult players.

Measures

A digital questionnaire was distributed via Qualtrics (Qualtrics, Provo, Utah). Before distribution, the questions were reviewed by four JPN scouts and two scouts of a professional soccer academy – who were also included in the sample – to improve terminology, consistency, and clarity. In total, the questionnaire consisted of 8 questions (2 open-ended, 1 rank, and 5 multiple-choice questions) divided across three sections.

Table 1 presents the different questions and response scales per section of the questionnaire. Participants completed the questionnaire at their own discretion. The questionnaire opened on 11-03-2019 and closed on 31-05-2019. In the first section scouts were asked “at what age can you reliably predict if a player has the potential to participate in professional soccer?” The second section consisted of two questions asking scouts about the information they take into account when predicting performance. Finally, the third section contained five statements focusing on the extent to which scouts predict performance in a structured manner. Previous studies in other contexts (e.g. in job interviewing, Chapman & Zweig, 2005) found that applying structure was not a unidimensional construct, but consisted of different components. As such, we analysed the single-item scores, instead of treating the statements as one or multiple scales (see Table 1).

Table 1. Questions in the questionnaire, per section of the questionnaire.

Section	Question number^a	Question	Scale
Scouts perception of predicting performance	1	“At what age can you reliably predict if a player has the potential to participate in professional soccer?”	Age in years (e.g. 14 or 17 years old)
Attributes relevant for future performance	1	“Describe a maximum of five attributes that you take into account when observing a player in your respective age cohort and that you consider to be predictive of future soccer performance”	Open
	2	“Please rank the attributes you described in the previous question from 1 = most predictive to 5 = least predictive”	Rank
Scoring and combining information	1	“Before observing a player, I already know which attributes I will evaluate”	Likert (1 = never to 5 = always)
	2	“When observing a player, I evaluate each attribute I find important separately”
	3	“I evaluate different players – of the same age and playing position – on the same attributes”
	4	“After observing a player, I sum my scores on the independently evaluated attributes to form my final prediction”
	5	“After observing a player, I use my overall impression of the player’s attributes to form my final prediction”

^a The question number per section of the questionnaire.

Statistical analysis

We computed means and standard deviations to examine the spread in age at which scouts perceived they could predict if a player has the potential to participate in professional soccer. These responses were stratified according to the age cohorts typically observed by the scouts.

In order to assess the frequency, variety, and importance of the attributes that scouts considered predictive of future soccer performance, the first two authors simultaneously categorized each attribute based on its descriptive content. Five performance categories emerged when exploring the attributes, namely (1) technical, (2) tactical and perceptual-cognitive skills, (3) personality-related and mental skills, (4) physical, physiological, and motor skills, and (5) “miscellaneous” attributes. Similar categories are frequently identified in the soccer talent literature when discussing potential performance predictors (e.g. Murr et al., 2018b; Murr, Feichtinger, Larkin, O’Connor, & Höner, 2018a; Williams & Reilly, 2000). Answers that varied in description, but were similar in content and context, were grouped together in a single attribute construct (e.g. “positioning on offense” and “moving without the ball in offense”) based on previous literature (Larkin & O’Connor, 2017; Murr et al., 2018a; Murr et al., 2018b; Roberts et al., 2019b). Then, we assessed the frequency of each attribute, as well as the number of times that attribute was considered to be the most important predictor of future performance by a scout (i.e. being ranked as the first attribute). Finally, to assess the level of detail in the scouts’ answers, each attribute was either rated as “general” when describing a domain (e.g. “technical skills or abilities”) or “specific,” when describing a skill or ability (e.g. “pass accuracy”). In order to assess the inter-rater reliability of this coding process, a random sample of k = 90 answers (approximately 15%) were translated, grouped together, and rated on specificity, by the first and last author, independently. This yielded a Cohen’s Kappa of 0.94, which indicates excellent reliability. The remaining answers were coded by the first author.

Finally, to examine the extent to which scouts score and combine information in a structured manner, we first looked at the response percentages to each statement on the structure of the talent identification process. Then, we computed Spearman’s correlations between the statements. These correlations provide information on whether the scouts apply the different statements uniformly and consistently. For instance, do scouts who know beforehand which attributes to assess also assess each attribute separately?

Results

Scouts’ perceptions of predicting future performance

Figure 1 presents scouts’ answers on the age at which they can predict if a player has the potential to participate in professional soccer (i.e. predict future performance). The findings are stratified according to the age cohort in which each scout typically observed players. The results show that the average age at which scouts perceived they could predict future performance increased depending on the age cohort they observed players in. More specifically, scouts who typically observed U12 and younger players perceived, on average, that they could reliably predict a player’s future performance at 13.6 (SD = 2.10) years old; for scouts who observed U13-U15 players this was 14.2 (SD = 1.84) years old; for scouts who observed U16 – U18 year old players this was 15 (SD = 1.80) years old, and for scouts who observed adult players this was 16.8 (SD = 1.28) years old. Interestingly, most of the scouts (63 out of 125) observed players in the U12 and younger cohort. Thus, the largest group of scouts perceived they could predict future performance for players that were older (i.e. 13.6 years on average) than the players they typically observed in practice.

Figure 1. Age at which scouts perceive they can predict performance, stratified by age cohort of players scouted (error bars indicate ± SD). Each dot indicates a scout answer.

Attributes relevant for future performance

The attributes that scouts considered predictive of future performance were very similar across age cohorts (see supplemental material). Therefore, we present results for the total sample here.

Table 2 presents the frequency (k) with which each attribute was mentioned and the number of times each attribute was considered to be most predictive (1st) of future performance, grouped by performance category. The scouts mentioned a wide variety of attributes: after grouping similar answers together, a total of 59 attributes were identified.

Table 2. Attributes scouts considered predictive of future soccer performance, in terms of total frequency (k) and the number of times each attribute was considered most predictive (1st).

Performance category	Attribute	k	1st
Technical	Technical skills or technique with the ball^a	82 (50%)	34 (74%)
	Ball control	25 (15%)	6 (13%)
	(Skills related to) transitioning^a	11 (7%)	1 (>2%)
	(Skills related to) defending^a	9 (6%)	1 (>2%)
	Pass intention or accuracy	9 (6%)	1 (>2%)
	First touch	6 (%)	0 (0%)
	(Skills and abilities related to) attacking^a	5 (3%)	1 (>2%)
	Shooting or shot technique	5 (3%)	0 (0%)
	Two legged	3 (2%)	1 (>2%)
	Dribbling	2 (1%)	0 (0%)
	Applying pressure	1 (<1%)	0 (0%)
	Blocking	1 (<1%)	0 (0%)
	Building up offensively	1 (<1%)	0 (0%)
	Disrupting the offensive build up	1 (<1%)	1 (>2%)
	Preventing goal scoring opportunities	1 (<1%)	0 (0%)
	Scoring goals	1 (<1%)	0 (0%)
	Performance category total	163 (28%)	46 (37%)
Tactical and perceptual-cognitive	Game sense and awareness	53 (40%)	11 (39%)
	Speed of handling	23 (17%)	7 (25%)
	Positioning or moving without the ball	19 (14%)	2 (7%)
	Vision, perception, seeing teammates and opponents, gaze behaviour	19 (14%)	2 (7%)
	Decision-making	8 (6%)	5 (18%)
	Tactical skills^a	6 (5%)	0 (0%)
	Football intelligence	4 (>3%)	1 (4%)
	Performance category total	132 (22%)	28 (22%)
Physical, physiological, and motor skills	Physiological or motor skills^a	38 (30%)	15 (58%)
	Sprinting speed	36 (28%)	4 (15%)
	Physical attributes^a	23 (18%)	2 (8%)
	Coordination	7 (5%)	0 (0%)
	Body composition or athletic build	6 (5%)	2 (8%)
	Agility	4 (3%)	1 (<4%)
	Strength in duels	4 (3%)	0 (0%)
	Explosiveness	3 (2%)	0 (0%)
	Length	3 (2%)	1 (<4%)
	Mobility	2 (2%)	0 (0%)
	Movement rhythm	1 (1%)	1 (<4%)
	Stability	1 (1%)	0 (0%)
	Performance category total	128 (22%)	26 (21%)
Personality-related and mental skills	Winning mindset or mentality	32 (26%)	6 (33%)
	Drive or intrinsic motivation	31 (25%)	9 (50%)
	Personality-related attributes^a	17 (14%)	1 (<6%)
	Perseverance, resilience, or toughness	11 (9%)	1 (<6%)
	Behaviour on and off the pitch	7 (6%)	1 (<6%)
	Coachability or fast learner	7 (6%)	0 (0%)
	Assertiveness or dominance	5 (4%)	0 (0%)
	Coaching other players or leadership	5 (4%)	0 (0%)
	Positive attitude	4 (3%)	0 (0%)
	Performance or goal oriented	2 (<2%)	0 (0%)
	Focus or concentration	2 (<2%)	0 (0%)
	Self-confidence	1 (<1%)	0 (0%)
	Performance category total	124 (21%)	18 (14%)
Miscellaneous	Team understanding, involving teammates	12 (26%)	1 (14%)
	Communication	10 (21%)	0 (0%)
	Undefined^b	8 (17%)	3 (43%)
	X-factor	5 (11%)	0 (0%)
	Innate talent (nature)	3 (6%)	2 (29%)
	Adaptability	2 (4%)	0 (0%)
	Biological age	2 (4%)	0 (0%)
	Calendar age	2 (4%)	1 (14%)
	Appearance	1 (>2%)	0 (0%)
	Education level	1 (>2%)	0 (0%)
	Lifestyle	1 (>2%)	0 (0%)
	Performance category total	47 (8%)	7 (6%)
	Grand total	594	125

Notes: Results are presented as absolute number of answers with percentage in brackets. Percentages per attribute refer to the percentage within performance category, whereas percentages per "performance category total" row refer to the percentage of grand total number of answers. Note: the total frequency for the attributes does not sum to k = 625 (i.e. 5 × 125), because multiple scouts listed fewer than 5 predictors.

^a Indicates an answer that can be considered a “general” domain, rather than a more specific predictor.

^b Answers that did not contain enough content information to be considered a predictor and could not be assigned to a performance category (e.g. “matching the playing style of club […]”).

The nine most frequently named attributes were technical skills or technique with the ball (k = 82, 1st = 34), game sense and awareness (k = 53, 1st = 11), physiological or motor skills (k = 38, 1st = 15), sprinting speed, (k = 36, 1st = 4), winning mindset or mentality (k = 32, 1st = 6), drive or intrinsic motivation (k = 31, 1st = 9), ball control (k = 25 1st = 6), speed of handling (k = 23, 1st = 7), and physical attributes (k = 23 1st = 2). Thus, scouts provided both general, non-specific attributes (e.g. technical skills or technique with the ball and physiological or motor skills) and more specific attributes (e.g. sprinting speed, ball control, and winning mindset or mentality).

Concerning the general performance categories, scouts mainly considered attributes in the technical performance category as predictors of future performance: A total of 163 (28%) answers belonged to this category. This was followed by 132 (22%) answers that belonged to the tactical and perceptual-cognitive skills, 128 (22%) to physical, physiological, and motor skills, 124 (21%) to personality-related and mental skills, and 47 (8%) to the miscellaneous category. Moreover, 46 of the 125 scouts (37%) ranked an attribute in the technical category as the most important predictor, followed by a tactical and perceptual-cognitive skill (n = 28, 22%), a physical, physiological, and motor skill (n = 26, 21%), a personality-related and mental skill (n = 18, 14%), and a miscellaneous attribute (n = 7, 6%). Thus, a technical skill was mentioned most often as the most important predictor. Tactical and perceptual-cognitive skills, physical, physiological, and motor skills and personality-related or mental skills were roughly equally distributed as the most important among the remaining scouts, and a small minority mentioned a miscellaneous attribute as most predictive.¹

Scoring and combining information

Figure 2 presents the response percentages to the statements on the different aspects of structure in scouts’ talent identification process. Overall, the scouts indicated that they applied a very structured process when observing players. Approximately 74% of the scouts indicated that they “always” or “very frequently” evaluated different players – of the same age and playing position – on the same attributes, and 73% indicated that they already knew which attributes they would evaluate before they observed a player. Moreover, 69% of the scouts indicated to always or very frequently evaluate different attributes separately, when observing a player. Although the scouts seemed to apply a structured approach in defining and evaluating separate skills and abilities, they mainly used their overall impression of the player’s attributes to form their final prediction, as 68% always or very frequently took this approach. Accordingly, a minority of 41% always or very frequently summed the independently evaluated attributes to form their final prediction.

Figure 2. Response percentages to the statements on the different aspects of structures that scouts apply when assessing players.

The Spearman’s correlations between different aspects of structure that scouts used when identifying talent were relatively low (between 0.03 and 0.45, see Table 5 supplemental material). This suggests that applying structure cannot be seen as a single construct, and that scouts did not uniformly apply all aspects when observing players. For example, we found a relatively small correlation between the statements “When observing a player, I evaluate each attribute I find important separately” and “Before observing a player, I already know which attributes I will evaluate” (r_s = 0.23, 95% CI = 0.05; 0.39).

Discussion

The current study examined three issues that are important to how soccer scouts identify talented players. Based on self-report data, we analysed at which players’ age soccer scouts perceive they can predict future performance; what attributes they consider to be important for future performance; and to what extent they score and combine assessments on these attributes in a structured manner.

Our results showed that the average age at which scouts perceive they could predict performance increased depending on the age cohort in which they scouted. The average age to predict performance fell within the age interval of these cohorts, with the exception of the largest cohort, that is, the U12 and younger cohort. Notably, there was a discrepancy between the player’s age at which many scouts in this cohort perceived they could reliably predict future performance (i.e. 13.6 years old, on average), and the actual age at which they scouted players (i.e. younger than 12 years old). This finding suggests that these scouts are aware of the idea that early indicators of later performance are often lacking or hard to predict (Abbott, Button, Pepping, & Collins, 2005; Den Hartigh et al., 2016). Yet, scouts do assess and advise on selection of players at younger ages.

One explanation for this discrepancy is that – given the difficulty of predicting future performance directly – those who scout in the younger age cohorts may be more concerned with finding the best current player, rather than finding the best player for the future (Ford et al., 2020). However, given that clubs invest substantial resources in developing these players over non-selected players, this approach seems to rely on the assumption that the best current young players are also those that have the highest potential for excellence in the future. It should also be noted that this assumption implies an inconsistency of thought: scouts are still indirectly making a prediction when assuming that the best current players are also the ones with the highest potential. Moreover, since the attributes needed for excellence are often unstable, develop non-linearly over time, and may not even be present in young players (Abbott et al., 2005; Den Hartigh et al., 2016; Simonton, 1999), selecting the best current players at a young age could harm the selection process. In sum, the finding that many scouts do not perceive they can predict performance for the players they scout, raises questions about the early (i.e. pre-pubertal) talent identification process (Güllich & Cobley, 2017).

Furthermore, although there seemed to be no apparent differences between scouts in the different age cohorts, we showed that scouts across cohorts consider a multidimensional range of soccer-related attributes when predicting performance. The five most frequently named attributes covered four major performance categories: technical skills or technique with the ball (i.e. technical) game sense and awareness (i.e. tactical and perceptual-cognitive skills), physiological or motor skills and sprinting speed (i.e. physical, physiological, and motor skills), and winning mindset or mentality (i.e. personality-related and mental skills). When examining the general performance categories, scouts mainly reported considering attributes in the technical performance category as predictors of future soccer performance. This was followed by tactical and perceptual-cognitive skills, physical, physiological, and motor skills, personality-related and mental skills, which were considered most important approximately equally often. The emphasis on technical attributes is encouraging, as these attributes have been shown to have relatively good predictive value in match play (Bergkamp, den Hartigh, Frencken, Niessen, & Meijer, 2020), and in specific technical tasks where they may be less influenced by maturational timing (Murr et al., 2018a; Vandendriessche et al., 2012).

On the other hand, the relative importance given to physical, physiological and motor skills differs from findings by Larkin and O’Connor (2017), Roberts et al. (2019b) and Jokuschies et al. (2017). For instance, sprinting speed was a frequently named attribute in our sample (named by 36 of the 125 scouts), but was excluded from the final list (together with agility and strength) by Larkin and O’Connor (2017), because it was not considered important enough by the coaches and scouts. It can also be argued that the tendency of clubs to systematically select older or more mature players indicates that scouts (implicitly) consider physical attributes as most important in practice. The emphasis on physical and physiological attributes in this way can be particularly problematic for young players, because of the large inter-individual differences that result from maturity status and relative age, which reduce after puberty (Deprez et al., 2015). Therefore, both biological and calendar age need to be taken into account when assessing the physical and physiological attributes of young players (Meylan, Cronin, Oliver, & Hughes, 2010).

Interestingly, scouts generally indicated that they predict performance by assessing the attributes in a structured manner. A majority of scouts indicated to (a) always, or very frequently, evaluate different players – of the same age and playing position – on the same attributes, (b) know which attributes they would assess before observing a player, and (c) evaluate different attributes separately. These aspects are important for maintaining high levels of inter and intra-rater reliability when assessing performance, and are therefore encouraging (Huffcutt & Arthur, 1994). However, there are three remarks regarding this finding.

First, while scouts claimed to systematically assess players on different attributes, it remains an open question how well they define those attributes, and if they do this explicitly or implicitly. It appeared that scouts often placed general domains on the attribute list (e.g. technical skills or technique with the ball) while fewer provided specific examples of skills and abilities that belonged to those domains. Thus, scouts may have had difficulty verbalizing in detail what attributes they considered important predictors of future performance, which suggests that they implicitly integrate various attributes in their mind. This would be in line with the way coaches identify talent (Christensen, 2009; Johansson & Fahlén, 2017), and is an indication of the holistic approach to predicting performance (Dana & Rick, 2006). For example, it is likely that skills and abilities considered to belong to “technique,” such as passing, dribbling, tackling, differ from scout to scout. Consequently, when assessing technique in this way, it may affect the reliability within and between scouts (cf. Chapman & Zweig, 2005).

Second, most scouts combined their assessments into a performance prediction based on their overall impression, as opposed to a sum of the independently assessed attributes. While predictions based on combining attributes according to a decision rule (e.g. summing scores on attributes) have been shown to outperform predictions based on overall impressions and intuition in holistic approaches (Arkes et al., 2006; Kuncel, Klieger, Connelly, & Ones, 2013), the latter are commonplace across selection contexts (Dana, Dawes, & Peterson, 2013). Therefore, it is not surprising that scouts in this study also applied this approach. Nevertheless, the predictive validity and reliability of scouts’ performance predictions may improve further if they use a decision rule to combine information (i.e. actuarial judgment, see Den Hartigh et al., 2018; for an explanation outside sports Meijer et al., 2020).

Finally, the low correlations between the statements suggest that scouts did not uniformly apply all aspects of structure. For example, most scouts who knew beforehand which attributes they were going to assess did not also evaluate different players – of the same age and position – on the same attributes, or evaluate each attribute separately. Thus, different scouts applied different aspects of structure, whereas literature suggests that predictions may become more consistent if scouts apply all aspects (Chapman & Zweig, 2005).

Limitations & conclusion

The main limitation of this study is that it assessed the talent identification process of scouts through self-report. This carries the risk that respondents are constrained in their self-knowledge (Paulhus & Vazire, 2007) or provide socially desirable responses. Including qualitative data could have provided additional insights into why scouts hold the perceptions that were found in this study and whether these align with what scouts do in practice (cf. Larkin et al., 2020; MacMahon et al., 2019; Roberts et al., 2019a). For example, in-depth interviews or think-aloud protocols could reveal what type of player scouts generally are selecting for (i.e. best player available or best long-term prospect), and their perception on how these selection strategies relate to each other (cf. Reeves et al., 2019). Additionally, observing scouts in practice could show to what extent their perceptions of applying structure align with what they actually do. Finally, an interesting avenue for future research is to consider the reliability and validity of scouts’ judgments. In such a design it would be necessary to collect the predictions of scouts and relate these to the future performance of players longitudinally (e.g. see whether players they picked actually reached the professional status).

A second possible limitation concerns the lack of detail in the predictors considered by scouts. This lack of detail may relate to the instruction in the questionnaire, as we did not want to steer scouts in a specific direction in section two of the questionnaire. Therefore, scouts were free to describe predictors in any way they wished, which resulted in varying levels of specificity for the attributes described. A final limitation is that we measured different aspects of structure using single item-scores, for brevity purposes. However, this meant that we were not able to compute reliability estimates over these items. Future research should consider measuring different aspects of structure with multiple items to compute reliability estimates (Chapman & Zweig, 2005).

The current study concludes the following regarding the process of talent identification in soccer scouts. First, previous literature has shown that early indicators of later performance can be unreliable (Den Hartigh et al., 2016; Simonton, 1999). In line with this literature, we showed that most scouts who observe younger players (i.e. U12 and younger) perceive they cannot reliably predict performance for the players they typically scout. Accordingly, we recommend that soccer organizations invest in the continuous (de)-selection of players across all age cohorts, and consider targeting post-pubertal players more often than is currently the norm (Güllich, 2014).

Second, considering the predictors that scouts say they find relevant, they value a multidimensional collection of attributes, but mostly account for general technical soccer attributes. Additionally, they seem to have difficulty formulating specific predictors of performance and likely integrate various attributes in their mind. Third, scouts report adopting a generally structured approach when scouting players, but do not apply the different structuring approaches uniformly, and mainly use their overall impression of the attributes to form their final predictions (i.e. holistic assessment). Given previous literature demonstrating that predictions based on overall “intuitive” impressions are non-optimal in terms of reliability and validity, we recommend that scouts are trained in a more consistent use of the different aspects of structure when predicting performance. For instance, soccer organizations could create more opportunities for scouts to train themselves in formulating specific predictors of future performance, and to systematically score and combine these predictors according to a decision-rule (Den Hartigh et al., 2018). We believe these recommendations will improve the reliability and predictive validity of scouts’ predictions in the future.

Acknowledgements

The authors would like to thank the heads of scouting of the Eredivisie clubs and talent performance coaches of Jeugdplan Nederland who assisted in reviewing early versions of the questionnaire and distributed the questionnaire to their scouts.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The dataset associated with the main analyses can be accessed through a Dataverse repository (https://doi.org/10.34894/LKDXFD). The data has been made available without restrictions and can be accessed by all interested parties.

Additional information

Funding

This research was partially funded by the Royal Dutch Football Association (Koninklijke Nederlandse Voetbalbond, KNVB, www.knvb.com). The KNVB did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Notes

1 Based on a suggestion by the reviewers, we conducted an additional analysis exploring the relationship between the five attributes named by each scout. Specifically, we examined whether there were common clusters of sequences in which the attributes, as categorized by their performance category, were listed. For example, do scouts that first list an attribute in the technical performance category also list a technical attribute as the second to fifth predictor? However, we did not find clusters that were shared among groups of scouts, instead finding a unique sequence for approximately 120 of the 125 scouts.