Assessing leadership decision-making styles: psychometric properties of the Leadership Judgement Indicator

Abstract

This study aimed to validate the Italian version of the Leadership Judgement Indicator, an unconventional instrument devoted to measurement of leaders’ judgments and preferred styles, ie, directive, consultative, consensual, or delegative, when dealing with a range of decision-making scenarios. After forward-translation and back-translation, its psychometric properties were estimated for 299 managers at various levels, who were asked to put themselves in the position of leader and to rate the appropriateness of certain ways of responding to challenge. Differences between several groups of managers, ranked in order of seniority, provided evidence for discriminant validity. Internal consistency was adequate. The findings show that the Italian adaptation of the Leadership Judgement Indicator has promising psychometric qualities, suggesting its suitability for use to improve outcomes in both organizational and selection settings.

Keywords:

• Leadership Judgement Indicator
• decision-making
• situational test
• scenarios
• psychometric properties

Introduction

As the concept of team-working has become increasingly important in organizational life, leadership, as the ability to lead and to exert interpersonal influence, has become a particularly critical requirement for creating an environment where individuals and teams have the capacity and opportunity to succeed.

The leadership model on which the Leadership Judgement Indicator (LJI)^1–3 is based is drawn from interactional theories. This perspective examines the interplay between leaders’ personality traits and the behavior and style they adopt, situational contingencies connected to the characteristics of the organizational context, the nature of the task concerned, and the abilities and experience of the people involved.^4,5

The LJI leadership model focuses on the ways in which the leader faces decision-making situations with regard to the team that he or she is seeking to lead or influence (see Figure 1). It provides a principle-driven approach that envisages which leadership style is most likely to be effective in any specific set of circumstances.

Figure 1 Decision-making styles underlying the Leadership Judgement Indicator.

The LJI assesses four main leadership styles, ie, directive leadership, consultative leadership, consensual leadership, and delegative leadership, each of which is divided into two facets, leading to eight LJI model substyles (unassisted directive leadership, researched directive leadership, one-to-one consultative leadership, group consultative leadership, chaired consensual leadership, team player consensual leadership, informed delegative leadership, and ballistic delegative leadership).

Directive leadership is generally most effective with a newly formed team, or one that is facing unfamiliar situations. It is a particularly useful style when an experienced leader is faced with many decisions. Failure to use this style may be perceived as a lack of focus and direction. On the other hand, too much directive orientation can lead team members to develop low esteem and do only as they are told and to lack initiative, creativity, or self-confidence. Unassisted directive leadership is a very leader-centered approach, based completely on the leader’s own ideas without any supporting information from colleagues. Researched directive leadership is employed when any necessary information is obtained from colleagues before decision-making.

Consultative leadership is good when the leader needs to take others’ views into account, but the ultimate decision needs to rest in the leader’s own hands. This is a particularly important style with a developing team because the members’ level of knowledge and understanding will be increasing but they may not yet have the necessary experience or alignment with the organization’s values to make optimum decisions on important unfamiliar issues. Low levels of consultation can lead to team members finding that their skills are underutilized. Overuse of this style can be seen as poor decision-making and is very time-consuming. One-to-one consultative leadership involves drawing colleagues’ point of views before the leader makes the decision, discussing the problem with team members individually without meeting with them as a group. Group consultative leadership involves gathering the team together and listening to colleagues’ opinions before making the decision.

Consensual leadership is particularly important for engendering ownership and commitment throughout an experienced team, especially when facing situations that need to be viewed from a number of different perspectives. Insufficient use of this orientation can result in loss of team skills and less involvement, which can have a damaging effect on motivation. Overuse of this style can lead to a perceived lack of clear leadership in which the leader can be considered as unable to make a decision without referring to others first. Chaired consensual leadership involves the leader leading a collaborative process whereby all team members participate in making the decision. Team player consensual leadership involves the leader becoming one of the team by arranging a team meeting to discuss and debate the problem with the chair rotating around the team.

Delegative leadership is particularly valuable with an experienced team that may have greater technical expertise than the leader on particular aspects of the job. If underused, this style can result in too little sharing of responsibility. This could lead to overload for the leader and to less self-confidence and a loss of team skills for the team members. If used too much, on the other hand, this style can lead to lack of control and loss of authority. It may lead to less respect for the leader and stress for the team members, as they may be unsure of their readiness for the whole responsibility for the specific task. Informed delegative leadership involves the leader meeting colleagues to provide them with necessary information, expectations, and objectives. The team can proceed then autonomously with problem-solving, but keeping the leader informed about the work in progress. Ballistic delegative leadership involves an initial meeting with the team members in order to establish the goals to be achieved. The team is then required to solve the problem, returning back to the leader only when a decision is required.

The present account aims to evaluate the psychometric qualities of the Italian version of the LJI, an unconventional instrument devoted to measure leaders’ judgment and preferred styles, ie, directive, consultative, consensual, or delegative, when dealing with a range of decision-making scenarios. In particular, the study aims to describe reliability, in terms of internal consistency and criterion-related validity involving seniority. It was hypothesized that more senior managers would show higher LJI scores than junior managers for two probable reasons: having greater experience and having been more rigorously selected on the basis of their leadership skills.

Materials and methods

Participants and procedure

The study included 299 managers at various levels, who were asked to put themselves in the position of leader and to rate the appropriateness of certain ways of responding to challenge.

Participants were drawn from a target population of workers who already had positions of organizational power and influence. Letters of invitation were sent to some companies to seek authorization to collect data. Based on the allowed licenses, subjects were asked to complete questionnaires according to the paper-and-pencil procedure.

Instrument

The LJI principally explores whether the leader can gear his or her personal style to the characteristics of the people involved and the nature of the task in a variety of decision-making contexts, and then come to an effective decision, and so improve performance.

The LJI assesses the respondent’s preference for using each of the four main styles within the leadership model. It then analyses this further, according to the eight substyles of the model. In addition, the instrument is used to identify the leadership judgment exercised by the participant in choosing when to use each of these styles and substyles. Since not one of these styles is any more universally applicable than another, successful leaders are those who use their judgment effectively in selecting an appropriate style for any particular set of circumstances.

Unlike traditional measures, situational tests present complex scenarios and offer a number of alternative behavioral answers to be assessed on the basis of judgment of appropriateness.⁶

The LJI comprises 16 leadership decision-making scenarios. For each scenario, four alternative courses of action are presented and the respondent is asked to rate the appropriateness of each option. The available options represent different leadership judgment styles. One of the possible solutions given is highly appropriate according to the LJI leadership approach, whereas the others are rather less appropriate and vary in their quality.

Here is an example of a scenario and the alternative responses.

Example

You manage a small business in which you have three employees. The office needs covering over the forthcoming school holidays and last year there were problems. All three employees seemed only interested in themselves and their choice of annual leave time created considerable disagreement. You anticipate similar difficulties this year, which could place you under pressure.

Available decision-making methods:

1. do not risk debate this year, but tell them when they can have leave

2. tell them the office should always be covered and get them to come up with a holiday schedule

3. call a meeting with all present and come to an arrangement that is acceptable to everyone

4. find out each person’s preferred holiday time and then you decide on the holiday times.

Respondents rate the appropriateness of each of the four options using the following rating scale:

1. Totally inappropriate

2. Inappropriate

3. Unsure

4. Appropriate

5. Highly appropriate.

In this example, a respondent might choose to rate option (c) as 2 because they believe this approach will only exacerbate self-interest and the arguments will continue. They may believe that (a) will produce a solution that nobody is happy with, so will rate it as 1. They may feel that solution (b) might just create the same problem as last year but is worth another attempt, so rate it as 4. However, they think that (d) is the best option in this circumstance, so rate it as 5.

Test administration is without time limits. However, the 64 items in the LJI usually take people 40 minutes to complete.

For each scenario, the respondents are asked to rate the quality of four ways of dealing with the situation, using a five-point Likert scale (1, totally inappropriate; 5, highly appropriate). The scoring procedure produces two types of scores: the judgment score, ie, how well individuals are able to assess the most effective leadership style as the most appropriate, and the preference score, ie, the frequency with which the person completing the LJI has endorsed the 4 (appropriate) or 5 (highly appropriate) ratings, thus revealing the extent to which one adopts a particular leadership strategy.

Data analysis

After forward-translation and back-translation, the psychometric properties of the LJI were estimated from the data collected. Internal consistency reliability was calculated using Cronbach’s alpha. Criterion-related validity involved analysis of variance of the mean LJI total judgment scores of the managerial groups examined in order of seniority. Correlations between preference scores reflecting the tendency to use each of the four management substyles were also explored.

Results

Participants demographics

Participants comprised 53.8% males and 46.2% females, with a mean age of 44.68±9.3 (range 24–75) years. As regards seniority, managerial groups were divided into executive managers (21.3%), senior managers (37.9%), employers (35.5%), and management consultants (5.3%) (see Figure 2). The mean number of people coordinated was 11.79±19.3 and their mean number of years working in their organization was 13.61±9.5 (range 0–44).

Figure 2 Managerial groups.

Internal consistency reliability

The LJI internal consistency in terms of coefficient alpha was 0.66. The reliability of the test is related to the correlation between the observed score and the true score; the square root of the reliability coefficient is a fairly close estimate of this correlation, so long as the sample size is larger than about 200.⁷ The reliability of the LJI was estimated from a sample of 299 managers at various levels. Thus, the correlation between our participants’ scores and their true levels of leadership judgment is 0.81.

Criterion-related validity

In order to determine criterion-related validity, we related LJI scores to a real-life measure. Participants were divided into managerial groups ranked in order of seniority. It was expected that the more senior managers would show higher LJI scores than junior managers, based on both greater experience and more rigorous selection processes for their job placement. Table 1 shows the LJI total judgment scores obtained by each occupational group. Based on analysis of variance, the average scores of the various groups seem different in the predicted direction. As expected the findings indicate that the groups differed in their performance on the LJI, with a gradual decrease in scores from the highest-ranked managers to the lowest-ranked management group [F(4, 269)=6.60, P<0.001].

Table 1 Descriptive Leadership Judgement Indicator total judgement scores for each group of managers

Occupation	Mean	SD	SE
Executive managers	0.66	0.29	0.04
Senior managers	0.59	0.29	0.03
Employers	0.54	0.30	0.03
Management consultants	0.30	0.30	0.08

Abbreviations: SD, standard deviation; SE, standard error of the mean.

Correlations between LJI styles

Table 2 shows the correlations obtained between preference scores, which reflect the tendency to use each of the four management styles. According to our results, both the directive and consultative styles (r =0.219, P<0.01) and the consultative and consensual styles (r=0.233, P<0.01) showed moderate positive correlations. A negative although significant low correlation was found between the consultative and delegative styles (r=−0.187, P<0.01) and between the directive and consensual styles (r =−0.188, P<0.01), and a smaller but still significant positive correlation was found between the tendency to use the delegative and consensual styles (r =0.128, P<0.05). No correlation was found for the tendency to use delegative and directive styles.

Table 2 Correlations between preference scores

	Directive preference	Consultative preference	Consensual preference	Delegative preference
Directive preference	–
Consultative preference	0.219**	–
Consensual preference	−0.188**	0.233**	–
Delegative preference	−0.033	−0.187**	0.128*	–

Notes:

* P<0.05

** P<0.01.

Adverse impact

To check for adverse impact, the average overall leadership performance of 161 males was compared with that of 138 females. The average total judgment score for the male sample was 0.59±0.30 and the average total judgment score for the female sample was 0.58±0.31. The difference between these groups is therefore approximately 0.04, which is very small indeed. The results of an independent-samples t-test [t (276)=0.287, P=0.774] revealed no statistically significant difference in test scores between males and females.

Representation of scores

Figure 3 shows the participants’ mean leadership preference scores. Figure 4 provides a picture of the mean leadership judgment scores for the group of participants. Substyles statistics offer another level of analysis to inform training needs. Substyles are shown to the right of the corresponding main style.

Figure 3 Averaged preference scores.

Figure 4 Averaged judgement scores.

Discussion

In view of a lot of earlier research suggesting the need to adapt psychologic tests for cross-cultural assessment in education, industrial, and clinical settings,^8–18 the current paper aimed to determine the psychometric properties of the Italian version of the LJI, a situational test developed to measure leadership judgment ability. The LJI enables assessment of a leader’s judgment and preferred strategies when in decision-making situations. In particular, the instrument measures the capacity to discern accurately the most appropriate leadership style whilst engaging with different people in different circumstances, as well as the level of preference for each leadership way of actions.

The findings of this study indicate that the Italian adaptation of the LJI has fairly good psychometric properties, with acceptable internal consistency. Reliability statistics show how accurate a person’s score is likely to be, ie, how closely an individual’s score on a measure reflects their true attitude or ability. According to our results, scores on the test provide a good approximation of a person’s true level of leadership judgment. A test–retest study to assess reliability in terms of temporal stability (ie, how stable scores are over time) is planned for the future.

Criterion-related validity was supported by empiric evidence that scores on the LJI are related to a person’s position within the management hierarchy. It seems that more senior managers (ie, individuals who are heavily involved in management) perform much better than junior managers, supporting the validity of the LJI as a tool for detecting an individual’s current level of management expertise. However, although in line with theoretical expectations, these findings should be interpreted with some caution. While outcomes indeed do provide some support for the underlying theory of the instrument and its ability to measure accurately, it is possible that seniority in management does not necessarily equate to good decision-making. Correlations between LJI styles are in the expected directions on the basis of the adopted theoretical model, providing support for construct validity.

Unlike traditional measures of ability, situational tests tend to minimize differences in performance between groups with a specific characteristic (eg, gender), mainly controlling for biases derived from a high level of adverse impact.^19,20 According to our outcome measures, males and females show very similar average test scores, thus excluding any adverse impact with respect to gender.

Additional studies are obviously needed, and a larger and probabilistic sample of participants is required to validate use of the LJI in both organizational and selection settings. Construct validation studies should be performed in order to verify that the test measures what it purports to measure (convergent validity) and also to check that it does not show appreciable correlations with other psychologic constructs to which it is theoretically unrelated (discriminant validity).

Application of the LJI could aid existing or potential persons in a position of influence to develop their ability to become more effective in their leadership decision-making, thereby optimizing performance, maintaining the motivation of followers, and fostering positive organizational outcomes.^21,22

Disclosure

The authors report no conflicts of interest in this work.