Does the Defining Issues Test Measure Ethical Judgment Ability or Political Position?

ABSTRACT

This article addresses the construct validity of the Defining Issues Test of ethical judgment (DIT/DIT-2). Alleging a political bias in the test, Emler and colleagues (1983, 1998, 1999, 2007), show that conservatives score higher when asked to fake as liberals, implying that they understand the reasoning associated with “higher” moral development but avoid items they see as liberally biased. DIT proponents challenge the internal validity of faking studies, advocating an explained-variance validation. This study takes a new approach: Adult participants complete the DIT-2, then evaluate the raw responses of others to discern political orientation and ethical development. Results show that individuals scoring higher on the DIT-2 rank others' ethical judgment in a way consistent with DIT-2-based rankings. Accuracy at assessing political orientation, however, is low. Results support the DIT-2's validity as a measure of ethical development, not an expression of political position.

Keywords:

• construct validity
• defining issues test
• moral judgment
• political orientation

Notes

1. S. Thoma, Center for the Study of Ethical Development, in recent personal correspondence.

2. Briefly, the six Kohlbergian stages are as follows (Rest, 1979, p. 9):

• Stage 1–Punishment and obedience orientation

• Stage 2–Naïve instrumental hedonism.

• Stage 3–Good-boy/girl morality of maintaining good relations, approval of others

• Stage 4–Authority-maintaining morality

• Stage 5–Morality of contract, of individual rights, and of democratically accepted law.

• Stage 6–Morality of individual principles of conscience.

• Stages 1–3 are characterized as Self-Interest, Stage 4 as Conventional, and Stages 5–6 as Principled.

3. The standard DIT contains five such A items, making available an additional Antisocial score that is not used in computing the ethical-judgment metrics. The Antisocial score represents a point of view critical of the Conventional order, but offering nothing constructive in its place (Bebeau & Thoma, 2003).

4. The reader may observe that an alternative phenomenon could account for such a correlation and must be ruled out: if judges evaluated targets whose responses agreed with theirs as having superior judgment, then the logic would be circular.

5. The option of evaluating the same target set on both criteria, while requiring less labor from participants, entails a threat of confounding. For example, after categorizing a target as having low ethical judgment ability, one would likely resist attributing their own political beliefs to that person.

6. An alternative explanation for this positive relationship between the rater's own putative “ethical development” score and greater accuracy in predicting others' level of “ethical development” is that raters may use similarity to their own responses as the benchmark for high ethical reasoning. Thus, high-scoring judges would tend to “accurately” assign high scores to similar target individuals, but low-scoring judges would “inaccurately” assign high scores to the targets similar to themselves. If so, then the logic of this study would be circular. To test that possibility, I generated, for each judge, a “similarity” measure between the judge and each of the ten targets he or she evaluated as to ethical judgment. The DIT-2 instrument includes five ethical scenarios, each with 12 factors to be rated as to their importance in the decision, a total of 60 items. The similarity metric is a Pearson correlation between the ratings by the judge and by the target individual. Each judge evaluated ten target individuals; across all 44 judges, the average correlation between a judge's agreement with a target and his or her ranking of the target was r(10) = .18 (N.S.). Moreover, a strategy of assigning higher ranks to targets who gave responses similar to the judge does not appear to have been effective; a correlation between the degree to which a judge's rankings were correlated with similarity-to-target, and the accuracy achieved, is negative and not significant (r [44] = −.18, p = .24).

7. The description of Stage-2 thinking in Rest (1979, p. 22) illustrates why one operating at this low level would have difficulty understanding either conventional or principled reasoning: “although each person in understood to have his own interests, an exchange of favors might be mutually decided. . . . The [central concept determining rights and responsibilities is the] morality of instrumental egoism and simple exchange: ‘Let's make a deal.’”

8. Eight of the 44 participants who completed the evaluations of ethical judgment did not complete the political evaluations. Deleting the eight participants from the analysis of ethical ratings does not change the conclusions.

9. Another indication why the judges might be expected to garner insights into the targets' political positions is found in a study quite outside the Kohlbergian paradigm, because it is purely empirical and does not rank moral values according to any system such as Kohlberg's ethic of justice. Graham, Haidt, and Nosek (2009) identify five sets of widely-shared “moral intuitions”: Harm/care, Fairness/reciprocity, Ingroup/loyalty, Authority/respect, and Purity/sanctity. The first two closely resemble Kohlberg's Principled thinking, while the others represent lower levels within the DIT. They find that liberals place more importance on the first two, while conservatives place approximately equal emphasis on all five. Information about such relative preferences is available from the targets' raw DIT responses.

Additional information

Notes on contributors

Charles D. Bailey

Charles D. Bailey is a Professor of accountancy and Arthur Andersen Chair of Excellence, University of Memphis.