How “none of the above” (NOTA) affects the accessibility of tested and related information in multiple-choice questions

ABSTRACT

Answering multiple-choice questions increases accessibility of the tested information and can improve accessibility of related information. However, multiple-choice questions with “none of the above” (NOTA) as a choice hurts accessibility of the previously tested information when NOTA serves as the correct answer (NOTA_correct). Would prompting participants to recall an answer when choosing NOTA, and then providing feedback, reduce the costs of NOTA_correct items? In the present experiments, participants answered general knowledge questions in multiple-choice (with a NOTA alternative) or cued-recall formats; half of the participants, when choosing NOTA, were prompted to provide an answer. Half of the participants received feedback. On a final cued-recall test assessing performance for previously tested and nontested related information, we found that NOTA_correct questions hurt performance for previously tested items as compared to the cued-recall condition; but they facilitated recall of related information. Feedback improved performance for all previously tested information, but performance was still worse for NOTA_correct questions than for cued-recall questions. The addition of a cued-recall component on NOTA questions did not influence accessibility. The results have implications for using NOTA in practice tests and help explain why using NOTA_correct items hurt learning, even with feedback, supporting a proactive interference account.

KEYWORDS:

• Multiple-choice
• testing effect
• none of the above
• retrieval blocking
• interference

Acknowledgments

These experiments were sponsored by Hillsdale College's LAUREATES (Laboratory for Advanced Undergraduate Research Education Adapted for Talented and Extraordinary Students) programme.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 NOTA_incorrect and NOTA_correct were called NOTA⁻ and NOTA⁺, respectively by Odegard and Koen (2007).

2 In one of the six counterbalancing conditions, for one question the correct answer was provided as an alternative in a question that was intended to serve in the NOTA_correct condition. Thus, for about 1/6 of the participants, there were 9 NOTA_correct questions and 11 NOTA_incorrect questions.

3 Mauchly's Test of Sphericity revealed significant covariance. Modified degrees of freedom and p-values are provided based on the Huynh-Feldt correction.