ABSTRACT
Objective
To adjust the decision criterion for the Word Memory Test (WMT, Green, 2003) to minimize the frequency of false positives.
Method
Archival data were combined into a database (n = 3,210) to examine the best cut score for the WMT. We compared results based on the original scoring rules and those based on adjusted scoring rules using a criterion based on 16 performance validity tests (PVTs) exclusive of the WMT. Cutoffs based on peer-reviewed publications and test manuals were used. The resulting PVT composite was considered the best estimate of validity status. We focused on a specificity of .90 with a false-positive rate of less than .10 across multiple samples.
Results
Each examinee was administered the WMT, as well as on average 5.5 (SD = 2.5) other PVTs. Based on the original scoring rules of the WMT, 31.8% of examinees failed. Using a single failure on the criterion PVT (C-PVT), the base rate of failure was 45.9%. When requiring two or more failures on the C-PVT, the failure rate dropped to 22.8%. Applying a contingency analysis (i.e., X2) to the two failures model on the C-PVT measure and using the original rules for the WMT resulted in only 65.3% agreement. However, using our adjusted rules for the WMT, which consisted of relying on only the IR and DR WMT subtest scores with a cutoff of 77.5%, agreement between the adjusted and the C-PVT criterion equaled 80.8%, for an improvement of 12.1% identified. The adjustmeny resulted in a 49.2% reduction in false positives while preserving a sensitivity of 53.6%. The specificity for the new rules was 88.8%, for a false positive rate of 11.2%.
Conclusions
Results supported lowering of the cut score for correct responding from 82.5% to 77.5% correct. We also recommend discontinuing the use of the Consistency subtest score in the determination of WMT failure.
Acknowledgments
The authors would like to thank the following individuals for their assistance in the completion of this research through the donation of data or consultation regarding statistical analyses: W. John Baker, Manfred Greiffenstein, Paul Green, and Roger Gervais, and John E. Meyers. We also wish to acknowledge the assistance of the three reviewers of this manuscript who helped improve the paper.
Disclosure statement
All authors provide expert consultation in personal injury litigation.
Correction Statement
This article has been corrected with minor changes. These changes do not impact the academic content of the article.