Field study of response accuracy and decision confidence with regard to lineup composition and lineup presentation

Abstract

Response accuracy with regard to lineup composition (present/absent target) and presentation (simultaneous/sequential), as well as decision confidence as a predictor and postdictor of response accuracy, was tested in a field experiment (N = 426). Overall response accuracy was 43%. It was not found significant main effects of lineup presentation or composition or a significant interaction between these variables in regard to response accuracy. By using both traditional methods of lineups’ efficiency comparison, and novel approach – ROC analysis, either procedure superiority was not revealed. Prospective confidence did not relate to response accuracy, but correct identifications were made with more retrospective confidence than were incorrect identifications. In simultaneously presented target absent lineups and sequentially presented target present lineups, correct identification resulted in a statistically significant higher level of retrospective confidence than did incorrect identifications. Participants were over-confident in their accuracy. The calibration curve, plotted for choosers, based on retrospective decision confidence was meaningful, but far from linear, while calibration curve based on prospective decision was flat. Thus the sequential lineup may not be superior to the simultaneous lineup and confidence may not be a very reliable indicator of accuracy in field setting, regardless of approach used for data analysis.

Keywords:

• Field study
• confidence
• accuracy
• calibration
• ROC

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. Hypothesis that the sequential procedure reduces mistaken identifications with little or no reduction in accurate identifications.

2. Answers of ‘do not know’ were not taken into account in further analysis. Logistic regression can be used in conjuction with ROC analysis (e.g. Andersen et al., 2014). Although in the majority of the research logistic regression is used as the solo method (see Grondlund et al., 2012), Gronlund and Neuschatz (2014) argue that ROC analysis should precede the use of logistic regression. Once ROC analysis reveals a discriminability difference, logistic regression can determine whether the dicriminability difference is due to a change in correct IDs or a change in false IDs (or possibly both).

3. D = (AUC1 –AUC2)/s, where s is the standard error of the difference between the two AUCs estimated by the bootstrap method (Mickes, Flowe, & Wixted, 2012). The number of bootstraps in this study was set to 2000.

4. Due to an insufficient number of participants, the calibration curve for nonchoosers was not plotted.

5. Calibration curves that were plotted separately for sequential and simultaneous lineups yielded similar results (Horry, Palmer, & Brewer, 2012), and a more linear C-A calibration curve was found in the sequential lineup. However, due to the small sample, this finding should be considered with caution.

6. The DR favored the sequential lineup, and the odds of guilt were higher (and the likelihood of misidentification was lower) if the suspect was identified through use of the sequential procedure.