Making sense of deviance: Identifying dissociating cases within the case series approach

Abstract

The case series approach in cognitive neuropsychology provides a means to test theories that make quantitative predictions about associations between different components of the cognitive system [Schwartz, M. F., & Dell, G. S. (2010). Case series investigations in cognitive neuropsychology. Cognitive Neuropsychology, 27, 477–494]. However, even when the predicted association is borne out the study may include outliers—observations that deviate significantly from the rest of the data. These outliers may reveal individual cases whose cognitive impairments dissociate from other cases included in the study. These dissociating cases can pose a significant challenge to the theory being tested. Using a recent case series that investigated the underlying causes of letter perseveration in spelling [Fischer-Baum, S., & Rapp, B. (2012). Underlying cause(s) of letter perseveration errors. Neuropsychologia, 50, 305–318], I discuss statistical and theoretical issues that arise when using outlier detection techniques to identify dissociating cases in a case series study.

Keywords:

• Cognitive neuropsychology
• Outlier detection
• Case series
• Single-case studies
• Associations and dissociations.

I am grateful to Gary Dell, Matt Goldrick, Michael McCloskey, Fred Oswald, and Brenda Rapp for helping me work through these issues of case series, outliers, and cognitive neuropsychology, and to David Kajander for reading through drafts. Portions of this work were presented in a symposium on case series investigations at the 50th annual meeting of the Academy of Aphasia. The remaining work was moulded by the discussion that took place at that meeting.

Notes

1 Fischer-Baum and Rapp (2012) describe a Mahalanobis distance procedure to detect outliers following Penny (1996). However, a further review of the statistics literature, and some simple simulations, has led me to the conclusion that the externally studentized residual test would have been more appropriate with our data set. The Mahalanobis distance technique measures how far a given data point is from the centre of mass of the whole data set, while the externally studentized residual test measures how far the data point is from the regression line. Consider a data set that includes a data point with a much greater rate of perseverative and nonperseverative intrusions than the rest of the individuals in the sample, but whose data point falls near the regression line. Given the theory being tested, it would be incorrect to conclude that the individual represented by this data point suffers a qualitatively different deficit from the others in the sample; instead, it appears that this individual has a quantitatively more extreme version of the same deficit. Yet, because this data point is far from the centre of mass of the whole data set, it will be identified as an outlier by the Mahalanobis distance technique. Because it falls near the regression line, it will not be identified as an outlier by the externally studentized residual test.

2 Error is larger for more extreme values of the predictor variable, causing the studentized residual to be smaller. Therefore, a larger difference between the observed and predicted values is needed for a point to be identified as an outlier at these extreme values (see Crawford & Garthwaite, 2006, for further discussion of this point).

3 One concern about this recursive application of the externally studentized residual test is that it will make false alarms more common; once one or two nonoutliers are incorrectly labelled as outliers, the technique may start to identify false alarms on other data points. The results of the simulation discussed below show that such false alarms are rare with this technique.

4 When there is no underlying association between perseverative and nonperseverative intrusions, the correlation is slightly negative; the rates of perseverative and nonperseverative intrusions are not entirely independent in this sample. The sum of perseverative and nonperseverative intrusion rates and the proportion of correct responses must be equal to 1.0. Therefore, if the perseverative error rate is very high (e.g., >50%) then the nonperseverative rate must be lower (e.g., <50%) and vice versa.

5 Keep in mind, an outlier should not be discarded as a false alarm simply because the researcher cannot think of a theoretical explanation for why that individual differs from the rest of the sample.

6 The degrees of freedom for the generalized externally studentized residual test is n – p – 1, in which n is the total number of data points, and p is the total number of parameters (2 in simple linear regression, >2 in multiple linear regression).