Official Position of the American Academy of Clinical Neuropsychology Social Security Administration Policy on Validity Testing: Guidance and Recommendations for Change

Abstract

The milestone publication by Slick, Sherman, and Iverson (1999) of criteria for determining malingered neurocognitive dysfunction led to extensive research on validity testing. Position statements by the National Academy of Neuropsychology and the American Academy of Clinical Neuropsychology (AACN) recommended routine validity testing in neuropsychological evaluations. Despite this widespread scientific and professional support, the Social Security Administration (SSA) continued to discourage validity testing, a stance that led to a congressional initiative for SSA to reevaluate their position. In response, SSA commissioned the Institute of Medicine (IOM) to evaluate the science concerning the validation of psychological testing. The IOM concluded that validity assessment was necessary in psychological and neuropsychological examinations (IOM, 2015). Objective: The AACN sought to provide independent expert guidance and recommendations concerning the use of validity testing in disability determinations. Method: A panel of contributors to the science of validity testing and its application to the disability process was charged with describing why the disability process for SSA needs improvement, and indicating the necessity for validity testing in disability exams. Results: This work showed how the determination of malingering is a probability proposition, described how different types of validity tests are appropriate, provided evidence concerning non-credible findings in children and low-functioning individuals, and discussed the appropriate evaluation of pain disorders typically seen outside of mental consultations. Conclusions: A scientific plan for validity assessment that additionally protects test security is needed in disability determinations and in research on classification accuracy of disability decisions.

Keywords:

• Malingering
• Social Security Administration (SSA)
• Disability determinations
• Performance validity test (PVT)
• Symptom validity test (SVT)
• Malingered pain-related disability (MPRD).

Background

Following the original publication of recommendations and criteria for detecting malingered neurocognitive dysfunction by Slick, Sherman, and Iverson (1999), the scientific community witnessed an explosion of research on malingering over the next 15–16 years (Sweet & Guidotti-Breting, 2013). During the same period of time, the Social Security Administration (SSA) developed a policy that discouraged the use of empirically derived cognitive performance and self-report validity measures, eventually denying the use of standard psychological tests that they called “malingering tests” for their inclusion of embedded validity indicators. On January 30, 2013, a United States Senator Tom Coburn, a physician, issued a letter to the Honorable Michael Astrue, Commissioner of the SSA, expressing concern about SSA’s policy to defund the ordering of performance and symptom validity tests (PVTs and SVTs) for psychological consultative examinations (PCE) and administrative appeals (Coburn, 2013).

Senator Coburn’s initiative was evaluated by the Inspector General (IG; O’Carroll, 2013), who reviewed that in the early 1990s, SSA discouraged the purchase of PVTs, continuing and updating this policy over succeeding years. In 2012, SSA’s Chief Administrative Law Judge (ALJ) issued a statement that ALJs could not order a PVT as part of a consultative examination (CE). While senior SSA officials offered criticisms of PVTs, the medical community, and in particular the two major neuropsychology academies (American Academy of Clinical Neuropsychology: AACN; National Academy of Neuropsychology: NAN), supported their use whenever secondary gain (e.g., compensation) was involved. Other agencies including the Veterans Administration and the Railroad Retirement Board, along with private disability insurers, regularly allow or require the use of PVTs and/or SVTs in their disability evaluation process. The IG encouraged the SSA to take into account the current literature on malingering and to seek external expertise and review of these issues. SSA agreed to do so through the Institute of Medicine (IOM), which held hearings on these issues in 2014, publishing their findings in April 2015 (Institute of Medicine, 2015). These findings laudably accepted the basic tenets of scientific research on validity examination, urging the SSA to adopt use of PVTs and SVTs within policy on CE and appeals. As of this writing, SSA has not responded to the IOM findings.

The purpose of this AACN-sponsored paper is to provide independent guidance and recommendations to SSA and policy-makers concerning the use of PVTs and SVTs in Social Security Disability (SSD) determinations. This work is written by contributors to the scientific literature concerning validity testing and its applications to disability evaluations. It is neither a discussion of nor a reaction to the IOM work.

SSA disability programs and definitions

The Social Security Disability Insurance (SSDI) Trust fund pays benefits to disabled workers (and their families), who satisfy its legal requirements, are “unable to engage in substantial gainful activity due to a medically determinable physical or mental impairment” severe enough to meet the listing requirements of the program, and have not yet attained retirement age (Board of Trustees, Federal OASDI Trust Funds, 2012). The Supplemental Security Income (SSI) program is a means tested¹ National federal assistance program administered by SSA that guarantees a minimum level of income for needy aged, blind, or disabled individuals (SSA [Social Security Administration], 2014b). The SSI program provides a uniform federal income floor, with optional state programs that supplement the income. The statistical reports for SSI and SSDI for 2013 both indicate that 60% of recipients under the age of 65 were diagnosed with a mental disorder (SSA, 2014a, 2014b).

The claimed disability “must have lasted or be expected to last for a continuous period of at least 12 months, or be expected to result in death” (Morton, 2010; p. 20, SSA, 2014b). The claimant has a burden to prove that due to mental and/or physical impairments, he or she is incapable of sustaining substantial gainful activity in a competitive work environment. A mere diagnosis does not translate into eligibility for benefits: the claimant’s particular condition must meet the SSA listing requirements for that condition (http://www.ssa.gov/disability/professionals/bluebook/). To meet an adult mental listing requirement, the impairment or combination of impairments, must be severe (not mild or moderate). The claimant’s ability to understand, carry out, and remember instructions; to perform activities with adequate persistence and pace; to respond appropriately to supervision and co-workers; and to adapt to work pressures are considered.

Why the current process needs improvement

Psychologists are involved in the disability determinations process in three ways: (1) as consultative examiners; (2) as consultants within the Disability Determination Services (DDSs) to assist in adjudication; and (3) as experts for ALJs in the appeals process. At the CE level, examiners must assert the validity of their findings, while determining if the findings are consistent with the self-report of symptoms, or with any other medical information provided. Within the DDSs, psychologists must determine the validity of the reports being reviewed, assign weight to opinions contained in reports, and decide whether to refer a case to the Fraud Unit. Opinions from treatment providers receive the most weight unless there is compelling evidence otherwise. Psychological experts in the appeals process examine all the medical records for a case, including the CE reports, and provide guidance to the ALJ while answering questions from the disability attorney representing the claimant.

Without validity testing, only a small proportion of SSD claimants are identified as providing non-credible cognitive test results (Chafetz, 2011a). Earlier work (Faust, Hart, & Guilmette, 1988; see Guilmette, 2013 for recent review) had already shown that psychologists are not successful at identifying protocols from individuals instructed to feign disabling problems, though they were confident in their ability to do so. The SSA policy preference to utilize observation of the claimant when challenged by tests (but not PVTs) distorts the analytic process in the CE. The validity problem is then compounded all the way up through the chain of determinations and appeals, as decisions are made without an accurate determination of the credibility of performance and symptom presentation. Moreover, giving greater weight to treatment providers, who typically have not used empirical methods for evaluating possible symptom exaggeration, and who, due to their treatment alliance, are unlikely to document suspicions of non-credible functional complaints, is problematic. For mental impairments, which are often based largely on symptom self-reporting, this policy effectively puts claimants in charge of medical evidence contained in their own statements without objectively examining that reporting for bias.

The SSR 96-7p Policy Interpretation Ruling is of substantial importance to adjudicating mental disability claims. It states: “No symptom or combination of symptoms can be the basis for a finding of disability, no matter how genuine the individual’s complaints may appear to be, unless there are medical signs and laboratory findings demonstrating the existence of a medically determinable physical or mental impairment(s) that could reasonably be expected to produce the symptoms.” Also, “When the existence of a medically determinable physical or mental impairment(s) that could reasonably be expected to produce the symptoms has been established, the intensity, persistence, and functionally limiting effects of the symptoms must be evaluated to determine the extent to which the symptoms affect the individual’s ability to do basic work activities. This requires the adjudicator to make a finding about the credibility of the individual’s statements about the symptom(s) and its functional effects” (Social Security Administration, 1999).

In particular, rulings (20 CFR 404.1529 (c) and 416.929 (c)) describe the kinds of evidence considered in assessing the credibility of the claimants’ statements. However, each of the seven kinds of “evidence” listed relies heavily upon the individual’s self-report: (1) Activities of daily living, (2) Duration, frequency, and intensity of symptoms, (3) Factors that aggravate symptoms, (4) Medications used, (5) Nature of the treatments received, (6) Other measures used to alleviate symptoms, and (7) Any other factors related to the relationship between symptoms and functional limitations. The obvious adjudicative problem is that if the claimant is exaggerating mental symptoms and associated functional limitations, he or she is also likely to bias his or her reports in responses to questions related to these evidence categories.

The DDS accuracy rate has been reported as being high (i.e., 95–97%, SSA, 2004). However, one must understand the limits of this accuracy determination method, so that inappropriate conclusions are not made. This accuracy rate is based upon the “net error rate,” which is the number of deficient cases (corrected plus non-corrected) in a sample, divided by the number of cases reviewed. Errors in the disability determinations process occur when the disability decisions are not supported by the medical evidence. The high-accuracy rate simply means that the medical evidence presented (including test findings and symptom reporting) fit the listing requirements for the specific disability 95–97% of the time. These figures have little or nothing to do with whether the symptom report or test findings were valid to begin with. Ultimately, it is the classification accuracy of disability decisions that will be most probative in determining how well the DDSs have done with respect to the SSA policy on validity (Chafetz, 2011a).

SSA’s prior rationale for not encouraging validity testing—necessity for change

Relying on prevailing scientific evidence and previous critiques of the SSA policy, Senator Coburn wrote that SSA’s policy against validity testing was out of place. Coburn (2013) cited the Chafetz (2010) challenge of SSA policy against validity testing (SSA rationale in italics):

(1)	Malingering cannot be proven with tests: This is a straw man argument in that no psychological construct or diagnosis (e.g., ADHD) can be “proven” with a test; the inferences about constructs involve probability statements. Empirically validated PVTs/SVTs improve the validity of conclusions drawn about the accuracy of cognitive test scores and subjective symptom complaints. These have obvious differential diagnostic and adjudicative value;
(2)	Observation and assessment of the claimant when challenged with various tasks is preferred: Except in the most egregious of cases, clinical judgment by experts has proven inadequate to determine malingering from mere observation of test scores or behavior (Guilmette, 2013; Heilbronner et al., 2009), leading to confirmation and cherry-picking biases (Guilmette, 2013), which involve the selection of those observations or presented symptoms that support the examiner’s opinion. A European study showed that 42.2% of psychiatric outpatients had a hidden agenda of secondary gain (including financial reward or avoidance of unpleasant situations), but only 6% of patients expressed their expectations of gain to their psychiatrists (van Egmond & Kummeling, 2002);
(3)	Individuals who over- or under-report symptoms may be doing so because of psychiatric disease or underlying personality problems: For any kind of problem, individuals may over- or under-report symptoms, which by itself does not constitute malingering. Clear guidelines (Heilbronner et al., 2009; Slick et al., 1999) coupled with relevant testing results provides an opportunity for determining the probability of malingering in individual cases;
(4)	Claimants who are malingering may have a genuine impairment: Frequently claimants have more than one kind of problem, and yet still may be producing non-credible results. It is for the consultative examiner to delineate the various issues in any case, and the job of the DDS to determine whether the claimant meets the listing requirements for the claimed disability. For both issues of psychiatric disease or genuine impairment, it is important to note that PVTs/SVTs are not blindly interpreted as indicators of malingering. Rather, they contribute valuable information about the probable accuracy of test scores and subjective complaints. Some persons may perform on PVTs/SVTs at levels that support inaccurate data. However, if the longitudinal record and credible behavioral signs support serious mental illness then the ultimate adjudication would reflect this. Established criteria (Slick et al., 1999) indeed reflect the need to evaluate other causes of test failure;
(5)	It is difficult to distinguish the functionally limiting effects of true impairment from evidence that is produced by fabrication or exaggeration: This assertion is substantially overstated and contrary to what has been demonstrated in known groups PVT validation studies. For example, many studies have demonstrated the ability to separate, with a high level of accuracy, persons who are providing non-credible effort on PVTs from credible patient samples who have been selected because they have documented substantial impairments, including: moderate to severe traumatic brain injury (Green, Rohling, Lees-Haley, & Allen, 2001; Rohling & Demakis, 2010), pain conditions (Etherton, Bianchini, Greve, & Ciota, 2005), and low (but not extremely low) IQ (Chafetz & Biondolillo, 2012; Chafetz, Prentkowski, & Rao, 2011). Moreover, depression has no significant effect on cognitive test performance in patients who pass validity testing (Rohling, Green, Allen, & Iverson, 2002).

The necessity for validity testing

Mental claims are challenging to adjudicate because determination of the credibility of the alleged mental symptoms and their impact on work-related functioning often lacks an adequate set of objective findings. As discussed in the Program Operations Manual System (SSA, 1999), “The finding on the credibility of the individual’s statements cannot be based on an intangible or intuitive notion about an individual’s credibility.” That is, subjective complaints need to be reasonably substantiated by objective behavioral signs and findings obtained at a cross-section in time, such as in the PCE, and across the longitudinal timeline. The psychological consultant employs hypothetico-deductive reasoning (i.e., formulating a hypothesis that can be falsified), and considers consistencies and inconsistencies when attempting to substantiate claims of severe disability. The appropriate use of SVTs and PVTs increases the likelihood of reaching valid conclusions about the credibility of reported impairments and ultimately leads to more accurate adjudication decisions.

Non-credible performance on the PCE often occurs in a different form than in other professional examinations. Neurologists are familiar with Waddell’s signs concerning low back pain (Waddell, McCulloch, Kummel, & Venner, 1980); psychiatrists hear atypical psychotic claims (Resnick & Knoll, 2008). These are non-credible symptoms presented to the medical examiner. In the PCE, however, the psychologist may be presented with non-credible cognitive complaints (e.g., claimed inability to recall birthdate) that purport to be about memory problems or low cognitive functioning.

Considering the SSA definition of disability, any claimant with a medically determined illness or condition must still prove that he/she cannot do ANY substantial gainful activity. The DDSs do not consider the claimant’s “own occupation” for benefits. As pointed out (Chafetz & Underhill, 2013), the claimant who has a severe yet treatable kidney disease, while unable to climb utility poles to perform electrical work, must still prove that he/she cannot perform any other occupation for which he/she is qualified including work that is not physically demanding (e.g., clerical work). Thus, even a medical (non-PCE) claim is about functioning (e.g., concentration, persistence, and pace). Given that a high percentage of claimants in physicians’ offices fail PVTs (Richman et al., 2006), it is likely that even medical claims would benefit from the suggested methods for the PCE.

Pain disorders require validity assessment

Pain disorders are not included in the mental listings, as they are categorized within the musculoskeletal, inflammatory, and immune system listings (Chafetz & Underhill, 2013). This means that malingered pain-related disability (MPRD; Bianchini, Greve, & Glynn, 2005) will occur outside the PCE. The goal of MPRD is the same as any malingered disability, to give the appearance of disability beyond that arising from the actual injury (Greve, Bianchini, & Brewer, 2013). As back pain is a frequent complaint involved in disability claims (Chafetz & Underhill, 2013), it is wise to be aware of the validity tests useful in assessing MPRD (Greve et al., 2013). Outside the CE for a DDS, financial incentive has been shown to specifically predict important outcomes, including overall treatment outcomes for pain (Rohling, Binder, & Langhinrichsen-Rohling, 1995) and even specific outcomes like spine surgery (Harris, Mulford, Solomon, van Gelder, & Young, 2005), demonstrating the importance that financial incentive has on pain complaints and responses to treatment for pain conditions. Recent research using formal methods for evaluating malingering in claimants referred for psychological evaluation for pain conditions suggests that there is a sizable minority of claimants who are non-credible in this pain group (20–50%, Greve, Ord, Bianchini, & Curtis, 2009). While SSA policy is written in a stringent fashion, requiring objective criteria on imaging and physical exam findings (e.g., atrophy) that support impairments, it still must be considered that claims of poor functioning may be too subjective to be considered without validity analysis. The desynchrony between the intensity of pain complaints, actual behavioral functioning in day-to-day activities, and objectively determined physical pathology may be related to many personal, personality, cognitive, and situational factors. Thus, the use of SVTs/PVTs are helpful in empirically evaluating a probable intent to over-report pain and related functional limitations that is motivated by secondary gain factors.

Definition of malingering

Malingering is the deliberate feigning or exaggeration of illness or injury for the purpose of gain (e.g., compensation or avoidance of duty/punishment). Authoritative expert guidance has been published (Slick et al., 1999), and supported by the NAN (Bush et al., 2005) and the AACN (Heilbronner et al., 2009). These position statements from the neuropsychology community concluded that failure to utilize validity testing in a medicolegal context is considered substandard practice.

Performance and symptom validity

PVTs are used to determine the accuracy of measures of actual ability, while SVTs help determine the accuracy of reporting of symptom experience (Larrabee, 2012). These terms are seeing wider use in the research community rather than less precise terms describing “effort” or “response bias.”

Malingering determination is a probability proposition

Formal criteria for a determination of malingering (Slick et al., 1999) led to tremendous growth in the empirical study of PVTs in the following 15 years. Using these criteria, researchers have been able to compare groups of persons that meet criteria for probable and/or definite malingering with various “credible” clinical samples. This has been referred to as the “known groups” method. This research incorporated diagnostic statistics,² and, as a result, diagnostic probability can now be quantified when PVTs and SVTs are failed. Using these methods, while taking into account the base rates of PVT failure in SSD claimants (Chafetz, 2008), an examiner can report to the DDS the probability of non-credible findings of any particular claimant. In concept and method, this is no different from the expert reporting to the trier-of-fact in a court of law the probability of malingering in any litigation or criminal case.

As PVTs must be rigorously developed to keep the rate of false positives low, there is a natural reduction in the sensitivity of these tests (i.e., the ability to detect invalid performance). This reduced sensitivity is offset by the use of multiple PVTs and SVTs. Because the cutting scores for individual PVTs and SVTs reflect truly atypical performance (i.e., non-compensation-seeking patients with serious problems can typically pass these tests), it is unlikely that a claimant with bona fide impairment will fail two or more of these measures, and even less likely that they will fail three or more (Larrabee, 2003). By contrast, as more PVTs and SVTs are failed, the likelihood of invalid performance increases (Larrabee, 2014). Using an invalid performance base rate of 40%, supported by empirical literature review (Larrabee, 2003; Larrabee, Millis, & Meyers, 2009), one can employ linking of likelihood ratios (Larrabee, 2008) to obtain PPP for multiple PVT failures. Using these mathematical methods, Larrabee (2008) showed that failure of two PVTs at a base rate of 40% yielded a posterior (after the test result) probability of invalid performance of 94.3%, whereas failure of three PVTs had a posterior probability of 98.8%.

Chafetz (2011b) analyzed what happens when one or more PVTs are failed in SSD claimants, comparing the mathematical method as recommended by Larrabee (2008) to an empirical (simple counting) method (Victor, Boone, Serpa, Buehler, & Ziegler, 2009). If one PVT was failed, the probability of malingering jumped from the base rate probability up to 84–91%, depending upon the classification accuracy of the PVT. With two failures, the probabilities jumped to about 93–97%, and with three PVTs failed the probabilities of malingering were over 99%. There was good agreement between the mathematical and simple counting methods in these disability claimants.

Larrabee, Greiffenstein, Greve and Bianchini (2007) reviewed research following known-groups and simulation designs and reached three conclusions. (1) Comparing performance on non-forced choice measures of ability, the performance of persons meeting definite malingering criteria was similar to the performance of non-injured persons simulating impairment, establishing that worse-than-chance performance was intentional, as it matched that of simulators who were known to be performing poorly on purpose. (2) The performance of the definite (below-chance) and probable malingerers did not differ on PVTs that were not forced-choice measures, establishing the equivalence of definite and probable malingering. (3) The work of Bianchini, Curtis, and Greve (2006) established that PVT failure was related to amount of external incentive, establishing that external incentive was reinforcing PVT failure.

Types of PVTs—embedded vs. free-Standing

Prior to the 1990s, research on techniques to identify non-credible performance on neurocognitive testing was sparse, but since that time there has been an explosion of publications addressing the development and validation of such techniques. Initially neuropsychologists focused on creating tests that had a single purpose in identifying failure to perform to true ability, so-called “dedicated” or “free-standing” PVTs. More recently, the field has moved toward validating techniques derived from standard neurocognitive tests (“embedded” PVTs), which have the potential for reducing test battery administration length. Embedded PVTs provide an evaluation of performance validity in “real time” (i.e., during actual test administration), rather than relying on dedicated PVTs that may have been administered up to hours before or after the standard tests.

Use of embedded PVTs also protects from “coaching” of, or “self-education” by, test takers regarding the presence of PVTs in the battery. While dedicated PVTs can readily be accessed by name in internet searches (along with information regarding actual test procedures), internet searches for standard cognitive tests show their primary purpose as measures of memory, attention, language, visual perceptual skills, etc., with their use as PVTs relatively unmentioned. Moreover, if proved valid, the actual cognitive test results can be interpreted for their original purposes.

Additionally, the field of neuropsychology is now coming to appreciate that malingering is not a “unitary construct,” and that test takers can elect to feign at different times during an examination. In particular, examinees may confine their attempts at feigning to those neurocognitive skill areas that they think are most impacted by the disorder they are claiming (Boone, 2009). In fact, it is atypical for a non-credible test taker to feign on every type of task measure administered (Boone, 2009), apparently reflecting the perception that it would appear implausible if he or she could not perform any tasks. Therefore, it is important that performance validity be repeatedly measured throughout an examination. Current practice guidelines indicate that PVTs are to be interspersed throughout a neurocognitive exam (NAN; Bush et al., 2005), including use of both “dedicated” and “embedded” measures (AACN; Heilbronner et al., 2009). Chafetz, Abrahams, and Kohlmaier (2007) and Chafetz (2012) have shown how embedded measures within the mental status and cognitive testing within a PCE are especially useful for these examinations. In particular, with appropriate cutoffs, Chafetz and Biondolillo (2012) have shown that well-motivated individuals with IQ between 60 and 75 pass these tests with little or no misidentification (zero false positives for most of the PVTs).

Most dedicated PVTs involve memory paradigms, but it is now understood that test takers can elect to feign thinking speed (“slowness”), attention, visual spatial, motor dexterity, multi-tasking and problem-solving, reading, math, and language deficits (Boone, 2009; Cottingham & Boone, 2010). Dozens of “embedded” performance validity indicators have now been validated across these various cognitive domains (see Boone, 2013, for listing) and are available for nearly every commonly used neurocognitive measure. While “embedded” PVTs, as a group, have lowered sensitivity relative to dedicated PVTs (although rates very considerably), they require no additional test administration time and best allow continuous measurement of performance validity across a neuropsychological exam. All PVTs, embedded or stand-alone, have problematic specificity (necessitating changes in cut-offs) in individuals with documented severe neurologic-, psychiatric-, or developmental-based cognitive problems and associated functional compromise.

The question arises as to whether “passed” PVTs “cancel out” any failures. However, when test cut-offs are selected to enhance specificity (i.e., only allowing a 10% false positive rate or less), sensitivity will be lowered, thereby rendering failed performances more informative than passing scores. That is, if specificity of individual PVTs is 90%, and sensitivity rates range from ≤50 to 80%, scores on individuals PVTs will be more effective in ruling in than ruling out non-credible performance.

The special case of the Minnesota Multiphasic Personality Inventory (MMPI-2-RF/MMPI-2)

Various self-reported inventories of personality, social, and emotional, and health functioning can be useful for SSD evaluations, including the Personality Assessment Inventory (PAI; Morey, 1991) and the Battery for Health Improvement-2 (BHI-2; Bruns & Disorbio, 2003). However, during the 2012 Senate hearings as to why SSA defunded the use of validity tests, it came to light that the MMPI-2 had been treated by SSA simply as a validity test, with no regard to its purpose and structure in assessing social and emotional functioning, and its use was defunded along with other validity tests (Coburn, 2013). Furthermore, in these hearings, the MMPI-2 was claimed to have weaknesses in its psychometric properties, limiting its applicability in persons with low IQ, low reading, or English as a second language.

The Minnesota Multiphasic Personality Inventory-2 Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2008/2011) is the latest version of the most widely used measure of psychological dysfunction. The MMPI-2-RF assists psychologists in determining whether, and to what extent, an individual is experiencing significant emotional, thought, or behavioral dysfunction. A fourth- to fifth-grade reading ability is required for valid completion of the MMPI-2-RF. An audio version of the test is available for individuals with lower reading skills. Most test takers can complete the inventory in 35–50 min.

Nine of the 51 MMPI-2-RF scales provide disability examiners with important information about the quality of the self-reported data provided by an examinee. As these scales are based on self-report, they are considered SVTs in their own right. These validity scales assess the extent to which a test taker was able to understand and respond relevantly to the test items, and, if so, whether the responses provide credible information about her or his psychological functioning. In cases where the validity scales indicate that the test taker has responded in a valid manner, the psychologist can have increased confidence in the findings indicated by the scores on the substantive scales from the MMPI-2-RF. If the validity scales raise questions about the quality of the information obtained, the examiner is able to take this information into account, and in some cases determine that the results are invalid and cannot be relied upon to obtain accurate information.

Use of validated, objective test findings ensures that disability claimants are treated fairly and consistently. An extensive research base is available to guide interpretation of MMPI-2-RF findings in disability evaluations. The Technical Manual for the inventory (Tellegen & Ben-Porath, 2008/2011) includes data on 1023 disability claimants. In addition, over 260 peer-reviewed studies report MMPI-2-RF results (for an up-to-date bibliography, see https://www.upress.umn.edu/test-division/MMPI-2-RF/mmpi-2-rf-references). It should also be noted that the previous version, the MMPI-2, with which SSA is most familiar and which was defunded by SSA (Coburn, 2013), also contained symptom validity scales that had been established as sensitive and specific (Ben-Porath, Greve, Bianchini, & Kaufmann, 2009; Greve, Bianchini, Love, Brennan, & Heinly, 2006), and therefore useful for disability examinations before the development of the MMPI-2-RF.

Malingering-by-proxy: non-credible findings in children

Child validity studies have lagged behind adult work in part because many practitioners believed historically that children could not or would not feign or exaggerate in an assessment setting. However, a sizable developmental psychology literature demonstrates that children are capable of deception by the preschool years and engage in deceptive acts quite frequently under the right circumstances (Talwar & Crossman, 2011). Thus, it is well documented that children and adolescents can deceive. A more important question in justifying the use of validity testing in children is whether or not they actually do deceive in assessment settings. Innumerable medical and psychological studies have now documented that children feign a whole host of difficulties during healthcare assessments including cognitive and academic problems, motor disturbance, vision and other sensory problems, seizures, psychosis, fever, skin conditions, respiratory problems, gastrointestinal upset, and orthopedic injury (see Kirkwood, 2015).

The children found to display the highest rate of non-credible effort during performance-based assessment are those undergoing PCEs for SSD. Chafetz and colleagues (Chafetz, 2008; Chafetz et al., 2007) found that 10% of children being evaluated for disability failed a PVT at below-chance levels, a rate that is only somewhat below that of adults (12–13%). Another 16% of children in the same study obtained chance-level performance and another 28–34% failed one or two PVTs. Thus, a large proportion of children in these disability samples demonstrate some evidence for malingering. The malingering is thought to be driven by the parents in most cases, and so would be considered “malingering by proxy,” which involves non-credible performance at the direction or pressure by others. A growing number of PVTs have been found to have sufficient empirical backing to justify their inclusion in batteries with school-aged children (Deright & Carone, 2015; Kirkwood, 2015).

Children’s lives can be diminished by parental pressure to pretend and maintain lower abilities than they are capable of, and a set of guidelines for reporting to child protection authorities in these circumstances have now been published (Chafetz & Dufrene, 2014). A large concern in these circumstances is the diminishment of education, but the problems can be broader, resembling the consequences of other forms of parental neglect/abuse.

Low-functioning abilities

It is important to recognize that no examiner or adjudicator wants to mislabel a claimant, determining that malingering has occurred when it has not. This scenario may happen when a claimant is low functioning or has other extremely limiting problems. The difficulty for the examiner or the adjudicator occurs as a dilemma within the determination itself, as low-IQ findings will occur to a greater extent as the claimant attempts to feign low functioning (Chafetz, 2008; Chafetz et al., 2007). In these cases, it is advisable to use PVTs and/or PVT cut-offs found to be more accurate for these individuals, to assist with use of computers on computer-administered tests for those claimants without experience in using a computer, and to adhere strictly to guidelines that more than one PVT failure is recommended for determination of probable malingering (Chafetz, 2015). Smith et al. (2014) found that existing PVT cut-offs generally had unacceptably high false positive rates in a low IQ population (≤75) with no motive to feign, but when cut-offs were adjusted to reduce false positive identifications, failure on multiple PVTs was rare. Similarly, Chafetz and Biondolillo (2012) observed that when using PVTs designed for low-IQ samples (60–75), well-motivated claimants do not fail PVTs (0% failed), except when the test has an ability component in addition to the effort component (6% failed). Also, it should be kept in mind that below-chance performance is interpreted the same way for low-functioning as well as high-functioning individuals, that is, as an indicator of intent to fail the PVT (Pankratz & Erickson, 1990).

Test security

Concerning PVTs in any compensation context, the accuracy of the findings and thus the valid interpretation of any validity testing result is dependent upon testing a claimant who is not aware of how to manipulate the findings. Indeed, in Detroit Edison Co. v. National Labor Relations Board (NLRB) (1979) the U.S. Supreme Court ruled on the public policy of test security for standardized psychological instruments (see Kaufmann, 2009), suggesting that test secrecy is critical to the validity of the examination process.

Therefore, it is most important for SSA not to be purchasing or recommending any specific PVTs for a CE. Instead, SSA should develop a scientifically based assessment policy that includes guidance for consultative examiners. Otherwise, if examinees or their advocates were able to determine in advance which PVTs will be used, there is a risk that examinees could conduct preevaluation research that would undermine their subsequent evaluation.

Summary

(1)	Following the seminal publication by Slick et al. (1999), there was an explosion of research on malingering over the next 15–16 years. During the same period of time, SSA developed a policy that discouraged examination of the validity of disability claims, eventually denying the use of standard psychological tests that they now deemed “malingering tests” for their use of internal validity indicators. Surveying the scientific breadth and depth of study on this problem, Senator Tom Coburn wrote to the Social Security Commissioner, the Honorable Michael Astrue, urging a change in policy. At the IG’s direction, SSA agreed to study the problem, charging the IOM with this task. The IOM concluded that the use of PVTs and SVTs was supported for the evaluation of disability claims requiring cognitive and non-cognitive psychological testing. Independently, the AACN has developed this expert guidance paper.
(2)	The SSA definition of disability requires a claimant to be unable to engage in any substantial gainful activity because of a medically determinable physical or mental impairment. The disability must have lasted or be expected to last for a continuous period of at least 12 months, or be expected to result in death. SSA does not deal with partial disability, nor are they concerned with whether the claimant cannot work just in his/her own occupation.
(3)	Under current SSA policy, the consultative process, along with the current internal DDS consultative review, and the expert analysis on appeals all have difficulty detecting the feigned CE except for the most egregious of feigned claims. The failure of clinicians to detect malingered cases is the rule rather than the exception, as research shows that clinicians are prone to various biases in judgment if they cannot use established scientific tools to guide judgment. SSA explanations for the policy of excluding PVTs and SVTs from the examination do not meet scientific standards for inclusion or exclusion of psychological tests.
(4)	Pain disorders are not recognized within the mental listings, and yet pain disorders are commonly feigned for compensation. PVTs and SVTs, along with psychological expertise, are necessary for the determination of MPRD.
(5)	The determination of malingering in any individual case involves probability determinations. Analysis of the probabilities following failure of validity testing shows a high probability of accurate determination of an invalid case when guidelines are strictly followed.
(6)	The use of embedded validity indicators within the examination process helps the examiner track the feigned elements of the examination in “real time” and across different cognitive processes. The current recommendation in a compensation-seeking context is to use multiple PVTs, both dedicated and embedded, interspersed throughout the exam, and covering multiple cognitive domains so that validity is repeatedly sampled.
(7)	The MMPI-2/MMPI-2-RF, which was defunded by SSA because it was deemed a “test of malingering,” is a helpful test within the disability process for the determination of social and emotional functioning. It contains several internal empirically supported validity scales to evaluate the credibility of the claimant’s responses concerning claimed symptoms. Other validated tests of self-reported symptoms, such as the PAI and the BHI-2, may also be useful.
(8)	Children can and do feign impairment on the examination process. In a compensation-seeking context, this feigning is frequently occurring at the direction or pressure of others, usually parents, and is therefore termed “malingering-by-proxy.” Forcing a child to engage in the feigning of disability can be damaging to the child’s development.
(9)	With low-IQ claimants, it is advisable to use PVTs and/or PVT cut-offs found to be more accurate for these individuals, to assist with use of computers on computer-administered tests for those claimants without experience in using a computer, and to adhere strictly to guidelines that more than one PVT failure is recommended for determination of probable malingering (Chafetz, 2015).
(10)	Test security is of utmost importance to valid assessment. SSA should not identify specific PVTs that should be purchased by the DDSs, but instead develop a scientifically based plan of validity assessment using established guidelines.

Recommendations for use of PVTs and SVTs in CE

(1)	It will be important for SSA to reverse its long-standing policy against the use of validity testing, adopting the recommendations, and standards for validity testing published by the national neuropsychology and psychology organizations.
(2)	All psychologist consultants and psychological examiners (PEs) working within the SSA disability program should be provided with specific training on how PVTs and SVTs can be used to enhance disability determination decisions. The DDSs should require PEs and internal consultative personnel to obtain continuing education, which can be supplied by the national and state psychology organizations. Non-psychologists should not be involved in this process due to test security concerns.
(3)	Rules should be established for protecting the security of all psychological tests including PVTs (see Kaufmann, 2009). Test security should be a part of the training of consultative examiners.
(4)	Specific dedicated PVTs or SVTs should not routinely be ordered by the DDSs or ALJs. PEs should instead be provided with more time and financial reimbursement, and asked to follow a scientifically designed plan for validity assessment. This will increase the likelihood that the correct adjudicative decision will be made because a more careful and empirically validated approach to assessing the accuracy of obtained cognitive test scores and self-reported complaints has been obtained.
(5)	Currently, the only information on validity of the CE has been done independently of SSA. As information on the validity of examinations would be of great benefit to the understanding of the disability process, SSA statisticians should begin sampling, coding, and archiving findings from CE under guidance from established psychology professional organizations. These data should be made available to researchers from these organizations, with funding by SSA for the further development of expert research and development of the CE. These data can also be used regionally and locally to maintain testing standards, and to help determine classification accuracy of disability decisions (Chafetz, 2011a).

Disclosure statement

Yossef Ben-Porath is a paid consultant to the MMPI Publisher, the University of Minnesota, and Distributor, Pearson. As co-author of the MMPI-2-RF, he receives royalties on sales of the test.

Drs. Chafetz and Williams both previously derived income for consulting for SSA but no longer do so.

Drs. Boone, Chafetz, Kirkwood, and Larrabee receive royalties from their books mentioned in this publication.

Notes

¹ Means testing involves a determination of whether the person is eligible for benefits based on whether the person or their family has the means to do without the benefits.

² Sensitivity, specificity, base rate, and positive and negative predictive power (PPP and NPP). The diagnostic formula for PPP, which is the certainty of being right when determining malingering with a PVT, is True Positives/(True Positives + False Positives). In the comparison between known malingering groups against credible clinical samples, cut-off scores are set so that approximately 90% of the bona fide impairment group is correctly identified as providing valid performance. The investigator also specifies the clinical characteristics of the 10% false positive cases (e.g., need for 24 h supervised living) so that a clinician utilizing the PVT can see if their examinee possesses characteristics that would indicate false positive identification.