The Birthday Party Test (BPT): A new picture description test to support the assessment of simultanagnosia in patients with acquired brain injury

Abstract

There is broad consensus on the utility of complex pictures in the assessment of simultanagnosia in patients with acquired brain injury (ABI). To overcome various shortcomings of current instruments, we have developed the Birthday Party Test (BPT); a picture description test that contains a neutral scene, a balanced representation of events, and provides clear instructions and a scoring-aid. We have applied the BPT in a large group of patients with ABI (n = 502) and in an age-matched healthy control group (n = 194). Our results show that performance on the BPT was associated with a range of descriptive, neuropsychological and clinical characteristics and that poor test performance appeared to be more common in patients with etiologies that have an increased risk of bilateral damage. Furthermore, we assume a high correspondence between test performance on the BPT and the assessor’s clinical judgment of likely having simultanagnosia in preliminary analyses. This study shows the potential usefulness of the BPT to support diagnostic decision making in simultanagnosia. The BPT is made freely available to facilitate its broad application in the clinical assessment of patients with visual impairment and to enable a further evaluation of its utility and validity in future studies.

Keywords:

• Assessment
• acquired brain injury
• birthday party test
• picture description test
• simultanagnosia
• screening
• visual perception

Since its first introduction in 1909 (Bálint, 1909), simultanagnosia has often been studied in the context of Bálint syndrome (Clavagnier et al., 2006; Coslett & Saffran, 1991) which represents a neurological disorder that is characterized by symptoms of optic ataxia, ocular apraxia and simultanagnosia. Simultanagnosia is defined as a disturbance of visual attention that becomes clinically evident in an individual by an inability to interpret the overall meaning of a visual scene, despite the individual being able to perceive and recognize the discrete elements that make up the scene (Luria et al., 1963; Rizzo & Vecera, 2002; Wolpert, 1924). This inability cannot be explained by a reduced visual acuity or a visual field defect.

The term simultanagnosia was proposed by Wolpert in 1924 (Wolpert, 1924), but the condition was preceded by various other names, including fatigability of attention (Bálint, 1909) and visual disorientation (Holmes & Horrax, 1919). In the vast majority of cases, simultanagnosia is caused by bilateral damage to the parietal and superior occipital cortex (Devinsky et al., 2008) and is more likely to occur in neurodegenerative diseases (Coslett & Saffran, 1991), such as posterior cortical atrophy (Benson et al., 1988; Kaeser et al., 2015; Neitzel et al., 2016), than in other neurological conditions (Devinsky et al., 2008). Diverse mechanisms have been suggested to underlie simultanagnosia which include, among others, a slowing of visual processing (Duncan et al., 2003), a restricted window of visual processing (Dalrymple et al., 2010; Michel & Henaff, 2004) and an inability to shift attention (Farah, 1990). Up till now, however, there is no consensus on its actual cause (Neitzel et al., 2016).

This may be related to the fact that there is a shortage of adequate and standardized assessment methods which makes the determination of the validity of research findings on simultanagnosia very difficult. So far, no tests were specifically developed and validated for the clinical assessment of simultanagnosia. Several approaches are currently being used in research and clinic to support the diagnosis of simultanagnosia (e.g. the hierarchical Navon letters (Navon, 1977)) with one of the most widely applied and accepted tools being a picture description test. Several picture description tests are available (e.g. the Telegraph Boy Picture from the Binet scale or the Binet-Bobertag Picture). Similar to all pictures is that a visual scene is depicted in which a few events take place. Probably the most well-documented and well-known picture is the Cookie Theft Picture (CTP). The CTP is part of the Boston Diagnostic Aphasia Examination (Goodglass & Kaplan, 1983) and is a picture stimulus that was designed to elicit a short narrative description on confrontation from patients with language dysfunction. In the context of simultanagnosia, the CTP is often used as a supplementary diagnostic tool in order to support diagnostic decision making (e.g. Braet & Humphreys, 2009; Clavagnier et al., 2006; Coslett & Lie, 2008; Dalrymple et al., 2013; Denburg et al., 2009; Huberle & Karnath, 2006; Kaeser et al., 2015; Shenker & Roberts, 2016).

As specific diagnostic criteria for simultanagnosia do not exist, diagnosis is usually based on clinical observation of the patient, supported by assessment tools, such as picture description tests. Interpretation of these picture descriptions are mostly qualitative, in accordance with the definition of simultanagnosia. As patients with simultanagnosia fail to localize objects, even when these objects are actually seen (Farah, 1990), the ability of these patients to relate objects to each other in space is impaired. In line with this, a patient with simultanagnosia would describe the picture in a piecemeal fashion, correctly describing the details of the scene depicted, but failing to integrate the different details and to derive its overall meaning (Barton, 2014; Beh et al., 2015; Caravaglios et al., 2015; Clavagnier et al., 2006; Coslett & Lie, 2008; Denburg et al., 2009; Digre & Varner, 2009; Girkin & Miller, 2001; Huberle & Karnath, 2006; Shenker & Roberts, 2016).

There are good reasons to assume that a picture description test, in general, may represent an adequate approach for the assessment of simultanagnosia. First, as described above, simultanagnosia becomes clinically evident through a patient’s inability to perceive a visual scene as a whole, despite the patient being able to perceive and recognize its individual elements. On this account, a picture description test could serve to support diagnostic decision making in simultanagnosia. Consequently, a picture description test was recommended by numerous studies as an appropriate means for the assessment of simultanagnosia (e.g. Beh et al., 2015; Das et al., 2007; Haque et al., 2018; Kline & Bajandas, 2013; Mevorach et al., 2014; Rizzo, 2000; Tranel, 1994). Also in a recent study using the Delphi methodology to determine the contents of a screening tool for visual perceptual disorders, internationally recognized experts unanimously agreed on the use of a picture description test for the assessment of simultanagnosia (De Vries et al., 2018).

Notwithstanding its frequent use in the assessment of simultanagnosia, currently available picture description tests have not been designed in an optimal fashion for diagnosing patients with visual disorders and may, therefore, come with several consequences that limit their utility for this particular application. The most relevant shortcoming is that events depicted in the pictures are not balanced with respect to their spatial distribution, making it sensitive to influences of other visual or attentional deficits such as hemianopia or neglect. An adequate picture description test to be applied in the assessment of simultanagnosia requires a balanced distribution of details and events with regard to their spatial location within the picture. This will allow for description in an asymmetric fashion in the case of a co-existence of other visual or attentional deficits (e.g. hemianopia or neglect). Therefore, details and events should be presented in all quadrants of the scene, so that the test can be administered to patients with such other deficits as well.

Another difficulty in the use of currently available complex pictures or picture description tests for the assessment of simultanagnosia is that clear instructions for administration and scoring are lacking. This causes administration and interpretation to vary considerably across studies. For example, a considerable amount of studies using the CTP for the assessment of simultanagnosia did not specify which instructions were used or did not follow the original instructions as formulated by Goodglass and Kaplan (e.g. Barton et al., 2007; Clavagnier et al., 2006; Coslett & Lie, 2008; Dalrymple et al., 2011; Denburg et al., 2009; Huberle & Karnath, 2006; Walsh et al., 2012). This makes it difficult to draw well-founded conclusions about individual patients and makes it even harder to compare patient performances across studies. These limitations may have contributed to the fact that the vast majority of research available used picture description tests for descriptive purposes only, i.e. to define and characterize groups. To date, we are aware of only one study that included the CTP as one of its main outcome measures for the assessment of simultanagnosia (Hoffmann et al., 2002).

In this study, we present the Birthday Party Test (BPT); a new assessment tool which includes a complex picture specifically designed to support the assessment of simultanagnosia. In addition to the test (including scoring instructions), we present the first results obtained with it, and demonstrate its utility to support decision making in simultanagnosia.

Methods

Participants

Patients with acquired brain injury

Patients with acquired brain injury (ABI) were recruited for this prospective study between 2015 and 2017 via 18 outpatient institutions of Royal Dutch Visio, a center of expertise for blind and partially sighted people in the Netherlands. Patients were referred by local physicians to Royal Dutch Visio at various stages post-injury for clinical assessment and treatment planning. Participation in the present study was part of regular care for patients as it did not require any changes of clinical routines (e.g. assessment, treatment). Therefore it was judged by the Medical Ethical Review Board of the University Medical Center Groningen to be exempted from the Dutch Law on Medical Scientific Research involving Human Beings (WMO). The study is performed in accordance with the latest version of the Declaration of Helsinki with regard to ethical standards of research involving human subjects.

Inclusion criteria for patients were (i) evidence for brain injury in the present or past that was acquired after the age of 18, and (ii) normal or corrected to normal visual acuity, that is, binocular performance on the visual acuity chart that is not worse than the line corresponding to an acuity of 0.8 (Snellen equivalent 6/7.5 or 20/25, LogMAR 0.1; World Health Organization, 2018). Best corrected monocular visual acuity was evaluated if binocular visual acuity was not obtainable, e.g. because of loss of vision in one eye.

A total of 949 individuals were considered for inclusion in this study. All of these patients were referred because of visual complaints or visual impairments determined by a medical specialist. Of those, 447 individuals were excluded because of several reasons (see Figure 1 for a flow chart): For 137 individuals no evidence for acquired brain injury was available (i.e. absence of a neurological report or no neurological abnormalities found). In 14 patients, the brain injury had a congenital origin or was acquired before the age of 18 years. Visual acuity of 36 patients was not known and 203 patients had a best corrected visual acuity of <0.80. Furthermore, 57 patients with ABI had to be excluded from further analyses because of incomplete data on the BPT. Exclusion of these individuals resulted in a total sample size of 502 patients with ABI.

Figure 1. Flow chart regarding participant selection.

The age of patients with ABI ranged from 18 to 93 years, with a mean age of 60.8 years (SD = 14.3). Educational level of patients with ABI ranged from 1 to 7, with a mean of 5.1 (SD = 1.2) according to the Dutch education scale ranging from 1 (elementary school not finished) to 7 (university degree; Verhage, 1965). The patient group with ABI represented a heterogeneous group of patients with various causes for brain injury, including traumatic brain injury, ischemic or hemorrhagic cerebrovascular accident, brain tumor and neurodegenerative diseases (Table 1).

Table 1. Characteristics of participants.

	Healthy participants (Total N = 194)		Patients with ABI (Total N = 502)
	N		N		p-Value
Age in years, mean (SD)	194	60.30 (13.76)	502	60.77 (14.32)	.694^h
Gender, N female (%)	194	121 (62.37)	502	232 (46.20)	<.001^i^,*
Handedness, N right (%)	194	168 (86.60)	495	445 (88.60)	.011^i^,*
Education^a, mean (SD)	194	5.61 (0.81)	496	5.12 (1.17)	<.001^j^,*
Etiology, N (%)			502
CVA				329 (65.5)
CVA nos				32 (9.7)
iCVA				250 (76.0)
hCVA				37 (11.2)
i + hCVA				10 (3.0)
Side of lesion^b			329	127/107/34/61
TBI				31 (6.2)
Tumor				27 (5.4)
Parkinson’s disease				20 (4.0)
MS				27 (5.4)
Multiple neurological etiologies				22 (4.4)
Other–single neurological etiology^c				35 (7.0)
Unclassified^d				11 (2.2)
Time since diagnosis in months, median (range)^e			458	6 (0–469)
Visual field defect, N yes (%)^f			386	275 (71.2)
Impaired contrast sensitivity, N (%)^g			395	13 (3.3)

CVA nos: cerebral vascular accident not otherwise specified; iCVA: ischemic cerebral vascular accident; hCVA: hemorrhagic cerebral vascular accident; i + hCVA: ischemic and hemorrhagic cerebral vascular accident; TBI: traumatic brain injury; MS: multiple sclerosis.

^a Highest completed education according to Verhage (1965) ranging from 1 (<6 years of primary education), 4 (less than four years of lower secondary education) to 7 (university degree).

^b Restricted to patients with CVA; right/left/bilateral/unknown.

^c Single etiology other than the ones specified, e.g. posterior cortical atrophy, subarachnoid hemorrhage.

^d Diagnosis unknown at time of referral.

^e In case of multiple neurological disorders, time in months since most recent diagnosis.

^f An absolute binocular visual field defect within 30 degrees from the center.

^g Impaired contrast sensitivity (Log) <1.40.

^h t-Test for comparisons of means.

ⁱ χ²-test for nominal data.

^j Mann-Whitney for ordinal data.

* Significant at p < .05.

Healthy participants

Ethical permission for the assessment of healthy individuals was obtained from the Ethical Committee Psychology (ECP) affiliated to the University of Groningen, the Netherlands. Between 2016 and 2018, 281 individuals were recruited from the local community via public announcements, community centers, advertisements in local newspapers and by word-of-mouth. Inclusion criteria for healthy participants were (i) age of 18 years or older, (ii) no self-reported history of neurological disorders, (iii) no indication for cognitive decline as assessed with the Mini Mental State Examination (MMSE; cutoff score <24; Folstein et al., 1975; Kok & Verhey, 2002), (iv) self-reported normal or corrected-to-normal vision, and (v) good command of the Dutch language. Of these 281 individuals, three individuals reported a history of a relevant neurological disorder, one individual performed below the cutoff score (<24) on the MMSE and 11 individuals reported an ocular condition (e.g. cataract, glaucoma). Finally, eight healthy individuals had to be excluded because data on the BPT or the CTP were incomplete. Excluding these individuals resulted in a sample size of 258 healthy individuals. To allow for comparison between groups, these individuals were matched as good as possible to patients with ABI on age and education. From the total of 258 healthy participants, participants were randomly selected in such a way that, compared to patients with ABI, the healthy comparison group contained for each 10 years of age a similar proportion of participants with, as far as possible, comparable educational levels. The sample size of the group of healthy participants was, therefore, reduced to 194 participants which served as a comparison group to patients with ABI.

The age of the healthy participants ranged from 21 to 87 years, with a mean age of 60.3 years (SD = 13.8). Educational level of healthy participants ranged from 3 to 7, with a mean of 5.6 (SD = 0.8). Healthy participants and patients with ABI did not differ significantly in age, however, the healthy participant group contained significantly more females, was less often right-handed and was more highly educated than patients with ABI (Table 1). Participation of healthy participants was voluntary and not paid.

Materials

Birthday party test

The Birthday Party Test (BPT) includes a complex picture that was designed by a professional cartoonist (Figure 2; for a copy in actual size we refer to the Supplementary materials). The cartoonist was shown existing complex drawings for comparison reasons and was instructed with specific criteria, taking into account the shortcomings of previous complex pictures: First, the picture should be a two-dimensional line drawing in landscape-format depicted in black lines on a white background for maximal contrast. Second, the picture should represent a scene of a situation encountered in everyday life, in which, like other complex pictures, several events take place. Third, to reduce possible biases on the individual’s test performance due to lateralized disorders (e.g. hemianopia or visual neglect), events (i.e., actions/relations) should be presented in each of the four quadrants of the picture. Furthermore, the correct perception of some events should require the integration of visual elements that are presented at different spatial locations in the picture (e.g. the action/relation that the woman points toward the mouse). Finally, the picture should avoid stereotypes.

Figure 2. The BPT (reduced size).

Instructions were standardized and read aloud to the participant “Please tell as accurately and thoroughly as possible what you see on the picture and what is going on in the picture”. In case the participant mentioned only a few items of the picture, only details without integrating the details or was not giving further responses for some time without indicating to be finished, the participant was encouraged by the assessor to give more information according to the following, again standardized, instruction: “Is there more to see?” or “Could you perhaps tell me more?” (the original Dutch instructions are provided in the Supplementary materials).

As there is no preexisting systematic scoring system for the assessment of simultanagnosia using a complex drawing, diagnosis of simultanagnosia is usually based on a qualitative evaluation. For the Cookie Theft Picture, a scoring aid was developed for the assessment of language dysfunctions (Croisile et al., 1996). We followed a similar design with regard to the subdivision in persons/objects and actions/relations. The scoring aid presents a complete list of the items that are depicted on the BPT (see the Supplementary materials for a copy of the scoring aid). The scoring aid of the BPT contains 40 items, each allocated to one of the three categories persons/animals (n = 9), objects (n = 18) and actions/relations (n = 13). This subdivision in persons/animals, objects and actions/relation was considered as important. There is evidence that objects and in particular the relation between objects are of relevance in this context. It was shown that patients with simultanagnosia fail to localize objects, even when these objects are actually seen (Farah, 1990). Accordingly, if a patient mentions objects only, but not the relation between objects, then this is an indication of simultanagnosia. Disturbed spatial cognition can therefore result in a piecemeal description of the picture. One point is awarded for each correctly reported item. Synonyms or descriptions are regarded acceptable, as we are interested in perceptual impairments, not language dysfunctions. Of note, if an action/relation is reported by the participant, points are also awarded for the corresponding items of the categories persons/animals and objects that are mentioned in the context of the action/relation. For example, the response “The woman points towards the mouse” is awarded with a point for the correctly reported action/relation, as well as with two more points for the correctly reported persons/animals, that is “woman” and “mouse”. It can be derived from the scoring aid if additional instructions (encouragement) have been given, in order to differentiate between items that have been reported spontaneously and items that have been mentioned after encouragement (by using the specified instructions). Outcome measures of the BPT are (i) the number of correctly reported items per category, (ii) the total number of correctly reported items, as well as (iii) the total time in seconds taken to perform the test (registered from the moment the picture is presented to the participant until the participant indicates being finished describing the picture). We consider the number of actions/relations (NoA-BPT) as the main outcome variable of the BPT as it requires the integration of visual elements into a coherent whole, i.e. the depicted scene. Further information on test administration and scoring (including examples of scoring) can be found in the Supplementary materials.

After having completed the test, the assessor was asked for a clinical judgment (see Supplementary materials) that comprises four questions related to the participant’s performance on the BPT. The assessor was asked (i) whether the participant described the picture based on the details or based on the whole (relations), (ii) whether the participant had to be encouraged and, if so, (iii) whether the encouragement helped the participant to describe the picture more elaborately. Finally, the assessor was asked (iv) to judge on a four-point scale whether the participant in question may suffer from simultanagnosia, ranging from very unlikely to very likely. As no clear criteria for the presence of simultanagnosia exist, no guidelines to judge the presence of simultanagnosia were provided.

Cookie theft picture

The Cookie Theft Picture (CTP) was published as part of the Boston Diagnostic Aphasia Examination (Goodglass & Kaplan, 1983) and represents a picture of a scene consisting of several items. The participant is requested to describe the picture and the performance is scored regarding the number of items mentioned. As described earlier, test administration and scoring of the CTP differed across previous studies, therefore, the instructions with regard to the CTP as used in the present study were kept completely identical to the instructions given with regard to the BPT in order to increase comparability of both tests. Scoring was performed by awarding one point per correctly reported item. Scoring was facilitated by a scoring aid that presents a complete list of all items that are depicted on the CTP, which follows to great extent the scoring as proposed by Croisile et al. (1996). The scoring aid consists of 24 items, each allocated to one of the categories persons (n = 4), spaces (n = 2), objects (n = 11) and actions/relations (n = 7). Analog to the scoring of responses on the BPT, a report of an action/relation was also scored with regard to the persons, spaces or objects that are mentioned in this context. The number of actions/relations (NoA-CTP) was considered as the main outcome variable of the CTP.

Assessment of visual function

Visual function was assessed in patients with ABI. Visual acuity, visual field and contrast sensitivity were assessed binocularly at 500 lux. Visual acuity was assessed at four meters using an acuity chart (e.g. ETDRS 2000 Letter Chart; Ferris et al., 1982). A performance worse than the line corresponding to an acuity of 0.8 (Snellen equivalent 6/7.5 or 20/25, LogMAR 0.1) was considered as mildly impaired or worse (World Health Organization, 2018). Binocular and monocular visual fields were measured by perimetry (e.g. Goldmann perimetry or Humphrey Field Analyzer). A visual field defect was defined as (i) clearly present (i.e. an absolute binocular defect within 30 degrees from the center), (ii) borderline (e.g. a relative binocular defect within 30 degrees from the center or relative/absolute peripheral field loss outside the central 30 degrees) or (iii) absent. Contrast sensitivity was tested using the Gecko Test (Kooijman et al., 1994) or the Vistech chart (Ginsburg, 1984). Contrast sensitivity was considered as moderately impaired when the log sensitivity was below 1.40.

Procedure

The BPT was designed as part of a screening battery for the assessment of visual perceptual disorders following ABI (for further details on the composition of the battery, see De Vries et al. (2018)). Both patients with ABI and healthy participants performed the screening battery including the BPT. For all participants, the BPT was presented in A4-format on a digital Wacom tablet (programmed by Metrisquare B.V.).

Patients with acquired brain injury (ABI)

Visual perceptual assessment in patients with ABI was performed by experienced neuropsychologists or trained graduate students supervised by an experienced neuropsychologist. Simultanagnosia is a rare condition, however, considering that patients are referred to Royal Dutch Visio because of visual complaints, clinicians working at Visio are experienced at the (differential) diagnosis of agnosia and other visual perceptual disorders. Before inclusion in this study, all participating clinicians received a lecture (given by JH) on perceptual disorders, including definition and characteristics of simultanagnosia. We explicitly did not train assessors on how to come to a judgment on the presence of simultanagnosia based on the scores on the BPT, in order to prevent bias.

Prior to visual perceptual assessment, patients with ABI were administered an assessment of visual functions, including visual acuity, contrast sensitivity and visual field, as performed by experienced optometrists/orthoptists.

Healthy participants

Healthy individuals were assessed at the Department of Psychology of the University of Groningen, the Netherlands. Assessments of healthy individuals were performed by trained graduate students. Students that participated in the assessment of healthy controls all received a lecture on visual perceptual disorders (by JH) in a previous year to their participation and received additional instructions on the characteristics of simultanagnosia shortly before their participation in the study. Informed consent was obtained from all healthy participants prior to their assessment. After completion of a demographic questionnaire and administration of the MMSE, participants performed a screening battery for visual perceptual disorders, which included a digital version of the BPT. Finally, all healthy participants additionally performed the CTP, presented on A4-paper.

Statistical analysis

Before any analyses were performed, outliers on the outcome measure time of the BPT and CTP (defined as values exceeding more than three times the absolute deviation from the median (Leys et al., 2013)) were considered for exclusion. Based on this approach, eleven healthy controls needed more time than three times the absolute deviation from the median to complete the BPT and/or the CTP. For patients with ABI, this number was 23. Closer inspection of the data showed, however, no indication of errors in test administration or other arguments that could legitimate the exclusion of these participants. Consequently, outliers were not excluded.

Summary statistics (i.e., measures of central tendency and variability) were calculated for all variables of the BPT and the CTP per group. Spearman correlation coefficients were computed between the main outcome measure NoA-BPT and descriptive, medical, and neuropsychological characteristics of the participants. Correlation analyses were performed separately for patients with ABI (including age, gender, education, visual field defects, contrast sensitivity and the clinical judgment of the assessor of the participant’s performance on the BPT) and healthy individuals (including age, gender, education, and the performance on the CTP). Correlation analyses were preferred over multivariate regression models in order to identify the entire association per pair of variables, and to avoid a drop in sample size within multivariate statistics because of missing values. Alpha level was set to 0.01 in order to control for type-1 error inflation in multiple testing. Furthermore, correlation analyses were mainly interpreted on the basis of effect size. In this respect, small effects (r < 0.30), medium effects (0.30 ≤ r < 0.50) and large effects (r ≥ 0.50) were distinguished according to Cohen (Cohen, 1988).

Next, a cutoff score on NoA-BPT was determined to classify individual scores into impaired and not impaired. The cutoff indicating impairment was determined based on a commonly accepted categorization of ability levels (Lezak, 2012), that is if a score in the NoA-BPT falls 2.0 SD below the mean of healthy individuals (i.e., percentile ≤2). This cutoff was applied on test performances of patients with ABI in order to determine the number of patients and to further characterize the patients who show impaired performance on the BPT. Furthermore, in order to evaluate the test performance of patients with ABI who likely suffer from simultanagnosia (Sim; based on the clinical judgment of the assessor “likely” or “very likely” having simultanagnosia), two comparison groups were formed. The two comparison groups represented (i) a group of patients with ABI unlikely suffering from simultanagnosia (NoSim; according to the clinical judgment by the assessor “unlikely” or “very unlikely” having simultanagnosia), and (ii) a group of healthy individuals (HC). The two comparison groups were selected based on age, educational level and, if applicable, visual field defect and etiology of ABI. More specifically, NoSim was formed in such a way that the distribution of the etiologies in this group was the same as in Sim. In order to represent less common etiologies better in NoSim, we randomly selected for NoSim twice as many individuals of each etiology compared to Sim. Because assumptions for parametric testing were not fulfilled for all variables, the three groups were compared on all outcome measures of the BPT by means of Kruskal-Wallis tests (omnibus comparisons) and Dunn’s post-hoc tests for pairwise comparisons. Significance levels were Bonferroni adjusted in order to correct for multiple testing. Effect sizes of pairwise comparisons were indicated by Cohen’s r (Cohen, 1988).

Moreover, for the group of patients with ABI, Receiver Operating Characteristics (ROC) analysis was performed in order to explore the association between the NoA-BPT and the assessor’s clinical judgment of the patients (“likely” or “very likely: having simultanagnosia). The strength of the association (i.e. correspondence in classification) is determined by the area under the curve (AUC), indicating the probability that a NoA-BPT score drawn at random from the first sample (i.e. patients unlikely having simultanagnosia) is higher than a score drawn at random from the second sample (i.e. patients likely having simultanagnosia) (Rice & Harris, 2005).

Results

Descriptive analysis

Figure 3 presents the distribution of scores of the main outcome variable NoA-BPT for healthy individuals and patients with ABI. Scores for both groups are approximately normally distributed, whereas a slightly larger spread of scores is observed for patients with ABI, with a narrower distribution around the mean for healthy individuals. Descriptive statistics of the distributions of the remaining variables of the BPT are presented per group in Table 2. Patients with ABI have slightly lower mean scores and a slightly increased spread in all variables of the BPT compared to healthy individuals. Furthermore, patients with ABI needed almost double the time to complete the test.

Figure 3. Distribution of scores on NoA-BPT for healthy individuals (n = 194) and patients with ABI (n = 502).

Table 2. Test performance of patients with ABI and healthy participants.

	N	Min	Max	Median	Mean	SD
Healthy participants (n = 194)
Cookie Theft Picture
Number of persons	194	1	4	4.0	3.48	0.56
Number of spaces	194	0	2	2.0	1.69	0.57
Number of objects	194	3	11	9.0	8.38	2.02
Number of actions/relations	194	2	7	5.0	5.06	0.97
Total score	194	10	24	19.0	18.61	2.82
Time to complete in seconds	193	17.0	369.0	70.0	76.26	39.52
Birthday Party Test
Number of persons/animals	194	3	9	8.0	7.78	1.52
Number of objects	194	2	18	12.0	11.42	3.99
Number of actions/relations	194	1	12	7.0	7.29	1.94
Total score	194	7	37	27.0	26.49	5.71
Time to complete in seconds	193	26.0	272.0	77.0	87.96	43.09
Patients with ABI (n = 502)
Birthday Party Test
Number of persons/animals	502	0	9	9.0	7.69	2.09
Number of objects	502	0	18	12.0	11.17	4.39
Number of actions/relations	502	0	13	7.0	7.02	2.61
Total score	502	6	40	27.0	25.88	7.12
Time to complete in seconds	502	38.0	1192.4	132.0	150.8	83.4

Correlation analyses in healthy individuals revealed that the main outcome variable NoA-BPT showed a positive association with education, as well as the NoA-CTP (see Table 3). These correlations were of small size. In patients with ABI, a positive significant association was found between NoA-BPT and education, whereas negative significant associations were found for age and the clinical judgment of the performance on the BPT (see Table 3). The correlations with age and education were of small size, whereas a large effect was observed for the clinical judgment. A negative significant correlation (small size) was found between NoA-BPT and visual field. The correlation between NoA-BPT and contrast sensitivity was non-significant.

Table 3. Spearman correlation coefficients between the NoA-BPT and various descriptive and clinical characteristics of healthy individuals and patients with ABI.

	NoA-BPT
	N	r	p
Healthy participants
Age	194	–0.127	.077
Gender	194	0.068	.344
Education	194	0.229	.001*
NoA Cookie Theft Picture	194	0.199	.005*
Patients with ABI
Age	502	–0.195	<.001*
Gender	502	0.050	.265
Education	496	0.263	<.001*
Contrast sensitivity^a	395	–0.034	.496
Visual field defects^b	386	–0.147	.004*
Clinical Judgment^c	436	–0.502	<.001*

NoA: Number of actions.

^a Impaired ((Log) <1.40) or not impaired contrast sensitivity.

^b Presence or absence of an absolute binocular visual field defect within 30 degrees from the center.

^c Clinical evaluation of simultanagnosia on a four-point scale.

* Significant at p < .01.

Identification of cutoff and performance on NoA-BPT for different etiologies

Next, a cutoff score of 3 on NoA-BPT was determined based on the performance of healthy individuals, representing a score of two SD below the mean (i.e., percentile ≤2). Subsequently, the performances of patients with ABI on the NoA-BPT were classified as impaired if scored ≤3, and not impaired if scored ≥4. Table 4 presents the number and percentages of patients with ABI who show an impaired performance on NoA-BPT and who are classified as likely having simultanagnosia according to the clinical judgment. Patients with multiple etiologies and patients who were still unclassified at referral showed most often impaired performance on NoA-BPT (more than 18%), followed by patients who were allocated to the group of other etiologies (more than 14%). Of note, among the patients with other etiologies, even though only observed in a small group, four out of eight patients with posterior cortical atrophy (PCA) or Alzheimer’s disease (50.0%) revealed a performance in the impaired range. No increase has been observed in the number of impaired patients suffering from brain tumors or MS. With regard to the clinical judgment of likely having simultanagnosia, the largest proportions were found in the group of other and multiple etiologies. Again, the small group of patients with PCA/Alzheimer’s disease (n = 8), allocated to the group of “other etiologies”, contained a large proportion of patients likely having simultanagnosia (n = 6). Most of the remaining groups had prevalence rates of about 4–10%, whereas no likely simultanagnosia was observed in patients with MS. A bit more than three percent of all patients scored below the cutoff (≤3) on NoA-BPT and were also evaluated by their clinicians as likely having simultanagnosia. Highest percentages were again found in the groups with multiple (9.5%) and other etiologies (11.4%).

Table 4. Patients with ABI with a NoA score below the cutoff on the BPT and likely having simultanagnosia based on clinical judgment.

	NoA		Clinical judgment		NoA and clinical judgment
Etiology	Total N	N below cutoff (%)^a	Total N	N with likely simultanagnosia (%)^b	Total N	N below cutoff and likely simultanagnosia (%)^c
CVA	329	28 (8.5)	290	15 (5.2)	322	9 (2.8)
TBI	31	1 (3.2)	25	1 (4.0)	31	0 (0.0)
Tumor	27	0 (0.0)	21	1 (4.8)	27	0 (0.0)
Parkinson’s disease	20	2 (10.0)	17	1 (5.9)	20	1 (5.0)
MS	27	0 (0.0)	24	0 (0.0)	27	0 (0.0)
Multiple etiologies	22	4 (18.2)	17	2 (11.8)	21	2 (9.5)
Other	35	5 (14.3)	32	7 (21.9)	35	4 (11.4)
Unclassified	11	2 (18.2)	10	1 (10.0)	11	1 (9.1)
Total group	502	42 (8.4)	436	28 (6.4)	494	17 (3.4)

NoA: Number of actions; CVA: cerebral vascular accident; TBI: traumatic brain injury; MS: multiple sclerosis.

^a Number of patients with a score below the cutoff (≤ 3) on number of actions, representing a score equally or lower than 2 SD below the mean of healthy participants (2th percentile).

^b Number of patients with likely/very likely simultanagnosia according to clinical judgment.

^c Number of patients with both a score below the cutoff (≤ 3) on number of actions and likely/very likely simultanagnosia according to clinical judgment.

Group comparisons on NoA-BPT

In order to characterize the performance on the BPT of those patients with ABI who likely suffer from simultanagnosia (Sim, n = 28), two comparison groups were selected based on age, educational level, and, if applicable, etiology of ABI. The two comparison groups represented patients with ABI unlikely suffering from simultanagnosia (NoSim, n = 56) and healthy controls (HC, n = 56). The three groups did not differ significantly from each other in terms of age, gender and education (Table 5). Also, both patient groups (Sim and NoSim) did not differ significantly in terms of visual field defects, time since injury and the different etiologies of ABI were equally represented. A statistical comparison of the outcome measures of the BPT (Table 6) revealed significant group differences in each of the variables assessed, except for the number of persons/animals. Sim mentioned significantly fewer objects, fewer actions/relations, obtained a lower total score and needed also more time to complete the test compared to NoSim and HC. Small to medium effect sizes were observed for the number of objects, whereas large effects were revealed for number of actions/relations, total score and time to complete the test. HC and NoSim did not differ significantly from each other on either of the measures of the BPT (negligible to small differences), except for the time to complete the test, with more time needed by NoSim (large effect size).

Table 5. Characteristics of healthy controls (HC), patients with ABI unlikely having simultanagnosia (NoSim) and patients with ABI likely having simultanagnosia (Sim).

	HC (N = 56)		NoSim (N = 56)		Sim (N = 28)		p-Value
	N		N		N
Age in years, mean (SD)	56	67.5 (9.3)	56	66.4 (10.0)	28	67.3 (9.5)	.820^e
Gender, N female (%)	56	30 (53.6)	56	24 (42.9)	28	13 (46.4)	.518^f
Education^a, mean (SD)	56	5.4 (0.9)	56	5.1 (1.0)	27	5.1 (1.2)	.296^g
Visual field defect, N (%)^b			39	28 (71.8)	21	17 (81.0)	.438^h
Time since diagnosis in months, median (range)^c			52	6.0 (0–285)	26	7.5 (3–67)	.079^e
Cause of ABI^d			56	30/2/2/2/4/14/2	28	15/1/1/1/2/7/1

HC: healthy controls; NoSim: patients with ABI with unlikely/very unlikely simultanagnosia; Sim: patients with ABI with likely/very likely simultanagnosia.

^a Highest completed education according to Verhage (1965) ranging from 1 (less than 6 years of primary education), 4 (less than four years of lower secondary education) to 7 (university degree).

^b Presence of an absolute binocular visual field defect within 30 degrees from the center.

^c In case of multiple neurological disorders, time in months since most recent diagnosis.

^d CVA/TBI/Tumor/Parkinson/Multiple etiologies/Other/Unclassified.

^e ANOVA/t-Test for comparisons of means.

^f χ²-test for nominal data.

^g Kruskal-Wallis for ordinal data.

^h Mann-Whitney for ordinal data.

Table 6. Group comparisons between healthy controls (HC), patients with ABI unlikely having simultanagnosia (NoSim) and patients with ABI likely having simultanagnosia (Sim) with regard to their performance on the BPT.

					Pairwise comparisons^b
	HC (n = 56)	NoSim (n = 56)	Sim (n = 28)	Omnibus test^a	Sim vs. HC		Sim vs. NoSim		HC vs. NoSim
BPT variable	M (SD)	M (SD)	M (SD)	H (df ), p	Z, p	Cohen’s r	Z, p	Cohen’s r	Z, p	Cohen’s r
Number of persons/ animals	7.54 (1.49)	7.84 (1.84)	7.07 (1.76)	8.58 (2), 0.014	–	–	–	–	–	–
Number of objects	10.68 (3.89)	11.16 (4.01)	7.89 (4.27)	10.89 (2), 0.004*	2.75, 0.006**	0.300	3.19, 0.001**	0.348	.54, .586	0.051
Number of actions/ relations	7.30 (1.75)	7.02 (2.15)	2.96 (2.32)	47.68 (2), < 0.001*	6.67, <0.001**	0.728	5.77, <0.001**	0.630	1.10, .270	0.104
Total score	25.52 (5.22)	26.02 (6.00)	17.93 (6.61)	29.20 (2), < 0.001*	4.77, <0.001**	0.520	5.07, <0.001**	0.553	.37, .713	0.035
Time to complete in seconds	87.53 (44.25)	152.95 (68.91)	275.54 (208.73)	65.82 (2), < 0.001*	7.77, <0.001**	0.853	3.41, 0.001**	0.372	5.34, <0.001**	0.507
Impairment on number of actions/relations, N (%)^c	–	2 (3.6)	17 (60.7)	–	–	–	–	–	–	–

HC: healthy controls; NoSim: patients with ABI with unlikely/very unlikely simultanagnosia; Sim: patients with ABI with likely/very likely simultanagnosia.

^a Kruskal-Wallis test.

^b Dunn’s post hoc test.

^c Number of patients with a score below the cutoff (≤3) on number of actions, representing a score equally or lower than 2 SD below the mean of healthy participants (2th percentile).

* Significant at Bonferroni corrected significance level of 0.05/5 = 0.01. **Significant at Bonferroni corrected significance level of 0.05/3 = 0.017.

Association between NoA-BPT and assessor’s clinical judgment of likely having simultanagnosia

Finally, a ROC analysis demonstrated a high correspondence between the NoA-BPT and the assessor’s clinical judgment of likely having simultanagnosia. The NoA-BPT could successfully distinguish between patients with ABI who, according to the assessor’s clinical judgment, likely suffer from simultanagnosia and patients who unlikely suffer from simultanagnosia, p < .001; SE = 0.032, 95% CI [0.847, 0.971]. Classification correspondence was high as shown by area under the curve (AUC) statistics of 0.91. A graphical illustration of the classification correspondence of the NoA-BPT is depicted in Figure 4. It must be noted that the AUC derived from the ROC analysis is likely inflated because patients’ BPT performance may have influenced the assessor’s clinical judgment. For this reason, we refrain in this analysis from deriving optimal cutoff points on the NoA-BPT for diagnostic purposes. Instead, we like to demonstrate how various scores on NoA-BPT correspond to the clinical judgment and present classification statistics of sensitivity and specificity for future replication (Table 7).

Figure 4. Receiver operating characteristics curve indicating the correspondence between the BPT number of actions (NoA) and the assessor’s clinical judgment of patients with ABI likely (n = 28) and unlikely (n = 408) suffering from simultanagnosia.

Table 7. Classification statistics for the identification of patients with likely simultanagnosia for various cutoffs of NoA on the BPT.

Cutoff	Sensitivity	Specificity
≤2	46.4	98.8
≤3	60.7	95.8
≤4	78.6	90.4
≤5	85.7	80.1
≤6	92.9	65.9
≤7	96.4	48.3

NoA: Number of actions.

Discussion

In this study, we presented the newly developed BPT; a picture description test that is specifically designed for the assessment of simultanagnosia in patients with ABI. We showed the utility of the BPT by applying it to a large group of patients with ABI and healthy controls.

The number of reported actions/relations on the BPT is the most useful indicator of simultanagnosia

As simultanagnosia is defined as an inability to integrate individual elements of a scene into a coherent meaning despite intact recognition of visual details, there is good reason to assume that NoA-BPT is the most critical variable for clinical evaluation, and is more informative than other variables such as number of persons, animals, or objects. Indeed, comparing patients with ABI likely having simultanagnosia (Sim) with patients with ABI unlikely having simultanagnosia (NoSim) and healthy controls (HC) revealed that NoA-BPT appears to be a useful indicator of simultanagnosia. Sim yielded significantly lower scores, substantiated by large effect sizes, compared to both NoSim and HC. Furthermore, no difference was found between the three groups for the number of persons/animals. This was in line with our expectation, because patients with simultanagnosia are able to recognize individual parts of a scene but no more than one at a time. Surprisingly, we found decreased scores for the Sim group on the number of objects compared to NoSim and HC, however, effect sizes were smaller than the effects found for NoA-BPT. This difference in number of objects was not related to decreased object recognition in patients with ABI in general, as no significant difference was observed between NoSim and HC for number of objects. Object identification has been found to be relatively spared in patients with simultanagnosia (Mazza, 2017; Thomas et al., 2012). However, Rennig and Karnath (2016) found that object perception in patients with simultanagnosia is dependent on object size, meaning that object recognition may decrease with increasing stimulus size. As the objects depicted in the BPT are relatively small, the reduced recognition of objects found in Sim can presumably not be explained by the size of objects depicted in the BPT.

Slow performance on the BPT is not a criterion for simultanagnosia

Simultanagnosia has been explained by Duncan and colleagues (Duncan et al., 2003) as a slowing of visual processing. Congruent with this, Sim needed significantly more time than NoSim and HC to complete the task in our study. On the other hand, NoSim was also significantly slower in completing the task compared to HC. This is not surprising considering that for decades, it has been demonstrated that ABI results in slowed performance on cognitive tasks. For example, processing speed deficits are found to be present in multiple sclerosis (DeLuca et al., 2004), stroke (Lesniak et al., 2008) and TBI (Hillary et al., 2010; Madigan et al., 2000). Therefore, the observed slowing on the BPT seems to be a common symptom of people with ABI in our sample rather than an indicator of simultanagnosia. Yet, these results provide further support for the utility of the BPT as an indication of simultanagnosia as NoSim differed significantly from HC only on the time to complete the test, not on other variables of the BPT.

Sensitivity of the BPT to demographic and clinical variables

NoA-BPT appeared to be sensitive to demographic variables (with younger age and higher education indicating better performance) and clinical variables (i.e. a negative association with the clinical judgment about the presence of simultanagnosia and a positive association with the performance on the Cookie Theft Picture (CTP)). If both the BPT and the CTP provide indications for simultanagnosia, considerable associations can be expected. Indeed, a significant association was found between both measures, but the correlation was of small size. This does not necessarily indicate that both tests measure different constructs, but it may show that they are differently designed. Where the BPT allows for more variation, a more limited range in score was found on the CTP. This limited range in scores on the CTP might have resulted in an underestimation of the true association, as in the present study this association could unfortunately only be explored within the group of healthy individuals, in which one would not expect simultanagnosia to be present. Larger associations might be observed in patients with ABI, which could be subject for future research.

Increased rates of poor BPT test performance in patients with multiple etiologies of ABI

Inspecting test performance of various subgroups of patients with ABI revealed considerable group differences regarding the NoA-BPT. Poor performance (equal or lower than 2nd percentile) was most common in patients with multiple etiologies (18%) and in patients who were still unclassified at time of referral (18%). Also, in the group with multiple etiologies relatively more patients have been suspected to suffer from simultanagnosia as evaluated by the assessor. As the likelihood of bilateral damage may be increased in this group, this finding is in accordance with previous research reporting that simultanagnosia (i.e. the dorsal variant) is caused primarily by bilateral damage (Devinsky et al., 2008). Also, in the group of patients that did not fit the most common causes of ABI (i.e. “other etiologies”), relatively more patients performed below the 2nd percentile. Interestingly, out of the 502 patients with ABI, 28 patients have been assumed to suffer from simultanagnosia, with higher relative frequencies of those having Alzheimer’s disease or PCA. This is not surprising, as simultanagnosia is more likely to occur in neurodegenerative diseases (Coslett & Saffran, 1991) and is considered one of the core cognitive features of PCA (Crutch et al., 2017), affecting over 80% of patients (Kas et al., 2011; Singh et al., 2015; Tang-Wai et al., 2004). This is a similar rate to the one found in our sample, even though we had access to only a small number of patients with an established diagnosis of PCA. Furthermore, it must be noted that patients with PCA often report vague complaints while ophthalmological and/or neurological examinations are relatively normal (Beh et al., 2015). Therefore, diagnosis of PCA might be missed or delayed. Thus, it appears not unlikely that this is the reason why the rate of impaired performance is elevated in the group “unclassified”, as these are the patients whose neurological diagnosis was still unknown at time of referral, but did suffer from visual complaints.

The number of reported actions/relations on the BPT corresponds to clinical judgment of likely having simultanagnosia

ROC analysis demonstrated the utility of the BPT (i.e. NoA) to distinguish patients with ABI who, according to the assessor’s judgment, unlikely suffer from simultanagnosia from those likely suffering from simultanagnosia. A strong association between scores on NoA-BPT and the clinical judgment can be assumed based on the AUC value of 91% and various scores of sensitivity and specificity, although it must be noted that the assessors were not blind to BPT performance, which may inflate the association between both variables. For this reason, optimal cutoff points cannot be derived from this analysis as classification accuracy would likely be overestimated.

Limitations

Our study has some limitations. First, healthy participants of the present study differed significantly from patients with ABI on several characteristics, including education. This could be a problem as the NoA-BPT was shown to vary as a function of education. Even though the association was only small, it may have distorted the determination of a cutoff indicating impaired test performance on the basis of the healthy participants.

The goal of the ROC analysis was to demonstrate to the reader how various scores on NoA-BPT corresponds to the clinical judgment of a large group of assessors, but not to investigate the accuracy of NoA-BPT in diagnosing simultanagnosia. However, as a second limitation, clinical judgment of likely having simultanagnosia was not independent from test performance in the BPT, as the assessors administering the test were also making the clinical judgment about the patient. Nevertheless, clinicians were not instructed how to derive a clinical judgment. In clinical practice, a clinical evaluation is based not only on a particular test score but also on other information, such as neurological information (e.g. localization of brain damage), anamnestic- or hetero-anamnestic information (e.g. a patient complains that an object or person that is seen clearly suddenly vanishes out of sight), observations as well as performance on other neuropsychological tests. Here, the clinical judgments were therefore presumably not exclusively based on one single test. However, as validated tests for the assessment of simultanagnosia are not yet available and because diagnosis is primarily based on clinical observation, examining the association between test performance on the BPT and the clinical judgment was, especially in the current phase of the BPT, an examination of interest. Yet, group comparisons between Sim and NoSim as well as the ROC analysis have to be interpreted with caution, as differences between the groups likely and unlikely having simultanagnosia and the discriminative value of NoA-BPT may be overestimated.

Related to this, even though raters have been trained by a lecture on visual perceptual disorders and additional explanation on the characteristics of simultanagnosia to ensure accuracy of their judgements, the agreement between raters in their judgment has not been examined. Therefore, it is possible that the individual thresholds for the judgment of the presence of simultanagnosia may differ between clinicians.

Fourth, the argument could be made that participants who needed more than three times the absolute deviation from the median to complete the BPT or CPT had an advantage in terms of greater time allotted for visual scanning. More time could possibly improve the detection of objects, but it is not expected that it would improve the detection of relationships between those objects in patients with simultanagnosia.

Finally, since data on injury severity of the TBI patients was not available, this variable was not taken into account in the analyses. Possibly, this could be of influence on test performance, e.g. patients with a mild TBI may perform differently than patients with severe TBI.

Conclusions

Many of the shortcomings of previous instruments in the assessment of simultanagnosia were considered in the development of the BPT presented in this paper. The test was specifically developed for simultanagnosia, provides clear instructions for administration and scoring, depicts neutral scenes and contains a balanced presentation of events in regard to their spatial presentation. The BPT is freely available to all readers without restrictions in order to facilitate research on and clinical use of the BPT and enable further evaluation of its utility and validity.

Our first psychometric investigations of the BPT in a large group of patients with ABI and healthy controls, as presented in this study, appear promising and provide the footing to a systematic observation of behavior that is typical for simultanagnosia. However, a validation of the test is not yet provided. As many psychometric characteristics of the instrument are still unknown, such as its concurrent (e.g. association to CTP scores in a clinical sample) and discriminant validity, its criterion validity toward an independent diagnostic decision, or its reliability in repeated assessments, future studies are needed.

Acknowledgements

We are grateful for the contribution to this project made by all participants, the staff of Royal Dutch Visio (in particular Hilde Biemans and Teatske Hoekstra), the Psychology graduate students of the University of Groningen and Ben Vaessen (Metrisquare B.V.).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by InZicht–ZonMW under Grant 94310003.