Diagnostic utility of Brain Injury Guidelines (BIG): systematic review and meta-analysis for prediction of neurosurgical intervention in traumatic brain injury

ABSTRACT

Background

The Brain Injury Guidelines (BIG) categorize the severity of Traumatic Brain Injury (TBI). The efficacy of BIG in predicting radiological deterioration and the necessity for neurosurgical intervention remains uncertain, as there is a lack of examination of pooled data from current literature despite validation in numerous single and multi-institutional studies. The aim of this study was to analyze existing studies to determine the diagnostic accuracy of BIG scoring criteria.

Methods

A systematic review and meta-analysis was conducted in accordance with PRISMA guidelines (PROSPEROID CRD42021277542). Three databases were searched, and articles published from 2000 to October 2022 were included (last search date: 25 November 2022). Pooled sensitivity and specificity were calculated using random effects meta-analysis.

Results

Of the 1130 articles identified, 13 were included in the analysis (9032 patients – 1433 BIG1, 2136 BIG2 & 3189 BIG3). A total of 2274 patients were not classified under either group. Pooled sensitivity for predicting neurosurgical intervention was 1.00 (95%CI:1.00–1.00), and 0.98 for radiological deterioration (95% CI: 0.927–0.996). The specificity in predicting radiological deterioration was 0.18 (95% CI: 0.16–0.21) and 0.05 for neurosurgical intervention (95% CI 0.05–0.05).

Conclusions

The BIG score is highly sensitive at excluding TBI cases that do not require neurosurgical intervention; however, BIG-2 and BIG-3 might not be useful for ruling in TBI patients who require neurosurgical intervention.

KEYWORDS:

• Traumatic brain injury
• BIG
• neurosurgery
• TBI
• intervention

Introduction

Traumatic Brain Injury (TBI) is common, with an estimated 1.4 million cases each year in England & Wales and is the most common cause of death under the age of 40 (1). The current NICE guidelines post stabilization advocate the use of computed tomography (CT) based on clinical assessments such as: GCS < 13 on assessment in the emergency department, suspected open/depressed skull fractures, post-traumatic seizure, focal neurological deficit, any signs of basal skull fracture or more than episode of vomiting. Neurosurgical consultation is standard care for TBI based on the abnormalities on imaging (2).

The Brain Injury Guidelines (BIG) was developed by Joseph et al., (2014) in order to better manage resources and stratify patient care for the treatment of TBI (3). BIG classifies patients into one of three categories based on CT scan findings, clinical history (loss of consciousness, anticoagulation therapy and intoxication), physical examination (GCS on admission, pupillary exam and neurological examination), and the need for neurosurgical intervention. BIG provides a method to categorize and treat TBI (4).

The presence of scoring criteria such as BIG is important in the management of TBI due to its high cost and incidence rate. Scores are likely to reduce unnecessary neurosurgical referrals and improve overall patients’ satisfaction with a possibility of early discharge (5). While primary studies have locally validated the use of BIG (6), a pooled diagnostic analysis on the use of BIG by assessing the pooled sensitivity, and specificity, has not been performed. The aim of this study is to evaluate at the overall diagnostic utility of BIG categories in the prediction of neurosurgical intervention and clinical deterioration.

Review question

The review aims to address the question: How effective is BIG in classifying patients according to the severity of their TBI?

Material and methods

Search strategy and selection criteria

We conducted a systematic review and meta-analysis according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Guidelines.

We searched PubMed, Embase, Clinical trails.Gov and Cochrane database of systematic reviews for full-text articles published in English (Search date 25 November 2022). Search terms used a combination of the terms ‘Traumatic brain injury’ and ‘neurosurgery,’ and their associated synonyms. The full search strategy for all databases can be found in Appendix Tables A1–A3. The Population, Intervention, Comparator, Outcome, Study Design criteria was used, and the inclusion criteria is shown in Table 1. We included the studies of adults (≥18 years) that specifically mentioned the term Traumatic Brain Injury (or TBI in Adults) in the title, or abstract. We excluded studies that reported pediatric TBI, or TBI studies that did not evaluate neurosurgical interventions. We excluded studies that were conference abstracts or case reports. We included TBI studies that specifically mentioned the term Brain Injury Guidelines (or BIG) in the title or abstracts, classified patients according to the BIG criteria, and included results on diagnostic accuracy (either for radiological, clinical deterioration, or neurosurgical intervention).

Table 1. BIG criteria.

Variables	BIG 1	BIG 2	BIG 3
Loss of consciousness	Yes/No	Yes/No	Yes/No
Neurologic examination	Normal	Normal	Abnormal
Intoxication	No	Yes/No	Yes/No
Coumadin, Aspirin, Plavix (CAMP)	No	No	Yes
Skull fractures	No	Non-displaced	Displaced
Epidural hemorrhage	≤4 mm	5-7 mm	≥8 mm
Intraparenchymal hemorrhage	≤4 mm, 1 location	3-7 mm, 2 locations	≥8 mm, multiple locations
Intraventricular hemorrhage	No	No	Yes
Subarachnoid hemorrhage	Trace	Localized	Scattered
Subdural hemorrhage	≤4 mm	5-7 mm	≥8 mm
Treatment
Hospitalization	No. observation for 6 hours	Yes	Yes
Repeat head CT	No	No	Yes
Neurosurgical consultation	No	No	Yes

Two reviewers (SK, CSG) independently screened titles, abstracts and full texts to include articles. If reviewers failed to reach consensus, a third author was sought for clarification.

Data extraction

Data extraction was completed by two authors independently (SK, CSG). The following data were extracted from included studies: Year published, journal, type of study (Randomized Control Trial [RCT] or observational study), single/multi center, number of patients with TBI, number of BIG-1,2,3 patients, and number of these patients that required neurosurgical intervention. Clinical and radiological deterioration were also recorded. Primary outcomes were sensitivity, and specificity of each BIG category.

Quality assessment

Quality assessment was completed by two reviewers independently (SK, CSG). Retrospective studies were classified according to the Newcastle–Ottawa Scale.

Statistical analysis

Baseline characteristics were presented as descriptive frequencies. For meta-analysis, we used random effects models of variables and endpoints. Bivariate summary receiver operating characteristic (SROC) curves and point estimates of sensitivity and specificity were computed according to Reitsma et al., using a linear mixed effects model with known variances of random effects (7). We evaluated the performance of BIG for radiological, clinical deterioration, and neurosurgical intervention points, by additionally collecting crossover diagrams, and diagnostic odds ratios (DOR). We summarized findings using pooled forest plots for sensitivity and specificity, and ROC plane plots. When studies were adequate, bivariable analysis and a SROC curve were presented. When studies were nondivergent, univariable analysis and ROC plane plots were used. We carried out an additional sensitivity analysis by carrying out analysis for studies at high risk of bias.

Data analysis of descriptive statistics was performed using SPSS (Version 27; IBM; Armonk; NY; USA). Both R statistics (Rstudio Version 4.0.1) and MetaDiSc 2.0 (http://www.metadisc.es/.) (8) was used to perform meta-analysis.

Results

Study details

After removal of duplicates, 236 studies were identified. After full-text assessment, 13 full-text studies were assessed for inclusion and were finally included, as shown in Figure 1. Majority of the studies were retrospective cohort studies (n = 11).

Figure 1. PRISMA flow diagram of study selection for inclusion in this review and meta-analysis.

Baseline characteristics

The baseline characteristics of included studies are summarized in Table 2; the most common country of published studies was the United States of America (69.2%, n = 9). The total number of TBI patients was 9032, of which 6758 (74.8%) were classified into one of the BIG categories. The remaining patients were not classified into any BIG category, and hence were excluded from the analysis. The median number of patients included per paper was 477 (IQR 13.0–32.0). The majority of the patients were classified as BIG-3 (47.2%), followed by BIG-2 (31.6%) and BIG-1 (21.2%) (Table 3).

Table 2. The table summarizes the baseline characteristics and outcomes measured for all the studies included in the final analysis.

Characteristics	N (%)
Country of origin
USA	9 (69%)
UK	2 (15%)
Canada	1 (3%)
Sweden	1 (3%)
Study design
Retrospective	11 (93%)
Prospective	2 (7%)
Total Number of patients	9032
Number of BIG patients	6758
BIG 1
Patients	1433 (21%)
Neurosurgical intervention	14 (0.98%)
Radiological intervention	37 (2.60%)
Clinical deterioration	16 (1.10%)
Deaths	1 (0.07%)
BIG 2
Patients	2136
Neurosurgical intervention	213 (10%)
Radiological deterioration	90 (4.20%)
Clinical deterioration	222 (10.40%)
Deaths	2 (0.09%)
BIG 3
Patients	3189
Neurosurgical intervention	494 (15.50%)
Radiological deterioration	602 (18.90%)
Clinical deterioration	270 (8.50%)
Deaths	42 (1.30%)

Table 3. The table summarizes the total number of patients and classification of patients under BIG in included studies.

Study	Country	Total number of TBI patients	Number of BIG 1	Number of BIG 2	Number of BIG 3
Joseph et al. 2014 (3)	USA	1232	121	313	798
Joseph et al. 2014 (9)	USA	254	148
Joseph et al. 2015 (10)	USA	796	170	175	451
Martin et al. 2018 (11)	USA	332	115	25	192
Marincowitz et al. 2020 (12)	UK	1569	57	Not classified	Not classified
Khan et al. 2020 (4)	USA	269	98	171	Not classified
Wheatley et al. 2021 (13)	USA	222	176	42	4
Vestlund et al. 2022 (14)	Sweden	538	8	Not classified	Not classified
Murali et al. 2021 (15)	USA	214	88	126	Not classified
Gribbell et al. 2021 (16)	USA	49	6	5	38
Tourigny et al. 2022 (17)	Canada	477	40	168	269
Marincowitz et al. 2022 (6)	UK	921	105	816	Not classified
Joseph et al. 2022 (18)	USA	2033	301	295	1437

BIG scores for predicting for neurosurgical intervention

The prediction for the need for neurosurgical intervention was calculated and shown in Table 4. Sensitivity and Specificity was calculated for each BIG score to evaluate the use of the guideline in predicting neurosurgical intervention. The number of studies that reported number of patients classified under each BIG criteria varied. The sensitivity for BIG 1 was 1.00 (95% CI 1.00–1.00) (Figure 2), whereas for BIG 2, the sensitivity was 0.07 (95% CI 0.00–0.12) (Figure 3) and for BIG 3, the sensitivity was 0.125 (95% CI 0.07–0.22) (Figure 4). The SROC curve is shown for BIG 1 and BIG 2; however, there was no SROC curve generated for BIG 3 (Supplementary Figure S1–figure 2). The DOR was high and negative likelihood ratio (LR-) was zero for BIG 1.

Figure 2. BIG 1 sensitivity plot for neurosurgical prediction.

Figure 3. BIG 2 sensitivity plot for neurosurgical prediction.

Figure 4. BIG 3 sensitivity plot for neurosurgical prediction.

Table 4. The table summarizes the bivariate model summary statistics for BIG score predicting neurosurgical intervention.

BIG 1
Parameter	Estimate	95% LCL	95% UCL
Sensitivity	1.00	1.00	1.00
Specificity	0.05	0.05	0.05
DOR	34699168.35	34245933.75	35158401.37
LR+	1.05	1.05	1.05
LR-	0	0	0
FPR	0.95	0.95	0.95
BIG 2
Parameter	Estimate	95% LCL	95% UCL
Sensitivity	0.007	0	0.123
Specificity	1.00	0	1.00
DOR	Inf	NA	Inf
LR+	Inf	NA	NA
LR-	0.993	NA	NA
FPR	0	0	1
BIG 3
Parameter	Estimate	95% LCL	95% UCL
Sensitivity	0.125	0.07	0.22
Specificity	1.00	0	1.00
DOR	111577408226953000	0	Inf
LR+	Inf	NA	NA
LR-	0.875	0.807	0.95
FPR	0	0	1.00

DOR = diagnostic odds ratio; LR+ = likelihood ratio for positive test results; LR- = likelihood ratio for negative test results; FPR = false positive rates.

BIG scores for predicting radiological deterioration

The prediction for radiological deterioration was calculated and shown in Table 5. Sensitivity was calculated for each BIG score to evaluate the use of the guideline in predicting radiological deterioration. Only a few studies reported radiological deterioration. The sensitivity for BIG 1 was 0.98 (95% CI 0.93–0.99) (Figure 5), whereas for BIG 2, the sensitivity was 0.09 (95% CI 0.05–0.13)(Figure 6) and for BIG 3, the sensitivity was 0.22 (95% CI 0.21–0.24) (Figure 7). The SROC curve is shown for BIG1 and BIG2 in Supplementary Figures 3 and 4; however, there was no SROC curve generated for BIG 3.

Figure 5. BIG 1 sensitivity plots for radiological deterioration.

Figure 6. BIG 2 sensitivity plots for radiological deterioration.

Figure 7. BIG 3 sensitivity plots for radiological deterioration.

Table 5. The table summarizes the bivariate model summary statistics for BIG score predicting radiological deterioration.

BIG 1
Parameter	Estimate	95% LCL	95% UCL
Sensitivity	0.98	1.00	1.00
Specificity	0.05	0.05	0.05
DOR	12.59	2.52	62.95
LR+	1.20	1.15	1.25
LR-	0.09	0.02	0.46
FPR	0.82	0.79	0.84
BIG 2
Parameter	Estimate	95% LCL	95% UCL
Sensitivity	0.09	0.05	0.13
Specificity	0.99	0.92	0.99
DOR	4.10	1.44	11.63
LR+	3.83	1.36	10.81
LR-	0.94	0.91	0.96
FPR	0.02	0.01	0.08
BIG 3
Parameter	Estimate	95% LCL	95% UCL
Sensitivity	0.22	0.21	0.24
Specificity	0.98	0.92	0.99
DOR	16.81	3.27	86.41
LR+	13.33	2.67	66.63
LR-	0.79	0.77	0.82
FPR	0.02	0.03	0.08

DOR = diagnostic odds ratio; LR+ = likelihood ratio for positive test results; LR- = likelihood ratio for negative test results; FPR = false positive rates.

Mortality prediction

Seven papers included reported death for any of the BIG groups, with only four of the studies looking at all three BIG categories. There was 1 death under BIG-1 (n = 1433), two deaths under BIG-2 (n = 2136) and forty-two deaths under BIG-3 (n = 3189) that were reported in the included studies.

Risk of bias

The risk of bias assessment is according to the Newcastle Ottawa Scale. Two studies scored a full 9, nine studies scored 8 out of 9 and two studies scored 7 out of 9.

Discussion

Summary of findings

This systematic review and meta-analysis is the first to synthesize the diagnostic accuracy of BIG scores for neurosurgical intervention. By assessing 13 studies, we identified a high sensitivity in predicting both the requirement for neurosurgery and radiological deterioration. The specificity was low due to the high number of false positives. Given sizable patient cohorts are still lacking in literature, we pooled together all studies that used BIG to ascertain the efficacy in the use of this guideline.

Comparison with literature

While the literature is limited on the efficacy on the use of BIG for predicting neurosurgical intervention for TBI. The Canadian CT Head Rule (CCHR) was developed as to help rule out the presence of intracranial injuries that would require neurosurgical intervention without the need for CT imaging (19). This is similar to the New Orleans Criteria (NOC), which is a tool used to determine the appropriateness of neuroimaging in the emergency department in patients with TBI.

Previous studies have attempted to establish the efficacy of these guidelines in the prediction for the need for neurosurgical intervention. A study by Bouida et al. (2013) found that the sensitivity and specificity for the need for neurosurgical intervention were 100% for the CCHR and 82% and 26% for the NOC. They found the NPV to be 100% and 99% and the PPV to be 5% and 2%, respectively, for the CCHR and NOC (20). Other studies have also found similar results with a sensitivity of 100%, with the specificity values varying between 48% and 80.7% for CCHR and 9.6% and 15.2% for the NOC (21–23). Another study by Gillespie et al. (2020) used radiological criteria to determine the need for neurosurgical intervention in mild TBI cases (5). The scoring system was based on purely radiological criteria, based on a radiologist/emergency medicine clinicians reporting. This criteria had a sensitivity of 99% and specificity of 51.9% for determining the need for surgical intervention.

While our study shows that the BIG 1 sensitivity is keeping with literature at an average of 99%, and the specificity is low at 9.4%, which is similar to NOC but much lower than the CCHR. The sensitivity for BIG 2 and BIG 3 are low at 7% and 12.5%, respectively. This is because BIG was developed as a screening tool to exclude patients who do not require neurosurgical intervention, rather than to identify those who need treatment.

This is likely due to various factors, such as the larger population of 9032 patients in the pooled analysis, with other studies varying between 368 and 1822 patients. The CCHR takes into account clinical factors such as episodes of vomiting, age, post-traumatic seizures and antiplatelet/anticoagulant medication. The BIG guidelines place more emphasis on type of injury such as subarachnoid hemorrhage, subdural hemorrhage alongside neurological examination and loss of consciousness (3), which are often managed conservatively, resulting in a lower specificity value.

Clinical and research implications

Our results have several implications for practice and research. The success rate on the use of BIG for predicting neurosurgery and radiological deterioration in patients are established in our results and could aid clinicians in the treatment of patients with TBI.

Though we highlight the efficacy of BIG compared with other guidelines, there are certain aspects that need to be addressed. The specificity of BIG is low compared with the other guidelines in place – the implications of a high sensitivity and low specificity in prediction of neurosurgery results in many patients who do not require neurosurgery being subject to further investigation that they do not require. This is especially true in the case of BIG-2 & 3 patients, with most studies not accurately predicting the need for neurosurgical intervention. It is important to consider the possibility of information bias when calculating specificity, as only 6/13 included studies had sufficient data for pooled analysis, which could explain the overall lower value.

The elevated sensitivity of the BIG enables its utilization as an early detection tool for identifying patients with TBIs characterized by mild or subtle presentations. This heightened sensitivity is instrumental in circumventing the risk of overlooking individuals who may require neurosurgical intervention, particularly in cases where clinical manifestations are less pronounced or initially overlooked. BIG could serve as a critical gatekeeper in ensuring that patients in need of specialized neurosurgical care are promptly identified and triaged accordingly. This preventive approach mitigates the likelihood of missed diagnoses and delays in intervention, thereby improving patient outcomes and prognoses.

The calculation of radiological deterioration seems to strongly agree with CCHR and NOC by having a similar sensitivity value, BIG 2 & 3 seems to be highly specific in identifying radiological deterioration. Studies have found a specificity ranging between 0.29% and 0.40%, which is considerably lower to what our study has shown (17,24). Further studies are needed to definitively answer the question on the overall efficacy of BIG, as both selection and observer bias may be prevalent, with many included studies in the review only looking at BIG-1 patients.

Limitations

This study has several limitations. Firstly, all studies included were retrospective, precluding pooled analysis of prospective studies. Additionally, although 13 studies were included in the review, a significant number of studies did not provide sufficient information to predict the use of BIG. The BIG scoring system is recent, and hence, there are very few clinical studies that have investigated its efficacy, unlike the CCHR and NOC. In addition, because adverse outcomes in BIG 1 and 2 class TBI are rare, larger numbers are needed to truly understand the safety of this approach in order to obtain sufficient event data. Furthermore, most of the included studies lack follow-up, with some being pilot studies (13,16). We also excluded full-text papers that are not available in English, restricting paper eligibility.

Conclusion

The BIG criteria are highly sensitive and could be used as an adjunct tool by neurosurgeons to determine the appropriate treatment for patients with TBI. Although the low specificity of the BIG may be recognized as a potential limitation or area for improvement, the strength of BIG lies in its high sensitivity, which enables the identification of most individuals with TBIs who may benefit from further evaluation or treatment. As a screening tool, the overarching goal of BIG is to ensure that no patients in need of neurosurgical intervention are inadvertently missed, thereby prioritizing patient safety and timely management of TBIs. BIG remains a valuable tool for clinicians in identifying individuals at risk of neurosurgical complications following TBI, ultimately optimizing patient care and outcomes.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

All data generated or analyzed during this study are included in this published article.

Supplemental data

Supplemental data for this article can be accessed online at https://doi.org/10.1080/02699052.2024.2375593

Additional information

Funding

The author(s) reported that there is no funding associated with the work featured in this article.