https://orcid.org/0000-0003-1654-6844
Abstract
Background
The diagnostic value of C-reactive protein (CRP) for appendicitis in children has not been evaluated in primary care. As biochemical responses and differential diagnoses vary with age, separate evaluation in children and adults is needed.
Objectives
To determine whether adding CRP to symptoms and signs improves the diagnosis of appendicitis in children with acute abdominal pain in primary care.
Methods
A retrospective cohort study in Dutch general practice. Data was collected from the Integrated Primary Care Information database between 2010 and 2016. We included children aged 4–18 years, with no history of appendicitis, presenting with acute abdominal pain, and having a CRP test. Initial CRP levels were related to the specialist’s diagnosis of appendicitis, and the test’s characteristics were calculated for multiple cut-offs. The value of adding CRP to signs and symptoms was analysed by logistic regression.
Results
We identified 1076 eligible children, among whom 203 were referred for specialist evaluation and 70 had appendicitis. The sensitivity and specificity of a CRP cut-off ≥10 mg/L were 0.87 (95%CI, 0.77–0.94) and 0.77 (95%CI, 0.74–0.79), respectively. When symptoms lasted > 48 h, this sensitivity increased to 1.00. Positive predictive values for CRP alone were low (0.18–0.38) for all cut-off values (6–100 mg/L). Adding CRP increased the area under the curve from 0.82 (95%CI, 0.78–0.87) to 0.88 (95%CI, 0.84–0.91), and decision curve analysis confirmed that its addition provided the highest net benefit.
Conclusion
CRP adds value to history and physical examination when diagnosing appendicitis in children presenting acute abdominal pain in primary care. Appendicitis is least likely if the CRP value is < 10 mg/L and symptoms have been present for > 48 h.
KEY MESSAGES
Testing CRP in children presenting with acute abdominal pain in primary care adds value to clinical features of appendicitis.
Appendicitis is unlikely if the CRP value is < 10 mg/L and symptom duration is > 48 h.
When suspicion of appendicitis is low, a CRP test is not clinically relevant.
Introduction
Acute abdominal pain is a common symptom reported in 9% of consultations with children in primary care [Citation1]. Although appendicitis is rare in these children (<5%) and may even resolve spontaneously, it can progress to perforation and death if undiagnosed [Citation2–4]. It remains a diagnostic challenge for general practitioners (GPs) to differentiate appendicitis from common self-limiting or functional abdominal conditions that present similarly [Citation5]. This is compounded by the difficult trade-off between trying to avoid unnecessary investigation and referral for abdominal surgery and not missing a case of appendicitis. Thus, a simple and readily available test could help GPs reduce doubt.
C-reactive protein (CRP) levels increase rapidly during acute inflammation [Citation6]. In specialist care, CRP is of moderate diagnostic value for appendicitis, having a sensitivity of 0.62–0.85 and a specificity of 0.59–0.94 at a ≥ 10 mg/L cut-off value [Citation7–10]. As a readily available point-of-care test (POCT), CRP is often used by GPs, including for children with acute abdominal pain [Citation11]. However, the diagnostic accuracy of CRP has not been determined in primary care and we cannot generalise from the results for specialist care because of differences in patient spectrum, i.e. disease prevalence, severity, and distribution [Citation2,Citation12].
We aimed to determine the diagnostic characteristics of CRP testing for appendicitis in primary care and to assess the value of adding CRP to basic clinical assessment.
Methods
Design
In this retrospective cohort study, we included children with acute abdominal pain who underwent CRP tests ordered by a GP between November 2010 and November 2016. Data was sourced from the Dutch Integrated Primary Care Information (IPCI) database, which contains pseudonymised medical records for 1.5 million patients from 600 practices across the Netherland and has been used extensively for research [Citation13]. We used data from three of the six software platforms within the IPCI database for this study. Data from the other three software platforms had already been used in another research project about the management of acute abdominal pain because these contained extra secondary care information, which was not evaluated in the present study [Citation14].
Study population
The International Classification for Primary Care (ICPC) is used for diagnostic coding in Dutch primary care. We manually reviewed the first patient contact in the study period that met all of the following criteria at the time of contact: the patient received a gastrointestinal diagnosis (ICPC codes D01 through D99); abdominal pain was mentioned in the free text record; the patient had been registered in that practice for at least 12 months; the patient was aged 4–18 years; and the GP obtained a CRP. We subsequently reviewed the identified contacts and retained only patients presenting with recent acute abdominal pain (i.e. the presenting symptoms started ≤1 week before the consultation). Patients with a history of appendicitis or appendectomy were removed.
CRP test and clinical features
CRP levels were extracted automatically from laboratory results or manually from free-text entries. Data for age, gender, and body temperature were extracted automatically, while data for another 18 clinical features (symptoms and signs) described in seven clinical prediction rules were extracted by manual review [Citation15]. Nausea and vomiting were combined into one variable consistent with most prediction rules. Elevated temperature and temperature ≥ 37.3 °C were combined into one variable according to the Alvarado score [Citation16]. Based on a Dutch guideline, rebound tenderness, guarding, rigidity, and pain at jarring motions were combined as ‘peritoneal irritation’ [Citation2,Citation8]. Coders determined whether each clinical feature was present, absent, or not recorded (Supplementary Table 1). If in doubt, the coders discussed with an experienced GP (CGHB) or within the research team when doubt remained until consensus was reached.
Appendicitis
The outcome of interest was appendicitis diagnosed by a medical specialist within six weeks after the initial consultation. The absence of appendicitis was based on either the secondary care specialist report or the GPs medical records during this period. When coders were in doubt, an expert panel of two experienced GP’s (MYB, CGHB) verified the diagnosis based on free text records and letters from the medical specialist.
Statistical analysis
Diagnostic test characteristics of CRP
We calculated the following diagnostic characteristics for CRP testing with their 95% confidence intervals (95%CIs): sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, positive predictive value, and negative predictive value. We determined that a sample of 1397 children with acute abdominal pain was needed to include 61 children with appendicitis and allow us to calculate sensitivity with sufficient precision (half-width of the 95%CI is 0.1). This was based on the reported 4.4% prevalence of appendicitis in primary care2 and an expected sensitivity of 0.80 using the typical 10 mg/L cut-off [Citation7,Citation17]. Subgroup analyses were performed by gender, age (4–8, 9–12, and 13–18 years), and symptom duration (<24, 24–28, and >48 h) [Citation18]. We also calculated the test characteristics for CRP cut-off levels of ≥6, ≥20, ≥30, ≥40, ≥50, ≥80 and ≥100 mg/L. However, we did not consider a cut-off level of ≥5 mg to be informative because POCT values <5 mg/L are recorded as 5 mg/L in the Dutch Integrated Primary Care Information database.
Added value above symptoms and signs
We analysed the value of adding CRP to clinical features by comparing a logistic regression model that contained clinical features only (basic model) with one that added CRP to the basic model. The dependent variable in both models was appendicitis. Predictors recorded in > 50% of children were entered into the basic model without further selection as all predictors were based on literature and clinical practice [Citation19,Citation20]. To evaluate the performance of both models and hence the added value of CRP, we compared the areas under the curves (AUCs) and used decision curve analysis. Net benefits were calculated for a range of thresholds, with an upper limit of 0.40 [Citation21].
Missing values
There was no missing demographic, CRP, or diagnostic data but there were missing values for some clinical predictors. We assessed the missing data mechanisms and patterns to exclude missing not at random [Citation22], and if excluded, used multiple imputation (predictive mean matching, 10 iterations) with all clinical features, referrals, and outcomes to predict the missing data and construct 20 data sets. Rubin’s rule was used to calculate pooled AUCs with 95%Cis [Citation23], and DeLong’s method was used to test the difference between models in each imputed dataset [Citation23]. If the decision curves were similar for the 20 imputed datasets, we presented one randomly selected figure. A sensitivity analysis was performed by comparing the AUC of both models with complete case analysis only and a zero imputation analysis dataset in which missing values were replaced with zero (i.e. the assumption that the missing clinical predictor was absent). STATA/SE 16.1 (Stata Corp, USA) was used to calculate CRP-test characteristics, compare AUCs and construct decision curves. IBM SPSS version 26.0 (IBM Corp, Armonk, NY, USA) was used for all other analyses.
Results
Study population and diagnosis
We identified 2741 children for manual review. Among these, 1076 had presented for the first time with acute abdominal pain and had CRP levels measured, with 70 (6.5%) having appendicitis (13 had a perforated appendix). The prevalence of appendicitis was 10.2% (95%CI, 7.8%–13.4%) in boys and 3.7% (95%CI, 2.5%–5.5%) in girls (). Other emergencies were detected, including one case each of hydronephrosis, intussusception, abdominal lymphoma, and pneumonia. In total, 265 of the 1076 children (25%) had no missing predictors, and except for having a greater prevalence of appendicitis, these were comparable to patients with missing values (Supplementary Table 2).
Table 1. Patient characteristics clinical features and CRP by diagnosis.
CRP
The median CRP value was higher in children with than without appendicitis at 42 mg/L (IQR: 18–83) and 5 (IQR:2 − 8 mg/L, respectively (). All children with a perforation had CRP levels > 20 mg/L (median 96 mg/L). At a cut-off of ≥ 10 mg/L, the sensitivity for appendicitis was 0.87 (95%CI, 0.77–0.94) and the specificity was 0.77 (95%CI, 0.74–0.79). The positive predictive value for CRP alone ranged from 0.18 (95%CI, 0.14–0.22) to 0.38 (95%CI, 0.22–0.56) at the minimum and maximum cut-off values of 6 mg/L and 100 mg/L, respectively (). When symptoms had lasted < 24 h, the sensitivity of CRP was only 0.67 (95%CI: 0.41–0.87) but when they had lasted > 48 h, it increased to 1.00 (95%CI, 0.89–1.00); by contrast, the specificity remained stable at 0.77 (0.71–0.82) and 0.75 (0.71–0.79), respectively. The diagnostic test characteristics were similar between gender and age groups (Supplementary Table 3).
Table 2. Test characteristics of CRP for appendicitis in children with acute abdominal pain at different cut-off levels.
Value of adding CRP to clinical features
The following predictors were recorded in > 50% of the participating children and were included in the basic model: pain duration (76%), elevated temperature (68%), peritoneal irritation (67%), right lower quadrant tenderness (65%), bowel sounds (54%), and nausea/vomiting (53%) (Supplementary Table 4). When all predictors of the basic model were negative, the predicted risk of appendicitis was 0.002; adding CRP in this context increased the risk of appendicitis to 0.013 for a CRP value of 100 mg/L. Notably, adding CRP to the basic model increased the AUC significantly from 0.82 (95%CI, 0.78–0.87) to 0.88 (95%CI, 0.84–0.91) (). In the sensitivity analyses, the AUC still increased significantly from the basic model to the basic plus CRP model, as follows: from 0.82 (95%CI, 0.72–0.91) to 0.89 (95%CI, 0.82–0.97) using complete case analysis (n = 219) and from 0.84 (95%CI, 0.80–0.89) to 0.89 (95%CI, 0.86–0.93) using zero imputation. The decision curves indicated that the net benefit of the model with CRP added was higher than that for the basic model alone at each referral threshold ().
Figure 1. AUC curve of randomly chosen datasat # 14, comparing the AUC of the basic model and the model with CRP.
Figure 2. Decision curve showing the net benefit of referral based on the basic prediction model with and without CRP (imputed dataset #14). In 15 of the 20 imputed datasets the basic model with the addition of CRP had greater net benefit for all threshold probabilities compared with the basic model alone.
Discussion
Main findings
When differentiating appendicitis in children presenting with acute abdominal pain in primary care, a CRP cut-off at ≥ 10 mg/L had a sensitivity and specificity of 0.87 and 0.77, respectively. However, the sensitivity increased to 1.00 when symptoms had been present for > 48 h, with a negative test making appendicitis less likely. It was notable that all children with perforation had a CRP > 20 mg/L but the utility of this finding will need further investigation. Adding CRP to the basic clinical model increased the AUC from 0.82 to 0.88, with decision curves confirming the added benefit.
Comparison with existing literature
We found no other study looking at the diagnostic value of CRP for appendicitis in children with abdominal pain in primary care. Interestingly, registration data could be used because enough CRP tests were performed despite not being recommended in the Dutch guideline [Citation2]. At the ≥ 10 mg/L cut-off, although the sensitivity was higher than reported in specialist care (0.87 vs 0.62–0.85), the specificity was comparable (0.77 vs 0.59–0.94) [Citation7–10]. Sensitivity was also higher among children with symptoms for > 48 h, consistent with other research showing that the sensitivity and discrimination of CRP increased over the first few days [Citation24]. Consequently, a low CRP value should be interpreted with caution if symptoms have only developed recently.
The decision curve confirmed that adding a CRP test was beneficial across a range of clinically reasonable referral thresholds. Adding a CRP test may therefore improve decision-making for GPs who adopt both high (to avoid negative referrals) and lower (to avoid missing appendicitis) referral thresholds [Citation21]. Although no previous study has separately evaluated the value of adding CRP to other appendicitis features, we note that it was selected for use as a predictor in the Appendicitis Inflammatory Response score in secondary care based on logistic regression analysis [Citation25]. In that setting, CRP is tested routinely in cases of suspected appendicitis, with imaging recommended before deciding to perform appendectomy [Citation26]. The present study adds to the existing literature, demonstrating a clear benefit from adding CRP to signs and symptoms when predicting appendicitis in children in primary care.
Strengths and limitations
This study benefitted from including enough patients with appendicitis to calculate sensitivity with sufficient precision. A prospective cohort study would not have been feasible due to the low prevalence. Although we included fewer patients than required by our sample size calculation, the prevalence of appendicitis was higher than expected [Citation2], possibly because GPs used the CRP test when they had a higher suspicion of appendicitis. Nevertheless, these results are only applicable to children with acute abdominal pain in whom the GP considers ordering a CRP test. We also used ≥ 10 mg/L as the main cut-off level despite there being no consensus on the optimal cut-off in acutely ill children [Citation27]. Although adding CRP to the model resulted in a statistically significant increase in the AUC, this does not necessarily imply clinical relevance. Therefore, decision curve analysis enabled us to quantify the clinical benefit of adding CRP to the model [Citation28].
Using routine healthcare data introduced essential limitations [Citation20]. First, the clinicians who coded the final diagnosis were not blinded to the CRP values, potentially leading to information bias and diagnostic accuracy being overestimated. We used specialist reports to ascertain the final diagnosis, when available but had to rely on free text entries in some cases. Second, one predictor had 47% of its values missing, which we handled by multiple imputations. However, sensitivity analyses using zero imputation and complete cases produced similar improvements in the AUCs after adding CRP. Third, because we evaluated routine practice data, not all patients will have received the same reference standard, with diagnosis verified by operation, imaging, or observation in different cases. This may introduce differential verification bias or workup bias that affected the test characteristics if mild or spontaneously resolving cases of appendicitis were misclassified [Citation29]. However, given that these children do not need an operation, missing them has limited clinical impact. Fourth, it can be challenging to select the proper patient population retrospectively. As the database contained problem-oriented records, we were able to select children with acute abdominal pain. However, we only selected children with a CRP test which implies that the GP was unsure about the diagnosis and that a CRP-test was available. Although generalising the results to all children with acute abdominal pain would introduce selection bias, the results can be generalised to children in which the GP is in doubt whether or not to refer. In a previous cohort study, children with appendicitis were less likely to be tested for CRP than children with appendicitis [Citation14]. Furthermore, as all consultations took place from 2010 to 2016, CRP-testing may have become more available to the GP. However, the Dutch guideline still advises against CRP-testing for children with suspected appendicitis. Finally, we did not analyse the diagnostic value of CRP for outcomes other than appendicitis, so our conclusions are limited to appendicitis. However, the task of the GP is not to diagnose acute appendicitis but to differentiate between severe and not threatening symptoms and signs on presentation in primary care. Since only four children with another diagnosis that needed emergency care were present, analysis for the outcome emergency (including appendicitis) would have yielded similar results.
Implications for research and practice
CRP adds value to symptoms and signs alone and may improve decision-making by GPs, but it should only be ordered when indicated by clinical history and physical examination. Indeed, a CRP test in children with acute abdominal pain but no signs or symptoms of appendicitis is meaningless. The GP can use either a POCT or an external laboratory, with both yielding similar results [Citation30], though POCT is available more rapidly and can reduce the risk of perforation.
None of the cut-offs had optimal sensitivity or specificity for safely excluding appendicitis. Even at ≥ 10 mg/L, 13% of cases had missed appendicitis (though perforation was less likely). However, conservative management may be supported at this threshold, especially if symptoms have lasted > 48 h. If the child is sent home on this basis, GPs should offer clear safety-netting advice about the diagnostic uncertainty, alarm symptoms, and need to reassess. At a cut-off level of ≥ 100 mg/L, only 2% of children without appendicitis tested positive, indicating that at this level, or possibly lower, referral is highly indicated. It should be noted that the confidence intervals around the sensitivity were relatively wide, which means that there is uncertainty about how often acute appendicitis will be missed. However, the number of negative referrals is determined by the false positive rate (1-specificity), which was estimated with high precision.
Given that CRP added value to routine assessment, we must now consider how to include it in a clinical prediction rule. It is also unknown if testing with or without a clinical prediction rule affects GPs decisions and patient outcomes. Therefore, before recommending CRP in primary care, the impact of its use on patient outcomes should be evaluated in a randomised controlled trial.
Conclusion
In conclusion, adding CRP to symptoms and signs may help GPs decide whether to refer a child with suspected appendicitis to secondary care. Studies are needed to evaluate whether this test can improve decision-making in children with acute abdominal pain.
Ethical approval
This study was approved by the Medical Ethics Review Committee (METc) of the University Medical Centre Groningen (METc2017/541) and did not fall under the scope of the Dutch Medical Research Involving Human Subjects (WMO).
Supplementary Table 1
Download MS Word (26.5 KB)Supplementary material: Decision curve analysis
Download MS Word (12.8 KB)Supplementary Table 4
Download MS Word (16.9 KB)Supplementary Table 3
Download MS Word (16.4 KB)Supplementary Table 2
Download MS Word (15.1 KB)Acknowledgements
We thank Dr Robert Sykes (www.doctored.org.uk) for providing editorial services. We also thank IPCI for providing the data.
Disclosure statement
The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.
Correction Statement
This article was originally published with errors, which have now been corrected in the online version. Please see Correction (http://dx.doi.org/10.1080/13814788.2022.2155364)
References
- Loening-Baucke V, Swidsinski A. Constipation as cause of acute abdominal pain in children. J Pediatr. 2007;151(6):666–669.
- Eizenga WH, Gieteling MJ, Berger MY, et al. Summary of the NHG guideline “abdominal pain in children”, the 100th NHG guideline. Ned Tijdschr Geneeskd. 2013;157(15):A6191. Available from http://www.ncbi.nlm.nih.gov/pubmed/23575296
- Humes DJ, Simpson J. Acute appendicitis. BMJ. 2006;333(7567):530–534.
- Andersson RE. The natural history and traditional management of appendicitis revisited: Spontaneous resolution and predominance of prehospital perforations imply that a correct diagnosis is more important than an early diagnosis. World J Surg. 2007;31(1):86–92.
- Lewis SRR, Mahony PJ, Simpson J. Appendicitis. BMJ. 2011;343:d5976.
- Gabay C, Kushner I. Acute-phase proteins and other systemic responses to inflammation. N Engl J Med. 1999;340(6):448–454.
- Shogilev D, Duus N, Odom S, et al. Diagnosing appendicitis: evidence-based review of the diagnostic approach in 2014. West J Emerg Med. 2014;15(7):859–871.
- Andersson REB. Meta-analysis of the clinical and laboratory diagnosis of appendicitis. Br J Surg. 2004;91(1):28–37.
- Yu CW, Juan LI, Wu MH, et al. Systematic review and meta-analysis of the diagnostic accuracy of procalcitonin, C-reactive protein and white blood cell count for suspected acute appendicitis. Br J Surg. 2013;100(3):322–329.
- Lycopoulou L, Mamoulakis C, Hantzi E, et al. Serum amyloid a protein levels as a possible aid in the diagnosis of acute appendicitis in children. Clin Chem Lab Med. 2005;43(1):49–53.
- Howick J, Cals JWL, Jones C, et al. Current and future use of point-of-care tests in primary care: an international survey in Australia, Belgium, The Netherlands, the UK and the USA. BMJ Open. 2014;4(8):e005611.
- Usher-Smith JA, Sharp SJ, Griffin SJ. The spectrum effect in tests for risk prediction, screening, and diagnosis. Br Med J. 2016;353:i3139.
- Vlug AE, van der Lei J, Mosseveld BMT, et al. Postmarketing surveillance based on electronic patient records: the IPCI project. Methods Arch. 1999;38(4):339–344. Available from http://www.schattauer.de.eur.idm.oclc.org/t3page/1214.html?manuscript=63&L=1.
- Blok GCGH, Veenstra LMM, van der Lei J, et al. Appendicitis in children with acute abdominal pain in primary care, a retrospective cohort study. Fam Pract. 2021;38(6):758–765.
- Nepogodiev D, Wilkin RJ, Bradshaw CJ, et al. Appendicitis risk prediction models in children presenting with right iliac fossa pain (RIFT study): a prospective, multicentre validation study. Lancet Child Adolesc Heal. 2020;4(4):271–280.
- Alvarado A. A practical score for the early diagnosis of acute appendicitis. Ann Emerg Med. 1986;15(5):557–564. [pii]
- Buderer NM. Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specificity. Acad Emerg Med. 1996;3bu(9):895–900.
- Dado G, Anania G, Baccarani U, et al. Application of a clinical score for the diagnosis of acute appendicitis in childhood: a retrospective analysis of 197 patients. J Pediatr Surg. 2000;35(9):1320–1322.
- Steyerberg E. Clinical prediction models. New York: Springer-Verlag; 2009.
- Oostenbrink R, Moons KGM, Bleeker SE, et al. Diagnostic research on routine care data: prospects and problems. J Clin Epidemiol. 2003;56(6):501–506.
- Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565–574.
- Gomer B. MCAR, MAR, and MNAR values in the same dataset: a realistic evaluation of methods for handling missing data. Multivariate Behav Res. 2019;54(1):153.
- Marshall A, Altman DG, Holder RL, et al. Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Med Res Methodol. 2009;9:57.
- Wu H-P, Lin C-Y, Chang C-F, et al. Predictive value of C-reactive protein at different cutoff levels in acute appendicitis. Am J Emerg Med. 2005;23(4):449–453.
- Andersson M, Andersson RE. The appendicitis inflammatory response score: a tool for the diagnosis of acute appendicitis that outperforms the Alvarado score. World J Surg. 2008;32(8):1843–1849.
- Bakker OJ, Go P, Puylaert JBCM, et al. Werkgroep richtlijn Diagnostiek en behandeling van acute appendicitis. Guideline on diagnosis and treatment of acute appendicitis: imaging prior to appendectomy is recommended. Ned Tijdschr Geneeskd. 2010;154:A303. Available from http://www.ncbi.nlm.nih.gov/pubmed/21262032
- Verbakel JY, Lemiengre MB, De Burghgraeve T, et al. Should all acutely ill children in primary care be tested with point-of-care CRP: a cluster randomised trial. BMC Med. 2016;14(1):131.
- Vickers AJ, van Calster B, Steyerberg EW. A simple, step-by-step guide to interpreting decision curve analysis. Diagn Progn Res. 2019;3(1):18.
- De Groot JAH, Bossuyt PMM, Reitsma JB, et al. Verification problems in diagnostic accuracy studies: consequences and solutions. BMJ. 2011;343(7821):d4770.
- Seamark DA, Backhouse SN, Powell R. Field-testing and validation in a primary care setting of a point-of-care test for C-reactive protein. Ann Clin Biochem. 2003;40(Pt 2):178–180.