Development and validation of a risk prediction model for hospital admission in COVID-19 patients presenting to primary care

Figures & data

Figure 1. Flow chart of in- and exclusions.

Table 1a. Patient demographics and clinical features are used for the development (n = 4,806) and validation (n = 830) cohort separately.

	Development cohort	Temporal validation cohort
Setting and location
COVID-19 centre Western South Limburg	2022 (42.1%)	NA
COVID-19 centre Eastern South and Central Limburg	2006 (41.7%)	NA
COVID-19 centre North Limburg	778 (16.2%)	NA
Regular GP practices North Limburg	NA	713 (85.9%)
Out-of-hours centre North Limburg	NA	117 (14.0%)
Demographics
Age (in years)	56 (18; 40; 70; 102)	56 (18; 45; 67; 95)
Sex, female	2710 (56.4%)	430 (51.8%)
Lifestyle and pre-existing comorbidity
Overweight	214 (4.5%)	27 (3.3%)
Current smoker	342 (7.1%)	10 (1.2%)
Type I Diabetes	15 (0.3%)	3 (0.4%)
Type II Diabetes	309 (6.4%)	85 (10.2%)
Diabetes, type unknown	160 (3.3%)	14 (1.7%)
Insulin-dependent diabetes	193 (4%)	37 (4.5%)
Hypertension	1287 (26.8%)	248 (29.9%)
Cerebrovascular event	217 (4.5%)	16 (1.9%)
Heart disease	463 (9.6%)	44 (5.3%)
Cardiac arrhythmias	331 (6.9%)	21 (2.5%)
Any chronic pulmonary disease (incl. asthma and COPD)	1214 (25.3%)	143 (17.2%)
Asthma	533 (11.1%)	47 (5.7%)
COPD	534 (11.1%)	39 (4.7%)
Kidney disease	118 (2.5%)	10 (1.2%)
Current malignancy	81 (1.7%)	11 (1.3%)
Malignancy in past	210 (4.4%)	12 (1.4%)
Use of immunosuppressive agents	308 (6.4%)	37 (4.5%)
Chronic corticosteroid use	223 (4.6%)	47 (5.7%)
NSAID use	105 (2.2%)	37 (4.5%)
Anticoagulation	776 (16.1%)	153 (18.4%)
COVID-19 status
Proven COVID-19^a	122 (2.5%)	821 (98.9%)
Probable COVID-19^b	4568 (95%)	9 (1.1%)
Contact with proven COVID-19 case	313 (6.5%)	333 (40.1%)
Contact with probable COVID-19 case	236 (4.9%)	440 (53.1%)

Reported statistics are median (min, first quartile, third quartile, max) or n (%).

^aProven COVID-19: confirmed COVID-19 according to a PCR or a rapid antigen test. Test availability was limited during the first wave in the Netherlands. ^bProbable COVID-19: ICPC codes were R83.03 Sars-CoV-2 in addition to other ICPC codes that may indicate COVID-19 (see Table S1, Table S2, Table S3). In the second-wave validation cohort, patients were excluded if they did not have confirmed COVID-19 according to GP or hospital records.

Table 1b. Reported symptoms and vital signs were recorded during the first GP consultation, for the development (n = 4,806) and validation (n = 830) cohort separately.

	Development cohort	Temporal validation cohort
Symptoms reported during the first GP consultation
Duration of symptoms (in days)	7 (0; 2.5; 14; 90)	7 (0; 4; 9; 35)
Sudden deterioration of symptoms	1340 (27.9%)	97 (11.7%)
Cough	3064 (63.8%)	544 (65.5%)
Sputum production	819 (17%)	119 (14.3%)
Haemoptysis	100 (2.1%)	23 (2.8%)
Dyspnoea	3095 (64.4%)	469 (56.5%)
Chest pain or pressure	1538 (32%)	152 (18.3%)
Sore throat	846 (17.6%)	93 (11.2%)
Loss of smell	259 (5.4%)	88 (10.6%)
Rhinitis	596 (12.4%)	94 (11.3%)
Deterioration of chronic illness symptoms (COPD, asthma, chronic cough)	598 (12.4%)	63 (7.6%)
Myalgia	590 (12.3%)	205 (24.7%)
Fatigue	1376 (28.6%)	333 (40.1%)
Shivering	448 (9.3%)	74 (8.9%)
Headache	876 (18.2%)	163 (19.6%)
Diarrhoea	469 (9.8%)	89 (10.7%)
Other abdominal complaints	1272 (26.5%)	300 (36.1%)
Confusion	159 (3.3%)	17 (2%)
Falling incident	77 (1.6%)	25 (3%)
Vital signs measured at the first GP consultation
Auscultation abnormalities	1145 (23.8%)	218 (26.3%)
Auscultation abnormalities one-sided	495 (10.3%)	106 (12.8%)
Auscultation abnormalities two-sided	650 (13.5%)	112 (13.5%)
Crepitations	592 (12.3%)	158 (19%)
Respiratory rate	18 (5; 15; 24; 40)	20 (10; 16; 30; 40)
Diastolic blood pressure	80 (38; 71; 90; 140)	80 (40; 70; 84; 110)
Systolic blood pressure	132 (80; 120; 148; 220)	127 (80; 118; 140; 180)
Heart rate	86 (30; 76; 100; 194)	87 (52; 78; 99; 150)
Body temperature	37.1 (34; 36.7; 37.6; 40.9)	37.4 (35.2; 37; 38; 40.8)
Peripheral oxygen saturation	98 (40; 95; 98; 100)	97 (56; 94; 98; 100)

Reported statistics are median (min, first quartile, third quartile, max) or n (%).

Table 2. Outcomes for the development (n = 4,806) and validation (n = 830) cohort separately.

	Development cohort	Temporal validation cohort
Hospital admission within 14 days	292 (6.1%)	126 (15.2%)
Pronounced dead by GP within 14 days (before hospital admission)	23 (0.5%)	7 (0.8%)
Hospital admission at any time during follow-up^a	294 (6.1%)	126 (15.2%)
ICU admission (any time)^b	60 (1.2%)	24 (2.9%)
In-hospital mortality (any time)^c	43 (0.9%)	16 (1.9%)

Reported statistics are median (min, first quartile, third quartile, max) or n (%).

^aHospital follow-up was obtained 2 weeks after the last inclusion of patients visiting general practice, and hence varied between patients from 2 weeks to 3.5 months.^bAmong admitted patients, we do not have complete follow-up on ICU admission of 1 (0.02%) and 18 (2.2%) patients of the development and validation cohort, respectively, because they were transferred to another hospital before any ICU admission took place. Consequently, ICU admission frequency may be underestimated.^cAmong admitted patients, we do not have a complete follow-up on in-hospital mortality of 9 (0.2%) and 25 (3.0%) patients of the development and validation cohort, respectively, because they were transferred to another hospital (before or after any potential ICU admission). Consequently, in-hospital mortality frequency may be underestimated.

Table 3. Model performance (leave-region-out cross-validation and temporal validation).

	Leave-region-out cross-validation	Temporal validation
C-index^a	0.84 (95% CI 0.83 to 0.86) (95% PrI 0.78 to 0.89)	0.79 (95% CI 0.74 to 0.83)
Calibration intercept^b	−0.08 (95% CI −0.98 to 0.52) (95% PrI −3.34 to 3.18)	0.02 (95% CI −0.21 to 0.24)
Calibration slope^a	0.89 (95% CI 0.71 to 1.07) (95% PrI 0.35 to 1.42)	0.82 (95% CI 0.64 to 1.00)

^a: a value of 1 is perfect, ^b: a value of 0 is perfect. CI: confidence interval. PrI: prediction interval. The CI quantifies the precision of the average performance, while the PrI reflects the variance of the performance across regions in the leave-region-out cross-validation.

Figure 2. Flexible calibration curve in the temporal validation cohort (n = 830, 133 events) showing the predicted probability of hospital admission versus the observed proportion with hospital admission. The histogram on the horizontal axis shows the distribution of predicted risks among patients with (1, bars pointing upward) and without (0, bars pointing downward) hospital admission. Very short bars at predicted probabilities >0.5 indicate high predicted admission risks are rare.