Diagnostic accuracy of routine liver function tests to identify patients with significant and advanced alcohol-related liver fibrosis

Abstract

Aims

Alcohol is the leading cause of cirrhosis, but most patients go undetected until decompensation occurs despite frequent contacts with the healthcare system. We aimed to evaluate the diagnostic accuracy of routine liver function tests compared with indirect and direct fibrosis markers and to assess doctors’ abilities to diagnose significant and advanced alcohol-related liver fibrosis.

Methods

This study was a retrospective evaluation of liver function tests for diagnosing alcohol-related liver disease compared to indirect fibrosis tests, the ELF test, and transient elastography. We also surveyed nine doctors who were presented with 225 patient cases from a cross-sectional, biopsy-controlled, single-centre study that evaluated diagnostic tools for alcohol-related liver fibrosis. The doctors assessed each case for significant (≥F2) or advanced (≥F3) fibrosis. We assessed inter-rater variability with Fleiss’ kappa.

Results

Routine liver function tests had poor diagnostic accuracy (highest area under the ROC curve for platelet count = 0.752) and poor sensitivities (10%–67%) when using the upper or lower normal limits as cut-offs. Indirect fibrosis indices performed significantly better but were still inferior to the ELF test and transient elastography. The nine doctors disagreed substantially in their predictions, with Fleiss’ kappa of 0.24 (95% CI0.22–0.26) and 0.51 (0.44–0.55) for significant and advanced fibrosis. All nine doctors exhibited poor case-finding abilities with sensitivities of 22–93%.

Conclusions

When using routine liver function tests, doctors may fail to diagnose more than half of all alcohol-overusing patients with advanced fibrosis, probably because they rely on upper and lower normal limits of routine liver function tests.

Keywords:

• Alcohol-related liver disease
• diagnostic accuracy
• cirrhosis
• family doctors
• fibrosis
• general practice

Introduction

Excess use of alcohol is a growing global problem, with a doubling in alcohol-related deaths in the United States over the past two decades [1]. An estimated 5–10% of patients with alcohol overuse develop severe alcohol-related liver fibrosis, making alcohol the main reason for advanced liver disease [2]. The average severe alcohol-related liver disease (ALD) patient has been in contact with the healthcare system for alcohol-related problems at the time up to diagnosis [3,4]. This provides ample opportunities for earlier disease detection during the up to 15 years of asymptomatic disease progression [3,5].

Family doctors constitute the gatekeepers for most patients with ALD [6]. As reliance on risk factors such as alcohol history and diabetes alone will significantly underestimate the prevalence of liver cirrhosis [7], family doctors rely heavily on a combination of routine liver function tests and alcohol history for ALD detection. An average family doctor does 45 lab tests per 10 patient-years, with routine liver function tests being the third most common test [8]. However, 75% of patients with alcohol-related cirrhosis are diagnosed at the time of decompensation, when survival is poor and abstinence has a little effect [9,10]. There is consequently the unexplored potential for early disease detection in primary care, paralleled by a gap in knowledge on how accurate primary care doctors assess fibrosis.

Prior studies in patients of mixed aetiologies have shown that reliance on abnormal liver function tests will miss most patients with asymptomatic cirrhosis, and that a combination of simple laboratory tests is more valuable than individual tests or clinical acumen [11–13]. However, simple laboratory indices such as Fib-4 are rarely directly available in primary care. Real-life studies suggest that elastography and the enhanced liver fibrosis test (ELF) can optimise referral pathways to improve the detection of early-stage cirrhosis patients in primary care [14–16], but these tests are rarely available outside specialist hepatology settings [17].

We aimed first to evaluate the diagnostic accuracy of individual routine liver function tests using the upper and lower normal limits as cut-offs and comparing them to direct and indirect fibrosis markers. To evaluate which clinical parameters and routine liver function test doctors rely on when making a diagnostic assessment, we then estimated the diagnostic accuracy and interobserver variance for detection of significant and advanced alcohol-related fibrosis among five family doctors and for hepatology specialists.

Methods

This study comprised a retrospective evaluation of routine liver function tests for diagnosing ASD and a survey of nine doctors who were presented with patient cases from a cross-sectional, biopsy-controlled single-centre study evaluating diagnostic tools for alcohol-related liver fibrosis (ethical approval ID S-20120071, S-20160021) [17,18]. Prior to study inclusion, all patients consented to participate after receiving oral and written information.

Biopsy-controlled cohort

The cohort comprised patients aged 30–75 years recruited from three hospital outpatient clinics and two municipal alcohol rehabilitation centres in the Region of Southern Denmark, described in detail elsewhere [17,18]. Patients were eligible for inclusion if they reported excessive use of alcohol (>21 units/week for men, >14 units/week for women) for at least one year. We excluded patients with competing liver disease, malignancy, severe comorbidity, and obvious cirrhosis making a liver biopsy redundant.

We performed all investigations on the same day in accordance with standard operating procedures. Investigations included medical and alcohol history, body mass index (BMI), blood pressure, routine liver function tests, indirect fibrosis tests (FIB-4, APRI, Forns, Fibrotest), the ELF test, and transient elastography (TE) with FibroScan 502 Touch (Echosense, France). Finally, we performed percutaneous, ultrasound-guided liver biopsy with a 17-gauge Menghini-needle in all patients, except for 15 patients with a liver stiffness below 6.0 kPa. These 15 patients were recruited after January 2016 when we changed our biopsy protocol due to data showing 100% NPV for advanced fibrosis in patients with TE < 6 kPa, making liver biopsy redundant in this setting. These 15 patients were labelled as not having advanced fibrosis but were removed from the analyses of significant fibrosis. An experienced pathologist staged histological fibrosis according to the Kleiner fibrosis score (F0–F4) [19].

Doctors’ questionnaire

We invited five family doctors as representatives of experienced doctors from primary healthcare centres at different geographical locations. They had minimum of 10 years of experience in primary care sees patients daily in 15-min consultations. We also invited four hepatologists from separate hospital departments of gastroenterology and hepatology; three senior hepatologists had worked in specialist hepatology units for over 10 years, and the junior hepatologist had three years of specialist training.

In October 2016, we designed a 225-page electronic questionnaire (one page for each patient) based on investigations from the biopsy-controlled study. We asked all nine above-mentioned doctors to complete the full questionnaire within one week. They did not have to complete the questionnaire in a single setting but could pause and resume at their convenience. Each page in the questionnaire contained information about one patient that would be routinely available in primary care:age, gender, BMI, blood pressure, routine blood tests; alanine transaminase, alkaline phosphatase, aspartate aminotransferase, albumin, bilirubin, gamma-glutamyltransferase, platelets, cholesterol, sodium, INR and full alcohol history. (Supplementary 1) The nine doctors were asked to decide whether they believed that the patients had 1) significant, or 2) advanced liver fibrosis. For the family doctors, we described significant fibrosis as ‘a substantial amount of fibrous tissue in the liver’ and advanced fibrosis as ‘cirrhosis or pre-cirrhosis’. The doctors knew that all patients were asymptomatic and had no previously known liver disease and that none had decompensated cirrhosis. The doctors were blinded to each other’s answers and the patients’ histological fibrosis stage.

Statistical analyses

We report data as means and standard deviations, medians and interquartile ranges (IQR), or counts and frequencies, depending on their distribution. We used upper or lower limits of normal for liver function tests as cut-offs. We assessed the diagnostic accuracy of continuous variables using the area under the receiver operating characteristics curve (AUC), with the DeLong test for AUC comparison. We evaluated the doctors’ discriminative ability to detect significant and advanced fibrosis by calculating the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). We report Fleiss’ Kappa for interobserver variance; this measures the agreement (0.0–1.0) between multiple raters for binary outcomes under the assumption that raters are chosen at random. A coefficient above 0.75 is considered good.

Finally, we explored which parameters the doctors relied on for diagnostic prediction by using multivariable logistic regression with backward elimination and including the variables in the doctors’ questionnaire. We considered a p-value below .05 as statistically significant. Statistical analyses were performed in Stata 16 (College Station, TX, US).

Results

Patient cohort

Of the 225 patients included from April 2013 to September 2016, 169 (75%) were male, 49 (22%) had advanced fibrosis (≥F3), and 112 (53%) had significant fibrosis (≥F2) (Table 1). The mean BMI was 27 ± 5 kg/m². The average patient had consumed an excess of alcohol for 11–20 years, with 116 (52%) still drinking at the time of inclusion. The patients with ongoing drinking consumed a median of 21 units (IQR 10–35) in the week leading up to inclusion (Table 1).

Table 1. Patient characteristics (n = 225).

	Median (p25–p75)
Gender, male (%)	169 (75%)
Age, years	56 (51–64)
Body mass index, kg/m^2	27 (23–31)
Abstinence, yes (%)	109 (48%)
Alanine transaminase, U/L	39 (21–47)
Alkaline phosphatase, U/L	105 (71–121)
Aspartate aminotransferase, U/L	48 (25–52)
Albumin, g/L	40 (39–44)
Bilirubin, µmol/L	12 (7–14)
Gamma glutamyltransferase, U/L	203 (38–215)
Platelets, U/L	236 (189–293)
Cholesterol, total mmol/L	5.0 (4.3–6.0)
Sodium, mmol/L	139 (138–141)
INR	1.0 (0.9–1.1)
APRI score	0.7 (0.2–0.7)
FIB-4	2.5 (1.0–2.8)
Forns index	5.5 (4.0–6.8)
Fibro test	0.35 (0.11–0.58)
ELF test	9.7 (8.7–10.5)
Transient elastography, kPa	14.5 (5.1–11.8)
Kleiner fibrosis stage (0–1/2/3/4)	113/63/16/33

INR: international normalized ratio; APRI score: AST to platelet ratio index, FIB-4: fibrosis-4 score; ELF test: enhanced liver fibrosis test. All data rounded to one decimal.

Diagnostic performance of individual liver function tests compared to direct and indirect fibrosis markers

The routine liver function tests with the best diagnostic performance for advanced fibrosis was INR (upper limit of normal cut point: 1.2) with an AUC of 0.78, however a sensitivity of 35% only. The routine liver function tests with the poorest performance were ALT (upper limits of normal cut point: 70 U/L for male, 45 U/L for female) with a sensitivity of 10% and a specificity of 86% (AUC of 0.48) (Figure 1).

Figure 1. Diagnostic performance of the routine liver function tests and non-invasive markers shown in receiver operating characteristics curves. AUROC: area under receiver operating curves; ALT: alanine transaminase; AST: aspartate aminotransferase; GGT: gamma glutamyltransferase; INR: international normalized ratio; APRI: aspartate to platelet ratio index; FIB-4: fibrosis-4 score; ELF test: enhanced liver fibrosis test.

Other algorithms, such as combined liver function tests, performed more accurately compared to stand-alone tests in advanced fibrosis (≥F3). In this population, both FIB-4 (cutpoint: 3.25) and the Forns index (cutpoint: 6.8) had AUCs of 0.82. Other tools of screening for liver fibrosis included transient elastography and the ELF test. Our data showed transient elastography had an AUC of 0.96 and a sensitivity of 100% when using 8 kPa as a cut-off. The ELF test (cutpoint: 10.5) had an AUC of 0.91 and a sensitivity of 75% (Figure 1).

For significant fibrosis (≥F2), the only combined liver function tests that outperformed the single liver function tests was Fibrotest (cutpoint: 0.58), which correctly classified 67%. Forns index (cutpoint: 6.8) correctly classified the same number of patients as GGT. As in advanced fibrosis, transient elastography and the ELF test both outperformed the routine liver function tests and combined scoring systems.

Doctors disagree substantially when diagnosing fibrosis

The nine doctors used different strategies to diagnose the patient cases described. Their overall diagnostic performance was moderate, with patients being correctly classified in 29–85% of cases. The family doctors had in general a high specificity of 75–95% for advanced fibrosis, thus ruling out disease at a high level, while the hepatologist, in general, had a higher number of correctly classified 88–93%, reflecting the ability better diagnose disease (Table 2, Figures 2 and 3).

Figure 2. Results of doctors, laboratory tests, and indirect and direct fibrosis markers as diagnostic test for significant fibrosis shown in four forest plots. N = 210, since 15 patients were not biopsied due to change in protocol, because of data showing NPV of 100% when TE below 6 kPa. TP: true positive; FP: false positive; FN: false negative; TN: true negative; ALT: alanine transaminase (cut-off: 45 as ULN for female, 70 as ULN for male); AST: aspartate aminotransferase (cut-off: 45 as ULN for female, 100 as ULN for male); GGT: gamma glutamyltransferase (cut-off: 75 as ULN for female, 115 as ULN for male); INR: international normalized ratio (cut-off: 1.1 as ULN for both female and male); APRI: aspartate to platelet ratio index (cut-off: 0.7 as ULN for both female and male); FIB-4: fibrosis-4 score (cut-off: 3.25 as ULN for both female and male); ELF test: enhanced liver fibrosis test; (cut-off: 7.7 ULN for significant fibrosis for both female and male).

Figure 3. Results of doctors, laboratory tests, and indirect and direct fibrosis markers as diagnostic test for significant fibrosis shown in four forest plots. N = 225. TP: true positive; FP: false positive; FN: false negative; TN: true negative; ALT: alanine transaminase (cut-off: 45 as ULN for female, 70 as ULN for male); AST: aspartate aminotransferase (cut-off: 45 as ULN for female, 100 as ULN for male); GGT: gamma glutamyltransferase (cut-off: 75 as ULN for female, 115 as ULN for male); INR: international normalized ratio (cut-off: 1.1 as ULN for both female and male); APRI: aspartate to platelet ratio index (cut-off: 1.0 as ULN for both female and male); FIB-4: fibrosis-4 score (cut-off: 3.25 as ULN for both female and male); ELF test: enhanced liver fibrosis test; (cut-off: 10.5 ULN for advanced fibrosis for both female and male).

Table 2. Diagnostic accuracy of the nine doctors.

	Significant fibrosis (≥F2)			Severe fibrosis (≥F3)
	Sensitivity	Specificity	Correctly classified (TP + TN)	Sensitivity	Specificity	Correctly classified (TP + TN)
Family doctor 1	93% (86–97)	12% [7–20]	55% (104 + 12)	61% (46–75)	75% (68–81)	72% (30 + 132)
Family doctor 2	60% (50–69)	69% (59–78)	64% (67 + 67)	45% (31–60)	95% (91–98)	84% (22 + 168)
Family doctor 3	58% (46–69)	56% (43–68)	57% (45 + 35)	25% (12–42)	94% (88–98)	48% (9 + 99)
Family doctor 4	76% (65–85)	79% (67–89)	77% (59 + 50)	69% (52–84)	85% (76–91)	51% (25 + 89)
Family doctor 5	72% (61–81)	75% (62–85)	73% (56 + 47)	64% (46–79)	75% (66–83)	45% (23 + 79)
Hepatologist 1	66% (57–75)	74% (65–83)	70% (74 + 73)	51% (36–66)	92% (87–96)	83 % (25 + 162)
Hepatologist 2	47% (38–57)	91% (83–96)	68% (53 + 89)	63% (48–77)	93% (88–96)	87% (31 + 164)
Hepatologist 3	22% (13–33)	86% (75–93)	50% (17 + 54)	64% (46–79)	90% (82–95)	83% (23 + 94)
Hepatologist 4	69% (58–79)	67% (54–78)	68% (54 + 42)	58% (41–75)	88% (80–93)	80% (21 + 92)

95% confidence intervals in parenthesis for sensitivity and specificity. Actual numbers in parenthesis for the correctly classified. TP: true positives; TN: true negatives.

We applied Fleiss’ kappa to compare the nine doctors as this is more accurate than other kappa assessments when comparing multiple assessors with a categorical outcome. Normally, a statistical approach would suggest at least 50% of the doctors would agree on a binary outcome. Our data showed a Fleiss’ kappa value of 0.50 (95% CI 0.44–0.55) in advanced fibrosis (≥F3), which equals moderate agreement amongst all nine doctors. For significant fibrosis (≥F2), we found a Fleiss’ kappa value of 0.24 (95% CI 0.22–0.26). If looking at the doctors in separate groups according to specialty, the family doctors had a kappa value of 0.16 and 0.57 for significant- and advanced fibrosis, respectively and the hepatologists had a kappa value of 0.33 and 0.46 for significant- and advanced fibrosis. If the doctors had equal answers in each case, the Fleiss’ kappa value would have been 1.00.

Doctors rely primarily on the platelet count when detecting advanced fibrosis

Of all the parameters given in the questionnaire, no variables were significant for all doctors. The Family doctors had either significantly relied on alkaline phosphatase (2 out of 5), or no variables. Family doctor 2 also relied significantly on albumin. All four hepatologists relied significantly on INR. Hepatologist 1 relied on bilirubin, GGT and INR. Both hepatologists 2 and 3 decisions were based on abstinence, alkaline phosphatase, and INR. Both had albumin borderline significant (p = .060). Hepatologists relied significantly on albumin and INR.

Discussion

In this study of 225 patients, we investigated the diagnostic accuracy of liver function tests to diagnose liver disease and combined this with a survey among four doctors using case presentations. We found that the general ability of liver function tests to diagnose liver fibrosis was poor. This suggests a high risk of under-referral to secondary care for patients at risk of liver fibrosis.

Previous studies have shown that most routine liver function tests perform poorly at diagnosing early stages of fibrosis [20,21], but their overall accuracy increases when monitoring end stages of ALD [11]. These studies raised concerns about the diagnostic tools currently available to family doctors and especially the challenges of diagnosing fibrosis without access to novel tools such as MRI, CT, TE or ultrasound [17,18,22].

Combined liver scoring systems

Previous authors have proposed a combination of liver function tests, such as the FIB-4, the Forns index, Fibrotest, and the APRI score, to reflect fibrosis in hepatitis C and B [23–25]. When these tests are applied in ALD patients, they have higher accuracy than routine liver function tests independently [11]. In our study, the best performing and available tests for doctors were the FIB-4 (0.82) and Forns index (0.82), but the ELF test (0.88) and Fibrotest (0.85) were significantly better at detecting both significant and advanced fibrosis. The ELF test has recently shown good diagnostic accuracy with an AUC of 0.91 [17,21]. However, the ELF test and the Fibrotest both combine several new biochemical markers that are currently unavailable to doctors in the primary care sector [26,27]. Furthermore, they are commercial markers and cost more than the routine liver function tests, making them less attractive for screening larger populations.

Diagnostic ability of the doctors

We found that the family doctors were poor at diagnosing both significant and advanced fibrosis and they disagreed substantially (kappa of 0.24 and 0.50) in their evaluation of the patient cases. If these results are typical of family doctors, they may help to explain the generally late diagnosis of cirrhosis and the diverse referral patterns from primary to secondary care.

The hepatologists were more accurate than the family doctors at diagnosing advanced fibrosis. Hepatologists are probably more aware of the evidence regarding liver function/synthesis from INR vs. inflammation/hepatocellular cell death from ALT. Although the APRI score, FIB-4, and Forns index were available to all doctors in this study, the specialists might be more familiar with them and therefore more likely to use them in predicting advanced fibrosis. The use of these scores appeared to provide a more accurate diagnosis.

A combination of variables makes the difference

Our data revealed that no variables were significantly used by all nine doctors, indicating that within this field there are no strong stand-alone markers for identifying liver fibrosis. However, all four hepatologists adjusted their answers according to the INR and often also albumin. They are variables that commonly decline during late-stage ALD and are used in prognostic scoring systems such as Child–Pugh score, and Meld–Na score. Several doctors used multiple more variables, this is neither a sign of poor education nor a lack of expertise, but more an indication of the challenges when combining several factors for diagnostics. This demonstrates the need for an easy-to-use scoring system that combines specific variables and results into a single value that can simplify the decision.

Our study evaluates the way doctors approach liver fibrosis in primary healthcare and hence the referral of patients to secondary healthcare. We found that routine liver function tests were inadequate, and their low accuracy means they do not provide sufficient help in the early detection of ALD. This could lead to under-referral to secondary healthcare and a large number of patients diagnosed late with a decompensating event. Conversely, it could also lead to over-referral family doctors are uncertain about how to interpret liver function test results and seek a second opinion from the specialist hepatologists. Because of poor diagnostic power in primary care, we end up having the wrong patients at the wrong time in the hospitals.

Strengths and limitations of this study

We included 225 patients from primary healthcare and alcohol rehabilitation centres who were considered representative of the patient population. As all patients were liver-biopsied, the true diagnosis was known to the study authors.

Limitations of this study are the small number of doctors and that the case representations did not present real-life patient/doctor consultations. This was a simulation in which the doctors received information in a questionnaire, and they did not see the patients in person. Further, the results of previous routine liver function tests were not available to the doctors. A family doctor would normally know their patients with abnormal liver function tests prior to the consultation and would have used additional information not included in the questionnaire.

Conclusion

Routine liver function tests do not provide adequate help for the diagnosis of significant or advanced alcohol-related liver fibrosis. The ability of family doctors to correctly diagnose liver fibrosis based on routine liver function tests is poor, and there is a high risk of under-referral to secondary care of patients at risk of liver fibrosis. For a more accurate diagnosis, it is possible to use currently available scoring systems such as the FIB-4 and Forns index, but these are not widely known among family doctors.

Author contributions

AK and MT conceptualized and designed the study. All authors conducted the study. KPL, TMH, and MT performed data analyses and drafted the paper. All authors edited the paper and approved the final manuscript. MT is guarantor.

Abbreviations
AAR	=	AST:ALT ratio
ALD	=	alcohol-related liver disease
ALK	=	alkaline phosphatase
ALT	=	alanine transaminase
APRI	=	AST-platelet ratio index
AST	=	aspartate transaminase
AUCROC	=	area under the receiver operating characteristics curve
BMI	=	body mass index
ELF	=	enhanced liver fibrosis test
FIB-4	=	fibrosis-4 index
GGT	=	gamma-glutamyl transferase
INR	=	international normalized ratio
IQR	=	interquartile range
NPV	=	negative predictive value
PPV	=	positive predictive value
TE	=	transient elastography

Acknowledgements

We thank Dr. A.S. Teisner, Dr. M.M. Lauridsen, Dr. J. Søndergaard, Dr. M. Svendsen, Dr. L. Søndergaard, Dr. P.A. Stenbøg, Dr. M.L. Andersen and Dr. T.R. Rudbaek for their expert contribution to the study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The study was funded by Innovation Fund Denmark, the European Union’s Horizon 2020 Research and Innovation Program [grant agreement number 668031], the Challenge Grant “MicrobLiver” [NNF15OC0016692] from the Novo Nordisk Foundation, University of Southern Denmark’s PhD stipends, and Region of Southern Denmark’s postdoc stipends. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.