Inter-reader agreement of interpretation of radiological course of bile duct changes between serial follow-up magnetic resonance imaging/3D magnetic resonance cholangiopancreatography of patients with primary sclerosing cholangitis

Abstract

Objectives: Interpretation of MRI/MRCP in primary sclerosing cholangitis (PSC) at a single time point has low inter-reader agreement. Agreement of interpretation of the dynamic course of duct changes in follow-up MRI/MRCP is of clinical importance but remains unknown. Our aims are therefore to assess the inter-reader agreement of interpretation of the course of duct changes in PSC and investigate if elimination of 3 D MRCP affects inter-reader agreement.

Materials and Methods: We studied 40 consecutive PSC-patients who underwent two liver MRI/MRCPs at two time points. Two readers independently evaluated the course of duct changes between the two time points in two imaging sets, one with and one without 3 D MRCP. The intraclass correlation coefficient (ICC) was calculated for evaluation of inter-reader and intra-reader agreement between the two time points and two imaging sets accordingly.

Results: Inter-reader agreement of the interpretation of the course of duct changes between the two time points was poor (ICC up to 0.224). Elimination of 3 D MRCP neither improved inter-reader agreement which was again poor (ICC up to 0.26) nor did it change considerably the way readers interpret the course of ducts changes (ICC for intra-reader agreement between 0.809 and 0.978).

Conclusions: Inter-reader agreement of the interpretation of radiological course of duct changes is poor in serial follow-up MRI/MRCP of PSC-patients. Elimination of 3 D MRCP does not increase inter-reader agreement but maintains an excellent intra-reader agreement for the interpretation of the dynamic course of bile duct changes.

Key points

• Inter-reader agreement of interpretation of radiological course of bile duct changes between serial follow-up MRI/MRCP examinations of patients with PSC is poor.

• Absence of 3D MRCP does not affect considerably the way readers interpret the radiological course of bile ducts changes.

• When MRCP is absent or of low quality, utilization of other sequences seems to be helpful as an alternative for bile duct evaluation.

Keywords:

• Magnetic resonance imaging
• cholangitis
• sclerosing
• bile ducts
• data visualization
• cholangiopancreatography
• reproducibility of results

Introduction

Primary sclerosing cholangitis (PSC) is a chronic, progressive inflammatory cholestatic liver disease of unknown origin leading to fibrosis and biliary cirrhosis. Patients with PSC are at high risk of developing hepatobiliary malignancies such as cholangiocarcinoma (CCA), hepatocellular carcinoma and gallbladder carcinoma [1–3].

Magnetic resonance imaging (MRI) with magnetic resonance cholangiopancreatography (MRCP) is generally accepted as the diagnostic test of choice for PSC. It has replaced endoscopic retrograde cholangiopancreatography (ERCP) due to known risks of complications of the latter such as cholangitis and pancreatitis [4,5]. At present, the role of imaging in surveillance of patients with PSC for early cancer detection is not well established, but cross-sectional imaging with ultrasound or MRI/MRCP every 6–12 months and serial CA 19-9 seems to be beneficial and is recommended by some experts in the field [6–8]. Early detection of CCA is of utmost clinical importance in the follow-up of patients with PSC. Dominant stricture which is defined as a stricture with a diameter of less than 1.5 mm in common bile duct (CBD) or less than 1 mm in the left or right hepatic duct and in the common hepatic duct (CHD) [9–11] is associated with a higher risk of CCA in patients with PSC [12,13]. A dominant stricture, however, is defined at ERCP and there is a lack of consensus regarding the applicability of this term at MRCP. Morphological/cholangiographic imaging findings in follow-up MRI/MRCP suggestive of CCA are development of new strictures and progression of length and/or grade of known strictures and/or upstream dilatation of the ducts.

A recent study performed by Zenouzi et al. [14] showed that inter-reader agreement between experienced clinicians of the interpretation of MRI/MRCP of PSC patients is low posing a challenge in utilization of MRI/MRCP as follow-up and/or surveillance method. In this study, the reviewers were asked to evaluate a short liver MRI protocol composed of a T2-weighted image of the liver and a 3 D MRCP. Although MRI/MRCPs were performed as part of a follow-up program, there was no comparison between serial MRI examinations but only interpretation of a single examination. In a follow-up setting, evaluation of the dynamic radiological course of bile duct changes in terms of progression of strictures and duct dilatation is motivated and adds clinically relevant information. Additionally, this study does not directly address whether the main reason for this low agreement is 3 D MRCP itself.

Therefore, the aim of this study is to evaluate the inter-reader agreement of the interpretation of radiological course of duct changes in terms of progression/regression of strictures and duct dilatation between two serial follow-up MRI/MRCP examinations of PSC patients. In addition, we investigate if elimination of 3 D MRCP from the imaging set affects inter-reader agreement.

Materials and methods

Patient population

For this regional ethics review board-approved retrospective study, we included 40 consecutive patients with a confirmed diagnosis of large duct PSC. The diagnosis of PSC was made based on established clinical guidelines, namely compatible clinical and biochemical findings, exclusion of secondary types of sclerosing cholangitis along with typical cholangiographic findings at MRCP or ERCP [15,16]. In our institution, we perform annual multiparametric liver MRI/MRCP and clinical and laboratory evaluation of patients with PSC as part of surveillance program for cancer. The standard liver MRI/MRCP protocol performed for this purpose, is in accordance with the recommendations made by the international PSC study group [17] and the consensus statement of ESGAR for imaging of CCA [18]. Patients in our study had undergone at least two liver MRI/MRCP examinations according to the standard protocol. The second MRI/MRCP was performed between January and June 2015 (time point 2) and the first MRI/MRCP was performed at any time prior (time point 1). The study group was composed of 29 male and 11 female patients with a median age of 47 years (range 23–79 years). The mean time between the two MRI examinations was 16 months (5.5–37 months). Basic demographic data are shown in Table 1.

Table 1. Demographic and laboratory data of study population (n = 40).

	Study population	Normal range (male/female)	N
Number of patients	40	–	40
Male/Female	29/11	–	40
Age	47 (23–79)	–	40
Time from diagnosis to second MRI (years)	6.8 (1–27)	–	28
Mean time between two MRIs (months)	16 (5.5–37)	–	40
Bilirubin (μmol/l)	12	<26	38
ASAT (U/L)	41.2	<44.7/<35.9	37
ALAT (U/L)	40.6	<70.6/<44.7	37
GT (U/L)	117.6	<82.3/<76.5	37
ALP (U/L)	123.5	<117.8	38
Albumin (gr/L)	38	36–48/36–35	37
INR	1	<1.2	35
Creatinine (μmol/L)	76	<100/<90	37
Number of patients undergoing ERCP between MRIs	6	–	–
Patients undergoing ERCP within 1 year after second MRI	7	–	–

All values expressed as medians unless stated otherwise.

All values are at the time of second MRI.

ALP: alkaline phosphatase; ALAT: alanine aminotransferase; ASAT: aspartate aminotransferase; GT: glutamyl transferase; INR: international normalized ratio.

Image acquisition

All MRI/MRCP examinations were performed using two different 1.5 T MRI scanners (Magnetom Aera and Magnetom Avanto, Siemens Healthineers, Erlangen, Germany) with 18-channel torso phased-array coil and fixed spine coil. For the purposes of the current study, we used only those sequences of the standard protocol that are important for the morphological evaluation of bile ducts (T2-weighted images, 3 D MRCP, T1-weighted images before and after contrast administration). Technical parameters of MRI sequences used in the study are presented in Table 2. Per oral contrast or spasmolytics were not used.

Table 2. Technical parameters of MRI sequences used in the study.

Sequence	Plane	Thickness (mm)	TE (ms)	TR (ms)	Scan time	Breathing technique
T2-w HASTE	Axial	4	124	1500	1–2 min	PACE
T2-w HASTE	Coronal	4	76	1080	1–2 min	PACE
T2-w 3D SPACE MRCP	Axial	2.5	685	4536	4–6 min	PACE
T2-w 3D SPACE MRCP	Coronal	1	685	3832	4–6 min	PACE
T1-w VIBE before contrast	Axial	2.5	1.92	4.29	17–22 s	BH
T1-w VIBE post contrast^a	Axial	2.5	1.92	4.29	20 min	BH

PACE: prospective acquisition correction navigator-triggered; BH: breath-hold.

^a 0.1ml/kg body weight of gadoxetic acid (Primovist, Bayer Healthcare); injection rate of 1 mL/sec followed by a bolus of 20 mL saline flush. Only portal venous phase, transitional phase (3 min) and hepatobiliary phase (20 min) was used for the study protocol.

Image evaluation

Two imaging sets per patient, one with and one without 3 D MRCP, were evaluated in two separate reading sessions in a random and independent fashion by two readers (A.G. and F.M. with 5 and 6 years of postresidency experience in abdominal radiology, respectively). Each imaging set included images from MRI/MRCP examinations from time points 1 and 2 and was assessed for the course of bile duct changes. A minimum two-week interval between readings was set to eliminate recall bias. Information on the diagnosis of PSC was available to the readers, but they were blinded to other patient data, such as clinical and laboratory findings. The imaging set with 3 D MRCP included: axial and coronal T2-weighted HASTE (Half-Fourier acquisition single-shot turbo spin echo), axial and coronal 3 D MRCP – including coronal maximum intensity projections (MIP), axial T1-weighted fat saturated pre- and dynamic post gadoxetic acid enhanced images in portal venous phase, transitional phase (3 min) and 20 min after contrast agent injection (Figure 1(a)). Imaging set without 3 D MRCP included all the above sequences apart from 3 D MRCP (Figure 1(b)). The readers were encouraged to apply minimum intensity projection (MinIP) on T1-weighted images in portal venous and transitional phase as an additional way of evaluating bile ducts. MinIP is a reconstruction algorithm that utilizes low signal intensity pixels from a data volume and generates two-dimensional images where these pixels are visualized. Readers were free to adjust window settings and slab thickness in 3 D MRCP and MinIP reconstructions in a commercially available PACS (picture archiving and communication system) workstation (SECTRA PACS, IDS 7, version 19.1, SECTRA AB, Linköping, Sweden).

Figure 1. Gadoxetic acid-enhanced liver MRI examination of a 65-year-old male patient with PSC. Sequences included in imaging set with MRCP is depicted in (a) and without MRCP in (b). Both readers evaluated each data set in two-time points simultaneously and at two separate reading sessions.

Bile ducts were categorized into three groups for the purposes of our study: 1. extrahepatic ducts extending from the ampulla of Vater to the primary biliary duct confluence (CHD and CBD), 2. left-sided ducts starting from the left main hepatic duct and extending to most peripheral bile ducts, and 3. right-sided ducts starting from the right main hepatic duct and extending to the most peripheral bile ducts. Groups 2 and 3 were considered intrahepatic ducts.

The two readers evaluated and registered the radiological course of strictures and dilatation of bile ducts in terms of progression, regression or stable disease between the two MRI examinations for imaging sets with and without 3 D MRCP separately using a 5-point scale (1 = obvious progression, 2 = subtle progression, 3 = unchanged, 4 = subtle regression, 5 = obvious regression). Progression was defined as presence of new stricture, visual progression of length and/or grade of stricture(s) and/or progression of dilatation of bile duct(s). Regression was defined as visual regression of length and/or grade of stricture(s) and/or regression of dilatation of bile duct(s). In case of mixed changes, that is, progression of dilatation but regression of stricture grade, status was registered as progression. If no discernible changes were noted, then the status was registered as unchanged.

Statistical analysis

The inter-reader agreement of the interpretation of radiological course of morphological duct changes between the two serial follow-up MRI examinations and the intra-reader agreement between the imaging sets with and without 3 D MRCP were analyzed according to the method presented and described by Shrout and Fleiss [19] which yields the intraclass correlation coefficient (ICC). ICC estimates and their 95% confidence intervals were calculated. Inter-reader agreement was calculated based on a two-way random effects model and intra-reader agreement based on a two-way mixed-effects model. Additionally, we calculated the absolute number of cases with agreement and disagreement between readers for each imaging set.

All analyses were carried out using two statistical packages, SPSS (IBM Corp., Released 2017, IBM SPSS Statistics for windows, version 25.0, Armonk, NY, USA) and STATA version 15.1 (STATA Corp LLC, Texas, USA).

Results

Patients in our study had elevated γ-GT and ALP (median =117.6 U/L and 123.5 U/L respectively). Six patients underwent ERCP between the two MRI examinations and seven of them within one year after the second MRI.

Reader A and B interpreted the course of duct changes between the two serial follow-up MRI/MRCP using the imaging set with 3 D MRCP as progression in CBD/CHD, right-sided bile ducts and left-sided bile ducts in 3/9, 8/8 and 13/8 patients, respectively, and as regression in 7/3, 5/2 and 2/2, respectively. Results of interpretation of course of duct changes between the two MRI/MRCP examinations for each reader, all bile duct groups and for imaging sets with and without 3 D MRCP are presented in Tables 3 and 4.

Table 3. Interpretation of radiological course of morphological changes of common bile duct and common hepatic duct (CBD/CHD), right-sided (RIHBD) and left-sided intrahepatic bile ducts (LIHBD) between two serial follow-up MRI examinations for imaging set with MRCP for reader A/B.

	Progression Reader A/B		Regression Reader A/B		Unchanged
	Obvious	Subtle	Obvious	Subtle
CBD and CHD	0/1	3/8	2/0	5/3	30/28
RIHBD	1/1	7/7	2/0	3/2	27/30
LIHBD	3/0	10/8	1/0	1/2	25/30

Table 4. Interpretation of radiological course of morphological changes of common bile duct and common hepatic duct (CBD/CHD), right-sided (RIHBD) and left-sided intrahepatic bile ducts (LIHBD) between two serial follow-up MRI examinations for imaging set without MRCP for reader A/B.

	Progression Reader A/B		Regression Reader A/B		Unchanged
	Obvious	Subtle	Obvious	Subtle
CBD and CHD	2/1	0/6	1/0	2/3	35/30
RIHBD	2/1	2/6	2/0	3/2	31/31
LIHBD	6/0	2/6	2/0	2/2	28/32

Poor inter-reader agreement of the radiological course of morphological bile ducts changes between the two MRI examinations in time points 1 and 2 was found for imaging set with 3 D MRCP with ICC values ranging between −0.449 and 0.224 (Table 5). Removing 3 D MRCP from the imaging set did not change how readers interpret the course of bile duct changes and intra-reader agreement between the two sets was almost perfect for both readers (Table 6). ICC was 0.815–0.962 for intrahepatic ducts and 0.809–0.978 for extrahepatic ducts, indicating strong agreement for both bile duct groups. Elimination of 3 D MRCP from the provided sequences did not change the grade of inter-reader agreement which was again poor. ICC values for inter-reader agreement with imaging set without MRCP ranged between 0.018 and 0.263 (Table 5). Absolute number of cases with agreement and disagreement between readers for each imaging set are presented in Table 7.

Table 5. Inter-reader agreement (ICC) between two serial follow-up MRI examinations for imaging sets with (+MRCP) and without MRCP (-MRCP) regarding interpretation of the radiological course of morphological changes of common bile duct and common hepatic duct (CBD/CHD), right-sided (RIHBD) and left-sided intrahepatic bile ducts (LIHBD).

Bile duct group	Inter-reader agreement, ICC (CI 95%)
	+MRCP	−MRCP
CBD/CHD	−0.449 (−1.567 to 0.204)	0.263 (−0.388 to 0.609)
RIHBD	0.074 (−0.757 to 0.511)	0.018 (−0.835 to 0.478)
LIHBD	0.224 (−0.443 to 0.587)	0.192 (−0.543 to 0.575)

Numbers in parentheses indicate 95% confidence intervals (CI).

Table 6. Intra-reader agreement (ICC) for readers A and B between imaging sets with and without MRCP regarding interpretation of the radiological course of morphological changes of common bile duct and common hepatic duct (CBD/CHD), right-sided (RIHBD) and left-sided intrahepatic bile ducts (LIHBD).

Bile duct group	Intra-reader agreement
	Reader A	Reader B
CBD/CHD	0.815 (0.650–0.902)	0.962 (0.929–0.980)
RIHBD	0.825 (0.669–0.907)	0.978 (0.959–0.988)
LIHBD	0.809 (0.638–0.899)	0.940 (0.886–0.968)

Numbers in parentheses indicate 95% confidence intervals.

Table 7. Absolute number of cases with agreement and disagreement between readers for interpretation of radiological course of morphological changes of common bile duct and common hepatic duct (CBD/CHD), right-sided (RIHBD) and left-sided intrahepatic bile ducts (LIHBD) between two serial follow-up MRI examinations for imaging sets with (+MRCP) and without MRCP (-MRCP).

	Agreement		Disagreement
	+ MRCP	− MRCP	+ MRCP	− MRCP
CBD and CHD	24	26	16	14
RIHBD	27	25	13	15
LIHBD	25	26	15	14

Discussion

In the present study, we show that there is poor inter-reader agreement of the interpretation of the radiological course of morphologic bile duct changes between two serial follow-up liver MRI/MRCP examinations of patients with PSC (Figure 2). This adds new information and is in line with previous findings by Zenouzi et al. [14] who examined inter-reader agreement of duct changes on a single MRI/MRCP examination with a short MRI imaging set. Our study shows that evaluation of progression of bile duct strictures and dilatation in serial MRI/MRCP examinations is at least as challenging as evaluation of bile ducts changes in a single examination and this despite addition of complementary MRI sequences. Addition of MRI sequences could theoretically help readers agree in a higher grade as it provides more tools for evaluation of stricture/dilatation where imaging findings can be assessed in a more thorough way and also be cross-checked between different sequences. The poor inter-reader agreement is worrisome as it questions the appropriateness of MRI/MRCP as tool for surveillance and follow-up in patients with PSC. Before examining the potential usefulness of MRI/MRCP in patients with PSC it is imperative that reproducible imaging features and techniques are available.

Figure 2. Coronal T2-weighted HASTE and 3 D MRCP (MIP) of a 38-year-old patient with PSC. Time-point 1 (b) and (d) and time point 2 (a) and (c). Reader A interpreted the radiological course of duct changes between the two time points as subtle regression whereas reader B as subtle progression.

In the study by Zenouzi et al. [14], the readers were provided with only two sequences, 3 D MRCP and T2-weighted images. The authors recognized the difficulty in the interpretation of even high-quality 3 D MRCP images. We assumed that 3 D MRCP in their study was the dominant sequence for bile duct evaluation and thus the main reason for the low inter-reader agreement. In our study, we hypothesized that elimination of 3 D MRCP from the imaging set could potentially actually help readers agree to a higher extent since the inherent difficulty of 3 D MRCP interpretation would no longer exist. In order to ensure, however, that the elimination of 3 D MRCP does not alter the way each reader interprets the course of bile duct changes, we calculated the intra-reader agreement between the imaging sets with and without 3 D MRCP for each reader. Interestingly, we found that elimination of 3 D MRCP does not influence the interpretation of the course of morphological bile duct changes (very strong intra-reader agreement with ICC values ranging between 0.809 and 0.978). This finding is very interesting and clinically relevant. In cases when 3 D MRCP is of low quality or not performed, radiologists may still be able to evaluate the course of bile duct changes by using alternative methods, such as 2 D T2-weighted sequences supplemented by T1-weighted contrast enhanced images after application of MinIP. To the best of our knowledge, this is the first study that utilizes MinIP in contrast-enhanced MRI for the evaluation of the bile ducts. In a previous report, the use of MinIP on contrast-enhanced multi detector computed tomography (MDCT) improved bile duct visualization in patients with biliary obstruction compared to conventional MDCT [20]. Figure 3 demonstrates such a visualization of bile ducts with gadolinium-enhanced T1-weighted images before and after application of MinIP and with 3 D MRCP.

Figure 3. Gadoxetic acid-enhanced liver MRI of a 42-year-old patient with PSC. 3 D MRCP (a), T1-weighted images in transitional phase (3 min after contrast injection) without (b) and with application of MinIP with 12 mm (c) and 16 mm (d) thick slab.

Another interesting finding is that the readers found that there were some patients with radiological regression of morphological bile duct changes between the two MRI examinations. This result is not in concordance with the study by Ruiz et al. [21] where none of their patients showed improvement of radiological findings. Some of our patients (n = 6) underwent endoscopic treatment (ERCP) between the two MRI examinations which at least partially explains this result. The high grade of difficulty of evaluation of MRI/MRCP or spontaneous radiological regression of ductal changes are other possible explanations.

Our study has limitations. The study population arises from a tertiary center which may limit the generalizability of our findings. Another limitation is the lack of standard of reference. However, we believe that this does not affect the present analysis since our aim was to evaluate the inter-reader agreement, namely whether readers report similar findings regarding the dynamic course of cholangiographic changes rather than the sensitivity and specificity of the imaging methods. Moreover, there are numerous different ways to attempt to increase the inter-reader agreement. For the scope of the study, we decided to evaluate only one of them, namely elimination of 3 D MRCP, although MRCP is mandatory for the diagnosis of PSC. We decided to do that intrigued by the findings of Zenouzi et al. [14] where 3 D MRCP had a major impact in the low inter-reader agreement. In addition, we did not investigate the added value of MinIP on contrast-enhanced T1-weighted MRI images for the evaluation of bile ducts in this study. Further studies are needed to explore the full potential of utilization of this algorithm.

In conclusion, our study shows that there is poor inter-reader agreement of the interpretation of the radiological course of morphological/cholangiographic bile duct changes in serial follow-up liver MRI/MRCP of patients with PSC. Elimination of 3 D MRCP and utilization of other sequences for bile duct evaluation does not increase inter-reader agreement but maintains an excellent intra-reader agreement for the interpretation of the course of bile duct changes.

Ethics approval

This study is approved by the institutional review board, Regionala Etikprövningsnämnden, Stockholm (EPN), D-nr 2011/824-31/2 and 2018/1494-31/3. The study conforms to declaration of Helsinki.

Patient consent

According to ethics approval every patient has given their informed consent to participate in the study.

Disclosures statement

The authors do not have any financial conflict of interest to disclose.

Funding statement

This study has received funding from Stockholm County Council and Cancer Research Funds of Radiumhemmet.

Abbreviations
MRI	=	Magnetic resonance imaging
MRCP	=	Magnetic resonance cholangiopancreatography
PSC	=	Primary sclerosing cholangitis
ICC	=	Intraclass correlation coefficient
CCA	=	Cholangiocarcinoma
ERCP	=	Endoscopic retrograde cholangiopancreatography
CBD	=	Common bile duct
CHD	=	Common hepatic duct
HASTE	=	Half-Fourier acquisition single-shot turbo spin echo
MIP	=	Maximum intensity projection
MinIP	=	Minimum intensity projection
PACS	=	Picture archiving and communication system
MDCT	=	Multidetector computed tomography

Acknowledgements

The biostatistician Per Näsman (KTH Royal Institute of Technology, Stockholm, Sweden) kindly provided statistical advice for this manuscript.