MRI quality assurance using the ACR phantom in a multi-unit imaging center

Abstract

Background. Magnetic resonance imaging (MRI) instrumentation is vulnerable to technical and image quality problems, and quality assurance is essential. In the studied regional imaging center the long-term quality assurance has been based on MagNET phantom measurements. American College of Radiology (ACR) has an accreditation program including a standardized image quality measurement protocol and phantom. The ACR protocol includes recommended acceptance criteria for clinical sequences and thus provides possibility to assess the clinical relevance of quality assurance. The purpose of this study was to test the ACR MRI phantom in quality assurance of a multi-unit imaging center. Material and methods. The imaging center operates 11 MRI systems of three major manufacturers with field strengths of 3.0 T, 1.5 T and 1.0 T. Images of the ACR phantom were acquired using a head coil following the ACR scanning instructions. Both ACR T1- and T2-weighted sequences as well as T1- and T2-weighted brain sequences in clinical use at each site were acquired. Measurements were performed twice. The images were analyzed and the results were compared with the ACR acceptance levels. Results. The acquisition procedure with the ACR phantom was faster than with the MagNET phantoms. On the first and second measurement rounds 91% and 73% of the systems passed the ACR test. Measured slice thickness accuracies were not within the acceptance limits in site T2 sequences. Differences in the high contrast spatial resolution between the ACR and the site sequences were observed. In 3.0 T systems the image intensity uniformity was slightly lower than the ACR acceptance limit. Conclusion. The ACR method was feasible in quality assurance of a multi-unit imaging center and the ACR protocol could replace the MagNET phantom tests. An automatic analysis of the images will further improve cost-effectiveness and objectiveness of the ACR protocol.

The complex nature and high accuracy demands of magnetic resonance imaging (MRI) instrumentation makes it vulnerable to technical and image quality problems. Although the question of appropriate technical quality assurance of MRI has not triggered as many national and international guidelines as the imaging methods using ionizing radiation, substantial work has been done. The Eurospin project in the 1990s recommended a set of standard phantoms [1]. The MagNET phantoms have been used in the evaluation program supported by the UK Government's Centre for Evidence-based Purchasing [2]. American Association of Physicists in Medicine (AAPM) published its quality assurance recommendation in 1990 [3]. There are also international standards for image quality measurement [4–8]. American College of Radiology (ACR) has its own phantom for accreditation purposes [9]. Several research groups have also suggested quality assurance protocols [10–15].

Large healthcare providers often have several MRI systems on several sites, possibly including mobile units. The sites typically use a common picture archiving and communications system, and the images are not necessarily read on the site where the imaging itself takes place. The technical and clinical quality of the images from different sites should fulfill the same standards when the indication for the imaging is the same. Therefore uniform and consistent technical quality assurance is essential. The MRI quality assurance protocol of our multi-unit imaging center currently consists of three parts: 1) Daily single-slice spin-echo image of a manufacturer-specific homogenous phantom, 2) coil tests performed using manufacturer-specific phantoms and instructions, and 3) annual measurement of MagNET phantoms [2]. This protocol has been used for several years and questions have arisen regarding the optimal time interval between different tests. Some measurements are time consuming and their clinical relevance has been discussed, since the sequences differ from those used in clinical practice. Thus new methods are continuously evaluated.

American College of Radiology (ACR) has built an accreditation program for the U.S. MRI sites [9]. The program includes measurements with a standardized phantom to estimate the technical quality of images. The phantom is also available for centers not taking part in the accreditation program. The phantom includes features that enable versatile image quality measurement and evaluation. An important advantage compared to our current quality assurance measurements is that the ACR protocol also recommends image quality acceptance levels for MRI sequences used in clinical practice.

The objectives of this study were to test the feasibility of quality assurance with the ACR MRI accreditation phantom in our organization and to explore the possibility to replace some of the existing quality assurance procedures with the ACR phantom test. The ACR method was selected for evaluation because of its internationally recognized position, as well as the short measurement time of the protocol.

Material and methods

The ACR MRI accreditation phantom is cylindrical and has an inside diameter of 190 mm and an inside length of 148 mm. It is filled with 10 mM NiCl₂ and 75 mM NaCl. It contains structures for measuring geometric accuracy, high-contrast spatial resolution, slice thickness accuracy, slice position accuracy, image intensity uniformity, percent signal ghosting and low-contrast detectability. The phantom images were acquired on 11 MRI systems of three major manufacturers following the ACR site scanning instructions [16]. The field strengths were 3.0 T (systems #1 and #2), 1.5 T (systems #3 to #10) and 1.0 T (system #11). All the systems were administered and operated by one regional imaging center. Apart from system #11, all the systems had been purchased or upgraded during the last seven years. The MRI systems were located in eight different hospital buildings and in a mobile unit (system #10). The clinically used head coil of each system was used in the measurements, with the exception of system #10 as the phantom was too large to fit inside that particular coil. The coils were 8- or 12-channel coils, apart from single channel coils on systems #6, #10 and #11. The phantom was carefully leveled inside the coil, and the center point of the phantom was aligned with the center point of the coil and the isocenter of the magnet with the positioning lasers. The anatomically shaped design of some coils prevented positioning of the phantom at the center of the coil. The measurement was performed twice on each system. The time interval between the two measurements varied from three to nine months.

According to the measurement protocol, a sagittal slice (locator) was acquired first. Then, axial sequences defined by the ACR and each site's own T1- and T2-weighted head sequences were acquired. In many cases the site sequences needed to be selected among several possible sequences used at the respective site. Number of slices, slice gap and slice thickness of the site sequences were determined by the ACR protocol, to ensure acquisition of the slices in correct positions for analysis. The sequence parameters are listed in Tables I–III. Pixel sizes for Tables II and III were calculated by dividing the field of view by the acquisition matrix. Image intensity correction options of multi-element coils were turned on in the ACR sequences whenever they were routinely used in clinical head sequences of the respective site, absent only on system #9. Parallel imaging was used in site T2 sequence of systems #2, #7 and #8.

Table I. Imaging parameters of the ACR sequences [16].

Sequence	TR (ms)	TE (ms)	FOV (cm)	Matrix	Averages	Number of slices	Slice thickness (mm)
Sagittal locator	200	20	25	256 × 256	1	1	20
Axial T1	500	20	25	256 × 256	1	11	5
Axial T2	2000	20/80	25	256 × 256	1	11	5

TR, repetition time; TE, echo time; FOV, field of view.

Table II. Imaging parameters of site T1-weighted sequences of different systems.

System #	Sequence	TR (ms)	TE (ms)	FOV (cm)	Matrix	Averages	BW (Hz/pixel)	Pixel size (mm)
1	SE	500	10	23 × 23	256 × 205	1	207	0.9 × 1.1
2^a	SE	500	8.4	23 × 20	256 × 218	1	201	0.9 × 0.9
2^b	SE	500	8.4	21 × 22	256 × 218	1	201	0.9 × 0.9
3	SE	491	14	23 × 20	256 × 224	2	130	0.9 × 0.9
4	SE	550	12	23 × 23	256 × 256	1	90	0.9 × 0.9
5	SE	550	8.7	23 × 23	256 × 256	1	150	0.9 × 0.9
6	SE	500	7.7	23 × 20	256 × 168	2	140	0.9 × 1.2
7	SE	475	14	23 × 23	240 × 180	2	122	1.0 × 1.3
8	SE	475	14	23 × 23	240 × 180	2	122	1.0 × 1.3
9	SE	180	14	24 × 24	256 × 224	2	61	0.9 × 1.1
10	SE	460	20	25 × 25	320 × 224	1	61	0.8 × 1.1
11	SE	297	15	23 × 23	256 × 205	2	100	0.9 × 1.1

TR, repetition time; TE, echo time; FOV, field of view; BW, bandwidth; SE, spin echo.

In FOV, matrix and pixel size, the first number refers to anterior-posterior direction and the second number to right-left direction.

^a,bThe first measurement was performed just after the installation of the system and the parameters were changed by the time of the second measurement.

Table III. Imaging parameters of site T2-weighted sequences of different systems.

System #	Sequence	TR (ms)	TE (ms)	FOV (cm)	Matrix	Averages	ETL	BW (Hz/pixel)	Pixel size (mm)
1	TSE	4000	80	24 × 24	400 × 316	1	13	213	0.6 × 0.8
2^a	TSE	6000	96	23 × 22	320 × 320	1	18	220	0.7 × 0.7
2^b	TSE	5120	94	22 × 22	448 × 314	3	13	223	0.5 × 0.7
3	TSE	4000	96	23 × 20	448 × 293	2	11	101	0.5 × 0.7
4	motion corrected TSE	4000	94	23 × 23	320 × 320	1	30	365	0.7 × 0.7
5	motion corrected TSE	4000	106	23 × 23	320 × 320	1	35	365	0.7 × 0.7
6	TSE	4000	99	23 × 20	256 × 244	2	11	100	0.9 × 0.8
7	TSE	2217	100	23 × 23	384 × 299	3	17	168	0.6 × 0.8
8	TSE	2217	100	23 × 23	384 × 299	3	17	168	0.6 × 0.8
9	FLAIR	8802	126	22 × 22	320 × 224	1	N/A	98	0.7 × 1.0
10	TSE	5200	101	25 × 25	448 × 320	2	16	122	0.6 × 0.8
11	TSE	2135	96	23 × 23	352 × 278	3	13	100	0.7 × 0.8

TR, repetition time; TE, echo time; FOV, field of view; ETL, echo train length; BW, bandwidth; TSE, turbo spin echo; FLAIR, fluid-attenuated inversion recovery.

The motion corrected TSE refers to BLADE/PROPELLER/MultiVane techniques. In FOV, matrix and pixel size, the first number refers to anterior-posterior direction and the second number to right-left direction. Inversion time of sequence of system #9 was 2200 ms.

^a,bThe first measurement was performed just after the installation of the system and the parameters were changed by the time of the second measurement.

The images (examples in Figure 1) were analyzed according to the ACR instructions by the same single observer and the results were compared to the ACR specifications [17]. Geometric accuracy was evaluated by measuring seven known distances in the images. High-contrast spatial resolution was assessed visually based on the distinguishability of hole-array pairs with hole diameters of 0.9 mm, 1.0 mm and 1.1mm. Slice thickness was calculated from the known ramp angle and slice position accuracy measurement was based on wedge visualization. Image intensity uniformity was calculated from pixel values inside a region of interest in a slice containing only uniform material. Ghosting values were calculated from regions of interest placed outside the phantom in the image. Low-contrast object detectability was visually assessed by calculating number of objects visible in four images with gradually decreasing contrast and object size.

Figure 1. Examples of ACR phantom images. In slice 1 the hole-array pairs are used for high-contrast spatial resolution and the ramps in the middle for slice thickness accuracy. Slice 5 together with measurements from slice 1 and the sagittal image are used for the geometric accuracy determination. Image intensity uniformity is determined from slice 7 and slices 8 to 11 are used for low-contrast object detectability measurement.

Results

Results of the seven evaluated parameters are presented in Figures 2–4 and Tables IV and V. The measured image intensity percent integral uniformities are shown in Figure 2. The ACR recommended acceptance values are 82% for 3.0 T and 87.5% for 1.5 T. Image uniformity was slightly below this value on both 3.0 T systems. In Figure 3 the measured image slice thicknesses are compared to ACR recommended acceptance limits of ± 0.7 mm. The results were outside the limits more often in the site T2-weighted sequences than in the other sequences. The mean observed slice thickness was 4.9 mm in ACR T2 sequences and 5.3 mm in site T2 sequences.

Figure 2. Image intensity percent integral uniformity of a. ACR T1, b. ACR T2, c. site T1 and d. site T2 sequences. The dash lines indicate the ACR recommended acceptance values for 1.5 T (87.5%) and 3.0 T (82%).

Figure 3. Slice thickness accuracy; the nominal value is 5 mm with ±0.7 mm tolerance, limits indicated by dash lines. a. ACR T1, b. ACR T2, c. site T1 and d. site T2 sequences.

Figure 4. Low contrast detectability of measurements 1 and 2 for a. T1-weighted sequences and b. T2-weighted sequences. The dash lines indicate the ACR recommended acceptance values for 1.5 T (9 objects) and 3.0 T (37 objects).

Table IV. High contrast spatial resolution of site sequences.

	T1		T2
system #	site, 1^st	site, 2^nd	site, 1^st	site, 2^nd
1	1.1	1.1	0.9	0.9
2	1.0	1.0	0.9	0.9
3	0.9	1.0	0.9	0.9
4	0.9	0.9	1.0	1.0
5	0.9	0.9	1.0	0.9
6	>1.1	>1.1	0.9	0.9
7	>1.1	>1.1	0.9	0.9
8	>1.1	>1.1	0.9	0.9
9	>1.1	1.1	1.0	1.1
10	1.0	1.1	0.9	0.9
11	1.1	>1.1	0.9	0.9

1^st and 2^nd refer to the two measurements. The values in the table indicate the size of smallest holes that were resolved (0.9, 1.0 or 1.1 mm). >1.1 indicates that the largest holes of 1.1mm were not resolved.

Table V. Percentages of the systems that passed the ACR recommended acceptance criteria in each test.

Sequence (measurement number)	ACR (1)	ACR (2)	Site (1)	Site (2)
Geometric accuracy	100	91	100	91
High-contrast spatial resolution	100	82	36	36
Slice thickness accuracy	100	91	55	55
Slice position accuracy	100	100	100	100
Image intensity uniformity	91	91	91	91
Ghosting	100	100	100	100
Low contrast object detectability	100	100	91	82

The systems achieved high-contrast spatial resolution corresponding to the acquired pixel size of 1.0 × 1.0 mm with the ACR sequences. This was also the ACR recommended acceptance value. The measured high contrast spatial resolution was generally lower for site T1-weighted sequences and higher for site T2-weighted sequences, when comparing with the respective ACR sequences (Table IV). There were differences also between anterior-to-posterior and left-to-right direction in some images. Low-contrast detectability results are shown in Figure 4. All the systems passed the ACR determined criteria, which was nine objects for 1.5 T and 37 objects for 3.0 T systems. The mean number of visible objects was 34 and standard deviation was 3.6 in the T1 sequences of the 1.5 T systems. There was larger variation in the T2 sequences, both between systems, ACR and site sequences, and different measurements of the same system; the mean number of visible objects was 26 and standard deviation was 7.3 in 1.5 T systems. All results of percent signal ghosting and slice positioning accuracy were within the ACR recommended acceptance criteria. In ghosting the criterion is 0.025% and the majority of the results were under 0.01%. In slice positioning accuracy the recommended acceptance criterion is ±5 mm. The measured deviations were between 0 mm and 3.6 mm. Generally the systems also passed the geometric accuracy test with limits of ±2 mm when measuring a known length of 190 mm, but two systems failed the test in one measurement.

Table V lists the results of passing or failing of each test in relation to the ACR recommended acceptance criteria. The tests that failed most commonly were high-contrast spatial resolution and slice thickness accuracy. The tests passed more often with the ACR sequences than the site sequences. The ACR instructions define that the overall test is passed if the criteria are met either with the ACR or the site sequences. The overall passing rate was 91% in the first and 73% in the second measurement.

Discussion

Despite well-designed manufacturer-specific service programs, it is essential to use standard phantoms in MRI quality assurance to enable uniform measurement of the systems in a multi-unit imaging center. In this study 11 MRI systems of a regional imaging center were measured twice with the ACR phantom to evaluate the feasibility of the ACR test for quality assurance of a large organization. Generally the ACR protocol was easy to perform and clearly instructed. The results showed that most of the systems operated at the level fulfilling the ACR recommended acceptance criteria. These observations were in agreement with another study by Chen et al. using the same phantom [15]. There were, however, some difficulties in applying the protocol into practice. The phantom did not fit inside all head coils. This was the case with system #10, but also with the 32-channel coil of system #2. The field-of-view was not allowed to be adjusted in site sequences, although in some cases it was too small to allow measurements of ghosting and geometric accuracy. The slice thickness results may have been affected by poor visibility of ramps in the phantom image with some T2-weighted sequences. In addition, there are acquisition parameters that are not defined for the ACR sequences. These include the receiver bandwidth, signal intensity correction methods and reconstruction filters. The choice of these parameters may have affected the results of low contrast detectability and image uniformity in this study. Pixel size differences and possible interpolation in reconstruction in clinical sequences had an effect on the results of the high contrast spatial resolution. Factors related to coil structure and B1 field may explain the performance of 3.0 T systems in the uniformity test.

The possibility to replace some of the existing quality assurance methods with the ACR method in our center was one of the objectives of the study. Currently a simple quality assurance test is performed with a homogenous phantom for the head coil every morning immediately after start-up of the system. Obvious faults with the system can be detected before the first patient of the day has been positioned on the scanner table. Due to well defined practice it is not relevant to replace the daily procedure with the ACR phantom test. The manufacturer specified coil tests cannot obviously be replaced either. However, the MagNET and the ACR phantom tests are both manufacturer independent and could therefore be interchangeable. The parameters measured by both methods are essentially the same. The ACR phantom test does not include a pure signal-to-noise ratio measurement and MagNET phantoms do not include structures for low-contrast detectability assessment. However, the signal-to noise ratio has a direct effect on the low contrast detectability. The advantages of the ACR phantom test compared to MagNET phantom tests are 1) fast acquisition of images (20 min), 2) inclusion of clinical sequences in the measurement protocol and 3) globally more recognized method. The reasons that speak for continuing with MagNET phantoms are 1) long measurement history in our organization, 2) measurement of three orthogonal slice planes and 3) in-house developed automatic analysis software that makes the image analysis fast and objective. One limitation of this study was that thus far only two measurements have been performed with the ACR phantom. Before final conclusion whether the ACR method has the potential to reveal changes in the performance of the systems better than the MagNET method, longer term follow-up of the ACR phantom measurements would be needed.

Both the measurement and the analysis should be reasonably fast to meet the demands of cost-effectiveness. In our organization the analysis of MagNET phantom images is automated which has made the analysis considerably faster and more objective. Developing automatic analysis software for the ACR phantom test would increase its cost-effectiveness and objectivity as well. The need for increased objectivity in the ACR image analysis is most obvious in slice thickness and low contrast object detectability measurements. The increasing role of anatomical and functional MRI in diagnosis, treatment planning and follow-up of cancer patients sets demands for objectivity and repeatability of quality assurance methods [18,19].

As Weinreb et al. [9] pointed out, clinical relevance of quality assurance is difficult to evaluate. In most quality assurance protocols the tests are performed with spin-echo sequences, which usually differ from the clinical sequences [1,2]. Connecting the quality assurance test to clinical image quality by using the same sequences is an advantage of the ACR method. It should also be noted, that the phantoms available for MRI quality assurance do not provide complete methods to evaluate clinical image quality. For example, none of the quality assurance protocols used or discussed in this study provides means to assess effectiveness of fat suppression, which is essential in clinical imaging. Even more important, the vast variety of contrasts in clinical images can not be mimicked by the current phantoms. The most challenging methods, such as functional MRI, diffusion and perfusion are pushing the systems to their limits and need dedicated quality assurance procedures [11].

Different sites of a multi-unit imaging center often have adopted their own conventions in imaging, even when the indication for imaging is the same. Still the minimum image quality level should be equal in all the sites. In practice the patient material varies from site to site and technical properties of the MRI systems may limit the achievable image quality. Clinical audit practices in the European Union have helped in approaching the objective of equal image quality with imaging methods using ionizing radiation, and there are recommendations and criteria to systematically evaluate clinical image quality from patient images [20,21]. Same kind of approach could be justifiable in MRI as well, and, for example the ACR accreditation program includes evaluation of clinical patient images in addition to phantom images. One possible future direction of this study would be to extend the quality assurance protocol by connecting the phantom results with the quality of patient images.

In conclusion, the ACR method proved feasible for quality assurance in a multi-unit imaging center and the ACR protocol could replace the MagNET phantom tests. The image acquisition procedure of the ACR test was fast and practical. Automatic analysis of the images will further improve cost effectiveness and objectiveness of the ACR protocol.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.