Comparison between automated and manual measurements of carotid intima-media thickness in clinical practice

Abstract

Background and aim:

The measurement of carotid intima-media thickness (cIMT) has been used as a marker of arterial wall disease. Manual measurements have been performed in most epidemiological studies, but, due to the introduction of new technologies, automated software has been increasingly used. This study aimed to compare manual versus automated cIMT measurements in common carotid (CC), bifurcation (BIF), and internal carotid (IC).

Methods:

Automated and manual cIMT measurements were performed online in 43 middle-aged females. Carotid segment measurements were compared by Bland–Altman plot and the variation and repeatability coefficients between observers were also determined for comparison.

Results:

The average timespan for manual measurements (57.30 s) were significantly higher than for automated measurements (2.52 s). There were no systematic errors between methods in any carotid segments. The variation coefficient was 5.54% to 6.34% for CC and BIF, 9.76% for IC, and absolute differences were 85% below 0.1 mm and 70% below 0.05 mm. Interobserver agreement showed no systematic error. The variation and the repeatability coefficients were better for the automated than manual measures.

Conclusion:

Although both methods are reliable for cIMT measurements, the automated technique allows faster evaluation with lesser variability for all carotid segments currently used in atherosclerosis research.

Keywords:

• intima-media thickness
• atherosclerosis
• carotid segments
• automated method
• manual measurement
• bland–altman plot

Introduction

Atherosclerosis and atherothrombotic disease represent the major causes of morbidity and mortality worldwide and they shorten life expectancy due to heart attack and stroke.^1,2 New technologies have emerged that allow identification of arterial wall disease progression, assessment of response to therapy and of the risk of the disease itself. The measurement of carotid intima-media thickness (cIMT) has been used in epidemiological and in risk stratification studies as a marker of subclinical atherosclerosis, a reliable substitute for a morbid event.³ Given that trials in prevention of coronary artery disease (CAD) events require thousands of subjects and five to ten years of follow-up, cIMT measurements permit smaller, lower-cost studies.⁴ Of the various noninvasive imaging methods available, cIMT measurement is currently recommended by the American Heart Association for inclusion in the evaluation of risk for development of cardiovascular disease in intermediate-risk patients classified by the Framingham risk score.^5–7

In 1986, Pignoli and colleagues⁸ identified the cIMT noninvasively using ultrasound and they concluded, at that time, that B-mode imaging was a useful tool for the detection and monitoring arterial wall changes. Since then, B-mode ultrasonography has been used to measure cIMT as a primary indicator of atherosclerotic disease, allowing better understanding of the development and progression of atherosclerosis. Given the strong correlations between atherosclerosis in various vascular beds, the measurement of cIMT is becoming increasingly accepted as a surrogate marker of generalized arterial disease.^9,10

About ten years ago, the Atherosclerosis Risk in Communities (ARIC) study enrolled 15,800 individuals and showed, after adjustments for covariates such as age, body mass index (BMI), smoking status, sex, race and blood pressure, that cIMT was still well correlated to total cholesterol, low density lipoprotein cholesterol, and triglycerides.¹¹ Then, in the year 2000, the American Heart Association Prevention Conference V stated that the measurement of cIMT add incremental information to traditional risk factor assessment in asymptomatic people, especially for those aged more than 45 years.¹²

The manual measurement (point-to-point measurement of B-mode images) of cIMT is the most common technique used in clinical practice, even though it is time-consuming and the results from these readings may be biased by the lack of expertise or by some subjective judgment of the observer. Even when the same images are measured twice but 12 months apart, one third variability in the measurements may be expected by manual measurement of cIMT. Thus, the development of automated measurement procedures carries an obvious appeal.^13,14 Previous studies have used the semiautomated and automated measurements of cIMT in common carotid, but none of them included online automated measurement software for cIMT.^15,16

In this context, the aim of this study was to compare the automated analyzing system for measurement of cIMT to manual measurements in predefined sections of the common carotid (CC), carotid bifurcation (BIF), and internal carotid (IC) using regular ultrasound equipment with standard software in a routine basis.

Subjects and methods

Study population

The study population consisted of 43 female patients, aging 38.90 ± 5.60 years, referred from a women’s health outpatient ward. All patients included in this study are participants of a clinical study concerning the effect of past gestational diabetes in subclinical atherosclerosis (patients and controls). In addition, this female group was homogeneous and free from atherosclerotic plaques and any other co-morbidity. Therefore, the measurement of cIMT could be useful to detect early atherosclerotic changes in this group. Informed consent was obtained from each participant before the enrollment procedure. The study protocol was approved by the Ethical Committee of the Federal University of Minas Gerais.

Study protocol

A GE Health Care Vivid 7 Dimension (Wauwatosa, WI, USA), high resolution ultrasound scanner was used with a high frequency (7, 10, or 12 MHZ) linear array transducer. For the examination of the carotid arteries, the necks of the subjects were turned slightly to the left or right side. The selected image was maximized and the gain settings optimized to visualize the far wall of the carotids in order to measure intima-media complex.

Both automated and manual measurements were performed as described: 1 cm distal of the flow divider in proximal IC, 1 cm proximal to the flow divider as BIF, and 2 cm or more proximal to the flow divider in CC.³ After freezing the image, 10 measurements were manually obtained by placing electronic calipers at the edge of the far wall of each segment. The cIMT data were taken online for every segment apart and the average of ten data point acquisitions was then calculated. The automated measure comprised an online measurement of multiple cIMT data points in seconds, and immediate results were provided as average, maximum, minimum, standard deviation, number of acquired data points, and distance. The commercial software algorithm is based on a comprehensive analysis of the two-dimensional vessel structure represented on an ultrasound image, rather than on a simple detection of grayscale gradients. This technique allows accurate detection of the interfaces without almost any influences by random irregularities of the ultrasound image. The operator sets the starting and ending point of the measurement area manually. The two lines along the boundaries of the IMT are automatically drawn (Figure 1).¹⁷

Figure 1 Representative images of automated and manual intima media thickness measurements.

Two experienced and well trained physicians randomly and blindly selected fifteen patients to perform the interobserver reproducibility of the manual and automated measurements.

Statistical analyses

The MedCalc package (Mariakerke, Belgium) was used for statistical analysis. Results were expressed as means ± standard deviation (SD) for CC, BIF, and IC cIMT measures. Manual and automated measurements were then compared by Bland–Altman plot for interpretation of method-comparison studies. The estimated bias (mean difference from average), SD, and the 95% confidence interval between the two measurements for the three carotid segments (CC, BIF, and IC) were obtained.

The images were stored in the Digital Imaging and Communications in Medicine (DICOM) and analyzed offline for interobserver evaluation. The agreement between observers for assessment of cIMT was determined as a relationship of the differences of the mean of the paired measurements, according to the Bland–Altman method. The variation coefficient was calculated according to the formula: (SD of the mean difference/√2) multiplied by 100 divided by the pooled mean values.¹³ For the repeatability coefficient we used the following formula: SD of the differences multiplied by two, which shows the expected variation in results for repeated measurements. According to the definition of repeatability coefficient given by the British Standards Institute, the mean difference must not be significantly different from zero, and the 95% of the differences are expected to lie within the range of ± 2 SD.¹⁸ The clinically accepted limits of agreement were defined as less than 0.10 mm.¹⁸

To compare the means of each carotid segment measurements and time duration for automated and manual measurements, unpaired Student’s t-test was used, and P < 0.05 was set as significant.

Results

The length measured in each carotid segment was 0.63 to 1.30 cm for CC, 0.49 to 1.07 cm for BIF, and 0.42 to 1.33 for IC, depending on the quality of the image, in which 10 points of manual measurements and an average of 199 automated measurements in subpixels were performed. After selection of the image, the average time for manual measurement was 57.30 ± 5.45 seconds, whereas automated method spent only 2.52 ± 0.38 seconds (p < 0.05).

The means and SD of manual and automated cIMT measurements obtained for CC, BIF, and IC were displayed in Table 1. No statistical differences were detected in any comparison (Table 1).

Table 1 Comparison of manual and automated measures in 43 patients using Bland–Altman plot, the percentage of differences greater than 0.05 and 0.10 mm in both measurements and the variation coefficient

	Common carotid average	Common carotid maximal	Carotid bifurcation average	Carotid bifurcation maximal	Internal carotid average	Internal carotid maximal
Manual measures (mm)	0.53 ± 0.07	0.64 ± 0.08	0.58 ± 0.11	0.77 ± 0.16	0.52 ± 0.20	0.69 ± 0.29
Automated measures (mm)	0.53 ± 0.06	0.65 ± 0.08	0.59 ± 0.11	0.78 ± 0.15	0.50 ± 0.17	0.69 ± 0.27
Mean difference of average (mm)	−0.003	0.01	0.013	0.003	0.01	0.005
95% limits of agreement (mm)	−0.10 to 0.09	−0.10 to 0.12	−0.09 to 0.11	−0.13 to 0.13	−0.15 to 0.12	−0.11 to 0.09
% of absolute differences <0.1 mm	97.7%	93%	90.7%	88.4%	86%	88.1%
% of absolute differences <0.05 mm	86.0%	74.4%	74.4%	69.8%	73.8%	79.1%
Variation coefficient	6.34 %	6.34%	6.10%	6.01%	9.68%	5.54%

The estimated bias and SD by Bland–Altman method between the automated and manual methods for average and maximum measurements of CC was −0.003 ± 0.050 mm and 0.010 ± 0.060 mm, for BIF was 0.013 ± 0.050 mm, and 0.003 ± 0.07 mm, and for IC was 0.01 ± 0.07 mm and 0.005 ± −0.05 mm, respectively. As shown in Figure 2, the mean bias in the average and maximum measurement of all carotid segments showed no systematic error.

Figure 2 Bland–Altman plot for the differences between average and maximum common carotid (CC), carotid bifurcation (BIF) and internal carotid (IC) measurements obtained by manual and automated method.

The evaluated segments in CC, BIF, and IC showed absolute mean differences varying from 0.003 to 0.01 mm with limits of agreement of −0.15 to 0.13 mm. More than 85% of the absolute differences between the automated and manual measurements were below 0.1 mm and around 70% below 0.05 mm. The variation coefficient ranged between 6.01 to 6.34% in CC and BIF and 5.54 and 9.68% in IC (Table 1).

Tables 2 and 3 showed the interobserver agreements of the manual and automated measurements. Based on the estimated bias and SD by Bland–Altman method, no systematic error was detected between two observers in both manual and automated measurements (Table 2). The repeatability and variation coefficients were lower in automated measures for all carotid segments, except for the average measurement of IC (Table 3).

Table 2 Interobserver agreement for manual and automated measurements of the carotid segments in 15 patients compared by Bland–Altman plot

	Common carotid average	Common carotid maximal	Carotid bifurcation average	Carotid bifurcation maximal	Internal carotid average	Internal carotid maximal
Observer 1 manual (mm)	0.50 ± 0.06	0.60 ± 0.13	0.57 ± 0.11	0.76 ± 0.18	0.49 ± 0.10	0.68 ± 0.12
Observer 2 manual (mm)	0.50 ± 0.06	0.62 ± 0.10	0.55 ± 0.06	0.72 ± 0.08	0.50 ± 0.13	0.72 ± 0.20
Manual interobserver difference/95% limits of agreement (mm)	<0.00001/−0.12 to 0.12	−0.03/−0.28 to 0.23	0.02/−0.10 to 0.15	0.04/−0.22 to 0.30	−0.007/−0.18 to 0.17	−0.05/−0.35 to 0.26
Observer 1 automated (mm)	0.51 ± 0.05	0.63 ± 0.06	0.56 ± 0.09	0.76 ± 0.14	0.47 ± 0.10	0.68 ± 0.13
Observer 2 automated (mm)	0.51 ± 0.05	0.65 ± 0.07	0.56 ± 0.07	0.74 ± 0.09	0.52 ± 0.14	0.75 ± 0.18
Automated interobserver difference/95% limits of agreement (mm)	<0.00001/−0.03 to 0.03	−0.02/−0.14 to 0.10	0.001/−0.07 to 0.07	0.03/−0.14 to 0.20	−0.04/−0.22 to 0.14	−0.07/−0.28 to 0.14

Table 3 Interobserver comparison of the repeatability and variation coefficients between manual and automated measurements

		Repeatability coefficients (in mm)		Variation coefficients
		Manual measurement	Automated measurement	Manual measurement	Automated measurement
Common carotid	Mean IMT	0.118	0.033	8.31%	2.28%
	Maximal IMT	0.265	0.125	15.35%	6.93%
Carotid bifurcation	Mean IMT	0.129	0.070	8.10%	4.39%
	Maximal IMT	0.267	0.174	12.84%	8.21%
Internal carotid	Mean IMT	0.182	0.184	12.91%	13.11%
	Maximal IMT	0.314	0.214	15.80%	10.58%

Abbreviation: IMT, intima media thickness.

Discussion

Our results showed that the development of automated methods for measuring cIMT in standard ultrasound equipment has importantly contributed to better reproducibility of results between different observers as well as reducing considerably the necessary time for image evaluation. As a matter of fact, the mean difference between the automated and manual methods for cIMT measurements was not clinically relevant in any of the carotid segments evaluated in the majority of cases, and carries no systematic errors. On the other hand, the automated method also enables us to reliably compare online data with other already published reports for different populations in percentiles, as has been recently proposed by the American Society of Echocardiography.¹⁹ The data expressed this way may help clarify the meaning of these numbers for practitioners, besides reducing misinterpretation of measurements.

The use of automated measures has been tested by different manufactures and most of them used a computerized offline program for tracing the edges in CC. Although the automated methods have been shown to be appropriate for the CC segment, automated measurements published for bifurcation and IC segments are still lacking.^20–22 Our findings with automated measurements clearly shed light on this field, pointing out to good reproducibility in all carotid segments without detectable systematic errors. Moreover, the repeatability and variation coefficient were also better for automated measurements, except for IC where measurements are expected to be worse due to difficulties concerning alignment of the ultrasound beam. We highlight that measurements of other segments by a reliable automated method is in clear advantage over measurements concentrating solely on CC. As bifurcation and internal segments are the first affected sites by atherosclerosis, a demonstration of an increased IMT measurement in these sites does not allow a precise differentiation between atherosclerosis and vascular hypertrophy but denotes a subclinical involvement of the carotid wall. Furthermore, most of the measurement bias was below the accepted clinical limits for daily practice.^20,23–25

Nowadays, the use of surrogate markers to predict cardiovascular events has become a reality, and technologies have to improve continuously to attend the needs of practitioners. In this regard, it is important to obtain fast and comparable IMT measurements to minimize errors. The automated measurement utilized in this study has shown live cIMT measurements by a standard ultrasound machine with less variability than manual measurements in most carotid segments, as it reduces the component of variability associated with both manual cursor placement and manual drawing of the wall. Our results added information about reproducibility in online automated IMT measurements, as the recent published study by Puchner and colleagues tested only the CC.¹⁷ Indeed, the automated edge tracking using subpixel interpolation determines edge boundaries at a resolution greater than monitor line resolution. However, we have to admit that, even in the hands of expert physicians on this field, automated measurements may sometimes not be available due to lack of image quality obtained.²⁶

Most of the large scale studies in cIMT have utilized manual measurements obtained by off-line video images. Only the Rotterdam study has determined the cIMT by both semiautomated and manual methods in the same population.^3,6,7,27 According to our results, the Rotterdam study concluded that manual measurements are time-consuming with larger variability between readers.^16,28,29 Therefore, the possibility of good quality online measurements with available software of good reproducibility in daily live exams would certainly expand information on cIMT on a regular basis for clinical practice.

We are aware of the limitations of our study. Despite the use of a homogeneous group of patients, this study included only women and we cannot assume that in men the results would be the same, even knowing that cIMT is thinner in female. All the interobserver measurements were derived from stored images and the reproducibility of the measurements could be different with different angles of insonation and depth of the vessel studied, especially in segments where the images are not supposed to be straight as in IC. The two observers were physicians trained in vascular ultrasound, and the variability based on different sonographers was generally larger than the variability based on readers in studies.¹³ In addition, online analysis requires utmost precision and skill. We tested only the automated software of GE Vivid 7, and could not assume that the agreements would be the same for other software. Nevertheless, some features of this study may increase the strength of our findings such as the homogeneity of our sample, the expertise of the sonographers who were blinded to study protocol, and the utilization of well-established protocols for both manual and automated cIMT measurements.

In conclusion, automated cIMT measurement available in standard equipments saves time and has a good reproducibility in many carotid segments currently used for research in atherosclerosis.

Disclosures

The authors report no conflicts of interest in this work.