Linear and angular measurements of computer-generated models: Are they accurate, valid, and reliable?

Abstract

Introduction: Traditional advanced imaging modalities such as CT and MRI are limited in their ability to perform accurate linear distance and angular measurements regardless of anatomical orientation. The construction of 3D models has been used to perform anthropometric analyses as well as in the reconstruction of rapid prototypes. We hypothesized that such measurements would be precise to within 2 mm or 2° of measurements performed with a coordinate measurement machine (CMM). We also hypothesized that there would be a high degree of interobserver reliability with these measurements.

Materials and Methods: Multiple aluminum screws were implanted in various positions in three foam pelvises which were subsequently scanned by CT and rendered as 3D models using a commercially available software package (Mimics). Linear and angular measurements were performed using a CMM machine, the software package, and a dial caliper or goniometer. The deviation of the measurements from the CMM data was compared using ANOVA. The interobserver reliability of both the manual and computer-generated measurements was calculated.

Results: The mean difference between the CMM distances and those measured manually and with the software was 2.12 ± 1.20 mm and 1.57 ± 1.05 mm, respectively. The mean difference between the CMM angular measurements and the angular measurements performed manually and with the software was 4.07 ± 4.70° and 1.62 ± 1.32°, respectively. In all cases, the manual measurements were significantly less accurate (p < 0.0001) and there was a high degree of interobserver reliability.

Conclusions: Computer-generated measurements taken from three-dimensionally reconstructed models are more accurate than manual measurements and are within 2 mm and 2° of measurements performed with a CMM. These measurements have high interobserver reliability.

• CT
• computer-generated model
• angular measurement
• Mimics

Introduction

Anatomical measurements are critical in the assessment of musculoskeletal deformities. Traditionally, direct measurements of post-mortem anatomical specimens have been the only available method for linear, angular, and volumetric measurements. Standard radiography has been used for linear measurements [1–3], but has the limitation of providing only two-dimensional information and no spatial resolution. Advanced imaging modalities such as CT and MRI are able to provide calibrated length measurements but do not routinely allow for measurement across multiple different sections. Additionally, angular measurements between lines and planes crossing multiple slices are not possible using standard software. Finally, both CT and MRI scans are prone to limitations resulting from variations in patient positioning in the scanner.

Recently, three-dimensional (3D) reconstructions have been performed on CT images to assist in the diagnosis of complex traumatic injuries, in surgical planning, and even in the analysis of predicted joint motion [4–7]. Additionally, CT reconstructions using specialized software packages have been used in the assessment of brain volumes in normal [8], [9] and pathological [10–14] states. Specialized software programs have become available which allow surface measurements, volumetric measurements, and angular measurements on models generated from CT or MRI images. Some software programs have evolved in their ability to perform virtual surgery on the rendered model. These surgical procedures can be realized intraoperatively using surgical navigation systems, rapid prototypes, or custom implants fabricated from the 3D data. However little data is currently available validating the accuracy of linear and angular measurements made on 3D computer models. Furthermore, there is a paucity of data validating the accuracy of angular measurements between planes in such software packages.

The optimal accuracy and precision of spatial data for clinical use is a point of some controversy. The scale of the surgical procedure and the proximity of vulnerable neurovascular structures are the major determinants of the expectations for these parameters. After a review of the literature in a number of different specialties which make use of surgical navigation, it was determined that the values of 2 mm or 2° represented a consistent threshold value for the minimum expected level of accuracy and precision.

We had four hypotheses for the present study: (1) Measurements performed in a 3D rendering software program (Mimics, Materialise, Ann Arbor, MI) would be valid and accurate to within the previously discussed threshold of 2 mm or 2° when compared to a “gold standard” measurement obtained using a coordinate measurement machine (CMM); (2) The distance measurements performed with the software program would be more accurate than those performed with a dial caliper, and the angular measurements would be more accurate than those performed with a manual goniometer; (3) There would be a high degree of interobserver reliability with both the manual and computerized measurements; and (4) Angular measurements between planes in the software program's anthropometric analysis package would be precise and accurate when used to measure known angles between planes on simple 3D polygon models constructed in a separate software program.

Materials and methods

Validation

Three identical foam pelvic models (Pacific Research Laboratories, Vashon, WA) were used for the experiment (Figure 1). Aluminum Philips-type screws were placed in 12 locations using a power drill. The outer diameter of the screw heads was 9 mm and they accommodated a 5-mm screwdriver head. The locations of the screws were the anterior superior iliac spine (ASIS), the anterior, superior, posterior and inferior acetabulum, and the pubic tubercles. All screws were placed bilaterally, and their exact position varied from pelvis to pelvis to allow a range of distances to be measured. All three pelvic models were then placed on a coordinate measurement machine (CMM) (Mitutoyo model BRT-504, measuring head model M11820, Aurora, IL). The coordinates of the central portion of each screw head were determined in a 3D coordinate system. Distances between points for a total of 45 different lengths for each of the pelvises were determined (a total of 135 measurements). The distances were calculated using the Pythagorean theorem where the distance D between two points with x, y and z coordinates is equal to the square root of the sum of the squares of the displacements on the x, y and z axes:Angular measurements were calculated for a total of 19 angles per pelvis (57 angles in total) by selecting the angle between two points with a third point as a vertex. All calculations were performed with a computational software program (Matlab, Mathworks, Natick, MA).

Figure 1. Anteroposterior (a) and lateral (b) views of a plastic pelvic model with aluminum screws inserted in predetermined anatomical points around the pelvis and acetabulum.

Manual linear and angular measurements

A standard 18-inch dial caliper (Aerospace) (Figure 2) was used to measure the selected distances between the screw heads. The measurements were converted to the metric system using a conversion of 25.4 mm per inch. The angles between the screw heads were measured with a manual goniometer. In some cases, it was not possible to measure the angles between the three screws with the goniometer if the line transected the pelvic model. In these cases, a best estimate was made with the manual goniometer.

Figure 2. Dial caliper used for manual measurements of point-to-point distances.

Computer-generated models

CT scans of the pelvic models were performed with 1-mm slices on a spiral CT scanner (Lightspeed QX/i, General Electric, Waukesha, WI). Image files in DICOM format were entered into the Mimics software program. Three-dimensional models of each of the pelvises were obtained using the Mimics software. The screws were identified on the CT images and were segmented as a separate layer (Figure 3).

Figure 3. Anthropometric analysis view generated in Mimics software package with screw heads segmented in red and pelvis segmented in green. [Color version available online.]

A special module of the software program allows linear and planar distance and angular measurements. This “anthropometric” module was used to measure all linear and angular measurements as described above (a total of 135 linear and 57 angular measurements).

Accuracy measurements

The accuracy of the computer-generated and manually measured distances and angles was determined by finding the difference between each measured value and that of the CMM data using millimeters for distances and degrees for angles. All angular and linear measurements were performed by three different examiners who were familiar with both the manual and computer-generated techniques.

Assessment of planar angular measurements

Simple 3D polygonal structures with known angular relationships were made in a computer-assisted design (CAD) software program (Solid Edge, UGS, Plano, TX). These 3D structures were imported into the Mimics software program. Using the Mimics anthropometric module, the planar surfaces of the 3D structures were established and the angles between the planes measured (Figure 4).

Figure 4. Polygonal structure generated in CAD software package and imported into Mimics with subsequent validation of interplane angular measurements. [Color version available online.]

Statistical analysis

Statistical analysis was performed using two-way factorial ANOVA to evaluate the linear and angular deviation from the CMM measurements. Statistical significance was set at p = 0.05. Post-hoc testing was performed using the Bonferroni-Dunn correction. The effect of the observer and the assessment technique were evaluated using this protocol. Interobserver reliability for measurements performed both manually and using the computer software was determined by calculation of intra-class correlation coefficients. All calculations were performed using standard statistical software (StatView, SAS Institute, Inc., Cary, NC, and SPSS v. 9.0, SPSS Inc., Chicago, IL).

Results

The average distance between screws was 126.9 mm (range: 33 to 260 mm). The average angular measurement was 98° (range: 18 to 180°).

Linear measurements

The mean difference between the CMM distances and those measured manually with a dial caliper was 2.12 ± 1.20 mm, and the mean difference between the CMM distances and the distances measured in the software was 1.57 ± 1.05 mm. The mean difference between the CMM angular measurements and the angular measurements made with a goniometer was 4.07 ± 4.70°, while the mean difference between the CMM angular measurements and the computer-generated angular measurements was 1.62 ± 1.32°. Based on the ANOVA, both the length and angular measurements with the manual technique were significantly less accurate than the computer-generated values (p < 0.0001 for both) (Figures 5 and 6). The effect of the evaluator was not significant for either the length or angular measurements. Intra-class correlation coefficients demonstrated an extremely high degree of interobserver reliability for the measurements of linear angles and linear distances using both manual and software measurement techniques (Table I).

Figure 5. Two-way ANOVA table for linear displacement with effect of observer and measurement technique (software versus manual) (error bars = 95% confidence interval).

Figure 6. Two-way ANOVA table for angular displacement with effect of observer and measurement technique (software versus manual) (error bars = 95% confidence interval).

Table I.  Intra-class correlation coefficients for manual and software measurements of linear distances and linear angles.

Measurement type	Parameter	Range
Software	Linear distance	0.9998–0.9999
Manual	Linear distance	0.9998–0.9999
Software	Linear angle	0.9984–0.9989
Manual	Linear angle	0.9844–0.9907

Assessment of planar angular measurements

Of the six polygonal structures imported into the Mimics program, the anthropometric analysis in Mimics demonstrated 100% accuracy in determination of the planar angles (Table II).

Table II.  Expected and measured interplane face angles of polygons imported into the Mimics software package.

Structure	Expected interplane angle (°)	Measured interplane angle (°)
Prism	60	60
Cube	90	90
Pentagon	108	108
Hexagon	120	120
Octagon	135	135
Decagon	144	144

Discussion

In this study, we compared linear distance and linear angular measurements obtained with hand instruments to those obtained with a computer-generated reconstruction. Our four hypotheses outlined previously were supported. The computer-generated distance and angular measurements were both within 2 mm or 2° of the “gold standard”, i.e., the measurement obtained using a coordinate measurement machine. The computer-generated reconstructions were also found to be more accurate for both distance and angular measurements than the manual measurements with a dial caliper or goniometer. The effect of the evaluator was not significant. Furthermore, there was a high degree of reliability between observers for both linear and angular measurements as well as for both manual and computer-generated measurements. To determine the accuracy of planar angular measurements, we imported known polygonal structures into the Mimic software and measured the angles between adjacent sides. We demonstrated 100% accuracy of the software program using this technique. This information is valuable in providing surgeons with confidence that computer-generated measurements, even when performed on 3D reconstructions, are more accurate than the use of even the most accurate and precise manual instruments. Computer-generated measurements provide additional value in their ability to obtain measurements regardless of what vital structures may lie in the measurement path and preclude manual measurements.

Previous studies on the validation of computer-generated models have focused on volumetric measurements [8], [12], [14–16]. We know of no published study that has validated the linear and angular measurements obtained between points in the Mimics software package. Furthermore, we know of no other study validating the angular measurements between planes obtained using Mimics.

This technology is well suited to further anatomical studies on skeletal structures such as the human pelvis. Based on the results of this study, kinematics of other structures such as the wrist joint can now be analyzed with confidence on the basis of linear as well as angular measurements [17], [18]. The use of rapid prototypes based on CT data has been widespread in orthopaedic surgery, mainly in the manufacture of custom implants in cases of major bone loss [19], [20] or in the treatment of anatomical locations having an irregular surface contour such as the femoral trochlea [21]. Additionally, this technology has been used in the pre-bending of plates for acetabular fixation, as well as in the manufacture of custom drill guides for placement of pedicle screws [22].

Limitations of the present study include the relatively small number of linear distances and angles measured (135 linear distances and 57 angles). Additionally, we used a dial caliper with an analog readout, which may have been less accurate than a digital caliper. The goniometer used for manual angular measurements was limited structurally by lines passing through the pelvis for some point combinations, while the computer software had no such limitations. Finally, the 3D polygons imported into the Mimics software package had face angles that varied in only one dimension, i.e., all planes were perpendicular to the base of the polygon. The accuracy of complex inter-plane angles that vary in more than one dimension cannot be confirmed by our methodology.

This work has implications for a broad range of future research applications. Our understanding of anatomy is evolving beyond descriptions of relative locations of anatomical structures into the realm of quantitative measurements of the size, location, and orientation of these structures. The foundation for all such future investigations rests in a validated, reliable computer-generated 3D software environment. For example, in the hip and pelvis, a number of underlying diseases such as femoroacetabular impingement (FAI) [23] and hip dysplasia [24–26] have been associated with osteoarthritis. These disorders are associated with abnormalities of acetabular coverage as well as acetabular version [27]. The ability to perform angular measurements between planes in 3D CT reconstructions of normal subjects will facilitate the establishment of normal anatomical values for acetabular version and coverage. We have established a methodology for performing such measurements by determining planes that run between the anterior and posterior wall of the acetabulum at various levels and are perpendicular to the floor of the pelvis. The angular difference between these planes and the sagittal plane is the acetabular anteversion at that specific level of the acetabulum (Figure 7) [28]. We have used this technique to superimpose a sphere onto the acetabulum with a total of seven levels, the fourth being the equator of the acetabulum. By measuring the anterior and posterior acetabular wall junctions with each of these levels, a quantitative measurement of acetabular version at each level can be achieved. Using this quantitative information, corrective osteotomies such as the Bernese periacetabular osteotomy can be performed precisely in the software environment (Figure 8) in such a way as to optimize the positional parameters of the acetabulum, restore normal anatomy, and presumably prevent the development of osteoarthritis. These same protocols can be applied to a variety of other anatomical locations including the cranium, orbits, and spine, as well as other joints throughout the body.

Figure 7. Clinical application of the validated interplane measurement protocol to determine variations in acetabular anteversion in a 3D CT-generated computer model of the pelvis. (a) Low magnification view. (b) High magnification view of right acetabulum demonstrating four anteversion planes as measured at different vertical positions of the acetabulum. The superior aspect of the acetabulum is divided into four vertical levels with the lowest level as the equator. Acetabular version at level 1 (green, long black arrow) shows retroversion of 7.75°; level 2 (white, small black arrowhead), level 3 (yellow, large white arrowhead), and level 4 or the equator (red, large white arrow) all demonstrate anteversion of the acetabulum of 19.47°, 21.52°, and 13.55°, respectively. [Color version available online.]

Figure 8. Presurgical planning and performance of a Bernese periacetabular osteotomy on a 3D model. This technique permits preoperative optimization of anterior and lateral acetabular roof coverage, and medialization with detection of bone interference. [Color version available online.]

Acknowledgments

The authors wish to thank Prof. Kazuo Yamazaki of UC Davis School of Engineering for the use of the coordinate measurement machine, Michael Akahori for assistance with the CMM machine, and Leo Palaima, the UC Davis Engineering School Shop Manager.