Three-dimensional reconstruction of extremity tumor regions by CT and MRI image data fusion for subject-specific preoperative assessment and planning

Abstract

This study was conducted to demonstrate the feasibility of three-dimensional (3D) reconstruction of extremity tumor regions for patient-specific preoperative assessment and planning by using CT and MRI image data fusion. The CT and MRI image data of five patients with solid tumors were fused to construct 3D models of the respective tumor regions. The reconstruction time and image fusion accuracy were measured, and the tumor features and spatial relationships were analyzed to enable subject-specific preoperative assessment and planning as guidance for tumor resection. The 3D models of the tumor regions, including skin, fat, bones, tumor, muscles, internal organs, nerves and vessels, were created with a mean reconstruction time of 103 minutes and fusion accuracy of 2.02 mm. The 3D reconstruction clearly delineated the tumor features, and provided a vivid view of spatial relationships within the tumor region. Based on this intuitional information, the subject-specific preoperative assessment and planning were easily accomplished, and all tumor resections were performed as planned preoperatively. Three-dimensional reconstruction using CT/MRI image fusion is feasible for accurate reproduction of the complex anatomy of the tumor region with high efficiency, and can help surgeons improve the preoperative assessment and planning for effective removal of tumors.

• Extremity tumor
• tumor region
• image fusion
• preoperative assessment
• preoperative planning

Introduction

Extremity tumors usually invade or abut multiple important anatomical structures due to their often large size [1]. The anatomical relationship between a tumor and peritumoral structures is complex, and its three-dimensional (3D) orientation is conceptually difficult to comprehend. Furthermore, features of the tumor, such as its shape, size and extent, are also characterized by 3D representation in the tumor region (also called the “region of surgical interest”). Therefore, the tumor features and relationships with surrounding structures must be firmly understood in order to effectively remove a tumor with disease-free margins and optimal preservation of peritumoral structures. To achieve this goal, detailed preoperative assessment and planning is necessary, and is accomplished by resorting to different forms of image data, especially computed tomography (CT) and magnetic resonance imaging (MRI) images [2–4].

CT and MRI have been widely used to provide anatomical information on the tumor region to facilitate preoperative assessment and planning [5–7]. However, neither CT nor MRI alone can provide excellent tissue contrast for each tumor-related structure. Image fusion combines the different sensitivities of CT and MRI images and offers surgeons more accurate information on the exact geometric and volumetric relationships between the tumor and peritumoral structures [8]. However, this information is two-dimensional (2D) in nature, and the surgeon must mentally integrate this 2D information into a 3D model before surgery [9]. This requirement for subjective mental integration can be avoided by using a preoperatively constructed 3D model of the tumor region that permits direct visualization.

In actuality, 3D reconstruction is a software-dependent image post-processing technique. CT- or MRI-based 3D reconstruction is limited to simultaneously reconstructing bony and non-bony structures in the tumor region. In contrast, CT/MRI image fusion can provide not only 2D fused images, but also hybrid 3D models assembled from CT- and MRI-based 3D models. To date, however, more emphasis has been placed on 2D fused images for clinical applications, especially for intraoperative navigation in surgery [8–10]. In contrast, few studies have focused on 3D reconstruction to improve the understanding of tumor anatomy and surgical planning, and, to our knowledge, there has been no previous report on reconstructing a 3D model of an extremity tumor region for subject-specific preoperative assessment and planning using the CT/MRI image fusion method. It is possible that the time-consuming reconstruction process and assumed unacceptable fusion error account for the limitation of its use in clinical applications.

The main purpose of this study was to demonstrate the ability to reconstruct complicated 3D extremity tumor regions based on the CT/MRI image fusion method, and also to display the value of visualized spatial information concerning tumor features and relationships in the tumor region for subject-specific preoperative assessment and planning. The 3D reconstruction time for each tumor was measured, and accuracy evaluation of the image fusion was also performed.

Materials and methods

Five patients with proven tumors, as shown in Table I, were included in this retrospective study. The study was approved by the institutional review board, and informed consent was obtained from the participating patients and their families. According to the patient's acceptance and treatment requirements, imaging examinations were conducted with CT (Brilliance 16; Philips, Cleveland, OH), MRI and magnetic resonance angiography (MRA) (Signa 1.5 T; GE, Milwaukee, WI). For each patient, the scanning parameters for CT, MRI and MRA were adjusted based on the tumor size to obtain clearer visualization of the structures of interest (Table II). During the examinations, all patients were in the supine position.

Table I.  Patient characteristics

No.	Age	Sex	Months after tumor onset	Tumor location	Histology
1	51	M	10	Right sacroiliac region	Chondroid chordoma
2	53	F	13	Right gluteal region	Well-differentiated liposarcoma
3	32	F	8	Right posterior pelvic region	Sciatic nerve sheath tumor
4	75	F	7	Right proximal thigh region	Well-differentiated liposarcoma
5	36	M	15	Left knee joint	Cyst

Table II.  Acquisition protocols and scanner specifics

Patient No.	Scanner	Voltage (kV)	Current (mAs)	Slice thickness(mm)	Matrix	Pixel (mm)	FOV (cm)	WI	Sequence	TR (ms)	TE (ms)	TNI
1	CT	120.0	266.3	1.3	512 × 512	0.773	39.6					91
	MRI			5.0	512 × 512	0.781	40.0	T2WI	SE	4000.0	88.1	34
2	CT	140.0	354.0	1.25	512 × 512	0.703	36.0					153
	MRI			5.0	512 × 512	0.840	43.0	T2WI	SE	8447.4	62	32
	MRA			1.5	704 × 704	0.568	40.0		3D TOF- SPGR	5.1	1.7	200
3	CT	120.0	168.0	1.5	512 × 512	0.641	32.8					143
	MRI			2.5	256 × 256	1.543	39.5	T2WI	SE	4000.0	88.1	32
4	CT	120.0	269.0	1.5	512 × 512	0.750	38.4					68
	MRI			5.0	528 × 528	0.718	37.9	T1WI	SE	500.0	20.0	32
5	CT	120.0	234.0	1.25	512 × 512	0.763	24.0					46
	MRI			2.5	512 × 512	0.352	28.0	T1WI	SE	342.0	17.6	34

FOV: field of view; WI: weighted image; SE: spin echo sequence; 3D TOF-SPGR: three-dimensional time-of-flight, fast spoiled gradient-recalled sequence; TR: time of repetition; TE: time of echo; TNI: total number of images.

Next, the volumetric image data from the CT and MRI(A) in DICOM file format were imported into Mimics 10.01 software (Materialise, Leuven, Belgium). In this software, the structures of interest were first segmented on the 2D images. CT images were used to segment bony structures because they had much higher CT values than non-bony structures, and the software could automatically extract their contours. MRI(A) images were used to segment non-bony structures, and careful manual editing was performed to delineate their boundaries due to the relatively low tissue contrast with the surrounding structures on MRI(A) images. Each structure was segmented and stored in a specific file (a so-called “mask”) to retain its anatomical information. After all the structures had been segmented on their 2D images, the 3D reconstruction was calculated. Finally, the computerized color 3D model for each structure was obtained. The 3D models of bony structures were developed from CT images, and the 3D models of non-bony structures were reconstructed from MRI(A) images (Figure 1). It was generally considered that the reconstruction time for the 3D model of a tumor region greatly affected its clinical applications, and that the time should be less than 4 hours [11]. Therefore, the segmentation time of the structures of interest was measured for each patient in this study.

Figure 1. Screen captures of the Mimics software showing the interfaces for reconstructing 3D models of the the structures of interest. (A) Bony structures reconstructed from CT images. (B) Non-bony structures reconstructed from MRI images.

An image-to-image registration between CT and MRI(A) image data for each patient was subsequently performed to correlate the anatomical information of the structures of interest from these complementary data sets. This registration allowed CT and MRI(A) images to be automatically fused by matching the segmented structures through the software. The reconstructed 3D structures from CT and MRI(A) images were further fused according to their in vivo spatial positions. Considering the different scanning parameters of CT and MRI(A) images, we preferred to use internal anatomical landmarks as registration reference points to determine the geometric transformation for the optimal image fusion [12]. At least six anatomical landmarks which were simultaneously visible on the corresponding CT and MRI(A) images were manually chosen to align the CT and MRI(A) images. Of these landmarks, bone islands and the femoral head centers were the most frequently chosen points. Based on the anatomical landmarks, which were automatically numbered by the software in the order in which they were chosen, the image fusion between two image data sets could be achieved (Figure 2).

Figure 2. Anatomical landmarks chosen during CT and MRI image fusion. (A) and (C) are CT images, (B) and (D) are MRI images, and (E) and (F) are fused CT/MRI images. p1 and p2 are the right iliac bone islands, p3 and p4 the right and left femoral head centers, and p5 and p6 the right and left external iliac artery centers. According to these corresponding landmarks on the CT and MRI images, all images are aligned with one another in a single fused image dataset.

Because only CT and MRI image data were available for patients 1, 3, 4 and 5, an image fusion between CT and MRI was necessary. In an attempt to perform multiple image fusions and present more spatial information regarding the peritumoral vessels, MRA images together with CT and MRI images were obtained for patient 2. Image fusion was performed twice: first between MRI and MRA, resulting in fused MRI/MRA images, and then between CT and the fused MRI/MRA.

The assessment of the image fusion accuracy was verified on the fused 2D images by using the contour overlay method: typically, bony surface contours served as good landmarks (Figure 3). The fused 2D images were initially assessed visually to confirm that the fusion had been successful. On each fused image, we measured the largest deviation of the contours of the same bone and took this as the image fusion accuracy for this image. As shown in Figure 3C, the measurement method was demonstrated on a fused image with large deviation. The measured values for all the fused images were then summed and averaged to obtain the mean value, which was considered to be the mean fusion accuracy of the image fusion. In addition, the mutual fitness relationship between 3D models was allowed to further assess the fusion accuracy (Figure 3). Here, the fusion accuracy of the 3D models was equivalent to that of the 2D images, because the 3D models were computed from the 2D images by the software. If the initial image fusion yielded a result with poor accuracy (mean fusion accuracy ≥ 2.5 mm), the landmarks were re-selected until a satisfactory fusion was achieved. In most instances, the fused images were satisfactory on the initial attempt. Because the image registration time was an important index for evaluating the efficiency of the adopted registration method [9], it was also measured for each patient in this study.

Figure 3. Assessment of image fusion accuracy. One method of assessment is to assess the overlay degree of the bone contours on the fused 2D images: (A) CT image: the black arrow shows the iliac contour; (B) MRI image: the white arrow shows the iliac contour; (C) The fused image with large deviation: horizontal arrows show that the iliac contours are not well overlaid and the vertical arrow shows the largest deviation of the iliac contours; (D) The accurately fused image: arrows show well-overlaid iliac contours. Another method is to assess the mutual fitness relationship between 3D models: (E) 3D tumor model: the arrow shows the vascular channel on the tumor surface; (F) 3D models of the tumor and peritumoral vessels assembled by the accurate image fusion: the arrow shows that one branch of the superior gluteal vessel only runs through the vascular channel. Fusion accuracy as in (D) and (F) is the satisfactory result for each image fusion.

Following the fusion of the 2D CT and MRI(A) images, the MRI(A)-based 3D models were incorporated directly into the CT-based 3D models, so an entire 3D model of the tumor region was generated to supply the visualized anatomical information. Tumor features such as shape, size, location, extent of invasion and vascular supply were analyzed in the preoperative assessment. The spatial relationships between the tumor and peritumoral structures were also carefully examined.

As for preoperative planning, these 3D models were used to simulate the actual surgical procedures in the software. First, models of each patient were inspected from the direction of the planned surgical approach, and along this trajectory structures were removed layer by layer from superficial to deep until the tumor was exposed. Then, different tumor stripping methods were tried in order to achieve full exposure of peritumoral structures. Marginal resection was planned for the chondroid chordoma in patient 1, and the planned resection margin was determined as 2 mm from the tumor edges [13]. Finally, after removal of the tumor, 3D models of peritumoral structures were again inspected by changing different angles to further perceive the structural relations. Throughout the simulation process, screenshots which recorded important anatomical views were saved and printed so as to be available in the operating room at the time of surgery.

Results

Three-dimensionsal bony structures of the pelvic bone, sacrum, coccyx, lumbar vertebrae, femur, patella, tibia and fibula were reconstructed from CT images, while non-bony structures, including skin, fat, tumor, muscles, internal organs, nerves and vessels, were reconstructed from MRI images. In patient 2, models of the pelvic vessels with their main parietal branches could also be obtained because of the high-quality MRA images (Table III). The entire reconstruction process accurately recorded the original coordinates of all the structures of interest as presented in their 2D images so that each 3D model reliably retained its anatomical features. The image fusions were successfully performed for all five patients, and the mean fusion accuracy was 2.02 mm (minimum value 1.15 mm; maximum value 2.42 mm). The total 3D reconstruction time for each patient, composed of segmentation and registration time, was determined, as shown in Table IV.

Table III.  Summary of reconstructed 3D structures of tumor region

	Reconstructed 3D structures
Patient no.	CT	MRI	MRA
1	Pelvic bone, sacrum, coccyx, lumbar vertebra (L5), femur	Tumor, subcutaneous fat, gluteus, iliacus, great psoas muscle, piriformis, bladder, prostate gland, rectum, external and internal iliac vessels
2	Pelvic bone, sacrum, coccyx, lumbar vertebra (L5), femur	Tumor, skin, subcutaneous fat, gluteus, piriformis, obturator internus, erector spinae, sciatic nerve	Main parietal branches of pelvic vessels
3	Pelvic bone, sacrum, coccyx, lumbar vertebra (L4,5), femur	Tumor, gluteus, piriformis, obturator internus, uterus, bladder, intestine, sciatic nerve, external and internal iliac vessels
4	Pelvic bone, sacrum, coccyx, femur	Tumor, subcutaneous fat, gluteus, tensor fasciae latae, rectus femoris, sartorius muscle, femoral vessels
5	Femur, patella, tibia, fibula	Tumor, skin, subcutaneous fat, vastus muscle, sartorius muscle, gracilis, semitendinosus, semimembranosus, biceps femoris, gastrocnemius, popliteal vessels

Table IV.  Summary of 3D reconstruction time (in minutes).

	Segmentation		Registration time
Patient no.	Time	No. of structures	MRI/ MRA	MRI/ CT	Total reconstruction time
1	115	22		11	126
2	102	21	5	8	115
3	92	20		9	101
4	85	21		5	90
5	77	18		4	81

Following the accurate image fusion, hybrid 3D models of structures of interest were created, and for each patient the entire 3D model of the tumor region was obtained. Moreover, to serve as a normal anatomical reference, the 3D model of the contralateral healthy anatomical region was also reconstructed. In different tumor regions various structures were included based on the tumor location (Table III). We could perform several manipulations in real time, including 360° rotations in any axis, cuts in any plane, and measurements of tumor size and volume in 2D or 3D configuration. Each 3D model of the tumor region allowed for individual structures to be added and removed independently. Furthermore, any structure in the model could be made semi-transparent and displayed in different colors, allowing for deeper relationships to be observed. Finally, the unique feature of the 3D model was that it allowed the simultaneous display of the 2D fused images in conjunction with the 3D model. In the tumor region, features of the tumor, including its shape, size, location, extent of invasion and (for patient 2 only) vascular supply, and its relationships with peritumoral structures were analyzed for subject-specific preoperative assessment as follows:

Patient 1. This was a male patient with a chondroid chordoma (Figure 4). The tumor appeared lobulated, and measured 8.8 × 8.4 × 6.1 cm with a volume of 206 ml. The tumor was located in the right sacroiliac region with an ill-defined margin. Superiorly, it extended to the proximity of the 2nd sacral vertebra; inferiorly, it extended into the greater ischiadic foramen at the level of the distal end of the 4th sacral vertebra. The tumor had invaded the sacral wing, the ipsilateral sacral foramen and the middle of the sacrum, without extending to the contralateral sacral foramen, and had partially destroyed the right iliac wing, while the greater ischiadic foramen was expanded. Posteriorly, the tumor neighbored the right gluteus maximus and medius, and anteriorly abutted the bladder, the right internal iliac vessels and the piriformis, where muscle distortion caused by the tumor was observed. Medially, the tumor adjoined the rectum, and laterally the right ilium and gluteus minimus.

Figure 4. 3D reconstruction of the tumor region for patient 1. (A) Superior view of the entire 3D model of the tumor region: “T” indicates the chordoma (in white). (B) The sacrum: the arrow shows the bony destruction. (C) Superior view with some structures cut away to show the spatial relationships between the tumor and peritumoral structures: (1) bladder; (2) rectum; (3) gluteus maximus; (4) gluteus medius; (5) piriformis; (6) right internal iliac vessels. “S” indicates the sacrum, “I” the ilium. (D) Posterior view. In this 3D tumor region the tumor features and relationships to peritumoral structures are clearly depicted.

Patient 2. This was a female patient with a well-differentiated liposarcoma (Figure 5). The tumor was ovoid, and measured 16.8 × 9.1 × 5.1 cm with a volume of 272 ml. The tumor was posteriolaterally located in the patient's right buttock, between the right gluteus maximus and medius with a well-defined margin. The blood supply to the tumor was mainly from the right superior gluteal artery, which had a communicating branch with the lateral femoral circumflex artery. The communicating branch anteriomedially descended and divided into three small branches above the femoral greater trochanter, yielding a vessel anastomosis with the corresponding branches of the lateral femoral circumflex artery. In addition, some small branches of the internal iliac artery joined the superior gluteal artery to participate in the blood supply (Figure 6). Compared with the left corresponding vessels, the right superior gluteal vessels were highly irregular, tortuous, hypertrophic and hypervascular. No communicating branch between the left superior gluteal artery and lateral femoral circumflex artery was found.

Figure 5. 3D reconstruction of the tumor region for patient 2. (A) Superior view of the entire 3D model of the tumor region. (B) Superior view with some structures cut away to show the spatial relationships between the tumor and peritumoral structures: (1) gluteus maximus; (2) gluteus medius; (3) gluteus minimus; (4) superior gluteal vessels. (C) Posterior view with some structures removed: (5) piriformis; (6) inferior gluteal vessels. (D) Oblique view accompanied by an axial slice of the fused images. Based on the 3D reconstruction, the shape, size, location and extent of the tumor are depicted in detail.

Figure 6. 3D model of the tumor blood supply. (A) The MRA appearance of pelvic vessels. (B) The 3D models of the pelvic vessels, tumor and bones. (C) The MRA appearance of the tumor blood supply from small branches of the internal iliac artery (1), superior gluteal artery (2), and a communicating branch of the lateral femoral circumflex artery (3). (D) The 3D models of the tumor blood supply, where (1), (2) and (3) are the same vessels shown in (C). The 3D model accurately reproduces the in vivo tumor-related vessels, and additionally demonstrates the spatial relationships between the tumor and peritumoral vessels.

Patient 3. This was a female patient with a sciatic nerve sheath tumor (Figure 7). The tumor was spherical, and measured 3.4 × 3.1 × 2.9 cm with a volume of 21 ml. This tumor arose from the right sciatic nerve with a well-defined margin and expanded into the greater ischiadic foramen. Anteromedially, the tumor neighbored the right inferior gluteal vessels and uterus; laterally, it was in contact with the right piriformis, and posteriorly it abutted the rectum.

Figure 7. 3D reconstruction of the tumor region for patient 3. (A) Tumor and peritumoral muscles and vessels: “T” indicates the tumor. (B) Posterior view with some structures cut away to show the spatial relationships between the tumor and peritumoral structures: (1) rectum; (2) uterus; (3) inferior gluteal vessels; (4) sciatic nerve; (5) gluteus maximus; (6) piriformis; (7) superior gluteal vessels. “S” indicates the sacrum, “I” the ilium. The visualized information offered by the 3D model of the tumor region can facilitate the preoperative assessment.

Patient 4. This was a female patient with a well-differentiated liposarcoma (Figure 8). The fusiform tumor measured 10.7 × 6.1 × 5.5 cm with a volume of 195 ml. This tumor lay beneath the subcutaneous fat with a well-defined margin and compressed the tensor fasciae latae into a thin muscular shell. Medially, the tumor neighbored the tensor fasciae latae, rectus femoris and sartorius muscle; posteriorly, it abutted the gluteus medius.

Figure 8. 3D reconstruction of the tumor region for patient 4. (A) Inferior view of the entire 3D model of the tumor region: “T” indicates the tumor. (B) Superior view with some structures cut away to show the spatial relationships between the tumor and peritumoral structures: (1) tensor fasciae latae; (2) subcutaneous fat; (3) sartorius muscle; (4) rectus femoris; (5) gluteus medius. This 3D virtual information can enable surgeons to conceptually understand the tumor characteristics in the tumor region.

Patient 5. This was a male patient with a knee cyst (Figure 9). The tumor was lobulated, and measured 7.5 × 5.8 × 2.1 cm with a volume of 43 ml. As with patient 4, this tumor also lay beneath the subcutaneous fat. Anteriorly, it neighbored the medial vastus muscle, and posteriorly it abutted the sartorius muscle and gracilis. Medially, the tumor was in contact with the femoral condyle.

Figure 9. 3D reconstruction of the tumor region for patient 5. (A) Superior view of the entire 3D model of the tumor region: “T” indicates the tumor. (B) Superior view with some structures cut away to show the spatial relationships between the tumor and peritumoral structures: (1) sartorius muscle; (2) gracilis; (3) popliteal vessels; (4) vastus medialis; (5) subcutaneous fat. “F” indicates the femur. The relationships between the tumor and peritumoral structures are easily obtained in three dimensions.

The subject-specific surgical planning for each patient could be simulated by the software. Proceeding along the planned surgical approach, with the removal order of different structures progressing from superficial to deep, the surgeons gradually became familiar with the anatomical layers around the tumor. The tumor and peritumoral structures were clearly demonstrated and the tumor features and spatial relationships could again be verified. Different tumor stripping methods could be rehearsed prior to surgery to determine which method could fully expose and safely manage the peritumoral structures. For example, for patient 2 we made sure that if the tumor was stripped from the lateral side the tumor-nourishing vessels could be easily exposed and ligated (Figure 10). For representative patient 1 with a malignant bone tumor, by inspecting the 3D model the spatial intraosseous and extraosseous extensions of the chordoma were clearly delineated, and the safety margins of the sacral and iliac osteotomy could be approximated as being 2 mm from the tumor edges. We could also move the resected tumor and osteotomized bone fragments to double-check how the tumor related to the surrounding structures (Figure 11).

Figure 10. Preoperative surgical planning for patient 2. (A) The entire 3D model of the tumor region. (B) and (C) The 3D model with the skin, subcutaneous fat and gluteus maximus removed, respectively, along the surgical approach. (D) The simulated tumor stripping method from the lateral to the medial side can fully expose the superior and inferior gluteal vessels, the communicating branch (white arrow) and the sciatic nerve. (E) The stripping method from the medial side limits the complete exposure of the aforementioned structures. (F) The tumor is moved laterally to double-check the anatomical relationship. The simulated surgical planning can give the surgeons a firm understanding of the anatomical layers around the tumor and make them more confident during the actual operation.

Figure 11. Preoperative surgical planning for patient 1. (A) The entire 3D model. (B) and (C) The 3D model with the skin, subcutaneous fat and gluteus maximus removed, respectively, along the surgical approach. (D) Preoperative osteotomy simulation with 2-mm safety margins from the tumor edges. (E) The resected tumor (in white) and osteotomized bone fragments from the sacrum and ilium. (F) The tumor and bone fragments are moved laterally to check the peritumoral structures again. The osteotomy simulation enables the surgeons to accurately define the osteotomy margins and safely resect the tumor at the time of surgery.

The saved screenshots were available in the operating room at the time of surgery as paper prints, and the reconstructed 3D models were also viewable on a computer monitor, where they could be manipulated to help the surgeons localize the tumor and other important structures if necessary. Compared to 2D images, the 3D models proved to be more intuitively useful when stripping the tumor and when assessing the tumor from areas in contact with vital structures. Based on the preoperative surgical planning simulated by the software, all the patients underwent effective tumor resections with tumor-free margins as planned (Figure 12), and the time required for procedures was reduced, especially the time for the intraoperative localization of structures of interest to the surgeons.

Figure 12. Intraoperative images of patient 1. (A) The posterior surgical approach. (B) and (C) The exposed tumor and peritumoral structures: “T” indicates the chordoma, “S” the sacrum, “G” the gluteus maximus, and “F” subcutaneous fat. The actual surgery faithfully reproduces the preoperative assessment and surgical planning presented by the 3D model.

Discussion

The tumor region has a complex spatial structure, of which only parts are visualized on sectional images. Despite the availability of 2D CT and MRI images with high resolution, they cannot directly provide surgeons with a view of the 3D tumor anatomy. A 3D model of the tumor region, in which the structures of interest are spatially represented, improves the vividness of the depiction [14]. We adopted a multimodality image fusion method for accurate 3D reconstruction of extremity tumor regions with regard to tumor features and spatial relationship observations, and this reconstruction facilitated the surgeons’ conceptual understanding of the tumor characteristics and enabled subject-specific preoperative assessment and planning. A reconstruction time of less than 2 hours is clinically acceptable.

To obtain the tumor-specific 3D model, CT and MRI images, which represent the gold standard in extremity tumor imaging for depicting pathological anatomy, are necessary. However, each imaging modality has distinct sensitivities for different tissues: CT scans show exquisite bony details, whereas MRI is superior to CT in delineating soft tissue [8]. Neither of them alone can simultaneously depict bone and soft tissue in detail, but CT and MRI scans are complementary in the diagnosis of neoplastic disease. The image fusion combines the advantages of both CT and MRI, and offers surgeons more accurate information on the exact geometric and volumetric relationship between the soft tissue structures and bony structures [15]. The CT/MRI image fusion offers three key advantages: (1) fused 2D images to provide anatomical information on the bony and non-bony structures; (2) a practical tool for accurate localization of the target tumor for real-time intraoperative navigation; and 3) a useful tool for fusing CT- and MRI-based 3D models to recreate the complex 3D tumor region for preoperative assessment and planning. At present, the first two advantages are widely generalized with broad acceptance [4], [16–18]; however, too little emphasis has been placed on the third advantage.

It is possible that the time-consuming reconstruction process and assumed unacceptable fusion error have restricted the clinical application of 3D reconstruction. In fact, the segmentation time accounts for the majority of the total reconstruction time (Table III), though it is inversely proportional to increasing reconstruction experience. For the tumor of patient 3, which was nearly the same as that of patient 1, the time was reduced by 23 min with the accumulation of segmentation experience. The mean time for image registration was approximately 8.4 min. This time is less than the 13.6 min reported by Wang et al. [9]. Different image registration methods may account for this discrepancy: our study was based on anatomical landmarks to fuse images [12], while Wang et al. used an automatic registration method; however, the final registered images sometimes required use of our method to adjust the fusion accuracy. In this study, we had a mean reconstruction time of 103 minutes. Moharir et al. [11] reconstructed 3D head and neck tumor regions, and for each 3D model a time of 12 hours was required. Based on their experience, a total reconstruction time of less than 4 hours was a realistic goal for clinical applications. Although we have reconstructed extremity tumors, as opposed to head and neck tumors, more structures are included in our study and our reconstruction time is considerably shorter than 4 hours. As there are no existing papers on the reconstruction of 3D extremity tumor regions as performed in our study, we have no control group for comparison. However, compared to 12 hours, our reconstruction time of 103 minutes is clinically acceptable. Comparisons between studies are complicated by differences in study conditions and methods for evaluating the image fusion accuracy. To date, there have been no studies reporting image fusion with 100% accuracy due to the different scanning parameters of CT and MRI(A). Here we used the mean fusion accuracy as a standard to compare this study to others. Many studies have shown accurate CT/MRI image fusions with a fusion accuracy of 1.25–2.0 mm in neurosurgery [8], [10], [12], [16]. Our mean fusion accuracy of 2.02 mm would seem to indicate that the image fusion performance of this study was relatively “inaccurate”. In fact, to avoid additional exposure or discomfort for the patients and reduce imaging costs, we obtained routine CT and MRI images (Table II). Furthermore, due to the large size of extremity tumors, we cannot obtain MRI images as thin as the 1- or 2-mm slices required for optimal image fusion [9], and the MRI image thickness (≥2.5 mm) in this study affected the image fusion performance. On the other hand, for relatively small brain tumors, image fusions with accuracy of 1.25–2.0 mm can be widely used in intraoperative navigation. However, considering the large volumes of the tumors studied, this 2.02-mm fusion error for preoperative assessment and planning is clinically negligible.

Our study differs from previous approaches to reconstructing 3D tumor regions which relied on either CT or MRI images alone. As mentioned previously, CT has limited soft tissue resolution, and MRI is limited in depicting bony structure. Combining the two modalities allows for accurate 3D reconstruction of bone, soft tissue, and vessels, and this method has been applied to the 3D reconstruction of head and neck tumors. In the study by Moharir et al. [11] three such tumors were accurately reconstructed for subject-specific preoperative assessment. Raappana et al. [19] reported the 3D reconstruction of a pituitary adenoma and its adjoining structures for preoperative surgical planning. For each patient, they used the 3D model as a “road map” during the operation to improve the safety and effectiveness of the surgery. Since this 3D reconstruction technique had been used successfully in complex tumors of the head and neck regions, the possibility that it might be feasible in other complex regions motivated our present study. Using the same method, we accurately reconstructed more 3D structures, including skin, fat, bones, tumor, muscles, vessels, nerves and pelvic viscera, and greatly reduced the 3D reconstruction time.

The most important advantage of this approach is that tumor-specific preoperative assessment and planning can be performed easily based on the 3D reconstruction. By examining 3D models, we can assess the tumor features which are not directly represented on 2D images. For the liposarcoma of patient 2, the tumor blood supply was accurately developed, which closely agreed with the appearance on MRA. Interestingly, the 3D model showed that the tumor was supplied by triple blood sources (Figure 6). However, it was difficult to distinguish which artery contributed to the tumor blood supply on MRA images due to the complex appearance of the vessel branches. The spatial relationships of the tumor to nearby structures of concern to the surgeons can also be clearly delineated by inspecting the 3D models. Particularly in the case of the tumors of patients 1 and 3, internal organs, bones, muscles and the internal iliac vessels surrounded the tumor from different directions, which would be difficult for the surgeons to understand conceptually by resorting to 2D CT and MRI images, whereas 3D models can provide the surgeons with the intuitional tumor orientation, making it unnecessary for them to spend a considerable amount of effort forming a mental 3D impression.

Based on the 3D reconstruction, the tumor-specific preoperative planning allows surgeons to operate in familiar territory at the time of the actual surgery [19]. We found that the preoperative planning simulated by the software was valuable for the tumor resection in all our patients as it provided better anatomical layers and stripping methods for the tumors. For example, in patient 2, we found most tumor-related vessels supplied blood from the anteromedial aspect of the liposarcoma, so the anteromedial aspect was a relatively “dangerous” region. As planned preoperatively, we stripped the tumor from the lateral side, and the branches of the superior gluteal vessels and the communicating branch could be clearly exposed and ligated, enabling easy removal of the tumor without a large hemorrhage or injury to the sciatic nerve. The 3D model of patient 1's chordoma assisted in the classification of the tumor as stage IIB [20], so a 2-mm margin of resection from the tumor edges was simulated preoperatively [13]. Marginal resection for this chordoma was successfully achieved, reliably reproducing the preoperative simulation. The subject-specific preoperative assessment and planning, based on the 3D model of the tumor region, greatly facilitated effective removal of the tumor and preservation of vital peritumoral structures during the actual surgery.

Our study has several important limitations. Firstly, although the 3D reconstruction time of 103 min is much shorter than that reported in other studies, there is room for improvement in the segmentation process to further reduce the prolonged reconstruction time. With continued improvements in the segmentation programs, this goal is attainable. Secondly, until recently, there was no imaging technique that could depict peripheral nerves with high sensitivity, so the extraction of nerve tissue depended entirely on manual segmentation, and for three patients we could not reconstruct the peritumoral nerves in three dimensions even using manual segmentation; effective reconstruction of the peripheral nerves relies heavily on the development of an appropriate imaging technique. Thirdly, only five patients were included in this study; more patients need to be recruited for a future study to further highlight the clinical significance of preoperative 3D re-construction.

In conclusion, we accurately reconstructed five complicated 3D extremity tumor regions, featuring several different types of tumors and multiple peritumoral structures, using CT/MRI image fusion, and the total reconstruction time was greatly reduced while maintaining clinically acceptable fusion accuracy. The 3D reconstruction clearly demonstrated the tumor features and spatial anatomical information in the tumor region. This 3D reconstruction method has proved helpful in improving the subject-specific preoperative assessment and planning for extremity tumors and in enabling surgeons to perform planned tumor resections effectively. We encourage surgeons to perform the 3D reconstruction process personally to improve their surgical management of complex extremity tumors.

Declaration of interest: This work was supported by a grant from the development fund of the Science and Technology of Pudong New Area, Shanghai, China (No. PKJ2008-Y07).