Quantitative analysis of velopharyngeal movement using a stereoendoscope: accuracy and reliability of range images

Abstract

Objective: We developed a novel method of producing accurate range images of the velopharynx using a three-dimensional (3D) endoscope to obtain detailed measurements of velopharyngeal movements. The purpose of the present study was to determine the relationship between the distance from the endoscope to an object, elucidate the measurement accuracy along the temporal axes, and determine the degree of blurring when using a jig to fix the endoscope. Methods: An endoscopic measuring system was developed in which a pattern projection system was incorporated into a commercially available 3D endoscope. After correcting the distortion of the camera images, range images were produced using pattern projection to achieve stereo matching. Graph paper was used to measure the appropriate distance from the camera to an object, the mesial buccal cusp of the right maxillary first molar was measured to clarify the range image stability, and an electric actuator was used to evaluate the measurement accuracy along the temporal axes. Results: The measurement error was substantial when the distance from the camera to the subject was >6.5 cm. The standard error of the 3D coordinate value produced from 30 frames was within 0.1 mm (range, 0.01–0.08 mm). The measurement error of the temporal axes was 9.16% in the horizontal direction and 9.27% in the vertical direction. Conclusion: The optimal distance from the camera to an object is <6.5 cm. The present endoscopic measuring system can provide stable range images of the velopharynx when using an appropriate fixation method and enables quantitative analysis of velopharyngeal movements.

Keywords:

• Accuracy
• craniofacial
• endoscopy
• image processing

Introduction

For a patient with a cleft palate to achieve normal speech, the palatal muscles must first be surgically reconstructed to obtain normal velopharyngeal function.[1,2] Indeed, the acquisition of normal velopharyngeal function is a large determinant of the outcome of subsequent speech therapy.[3] It is therefore important to accurately judge velopharyngeal function to ensure that the patient will achieve normal speech following the operation. Endoscopy is widely used to examine velopharyngeal function. However, evaluation of velopharyngeal incompetence using an endoscope only allows for qualitative assessment. Therefore, detailed analysis and objective evaluation of velopharyngeal movement is not currently possible.

Quantitative evaluation of velopharyngeal function using magnetic resonance imaging was recently performed.[4,5] However, magnetic resonance images are distorted if speech appliances are worn and cannot provide real-time feedback. To resolve these issues, we developed a method for accurate three-dimensional (3D) measurement of velopharyngeal movement using a stereoendoscope.

To illustrate the various aspects of velopharyngeal movement, we have been developing techniques with which to measure movements of the velopharynx in detail and methods with which to produce range images of the velopharynx using a 3D endoscope. These developments have made it possible to produce range images by incorporating pattern projection into a commercially available 3D endoscope.

Our previous study demonstrated that this method can be used to successfully produce accurate and reliable range images of the velopharynx.[6] This system provides range images of the velopharynx with a measurement error of approximately 2% for an object at 5 mm. However, this endoscopic measuring system is relatively large because the pattern projection system is incorporated into a commercially available 3D endoscope; therefore, it cannot be inserted through the nasal cavity.

To apply this measurement system to actual measurements of velopharyngeal movement, the endoscope must be inserted into the mouth. Several factors should be considered before this system can be clinically applied. For example, it is important to consider whether this method also enables accurate measurements of large and rapid movements. Additionally, the position change between the endoscope and object must be performed in a manner that avoids blurring of the endoscopic image. Therefore, the purpose of this study was to clarify the relationship between the distance from the endoscope to an object and the measurement accuracy along temporal axes as well as to elucidate the degree of blurring that occurs when using a jig to fix the endoscope.

Materials and methods

Hardware

The details of the endoscopic system are described in our previous article.[6] In brief, this endoscopic system comprises a 3D endoscope (3D-132D; Shinko Optical Co., Ltd., Tokyo, Japan), a camera control unit (3D-SP-2001; Shinko Optical Co., Ltd.), a 3D converter (SK-1057-3D-A; Shinko Optical Co., Ltd.), and a light source (CL-75X-II; Shinko Optical Co., Ltd.). Two 1/10-inch micro-charge-coupled device cameras were mounted onto the tip of the endoscope. A pattern projection system was then integrated into the commercially available endoscope. The light source was connected to the pattern, and the pattern was sent to the tip through optical fibers. Right and left camera images of the endoscope with the projected pattern were captured via an IEEE1394 firewire cable at a sampling rate of 30 frames per second after conversion from an analog to a digital signal with a converter (Canopus ADVC-110; Grass Valley Co., Ltd.; Kobe, Japan) connected to a workstation (Dell Precision T7500, CPU: Xenon 2.40 GHz, 6 core; Round Rock, TX). The 3D converter synchronized the right and left cameras of the endoscope.

Software

Because the original purpose of the 3D endoscope system was to obtain stereoscopic images, right and left images were integrated into one.avi video file as odd and even scanning lines, respectively. First, the video files were separated into right and left camera images (640 × 240 pixels for each camera video file), and interlace interpolation was then performed using linear interpolation. The distortion of the camera images was corrected using the Tsai algorithm.[7] After the camera images were rectified, the right and left images were stereo-matched, and disparities were calculated using the Birchfield technique.[8]

Determination of optimum distance from camera to object

Graph paper ruled into 1- × 1-mm squares was initially used as the test object with which to evaluate how the distance between the tip of the endoscope and the object affects the image quality. The graph paper was fixed onto a slide table of an electronic actuator (SUS Co., Ltd., Shizuoka, Japan) situated perpendicular to the axis of the endoscope (Figure 1). The distance between the slide table and the endoscope was automatically calibrated in each trial, and movement of the slide table was controlled mechanically using a personal computer. The distances from the endoscopic tip to the object were regulated every 0.5 mm from 2.0 to 8.0 cm. Range images of the graph paper were produced, and the calculated distances were compared with the true distance of 5.0 mm. The measurement error produced as a result of the difference in the distance between the graph paper and endoscope was investigated.

Figure 1. Electric actuator. The slide table position and movement speed are automatically controlled with a computer. The positioning accuracy is <0.1 mm.

Verification of stability

Six men (mean age, 27.8 years; range, 25–31 years) and four women (mean age, 27.3 years; range, 24–29 years) participated as volunteers in this study. We measured a tooth as a reproducible object because it is located near the pharynx. Range images of the maxillary first molar were produced (Figure 2). The volunteers were asked to sit in a chair and rest their head against a wall to fix the head posture. The endoscope was then secured using a fixation device (Figure 3). The fixation device comprised a commercially available light stand and multijointed arm. This system enabled arbitrary positioning. As described above, range images were produced for 30 consecutive frames per second. The coordinate values of the mesial buccal cusp of the maxillary first molar were recorded for each of the 30 frames, and the standard errors were calculated.

Figure 2. Range images of the maxillary first molar produced by the present system. The coordinate values of the cusp were recorded for all 30 frames, and the standard error was calculated. The left and right figures show the range images of the maxillary first molar by mesh and texture, respectively.

Figure 3. Fixation device comprising a commercially available light stand and a multijointed arm. This system enables arbitrary positioning.

Accuracy along temporal axes

A marking needle was fixed onto the slide table of the electronic actuator. The slide table was moved at various speeds and recorded from both the horizontal and vertical direction using the endoscope. The quantity of movement per frame was measured and compared with the corresponding theoretical value calculated from the frame rate. The accuracy along the temporal axes was evaluated by comparing the measured value with the theoretical value.

Ethics statement

This study was approved by the Institutional Review Board of Yamaguchi University Hospital.

Results and discussion

The relationship between the distance from the tip of the endoscope to the object and the associated measuring error are shown in Figure 4. The measuring error became substantial when the distance from the tip of the endoscope to the object was >6.5 cm. In contrast, when the distance from the camera to the object was <2 cm, the produced range images became remarkably small. If the tip of the camera was positioned approximately 2.0–6.5 cm from the object, accurate and sufficiently sized range images were produced. With respect to the stability of the range images, the standard error of the 3D coordinate value produced from 30 frames was within 0.1 mm (range, 0.01–0.08 mm) (Table 1). With respect to the accuracy along the temporal axes, the measurement error was 9.16% in the horizontal direction and 9.27% in the vertical direction (Table 2).

Figure 4. Relationship between the distance from the tip of the endoscope to an object and the measuring error. The measuring error became substantial when the distance from the tip of the endoscope to the object was >6.5 cm. The red dashed line shows the true distance (5 mm).

Table 1. Standard error of the coordinate values of the mesial buccal cusp of the maxillary first molar in 30 consecutive frames.

Volunteer No.	x coordinate (mm)	y coordinate (mm)	z coordinate (mm)
1	0.01	0.02	0.05
2	0.01	0.01	0.03
3	0.02	0.01	0.05
4	0.03	0.03	0.05
5	0.05	0.01	0.08
6	0.01	0.02	0.08
7	0.04	0.01	0.05
8	0.01	0.01	0.03
9	0.03	0.02	0.06
10	0.01	0.01	0.03

Table 2. Measurement accuracy along temporal axes.

	30 mm/s	60 mm/s	90 mm/s	Average (%)
Horizontal	1.19 (1.00)	2.07 (2.00)	3.02 (3.00)	9.16
Vertical	1.06 (1.00)	2.12 (2.00)	3.10 (3.00)	9.27

All rates are theoretical and given in millimeters. ( ): The corresponding theoretical value calculated from the frame rate.

We have developed a method that successfully produces accurate range images of the velopharynx. The use of the herein-described endoscope fixation device enables the capture of a stable, unblurred range image of the real-time movements of the velopharynx through the oral cavity of a patient. We confirmed the accuracy of this method in the present study using a tooth as the endoscopically viewed object because it is close in proximity to the velopharynx. Our aim is to use this system clinically in the future. When inserting an endoscope through the mouth, it is difficult to completely fix both the object and endoscope. Therefore, we stabilized each subject’s head onto the wall, fixed the endoscope in a fixation device, and evaluated the degree of blurring. Accurate, reliable range images were provided when the distance between the subject and camera was <6.5 cm. The present endoscopic measuring system provided relatively accurate and reliable range images of the velopharynx and enabled quantitative analysis of velopharyngeal movement.

Some studies have focused on the measurement accuracy of 3D endoscopes.[9,10] However, to the best of our knowledge, no studies have evaluated the capture of range images and evaluation of subject movement along the temporal axes. The herein-described 3D endoscopic system fills this knowledge gap and thus obtains accurate movement measurements. The speed of soft palate movement is approximately 70 mm/s in healthy adults, but is lower in patients with congenital velopharyngeal incompetence and submucous cleft palate.[11] This novel system will enable detailed evaluation of soft palate movement in such patients from the optimal viewpoint of the temporal axes.

Although various software-based methods are available to correct endoscopic blurring, we controlled for blurring using a fixation jig and obtained results within an acceptable range. Software-related problems associated with creation of the range image and the error associated with choosing a measurement point (quantization error) were settled in our previous article.[6] The standard errors were <0.1 mm, even with addition of the endoscope fixation error.

A limitation of this endoscopic measuring system is that it is relatively large; the pattern projection system is incorporated into a commercially available 3D endoscope. Therefore, it cannot be inserted through the nasal cavity. The development of a thin, lightweight appliance that can be inserted into the nasal cavity for measurement is needed to better understand the detailed movement of the velopharynx. Further evaluation of the fixation method is also needed. Nonetheless, we believe that this system can be used to accurately demonstrate changes in velopharyngeal function following surgical correction of velopharyngeal incompetence in patients with cleft palate.

In conclusion, accurate, reliable, and stable range images can be obtained by setting the tip of the endoscope 2–6 cm from an object and using the herein-described fixation device. This method enables quantitative analysis of velopharyngeal movements.

Acknowledgements

The authors thank Mr. Yuji Uchida from Ergovision Co., Ltd. (Osaka) and Mr. Tsuneo Fukuyo from Shinko Optical Co., Ltd. (Tokyo) for their helpful advice.

Declaration of interest

Asuka Nakano, Katsuaki Mishima, Ruriko Shiraishi, and Yoshiya Ueyama have no conflicts of interest or financial ties to disclose. This work was partly supported by the Japan Society for the Promotion of Science under a Grant-in-Aid for Scientific Research (B) No. 21390537, the Japanese Foundation for Research and Promotion of Endoscopy Grant, and the Suzuken Memorial Foundation.