Will haptic feedback speed up medical imaging? An application to radiation treatment planning

Abstract

Haptic technology enables us to incorporate the sense of touch into computer applications, providing an additional input/output channel. The purpose of this study was to examine if haptic feedback can help physicians and other practitioners to interact with medical imaging and treatment planning systems. A haptic application for outlining target areas (a key task in radiation therapy treatment planning) was implemented and then evaluated via a controlled experiment with ten subjects. Even though the sample size was small, and the application only a prototype, results showed that haptic feedback can significantly increase (p<0.05) the speed of outlining target volumes and organs at risk. No significant differences were found regarding precision or perceived usability. This promising result warrants further development of a full haptic application for this task. Improvements to the usability of the application as well as to the forces generated have been implemented and an experiment with more subjects is planned.

Medical imaging technology is rapidly advancing, resulting in more and better images, but also more data for the diagnosticians and radiation oncologists/physicists/dosimetrists to handle and analyze. While digitalization of the working environments has facilitated decision making and image interaction, it has also enabled new possibilities. Although most users rely upon visual feedback when interacting with a computer, this does not have to be the only feedback channel.

Vision, hearing, and touch are the senses used when interacting with computers [1]. The word haptics is derived from the Greek haptesthai, meaning “to touch” [2]. Via a haptic device and a haptically enabled application, users can feel and manipulate virtual objects [3]. Haptic feedback is synonymous with tactile feedback coupled with kinesthetic (force) feedback – together they mediate the sense of touch to the user.

The goal of this work was to explore if haptics could increase the speed of outlining target volumes and organs at risk and/or a greater precision could be achieved. To accomplish this, an experiment was performed to test the following hypotheses in comparing a haptic-visual interface to a visual only interface:

(H1) Haptic feedback will shorten the time required to complete the task; (H2) Haptic feedback will increase precision; (H3) Haptic feedback will increase perceived usability.

Methods and materials

Initially, field observations were conducted to gain knowledge about the users, the context in which they work, and the tasks which they perform in order to learn how medical images are used for diagnostics and treatment planning, and to understand what haptic feedback might add and how the haptic system should work. Subsequently, using a SensAble Technologies PHANTOM Omni haptic device (Figure 1) and their OpenHaptics toolkit [4], an application for drawing contours on typical medical images (in this case those derived from Computed Tomography (CT)) was designed and implemented in OpenDX [5]. A 2D version of the application was implemented since the potential users mostly work in 2D and, according to observations, were quite happy with this.

Figure 1.  SensAble Technologies’ PHANTOM® Omni™ Haptic Device.

As haptic feedback is a vector valued function, a function is needed to compute a feedback force (vector) from the data and in some cases from the motion of the pointer. A common type of haptic feedback is force based on the local gradient, in this case the voxel values near the pointer [6]. The gradient force propelled the haptic device (a pen-like object) toward a region of higher intensity or resisted the movement of the pen from a region of higher intensity. Thus a force field derived from image gradients gave information about the locations of the borders between voxels with different intensity values enabling the user to better feel the edge of the structure.

To control the speed with which the user could move the pen, and also to provide feedback in all parts of the object, we added a viscous force or viscosity – a resisting force derived from the direction of motion, the velocity, and a damping factor (the viscosity constant) associated with the device. This prevented the user from moving the pen extremely rapidly across the image, which again, enabled the user to feel the edges better.

The transition between virtual surfaces with less well defined gradients, such as different soft tissues, was weak compared to the transition between soft tissue and bone. With an exponential variation (since a logarithmic relation exists between physical stimulus intensity and perceived magnitude) a stronger variation was generated, but was not able to provide a force which distinguished between soft tissues where the intensity value magnitudes were similar. There was also a problem with mechanically generated noise when the gradient vector field was too weak or undefined. A simple filter, which set all small gradient values to zero, was created. It reduced, but did not completely eliminate the noise.

The application was evaluated in a controlled experiment. Since using a haptic device was new to all the users some of whom were not experts at contour drawing, it was thought best to eliminate any confounding factors that might be introduced by the slice choice. Therefore two slices from a CT scan of a plastic replica of the vertebra was chosen as the image data (Figure 2). The users’ task was to contour twice, a designated part of the vertebra in each of the two different slices.

Figure 2.  CT slices used for the experiment.

Experimental design

A within group design was used. The sample consisted of ten subjects: eight physicians (three orthopedic surgeons, two radiologists, three radiation oncologists) and two radiotherapy nurses. These ten subjects performed two predetermined tasks twice: once with a haptic and visual interface and once with a visual only interface. The subjects were randomly split into two groups starting with different modes to evenly distribute the effects of practice and fatigue. A test session lasted for about 30 minutes. Instructions were given on paper.

Training time was given in each mode in a session immediately prior to the experiment. The participants themselves chose the duration of the training period; however no one trained for more than five minutes. After finishing a mode (haptic visual/visual only), a log (which recorded the positions of the permanent glyphs) was saved and then a questionnaire was completed. Then the whole procedure started from the beginning again – now in the other mode. When completing the questionnaire after both test rounds, there was an additional page of questions.

Test environment

The test environment consisted of a computer with one monitor, mouse, keyboard, and a Phantom Omni Haptic Device. The Omni was placed in front of the screen while the mouse and the keyboard were moved to the right. The subjects were free to move the Omni if they liked.

The software used for the experiment consisted of two versions of the application, one with haptic force feedback enabled, and one without. The only difference between the two was that the forces were all set to zero in the version without force feedback. With haptic feedback enabled, the participant could feel the edges of the object when moving the pen (as apposed to without when the pen worked just like a mouse). On the display the presentation was the same.

A video camera was used to monitor the experiments. It filmed the screen and the hand as well as recorded the sounds (speech and surrounding sounds). Participants were aware that the sessions were being video taped.

Experimental task

Each participant was asked to, with the Omni (Figure 1), generate glyphs (as markers) along two different contours. As the subject moved the pen, a green glyph showed the participant the position in the image object on the screen. By pushing the blue button on the pen red permanent glyphs were generated. These were to function as coordinates for drawing a line which would resemble the contour. The instructions said that the user should choose the number of glyphs with this in mind. Each task was done twice on each slide (eight drawings in all).

Variables

By changing the style of the input device, the independent variable, two different modes were presented to the users: visual only mode, and haptic and visual mode.Three dependent variables were measured during the experiment:

• Time: how long (in seconds) it took to perform each task. However, the participants did not know that they were being timed as we did not want them to feel stressed.

• Precision: how well was the contour outlined? The best result from each mode was compared with a template.

• Perceived usability: how did the user feel about this experience? After each trial of the task the user was asked to fill out a questionnaire asking him/her about the usability of the specific mode.

Evaluation methods

Time was measured, in seconds, for each drawing separately: from when the user's first point appeared on the screen until the last one. The total time for all four drawings (two each of the two slices) in each condition was used for the analysis.

Precision was evaluated on the basis of curves drawn using the coordinates for the glyphs, which the participants generated in each task. (The curves were adjusted so that no points which could be a disadvantage were included, since the subjects were not able to erase glyphs.) For each task the best resulting curve from the haptic-visual mode and the best resulting curve from the visual mode were drawn in the same graph. Three subjects, not earlier involved in the tests or in the project, then looked at these graphs, comparing them with the true contour, and rated the result as either ‘equally good’ or ‘x is worse than y’. (These three subjects did not know which curves were from which mode.)

Perceived usability was evaluated from the questionnaires. For each participant the ratings on all questions were summed and then divided by the number of questions. These values were then used in the analysis. All answers used for this analysis were given on the seven-point scale. For the six sub-areas (performance, functionality, learnability, stability, ergonomics, and interaction), as there were different numbers of questions; the sum of the answers was divided by the number of questions.

For the analyses of the measures, task completion time and perceived usability, the results of ten subjects were analyzed using ANOVA. This is the appropriate statistical test as this experiment had a within group design [7]. The level of significance was set to 0.05 for the analyses in this study. The statistics program SPSS was used.

Results

Time

The first hypothesis considered whether haptic feedback would shorten the time required to complete the task. The results showed that task completion time differs significantly between the two conditions (F = 5.43, p < 0.05). The participants mean completion time (M) was about a minute shorter for the haptic-visual condition, M_HV=272 seconds, compared to the one for the visual condition, M_V=331 seconds. Total time varied between subjects as shown in Figure 3.

Figure 3.  Total drawing time for each subject. The figure shows how much of the time was spent for the drawing with haptic feedback in comparison to without. (First five represent the subjects who started in visual only mode.)

Precision

The second hypothesis stated that haptic feedback would increase precision. The hypothesis was not verified. Results are presented in Table I. In 60% of the total number of cases the results were rated as equally good and in 38% the results were better with the haptic force enabled.

Table I.  from the precision experiment where curves drawn from the generated glyphs were judged by three independent examiners.

Slice 1				Slice 2
	worse with haptic feedback	equally good	worse with NO haptic feedback	worse with haptic feedback	equally good	worse with NO haptic feedback
1		xxx			x	xx
2	x	x	x		x	xx
3			xxx		x	xx
4		xxx				xxx
5			xxx			xxx
6		xxx			xxx
7		x	xx		x	xx
8		xxx			xxx
9		xxx			xxx
10		xxx			xxx
sum	1	20	9	0	16	14
	3.30%	66.70%	30.00%	0%	53.30%	46.70%

Perceived usability

The last hypothesis stating that haptic feedback would contribute to usability was not verified. The different usability areas (performance, functionality, learnability, stability, ergonomics, or interaction), measured by the questionnaires, were analyzed both together as a total dimension and separately. The dimension perceived usability did not differ significantly (F = 0.21, p = 0.07) nor did any of the sub-areas. Although there were no significant differences, the results show a slightly more positive result for the total dimension (mean rating with haptic feedback was 5.4 compared to the mean rating without haptic feedback which was 5.3), as well as for all sub-areas except stability (see Table II).

Table II.  Results from the analysis of questionnaire data showing number of subjects (n), F-value (F), significance (p) and mean values (M).

				Haptic feedback	NO haptic feedback
Experienced usability	(n = 10)	F = 0.21	p = 0.07	M = 5.4	M = 5.3
Performance	(n = 10)	F = 2.06	p = 0.19	M = 5.2	M = 4.6
Functionality	(n = 10)	F = 0.01	p = 0.91	M = 5.3	M = 5.2
Learnability	(n = 10)	F = 0.28	p = 0.61	M = 5.4	M = 5.2
Stability	(n = 10)	F = 2.31	p = 0.16	M = 4.8	M = 5.5
Ergonomics	(n = 10)	F = 1.98	p = 0.19	M = 4.2	M = 3.6
Interaction	(n = 10)	F = 0.37	p = 0.56	M = 6.2	M = 6.0

Discussion

The use of haptic force feedback has been shown to have performance advantages, i.e., interaction becomes faster and more precise. For example, Sallnäs [8] has shown that users often feel more confident when they receive both haptic and visual feedback. Enriquez, et al. [9] showed that force feedback can provide safety and performance benefits in perceptually overloaded tasks such as driving. In medicine, haptic feedback has mostly been used for surgical training simulations [10]. Indeed, Ahlberg et al. demonstrated that simulator training significantly improved the initial learning curve for the simulator-trained over the control group [11].

Research in applying haptic feedback for treatment planning is just beginning, but appears promising. To increase efficiency and precision of dose planning, Olofsson et al. investigated the use of a haptic interface to Leksell GammaKnife® [12]. Contrary to most other haptic feedback applications, their implementation did not aim at improving tactile interaction and presence, but simply supplying additional information to help the physician place iso-centers at advantageous positions.

In an experiment with 15 medical students, Kuroda et al.[13] tested the use of a haptic device to feel a difference in stiffness between adjacent soft tissues in the rectum. The goal was to simulate rectal palpation, distinguishing between the prostate and the tissues lying in front of it using forces based on the relative stiffness of the tissues. They obtained a significant result that showed that forces based on the stiffness of neighboring objects can allow the user to perceive the tissues as different. However, they only used a simulated model for the tissues and not actual medical images.

The results of the experiment reported here suggest that haptic feedback can increase the speed of outlining target volumes and organs at risk. Even though the sample size was small, and the application only a prototype, the results were encouraging. They showed a significant difference for the time dimension and the hypothesis that haptic feedback would shorten the time required to complete the task was verified. The results are consistent with earlier studies [8] and suggest that the user has greater control when using more senses.

The other two hypotheses regarding improved precision and improved usability were not verified. However, the results indicated that haptic feedback can have a positive influence here as well. Remembering that the application only was a prototype where the forces were not that prominent and with some noise problem, the results showing equally good precision in 60% of the cases, and better precision with haptic feedback in 38% of the cases, is very positive. Also, when the test results for precision were grouped by test order (haptic + visual, or visual + haptic), it was noticed that for users starting with the haptic feedback the precision was equally good in both tests, while for users starting with the visual only, precision was better with haptic feedback.

In summary: less time was needed with just as good precision and usability – this shows the potential for further development of a haptic application. The experiment also resulted in some other information which is worth mentioning: A few of the participants had positive attitudes towards a haptic application, but in general there was more initial skepticism. However, the questionnaire answers and some comments showed that some participants’ attitudes were more positive after the test.

Future

These promising results show that it might be of interest to further develop the application. There are several things which can be done to create a better application. First and foremost the speed of the application should be increased as larger data volumes tend to make the interactive responsiveness slower. Second, an optional 3D interface should be added. However, the 2D interface should remain, as most physicians and other practitioners are used to working in 2D and often 3D applications demand much more of the computers (which, right now, in the clinical setting are frequently not optimized for manipulating large data volumes in 3D). More and better drawing functions and the possibility to erase the drawing when it is not right are also important features of a future application.

There is also the need for better/different forces so the application can be used for distinguishing between different soft tissues even if they are close in intensity value. Forces, which are associated with intensity values instead of just the tissue/organ boundaries, would perhaps be more suitable. The next experiment will be done with improved forces, a larger number of experimental subjects, and patient images instead of model (phantom) images.

Acknowledgements

This paper is based upon the thesis project conducted by the first author and published as:Eva Anderlind, Haptic Feedback for Medical Imaging and Treatment Planning, Master of Science Thesis, School of Computer Science and Communication, Royal Institute of Technology (KTH), Stockholm, Sweden, June 2006.