Development and functional demonstration of a wireless intraoral inductive tongue computer interface for severely disabled persons

Figures & data

Figure 1. Overview of the Inductive Tongue Computer Interface. The Inductive Tongue Interface transmits the 18 sensor signals wirelessly out of the mouth to the central unit, which processes the signals and transmits the processed information to the PC unit, which emulates a mouse/keyboard function. The board used for the PC unit includes a microcontroller and radio chip.[24]

Figure 2. Overview of the ITI parts. Top: The Two 10-layer PCB constituting the key pad (left) and the mouse pad (right), the battery and the PCB for the electronics. The scale is in centimeters. Middle: Top middle: The assembled ITI parts, showing the charger coil and the sensor PCB. Bottom middle: The battery and one of the flex prints used to connect the parts. Bottom: The fully embedded ITI dental appliance.

Figure 3. Overview of the main parts of the ITI signal processing and detection system. (a) The steps in the signal processing system. (b) The ideal principle of measurement: a band-pass filter was created by connecting a capacitor to each sensor coil. (c) During activation the impedance of the LC band-pass filter changed, and the activation was detected at the frequency, f_meas, indicated by the vertical punctured line. (d) The real frequency characteristics of the detection LC band pass filter with and without sensor activation.

Figure 4. Left: The stainless steel frame forming the base of the encapsulation of the electronic parts. Right: The activation unit glued to the tongue.

Figure 5. The filter structure used for removal of the baseline from the sensor signals.

Figure 6. The method for adjustment of the filter weights for the filter (Figure 5) for removal of the sensor offset and drift.

Figure 7. The sensors and the related typing functionality in mouse mode (left sensor layout) and keyboard mode for full alphabet text input (right sensor layout). Switching between keyboard and mouse modes was performed by activating sensor number 10 for 10 sec. Adapted from Lontis et al. [29]

Figure 8. Typing procedure: when a sensor on the keypad area was activated, in this case sensor 2, the first character related to that sensor was displayed as a visual feedback for the user. If the sensor was still activated after a time period, called dwell time 1 (D1), it could be typed during the following time period, dwell time two (D2), during which deactivation of the sensor would result in typing of an “a”. If the sensor was activated beyond D2, the next character related to sensor 2 which was “b” was displayed, and deactivation of the sensor would then cause a “b” to be typed. If the sensor was still not deactivated, the next character related to the sensor would appear, and this procedure would continue in a circular manner until the sensor was deactivated. If the sensor was deactivated during D1, the “a” would be deleted and no character would be typed. This gave the user a time period, D1, to slide on the keypad area, with visual feedback, without typing undesired characters.

Figure 9. The Matlab© based visual feedback used in the experiments. The given character string to be typed is shown with the currently desired character in capital. In the text field above, the characters typed by the subject are displayed. Correctly typed characters are shown in capital. A figure below the text field shows the position of the characters on the keypad area of the ITCI. As default, all sensors were marked with a green circle. When the activation unit was placed at a sensor/character for less than the time D1, the circle around the character turned yellow, and the character was temporarily typed at the cursor in the text field. When D1 was exceeded, the colour turned from yellow to red and deactivation of the sensor resulted in sustained typing of the character in the text field after which the cursor moved one step forward to provide feedback for the next character to be activated by the subject. If no deactivation occurred within the time D2, the next character related to the activated sensor was displayed.

Table 1. Tasks for typing with the Matlab interface.

	Trials of 30–60 s
1	.,?” ()@abcæådefghijkl
2	Mnopqrstuvwxyz
3	heprjå(lb, sagpd
4	repqg.åfb)usdt8æ
	“Wack-A-Mole” or “Born to Be Big”: 5–10 min
	Trials of 30–60 s
5	Aaaaaaaaa
6^a	abababab
7	Abcæå
8	.,?!”()@
9	Def
10	Ghi
11	Jkl
12	Mnoø
13	Pqrs
14	Tuv
15	Wxyz
15^b	Space
16	.ad.ad.ad
17	Gjmgjmgjm
18	ptw
19	type a character and delete
	Trial running–max 5 min
20	jeg er færdig! (use Delete)
21	jeg er færdig! –(use delete and easy write)
	Trials of 30–60 s
22	.,?()@abcæådefghijkl
23	Mnopqrstuvwxyz
24	heprjå(lb, sagpd
25	repqg.åfb)usdt8æ

^a Only performed by S1 and S2,

^b not used for data analyses, Trial 17–25 were only performed by S3 and S4.

Figure 10. The number of correctly typed characters in trials of 30 sec duration. Trials for the healthy subjects S3 and S4 using the keypad area. The results are for trial 1 to 17 in Table 1.

Figure 11. The number of correctly typed characters in trials of 30 sec. for the individuals with tetraplegia, S1 and S2 using the keypad area. The shown results are for trial 1 to 16 in Table 1.

Table 2. Mean and average typing times for all subjects in trials of 30 sec.

	Keypad correct characters mean/max/min [No.]	Keypad time/correct character mean/max/min [s]	Mouse pad correct characters mean/max/min [No.]	Mouse pad time/correctcharacter mean/max/min [s]
S1, Day 1	4.6/16/0 (SD 4.0)	6.5/–/1.9
S2, Day 2	3.7/7/1 (SD 1.9)	8.1/30/4.3
S3, Day 1	5.1/14/0 (SD 3.8)	5.9/–/2.1	5.4/13/1 (SD 3.24)	11.1/60/4.6
Day 2	5.7/15/1(SD 3.5)	5.3/30/2	4.8/12/1 (SD 2,6)	6.25/60/2.5
S4, Day 1	3/11/0 (SD 2.7)	10/–/2.7	1.4/4/0 (SD 1.3)	21.4/30/7.5
Day 2	9.2/17/4 (SD 3.8)	3.3/7.5/1.8	5.3/21/2 (SD 4.2)	5.715/1.4

Figure 12. The number of correctly typed characters for S3 and S4 using the on-screen keyboard with the ITCI mouse pad area. The trial number refers to the numbers in Table 1 + 24, e.g., the characters typed in Trial 29 are “aaa”. Trial durations of the trials are shown in the figure.

Lund ME, Caltenco HA, Lontis R, et al. A framework for mouse and keyboard emulation in a tongue control system. Paper presented at: Proceedings of the 31st Annual International Conference of the IEEE EMBC; 2009; 815–818.

 Google Scholar

Lontis R, Andreasen Struijk LNSA. Alternative design of inductive pointing device for oral interface for computers and wheelchairs. 34th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE EMBS, EMBC. 2012;2012:3328–3333.

 Google Scholar