Handheld motorized injection system with fiber-optic distance sensors and adaptive time-delay controller

Abstract

During the last decade, many ophthalmic therapeutic drugs have been clinically approved, and intraocular injection has been a common surgical intervention. Injecting drugs directly into the subretinal space is crucial to treat retinal complications effectively. Here, we report a handheld microinjector with two fiber-optic distance sensors and time-delay control (TDC) to mitigate nonlinear disturbances during the injection task. The conventional method exhibited a cosine error of approximately 77 μm at an angle of 45°, whereas our proposed needle reduced measurement errors to ∼6 μm. Also, TDC-based position regulation is designed to adaptively apply motor inputs by estimating disturbances during the handheld task and achieving fast system responses with minor control errors. Phantom studies show a maximum reduction of 26.5% in root-mean-square error (RMSE) compared to the existing approach. Moreover, ex-vivo experiments demonstrated superior and robust injection performance, resulting in an injection RMSE of 10.3 μm.

Keywords:

• Fiber-optic interferometric sensor
• needle-integrated distance sensor
• handheld microinjector
• time-delay control (TDC)

1. Introduction

Several inherited retinal diseases, such as macular degeneration and retinitis pigmentosa, have an estimated 2.7 billion carriers worldwide and are expected to affect 5.5 million people, causing heterogeneous ophthalmic conditions.^[1] Most retinal complications stem from the subretinal dystrophies at the photoreceptor layer or retinal pigment epithelium (RPE).^[2] The U.S. Food and Drug Administration has recently approved several therapeutic and genetic medications showing high efficacy on various retinopathies.^[3] Accordingly, the estimated trials of intravitreal drug injection have continuously increased, reaching 6 million cases in 2016.^[4] However, the intravitreal injection allows drugs to permeate the retina by diffusion, which demands large doses and prevents targeted drug delivery.^[2] On the other hand, the subretinal injection can directly deliver therapeutic agents beneath the retinal membrane with high concentration and low recurrence rate of complications.^[5] Reproducibility and accurate injection depths are the primary requirements of subretinal injection to prevent unforeseen ocular damages, such as choroidal rupture or drug reflux, by excessive and shallow injection depths, respectively.^[6]

Recent advancements in the robot-assisted surgical platform have allowed surgeons to perform safe subretinal microinjection with intraoperative optical coherence tomography (OCT).^[7,8] OCT is a classic medical imaging device that uses a broadband laser source to provide a real-time cross-section profile of the surgical site with an axial resolution of several micrometers.^[9,10] Various biomedical sensors using OCT have been exploited, taking advantage of sharing the light source of existing OCT equipment by a fiber-optic technique. For the microinjection task, common-path OCT technology utilizing the reflected signal at the optical fiber end as a reference signal has enabled compact, non-contact, and high-resolution distance measurement between the tooltip and the surgical lesion.^[11–17] In addition, motorized control of the surgical tooltip facilitates a precise microsurgical injection far beyond human dexterity.

The benchtop subretinal injection instruments consisting of several motors and multi-degree-of-freedom structures have been recently used in clinical trials for in-vivo animals^[18] and humans.^[19] The robot-assisted control maintained the needle tip within $<40 \mathrm{\mu m}$ from the target, compensating for $<12 \mu \mathrm{m}$ physiological retinal motions. The benchtop solutions can support a high level of autonomy; on the other hand, the handheld approaches are potentially in higher demand with a low-resource setup, intuitive operation, short training time, and fewer work steps.^[8] Several causes of disturbances exist in a clinical subretinal injection, such as heartbeats, breathing, and hand tremors. Among them, involuntary hand tremor is the most significant disturbance, having an amplitude and a frequency band of $0.1\mathrm{\text{mm}}$ and $6-30\mathrm{\text{Hz}},$ respectively.^[20] To implement the handheld microinjection system as clinically applicable as the benchtop instruments did, accurate position measurements and rapid rejection of disturbances are the two critical requirements.

The previous position measurements have been done by a one-dimensional (1D) fiber-optic OCT distance sensor.^[12–19] It must tolerate a cosine measurement error at clinically required injection angles up to $45°$ owing to an inevitable offset between the needle and the OCT sensor.^[21] Software-based approaches^[22,23] and hardware-based enhancements of the sensor signal^[24] have been implemented to overcome inaccurate position estimation. Although the abovementioned methods can achieve reliable position estimation at 90°, a cosine error must be considered to meet the clinical demand for the oblique subretinal injection. An alternative approach, such as 2D or 3D OCT images, can be generated using a miniature beam scanner^[25,26] or an external benchtop intraoperative OCT device, allowing for a more precise tracking of needle positions.^[11] Nonetheless, in the case of handheld robots, the substantial need for quick compensation of hand tremors has led to the prevalent utilization of A-line OCT signals transmitted through the fiber-optic sensor attached to the needle. As state-of-the-art control methods for handheld robotic micromanipulators, hybrid motor controllers using an optimized proportional (P), integral (I), and derivative (D) control gains with a latency-compensating motion estimator (ME) have been used to compensate for the involuntary hand tremor.^{[13–17,22,27]} The previous schemes could achieve a good root-mean-square-error (RMSE) of $<10 \mu \mathrm{m}$ at $90°;$ however, the handheld system for clinical use has been significantly hindered by a considerable increase in RMSE at oblique angles.^[28]

This study demonstrates a handheld motor-assisted microinjector with two OCT sensors and a robust adaptive controller for a handheld subretinal injection application. Two high-precision OCT sensors are fabricated by fusing a single-mode fiber (SMF) and gradient-index fiber (GIF) and deployed to calculate the actual needle tip position from the sample in a 2D cross-sectional plane. In addition, the adaptive time-delay controller (TDC) is designed to compensate for the high-order nonlinear terms, including the motor hysteresis and the hand tremor, aiming to expand the control bandwidth and rapidly react to abrupt hand motions. Both phantom studies and ex-vivo experiments were performed for tremor compensation and injection results, and the proposed method could achieve a good injection performance compared to the clinically available benchtop device.

2. Materials and methods

2.1. Handheld microinjection system

The configuration for the proposed microinjection system is shown in Figure 1(a). A handheld microinjector comprises a sensorized needle and a linear piezoelectric (PZT) motor (LEGS-LL1011A, PiezoMotor). The needle is made by attaching two OCT distance sensors to the 33-gauge microneedle, and the outer diameter is designed to be insertable through a 25-gauge surgical trocar. The PZT motor regulates the sensorized needle tip to the desired position with high precision and speed. The microinjection system includes an OCT engine (Swept Laser OEM engine, AXSUN), 2 × 2 fiber coupler (TW1064R5A1A, Thorlabs), motor driver (PMD101, PiezoMotor), syringe pump (Fusion 100, Chemyx), and workstation (Precision T7610, Dell). One detector acquires two combined interferometric signals of both OCT sensors at a $100\mathrm{\text{kHz}}$ sampling rate. Subsequently, batches of one hundred samples are processed by a graphics processing unit (GPU, GeForce GTX1060, Nvidia); spectrum averaging and fast Fourier transform (FFT) are performed to acquire OCT signal by parallel GPU programming (CUDA), resulting in a system operation speed of $1\mathrm{\text{kHz}}.$ The motor driver operates the four piezoelectric components within the motor by controlling them through RS232 serial communication using quadratic phase modulation. In addition, the maximum motor speed of $3.3\mathrm{\text{mm}}/\mathrm{s}$ was achieved by optimizing the motor’s resolution and speed. This speed surpasses typical hand tremor,^[20] making it well-suited for the control loop of the hand tremor compensation. Moreover, the syringe pump pushes the syringe with a minimum step size of $0.1 \mu \mathrm{m}$ to gently inject the drug through a hydraulic tube.

Figure 1. System configuration and the schematic of the handheld microinjection system for precise subretinal injection.

2.2. High-Precision OCT sensor

For a high-precision OCT distance measurement, several lenses, such as conical,^[29] spherical,^[30] rod types,^[31] and the GIF^[32] have been employed. Among these, the GIF has various advantages such as, compactness, low cost, reduced optical losses, and simple fabrication procedures. Here, we adopted the GIF and found its optimal length and polishing angle for the high-SNR OCT signal.

The schematics of the bare SMF and GIF-attached SMF (GIF-SMF) are shown in Figure 2(a). The radial gradient of the refractive index within the GIF bends the diverged light from the SMF, so the divergence angle, ${\theta }_{\mathit{\text{div}}},$ depends on the GIF’s length, $L_{\mathit{\text{GIF}}},$ and the gradient factor of the GIF, $g_{\mathit{\text{GIF}}}.$ According to the Gaussian beam-propagation method,^[32] the beam divergence angle of the GIF-SMF can be approximated as follows: (1) $${\theta }_{\mathit{\text{div}}}\cong a\sqrt{{\mathrm{\text{sin}}}^2\left(g_{\mathit{\text{GIF}}}{L}_{\mathit{\text{GIF}}}\right)+b^2{\mathrm{\text{cos}}}^2\left(g_{\mathit{\text{GIF}}}{L}_{\mathit{\text{GIF}}}\right)}$$(1) (2) $$a=n_{\mathit{\text{GIF}}}{g}_{\mathit{\text{GIF}}}{D}_0/2$$(2) (3) $$b=4\lambda /\pi n_{\mathit{\text{GIF}}}{g}_{\mathit{\text{GIF}}}{D}_0^2$$(3) where $n_{\mathit{\text{GIF}}}$ is the refractive index of the GIF, $D_0$ is a mode-field-diameter (MFD), and $\lambda$ is the wavelength, which are $1.493,$ $6.2 \mu \mathrm{m},$ and $1060\mathrm{\text{nm}},$ respectively. To enhance the signal-to-noise ratio (SNR) along the entire measurement range, ${\theta }_{\mathit{\text{div}}}$ must be minimized. Therefore, the optimal length of the GIF, $L_{\mathit{\text{GIF}}}^{*},$ can be calculated by taking the derivative on Equation (1)(1) $${\theta }_{\mathit{\text{div}}}\cong a\sqrt{{\mathrm{\text{sin}}}^2\left(g_{\mathit{\text{GIF}}}{L}_{\mathit{\text{GIF}}}\right)+b^2{\mathrm{\text{cos}}}^2\left(g_{\mathit{\text{GIF}}}{L}_{\mathit{\text{GIF}}}\right)}$$(1) concerning $L_{\mathit{\text{GIF}}},$ as follows: (4) $$L_{\mathit{GIF}}^{*}=\frac{\pi }{2g_{\mathit{GIF}}}\left(2\mathrm{k}+1\right),\mathrm{k}=0,1,2,\dots$$(4) where $\mathrm{k}$ is an integer, so $L_{\mathit{\text{GIF}}}^{*}$ has a period dependent on $g_{\mathit{\text{GIF}}}.$ For the fabrication of the GIF-SMF, we used the commercial SMF (1060XP, Thorlabs) and the GIF (GIF625, Thorlabs) with a $g_{\mathit{\text{GIF}}}$ of $4.944\mathrm{ }{\mathrm{\text{mm}}}^{-1};$ so, the $L_{\mathit{\text{GIF}}}^{*}$ has a period of $0.317\mathrm{\text{mm}}.$ The bare SMF has a nominal numerical aperture (NA) and MFD of $0.14$ and $6.2\pm 0.5 \mu \mathrm{m},$ respectively. The outer diameter of the GIF is the same as the SMF, $125 \mu \mathrm{m};$ however, the GIF has a larger core diameter and NA of $62.5 \mu \mathrm{m}$ and $0.275,$ respectively. The beam profiles captured from the SMF and the GIF-SMF along the z-axis are shown in the inset figures. We obtained several beam images from the fiber end, $z=0\mathrm{\text{mm}}$ to $z=3.5\mathrm{\text{mm}},$ where each image was captured using a high-resolution digital microscope (INSPEX2, Ash Technologies). The initial beam diameter of the SMF was measured as $8.6 \mu \mathrm{m},$ smaller than that of the GIF-SMF, $13.6 \mu \mathrm{m};$ however, the beam from the SMF rapidly diverged to $\sim 200 \mu \mathrm{m}$ at $z=1.8\mathrm{\text{mm}},$ whereas GIF-SMF was still $51.8 \mu \mathrm{m}$ at the same $z.$ It is also noticeable that the beam at $z$ of $3.5\mathrm{\text{mm}}$ is mostly scattered. In contrast, the reduced divergence angle in the GIF-SMF enables the beam’s intensity to be maintained at large $z$ values by nearly collimating the beam.

Figure 2. (a) Schematic diagrams of beam profiles exiting from the SMF and GIF-SMF along the z-axis. (b) Image of fusion splicer and fiber polisher. (c) Fiber fusing procedure. (d) Fiber polishing with a desired polishing angle, α. (e) SNR roll-of results from a mirror at 90° and an anodized aluminum plate at 45°.

Not only is the optimal GIF length important, but the polishing angle also plays a significant role in improving the OCT signal.^[33] We used a fusion splicer (S178, FITEL) and fiber polisher (NOVA Polisher, KrellTech) to fabricate a high-precision OCT sensor by adjusting $L_{\mathit{\text{GIF}}}$ and the polishing angle, as shown in Figure 2(b). The two types of optical fibers can be fused with a minimal optical loss of less than $-0.1\mathrm{\text{dB}},$ as shown in Figure 2(c). Then, the part of the GIF is cut and polished by the fiber polisher, as shown in Figure 2(d). The fiber polisher can control the polishing angle, $\alpha ,$ with an accuracy of $0.5°.$ If $\alpha$ equals zero, much back reflection is received from the fiber end; conversely, for a large $\alpha ,$ it is hard to obtain sufficient signals from the sample. The optimal $L_{\mathit{\text{GIF}}}$ and polishing angle were found at $0.32\mathrm{\text{mm}}$ and $2°,$ capable of reducing the ${\theta }_{\mathit{\text{div}}}$ of the SMF and GIF-SMF from $12.4°$ to $2.5°,$ respectively.

The analyses of the SNR roll-off are achieved by a mirror at various positions to compare the SNR between the SMF and the GIF-SMF, as shown in Figure 2(e). The angle-polished GIF-SMF could provide a higher OCT amplitude than the bare SMF, resulting in a distance measurement resolution of $8.5 \mu \mathrm{m}.$ An anodized aluminum plate is used to assess the SNR from the inclined subject as a sample and tilted at $45°.$ The SMF cannot detect the signal of the tilted aluminum plate more than $>20\mathrm{\text{dB}}$ from $2\mathrm{\text{mm}};$ however, the GIF-SMF can detect it through the entire measurable range with the SNR of $>20\mathrm{\text{dB}}.$ Hence, the proposed angle-polished GIF-SMF, in contrast to traditional lens attachment methods that solely consider beam shape, also considers the characteristics of common-path interferometric signals, facilitating an increased SNR of the OCT signals from objects at a distance or with a large inclination.

2.3. Sensorized needle

Two microscopic images of the fabricated needle are shown in Figure 3(a) and (b). Two OCT distance sensors are attached precisely to the opposite sides of the needle to avoid an optical cross-talk by using a home-build manufacturing zig. Each sensor has a different offset from the needle tip, which is determined to have enough measurement ranges for both position and angle using only one photodetector.

Figure 3. (a) and (b) the microscopic images of the fabricated sensorized needle. (c) Reference coordinates and system parameters of the needle.

The schematic of the coordinate systems attached to the sensorized needle is shown in Figure 3(c). Our goal is to measure the distance of the needle tip from the sample, $\mathit{\text{pos}}$ ($\overline{\mathit{\text{OS}}}$), and its angle, $\theta$ ($\angle O\text{S}{S}_1$), with the measured $d_1$ ($\overline{F_1{S}_1}$) and $d_2$ ($\overline{F_2{S}_2}$) by the two OCT sensors. ${\mathit{\text{pos}}}_1$ ($\overline{T_1{S}_1}$) and ${\mathit{\text{pos}}}_2$ ($\overline{T_2{S}_2}$) are defined by $d_1-d_{o,1}$ and $d_2-d_{o,2},$ respectively. $d_{o,1}$ and $d_{o,2}$ are predetermined to enable $\theta$ measurement from $40°$ to $140°$ and $\mathit{\text{pos}}$ measurement from $-0.8\mathrm{\text{mm}}$ to $1.8\mathrm{\text{mm}},$ respectively. $\theta$ and $\mathit{\text{pos}}$ can be calculated as follows: (5) $$\theta =\mathit{\text{atan}}\left(\right(D_N+D_F)/({\mathit{\text{pos}}}_2-{\mathit{\text{pos}}}_1\left)\right)$$(5) (6) $$\mathit{\text{pos}}=\frac{2D_N}{\left(2D_N+D_F\right)}\mathit{\text{po}}{s}_1+\frac{D_F}{\left(2D_N+D_F\right)}{\mathit{\text{pos}}}_2$$(6) where $D_F$ and $D_N$ denote the diameters of the OCT sensor and the needle, respectively. In our design, $D_F$ and $D_N$ are $0.125\mathrm{\text{mm}}$ and $0.220\mathrm{\text{mm}},$ respectively; so, the position error of $\pm 92.46 \mu \mathrm{m}$ is obtained at $\theta$ of $40°.$ The average retina thickness is approximately $300 \mu \mathrm{m},$^[34] so this cosine measurement error could lead to drug reflux or cause severe retinal damage. The previous OCT-based micromanipulators have not considered $\theta$-related cosine error;^[12–19] thus, an unavoidable measurement error of approximately $100 \mu \mathrm{m}$ arises from the distance between the needle tip and the sample. On the other hand, the proposed position estimation can correct this error by measuring the inclination angle, making it a more rigorous and accurate distance measurement method.

For the sensor calibration, multiple measurements at several reference positions and angles were acquired by a linear stage with $1 \mu \mathrm{m}$ resolution (Digimatic Micrometer Heads Series 350, Mitutoyo), as shown in Figure 4(a). The calibration process was observed using a digital microscope (Dino-Lite Edge, AnMo Electronics Corporation), as depicted in the inset image. For the reference position, ${\mathit{\text{pos}}}_{\mathit{\text{ref}}},$ we used the value obtained from the micrometer by moving the linear stage backward after bringing the needle tip into contact with the sample. In the case of the reference angle, ${\theta }_{\mathit{\text{ref}}},$ the microscopic image-based angle measurement was performed. The representative A-lines acquired at various angles in the calibration setup are shown in Figure 4(b). Because only one photodetector exists, two OCT signals obtained from the two sensors are superimposed. Thus, two similar OCT signal packets exist, and we applied a dual-surface detection algorithm ^[28] using thresholding and derivative analysis. This algorithm was designed to extract only the surface distance from the cross-sectional layer consisting of the acquired A-line. Estimating the surface layer involves relatively straightforward computation with high accuracy, rendering it compatible with the high-speed time delay estimation controller.

Figure 4. (a) The schematic diagram of the calibration setup. (b) Representative A-lines obtained by the implemented sensorized needle. (c) The calibrated angle measurement results from 40 ° and 140 °. (d) Comparison of the position measurement results at 45 °.

The calibration is performed to determine the optimal system parameters that can provide the least RMSE for both $\mathit{\text{pospos}}$ and $\theta .$ The angle measurement accuracy is analyzed from $40°$ to $140°$ with the angular step size of $10°,$ as shown in Figure 4(c). The square dots are the mean of the measurement errors at each reference angle, and the error bars represent the standard deviations; the calibrated results have the maximum mean and the standard deviation of $1.7°$ and $2.0°,$ respectively. The comparison between the two position measurements at $45°$ is shown in Figure 4(d). The previous position measurement cannot accurately measure the needle tip position with a cosine measurement error of $\sim 77 \mu \mathrm{m}.$ In contrast, the proposed position estimation could obtain fewer errors of $<6 \mu \mathrm{m}$ up to the reference position of $<0.8\mathrm{\text{mm}}.$ The measurable range at $45°$ is limited to $1.4\mathrm{\text{mm}}$ owing to the increased signal gap distance within the OCT’s measurable range of $3.7\mathrm{\text{mm}}.$ Thus, while traditional 1D position information is susceptible to cosine measurement errors, adding an extra fiber-optic sensor maintains sensor precision and high measurement accuracy. Moreover, it does not significantly increase system complexity, as two distance measurements can be obtained using a single detector.

2.4. Design and implementation of the time-delay-controller

Various control methods such as iterative gain scheduling,^[35] sliding-mode control,^[36] a hysteresis model-predictive controller,^[37] and the TDC ^[38] have been adopted to cope with the disturbances and inherent nonlinearities of the actuator. The TDC is an adaptive motor controller with a fast operation speed and is suitable for a handheld instrument with uncertain dynamics and considerable disturbances.

The presented system has two operation modes, as shown in Figure 5(a): distance locking and injection. In the distance-locking, the desired $\mathit{\text{pos}}$ value, ${\mathit{\text{pos}}}_d$, is denoted as $d_{\mathit{\text{dl}}};$ here, we set $d_{\mathit{\text{dl}}}$ to $500 \mu \mathrm{m}.$ $d_{\mathit{\text{inj}}}$ is the injection depth to which the motor must push the needle to reach the target layer. The value of $d_{\mathit{\text{inj}}}$ is determined by the OCT microscopy before injection, and ${\theta }_{\mathit{\text{inj}}}$ is the measured injection angle.

Figure 5. The schematic of the proposed control algorithm. (a) Two operation modes of the handheld microinjector: distance-locking and injection. (b) The block diagram of the position regulation feedback loop using the proposed time delay controller.

Once the control is started, the motor input $u(t)$ is applied to regulate $\mathit{\text{pos}},$ and $d_{\mathit{\text{PZT}}}$ is the displacement of the PZT rod; here, $u(t)$ controls the velocity of the PZT rod $\frac{d}{\mathit{\text{dt}}}{d}_{\mathit{PZT}}\left(t\right)$, as follows: (7) $$\frac{d}{\mathit{\text{dt}}}{d}_{\mathit{PZT}}\left(t\right)=\mathit{\text{mu}}\left(t\right)+f_{\mathit{PZT}}\left(·;t\right)$$(7) where $m$ is the scaling factor from the motor input to the output velocity, and $f_{\mathit{\text{PZT}}}(·;t)$ is an unpredictable nonlinear term. Due to the hysteresis of the PZT, the different scaling factors are measured along each direction as $m_{+}=11.7\pm 0.5 \mathrm{\mu m}/\mathrm{s}$ for the positive direction and $m_{-}=11.0\pm 0.4 \mu \mathrm{m}/\mathrm{s}$ for the negative direction, respectively. In addition, the $\frac{d}{\mathit{\text{dt}}}{d}_{\mathit{PZT}}\left(t\right)$ reaches its saturation value at the motor input of $\pm 280.$ As an estimated scaling factor, $\widehat{m},$ we used the mean value between $m_{+}$ and $m_{-};$ $\widehat{m}=11.35.$

A first-order dynamic equation is built by using the time-derivatives of $\mathit{\text{pos}},$ $d_{\mathit{\text{PZT}}},$ and $d_{\mathit{\text{Trem}}}$ as follows: (8) $$\frac{d}{\mathit{\text{dt}}}\mathit{\text{pos}}\left(t\right)=\frac{d}{\mathit{\text{dt}}}\left(d_{\mathit{PZT}}\left(t\right)+d_{\mathit{Trem}}\left(t\right)\right)$$(8)

Here, both velocities of the PZT motor and hand tremor give unpredictable disturbances to the system. We define $f_{\mathit{\text{sys}}}(·;t)$ as a combined nonlinear term including both the PZT motor and hand tremor; $f_{\mathit{\text{sys}}}\left(·;t\right)=f_{\mathit{\text{PZT}}}\left(·;t\right)+\frac{d}{\mathit{\text{dt}}}{d}_{\mathit{\text{Trem}}}\left(t\right).$ Therefore, Equation (8)(8) $$\frac{d}{\mathit{\text{dt}}}\mathit{\text{pos}}\left(t\right)=\frac{d}{\mathit{\text{dt}}}\left(d_{\mathit{PZT}}\left(t\right)+d_{\mathit{Trem}}\left(t\right)\right)$$(8) can be expressed by estimated values as follows: (9) $$\frac{d}{\mathit{\text{dt}}}\mathit{\text{pos}}\left(t\right)=\widehat{m}u\left(t\right)+{\widehat{f}}_{\mathit{\text{sys}}}\left(·;t\right)$$(9) where ${\widehat{f}}_{\mathit{\text{sys}}}$ is the estimated value of $f_{\mathit{\text{sys}}},$ which equals to $\left(m-\widehat{m}\right)\times u\left(t\right)+f_{\mathit{\text{sys}}}\left(·;t\right).$

Employing the time-delay estimation (TDE), ${\widehat{f}}_{\mathit{\text{sys}}}(·;t)$ can be approximated to ${\widehat{f}}_{\mathit{\text{sys}}}(·;t-L),$ where $L$ refers to the least time interval. The presented system is operated at $1\mathrm{\text{kHz}};$ however, the system delay, including computational and transport latencies, is measured as $5\mathrm{\text{ms}},$ so $L$ is regarded as $5\mathrm{\text{ms}}.$ It is still fast enough to apply the TDE to physiological hand tremors, ranging from $6\mathrm{\text{Hz}}$ to $30\mathrm{\text{Hz}}.$^[20] ${\widehat{f}}_{\mathit{\text{sys}}}(·;t-L)$ can thus be calculated by $\frac{d}{\mathit{\text{dt}}}\mathit{\text{pos}}\left(t-L\right)-\widehat{m}\times u\left(t-L\right),$ which are known values. The motor input in Equation (9)(9) $$\frac{d}{\mathit{\text{dt}}}\mathit{\text{pos}}\left(t\right)=\widehat{m}u\left(t\right)+{\widehat{f}}_{\mathit{\text{sys}}}\left(·;t\right)$$(9) can be expressed as follows: (10) $$u\left(t\right)={\widehat{m}}^{-1}(\frac{d}{\mathit{\text{dt}}}\mathit{\text{pos}}\left(t\right)-{\widehat{f}}_{\mathit{\text{sys}}}\left(·;t-L\right))$$(10)

The error, $e\left(t\right),$ is defined as ${\mathit{\text{pos}}}_d(t)-\mathit{\text{pos}}\left(t\right),$ and ${\mathit{\text{pos}}}_d$ is a constant. Here, the desired error dynamics of $\frac{d}{\mathit{\text{dt}}}e\left(t\right)+\mathit{\text{Ke}}\left(t\right)=0$ with a proportional control gain of $K$ can be applied, which returns the motor input for the TDC as follows: (11) $$u\left(t\right)={\widehat{m}}^{-1}(\mathit{\text{Ke}}\left(t\right)-{\widehat{f}}_{\mathit{\text{sys}}}\left(·;t-L\right))=K_{\mathit{TDC}}e\left(t\right)-{\widehat{m}}^{-1}\frac{d}{\mathit{\text{dt}}}\mathit{\text{pos}}\left(t-L\right)+u\left(t-L\right)$$(11)

Therefore, $u\left(t\right)$ uses one control gain $K_{\mathit{\text{TDC}}}$ ($=K/\widehat{m}$), and a nonlinear term can be calculated by the previously measured positions and motor inputs.

The block diagram of the proposed system is shown in Figure 5(b). We developed custom operation software based on Visual C++ and the Microsoft Foundation Class (MFC) library. This software comprises two synchronized threads: one for acquiring the OCT spectrum and the other for the OCT signal processing with the CUDA programming, $\mathit{\text{pos}}$ and $\theta$ calculation and applying the motor input. At the OCT signal processing block, two distance indexes from the processed OCT signal are first acquired by the surface detection algorithm using a low-pass filter, thresholding, and derivatives.^[28] Subsequently, $\mathit{\text{pos}}$ and $\theta$ are calculated and stored in a buffer with a size of $10.$ A temporal moving average filter is applied to the position buffer to reduce high-frequency noise. As a motion estimator (ME), the derivative of the filtered position buffer is used to obtain the delay compensation term.^[39] This estimator using the delayed motion buffer is remarkably efficient and robust when $L$ is small enough and has been adopted to estimate physiological motion for robot-assisted surgery.

The proportional gain $K_{\mathit{\text{TDC}}}$ is multiplied by the error to calculate the motor input. In addition, the TDE-based nonlinear term is calculated by considering the previous position and motor input, as indicated by the blue dotted box. The TDE result is combined with $K_{\mathit{\text{TDC}}}e(t),$ and the saturation function of $\pm 280$ is applied to apply motor input limits. The motor driver moves the PZT motor with a velocity of $\frac{d}{\mathit{\text{dt}}}{d}_{\mathit{\text{PZT}}}\left(t\right),$ and nonlinear disturbances are added from both $\frac{d}{\mathit{\text{dt}}}{d}_{\mathit{\text{PZT}}}\left(t\right)$ and $\frac{d}{\mathit{\text{dt}}}{d}_{\mathit{\text{Trem}}}\left(t\right).$

The step response of the microinjector is studied to optimize the control gain using three control methods: P controller with motion estimation (ME) (P + ME),^[28] PID controller with ME (PID + ME),^{[13–17,22,27]} and the proposed TDC. Ziegler and Nichols’s gain tuning method ^[40] is adopted to determine each control gain heuristically, as shown in Figure 6(a)–(c), and a step signal from $450 \mu \mathrm{m}$ to $550 \mu \mathrm{m}$ is used to compare three control methods in terms of the transient responses, such as rising and settling times and overshoot, as shown in Figure 6(d). For the P + ME controller, the optimal proportional gain, $K_P$ is selected to $3.3.$ It shows a rising time of $36.6\mathrm{\text{ms}},$ an overshoot of $0.09,$ and a settling time of $62.7\mathrm{\text{ms}}$ for the $5\mathrm{\%}$ criterion. The PID + ME controller is implemented by introducing the integration gain ($K_I)$ and derivative gains ($K_D$). $K_I$ and $K_D$ enhanced the system response; however, excessive overshoot and oscillation occur when considerable disturbances are applied. Therefore, the optimal combinations of PID gains are obtained at a slightly lower $K_P$ of $2.8$ with $K_I$ and $K_D$ of $0.15$ and $0.1,$ respectively. The PID + ME can achieve a faster settling time of $58.6\mathrm{\text{ms}}$ with less overshoot of $0.07$ than the results of the P + ME controller. The proposed TDC has one control gain, $K_{\mathit{\text{TDC}}},$ and it is noticeable that the proposed TDC has a minor overshoot and damped oscillations, as indicated by the asterisks. The optimized $K_{\mathit{\text{TDC}}}$ is $6.8,$ with the fastest settling time of $50.6\mathrm{\text{ms}}$ and the smallest overshoot of $0.02.$

Figure 6. The control gain optimization of (a) P, (b) PID, and (c) TDC. (d) The comparison of the step responses and (e) the motor inputs by the conventional methods and the proposed TDC method with the optimized control gain.

The recorded motor inputs for each control method during the step response are shown in Figure 6(e). As the asterisks indicate, the proposed TDC can react faster than other methods to get fewer overshoots and settling times for the same rising time. Subtraction of the TDE nonlinear term, ${\widehat{m}}^{-1}{\widehat{f}}_{\mathit{\text{sys}}}(·;t-L),$ allows applying the higher motor input with less damped oscillations, resulting in a faster response. The proposed TDC can effectively regulate the needle tip position by rapidly rejecting the disturbance with less overshoot.

3. Results and discussions

3.1. Control results on recorded hand tremors

We designed the experiment using the recorded tremor to validate control performances, as shown in Figure 7(a). The hand tremors are recorded for $>30$ seconds without applying the motor input, as shown in the inset. Subsequently, a position regulation task was performed where the system tracked a target position derived from the recorded hand tremor. An aluminum plate is mounted on a rotational stage to analyze the effect of the tilted angle on the control performance. Even though the same recorded tremor is applied, the control performances of each method are affected by the tilted angle. Ten recorded $\mathit{\text{pos}}$ are used as the desired position, and the tracking performance is evaluated at different reference angles from $45°$ to $135°$ by fixing the microinjector.

Figure 7. (a) Schematic diagram of the experiment on the recorded tremor. (b) Representative tracking results on the simulated hand tremor at 45°.

The part of the representative control results at $\theta$ of $45°$ is shown in Figure 7(b). The black line indicates the error when the motor input is not used. Then, each control method is applied to compensate for the same recorded tremor. The proposed TDC shows the fastest reaction to the sudden changes in disturbance than the other methods, as indicated by the asterisks in three inset graphs. The abrupt error changes stem from both the hand’s sudden movements and the motor’s non-uniform stroke. Hence, the proposed TDC enhances the system response to these instantaneous disturbances while preventing an extensive motor input.

The RMSE results of three control methods at each reference angle are shown in Figure 8(a). Ten recorded tremors are applied at each reference angle from $45°$ to $135°$ with $15°$ intervals. For all the control methods, the least RMSE is achieved when $\theta$ is $90°,$ and the control performance gets worse as $\theta$ deviates from $90°.$ The one-tailed t-test is conducted with the hypothesis that one RMSE dataset is statistically lower than the other. The $P$-value is the probability of rejecting the above hypothesis, so the lower $p,$ the more apparent the difference between the two datasets. One asterisk ($\mathrm{*}$) and two asterisks ($**$) indicate $p$ of less than $5\mathrm{\%}$ and $1\mathrm{\%},$ respectively. The proposed TDC is shown to have the lowest RMSE values for all angles with a confidence level of $>95\mathrm{\%};$ the minimum and maximum mean RMSEs are $5.5 \mu \mathrm{m}$ and $7.8 \mu \mathrm{m}$ at $45°$ and $90°,$ respectively. The corresponding values for the P + ME and PID + ME are $6.3 \mu \mathrm{m}$ and $8.1 \mu \mathrm{m}$ as well as $6.0 \mu \mathrm{m}$ and $8.1 \mu \mathrm{m},$ respectively.

Figure 8. (a) RMSE control results of each control method at several angles. (b) Comparison of position regulation results by each control method.

The frequency-domain analysis is also performed, as shown in Figure 8(b). The black line represents the FFT result of all recorded tremors (total time length is 35 minutes) without position regulation. There is an upwards convex trend in the 5 Hz to 20 Hz range, which agrees with the well-known bandwidth of involuntary hand tremors.^[20] It is also noticeable that the shape of FFT results is almost the same, which indicates that all the control results have similar residual errors on the same disturbances.

All the control methods significantly reduced the FFT amplitude in the decibel scale, and the TDC shows the least FFT amplitudes among all the frequency ranges. The inset graphs show the FFT amplitudes in the frequency ranges of $5\mathrm{\text{Hz}}$ to $14\mathrm{\text{Hz}}$ and $18\mathrm{\text{Hz}}$ to $26\mathrm{\text{Hz}},$ respectively. The FFT result of the proposed TDC gets lower than the other two control methods as the analysis frequency band increases. The results of the P + ME and the PID + ME become higher FFT values than the recorded tremor at $18\mathrm{\text{Hz}},$ which means the two control methods have high-frequency residual errors. On the other hand, the proposed TDC could show the reduction of the hand tremor over a wider frequency band up to $24\mathrm{\text{Hz}}.$ The TDC can efficiently drive the motor to compensate for high-frequency disturbance.

3.2. Injection results on an agarose-gel phantom study

An injection phantom made of $2\mathrm{\%}$ agarose gel is prepared to have an average eyeball curvature, as shown in Figure 9(a). As an experimental protocol, the needle is moved from the right to left sides of the agar phantom and is inserted at three injection points where the tilted angles are $45°,$ $90°,$ and $135°.$ The target injection depth is determined to be $250 \mu \mathrm{m}$ beneath the sample surface, so the desired $d_{\mathit{\text{inj}}}$ values for each injection are $-353 \mu \mathrm{m},$ $-250 \mu \mathrm{m},$ and $-353 \mu \mathrm{m},$ respectively. Distance locking is operated when we transversely move the needle location from the first to the last injection place. Thus, the needle tip position is maintained from the sample surface, and the measured angle continuously changes from $45°$ to $135°.$

Figure 9. Injection performances on the eyeball phantom made of 2 % agarose-gel. (a) Schematic of the injection experiment with three injection points at tilted angles of 45 °, 90 °, and 135 °. (b) The representative θ and pos measurement results during the one-cycle injection experiment. (c) The representative microscopic images, OCT images, and pos measurements during the injection at each desired point. (d) Comparison of RMSE results during the injection for each control method.

The representative $\theta$ and $\mathit{\text{pos}}$ measurement results with the proposed TDC are shown in Figure 9(b). Three injections were made at the ${\theta }_{\mathit{\text{inj}}}$ of $47°,$ $90°,$ and $133°,$ and the averages of $\mathit{\text{pos}}$ for each injection were $-341.8 \mu \mathrm{m},$ $-250.0 \mu \mathrm{m},$ and $-336.1 \mu \mathrm{m},$ respectively. The more inclination increases, the more considerable injection depth is required to get the same needle tip position from the sample surface. The microscopic and OCT images at each injected point are shown in Figure 9(c). The commercial OCT microscope (Callisto $930\mathrm{\text{nm}}$ OCT Imaging System, Thorlabs) is used to visualize the cross-section of the injected agar phantom, and the white dotted line indicates the target depth. The needle has been inserted to the consistent depths for all angles, and RMSE values are measured to $7.0 \mu \mathrm{m},$ $3.1 \mu \mathrm{m},$ and $4.8 \mu \mathrm{m},$ respectively. Furthermore, we compared the injection experiment results for each control method: P + ME, PID + ME, and TDC, as shown in Figure 9(d). The RMSE results of the TDC are statistically lower than the other two methods, with $>95\mathrm{\%}$ confidence level. The TDC’s best and worst averaged RMSE results are $3.1 \mu \mathrm{m}$ and $7.8 \mu \mathrm{m},$ respectively, resulting in average improvements of $26.5\mathrm{\%}$ and $18.5\mathrm{\%}$ from the P + ME and the PID + ME, respectively.

3.3. Injection results on ex-vivo porcine eye

A porcine eye is used as an ex-vivo sample to perform the subretinal injection, as shown in Figure 10(a). The porcine eye was achieved from the slaughterhouse on the day of slaughter. We observed the injection procedure through a microscope, and two illuminations from OCT sensors can be visualized, as shown in the inset image. The target layer is an RPE, and indocyanine green (ICG) is used as an injection material. The representative results of the measured $\theta$ and $\mathit{\text{pos}}$ are shown in Figure 10(b). Two control types can be clearly distinguished: distance-locking and injection and the injection has been performed at the measured injection angle of $45°.$ The calculated injection depth is $-338.3 \mu \mathrm{m},$ and the control RMSE is measured as $10.3 \mu \mathrm{m}.$ The needle was maintained under the retina for over $30$ seconds while gently perfusing ICG.

Figure 10. Injection results on an ex-vivo porcine eye. (a) Experimental setup image of the porcine eye injection and the microscopic image of the sensorized needle. (b) Representative θ and pos measurement results during the ex-vivo porcine eye experiment. (c) 3D OCT images of the injected ex-vivo porcine eye (c1) with cosine measurement error and (c2) without cosine measurement error. (d) 2D OCT images: (d1) en-face image of the injected layer and cross-section images of both (d2) non-injected and (d3) injected area.

The typical injection results are shown in Figure 10(c). The desired injection angle is set to $45°,$ so the cosine measurement error accounts for $\sim 77 \mu \mathrm{m}.$ This error made shallow injection depth, often causing drug reflux, as shown in Figure 10(c1). On the contrary, the proposed sensorized needle can compensate for the cosine measurement error, providing the accurate position estimation of the needle tip at an oblique angle. It results in a successful injection, as shown in Figure 10(c2). Both an injected hole and a subretinal bleb could be observed.

2D en-face and cross-sectional images of the injected lesion are acquired, as shown in Figure 10(d). The acquired en-face image can visualize the punched hole by the needle and injected ICG, as shown in inset Figure 10(d1). In the case of the non-injected area, an intact ILM, retina, choroid, and RPE layer can be observed, as shown in Figure 10(d2). On the other hand, the injected area has an incision by the needle and the injected ICG, as shown in Figure 10(d3). Hence, the ICG has been successfully injected at the RPE layer between the retina and choroid.

4. Conclusion

In this study, a sensorized handheld microinjector with an adaptive TDC was proposed and implemented to achieve precise robot-assisted subretinal injection. Two OCT distance sensors measure the position and angle between the needle tip and the target sample. The presented $\mathit{\text{pos}}$ estimation could mitigate inevitable cosine measurement errors. In addition, a TDC scheme was designed to compensate for disturbances effectively with the improved control RMSE, broaden control bandwidth, and rapidly respond to the abrupt involuntary motions. We demonstrated that the proposed system performed better than the conventional methods through the recorded tremor and the phantom injection experiments. Our proposed TDC method excels in its adaptive estimation of system nonlinearity and disturbances, which empowers it to provide rapid and robust control over a broad spectrum of hand tremor patterns. This effectiveness is consistently demonstrated through significant performance improvements in both the time and frequency domains. We examined the ex-vivo porcine eye to perform an actual injection of a drug to the RPE layer. The proposed method was able to accurately estimate the inclined needle tip position at the clinically demanded injection angle, so it can significantly alleviate both drug reflux and retinal damage. Moreover, the needle position can be precisely maintained at the desired depth with an RMSE of $10.3 \mu \mathrm{m},$ comparable to the recent benchtop robotic in-human study.

The proposed method is anticipated to serve as a crucial step toward a handheld microinjection system, facilitating precise motor regulation and accurate position estimation for clinical use. Here, we utilized two fiber-optic sensors to ensure compatibility with commercial intraoperative OCT, which provides real-time retinal cross-sectional imaging. However, as part of our future work, we are actively working on incorporating another distance sensor for 3D position estimation. To develop a more versatile sensorized needle, exploring optimal hardware and software design is crucial. It includes ensuring a needle with a small enough diameter to be compatible with standard surgical trocars, while enabling fast and precise position regulation control. Further studies are also required to integrate with a clinical subretinal injection procedure. In addition, these sensorized needles and TDC-based position regulations can be applied to other industrial or medical applications where accurate position estimation and disturbance rejection play essential roles.

Disclosure statement

No potential conflicts of interest was reported by the author(s).

Additional information

Funding

This work was supported by the DGIST R&D Program of the Ministry of Science and ICT (23-RT-01) and the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 10171504).