A new regularization approach for numerical differentiation

Abstract

It is well known that the problem of numerical differentiation is an ill-posed problem and one requires regularization methods to approximate the solution. The commonly practiced regularization methods are (external) parameter-based like Tikhonov regularization, which has certain inherent difficulties associated with them. In such scenarios, iterative regularization methods serve as an attractive alternative. In this paper, we propose a novel iterative regularization method where the minimizing functional does not contain the noisy data directly, but rather a smoothed or integrated version of it. The advantage, in addition to circumventing the use of noisy data directly, is that the sequence of functions constructed during the descent process tends to avoid overfitting, and hence, does not corrupt the recovery significantly. To demonstrate the effectiveness of our method we compare the numerical results obtained from our method with the numerical results obtained from certain standard regularization methods such as Tikhonov regularization, Total-variation, etc.

Keywords:

• Numerical differentiation
• Newton-type methods
• Mathematical programming methods
• optimization and variational techniques
• Volterra integral equations
• inverse problems
• Tikhonov regularization
• iterative regularization
• numerical analysis

1991 Mathematics Subject Classifications:

• Primary 65D25
• Secondary 49M15
• 65K05
• 65K10
• 45D05
• 45Q05

Acknowledgments

I am very grateful to Prof. Ian Knowles for his support, encouragement and stimulating discussions throughout the preparation of this paper.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 The technicality as to how far g can be weakened so that the solution of equation (1(1) $T_D\phi =g_1,$(1) ) is uniquely and stably recovered (finding the ϕ inversely) will be discussed later.

2 Again the domain space for the functionals $G_1$, $G_2$ and G are discussed in details later.

3 It can be further proved that it's also the first Fr$\acute{e}$chet derivative of G at ψ.

4 Hence $T_D^{\ast }$ is also a linear and bounded operator in ${\mathcal{L}}^2[a,b]$.

5 Again, it can be proved that it's the second Fr$\acute{e}$chet derivative of G at ψ.

6 Just for simplicity, we assume $g(a)=\tilde{g}(a)$.

7 In our experiments, we considered $\tau =1$ and the termination condition as $||T{\psi }_m-g_{\delta }|{|}_{{\mathcal{L}}^2}<\delta$.

8 As explained in the ${\mathcal{L}}^2$-gradient version of the descent algorithm for G, in Section 4.

9 To be consistent with ${}^6$, we kept $\tilde{g}(a)=g(a)$ and $\tilde{g}(b)=g(b)$, but can be avoided.

10 Where the test function is $g(x)=|x-0.5|$ on $[0,1]$ and the data set is 100 uniformly distributed points with $\sigma =0.01$.

11 the fluctuating occurs since the values of $||\tilde{g}-g_m|{|}_{{\mathcal{L}}^2}$ tends to decrease first (when approximating the exact g) and then increases (when making a transition from g to $\tilde{g}$) and eventually decreases (when trying to fit the noisy $\tilde{g}$, i.e. overfitting)