4,380
Views
4
CrossRef citations to date
0
Altmetric
Short Communication

A recursive algorithm for computing the inverse of the Vandermonde matrix

| (Reviewing Editor)
Article: 1175061 | Received 21 Oct 2015, Accepted 31 Mar 2016, Published online: 29 Apr 2016

Abstract

The inverse of a Vandermonde matrix has been used for signal processing, polynomial interpolation, curve fitting, wireless communication, and system identification. In this paper, we propose a novel fast recursive algorithm to compute the inverse of a Vandermonde matrix. The algorithm computes the inverse of a higher order Vandermonde matrix using the available lower order inverse matrix with a computational cost of O ( n 2 ) . The proposed algorithm is given in a matrix form, which makes it appropriate for hardware implementation. The running time of the proposed algorithm to find the inverse of a Vandermonde matrix using a lower order Vandermonde matrix is compared with the running time of the matrix inversion function implemented in MATLAB.

Public Interest Statement

The Vandermonde matrix and its inverse have been widely used in many applications, such as polynomial interpolation and signal processing. In this paper, a fast recursive algorithm is proposed to find the inverse of a Vandermonde matrix. We show that the inverse of a ( n + 1 ) × ( n + 1 ) Vandermonde matrix can be computed recursively using the inverse of a reduced size n × n Vandermonde matrix. The size of n × n Vandermonde matrix can be further reduced until we reach to a size small enough that the inverse can be computed easily.

Furthermore, the proposed algorithm can be used for finding the inverse Vandermonde matrix when the entries are observed sequentially. The proposed algorithm in each iteration uses the new entry and the previous inverse matrix to compute the inverse of the increased size Vandermonde matrix.

1. Introduction

The Vandermonde matrix and its inverse have been widely used in many applications, such as polynomial interpolation (Phillips, Citation2003), curve fitting (Wilf, Citation1958), system identification (Barker, Tan, & Godfrey, Citation2004), signal reconstruction (Olkkonen & Olkkonen, Citation2010), wireless communication (Wang, Scaglione, Giannakis, & Barbarossa, Citation1999), and signal processing (Ryan & Debbah, Citation2009). Computing the inverse of a Vandermonde matrix has been extensively studied over the last four decades. Tou (Citation1964) and Wertz (Citation1965) used the Lagrange interpolation formula and expressed the elements of the inverse of a Vandermonde matrix as the coefficients of a polynomial. Explicit formulas for finding the inverse of the Vandermonde matrix are given in Kaufman (Citation1969), Neagoe (Citation1996), El-Mikkawya (Citation2003), Respondek (Citation2016). Authors in Gautschi and Inglese (Citation1988) found lower bounds for the condition number of the Vandermonde matrices and showed that the bounds grow exponentially as the size of the matrix increases. Therefore, the Vandermonde matrices are usually ill conditioned and the methods proposed in Kaufman (Citation1969), Neagoe (Citation1996), El-Mikkawya (Citation2003) may fail to accurately compute the elements of the inverse matrix. In later works, fast algorithms are derived in O ( n 2 ) and O ( n 3 ) to compute the elements of the inverse of the Vandermonde matrix (Eisinberg & Fedele, Citation2006; Gohberg & Olshevsky, Citation1997).

In this paper, we propose a novel recursive algorithm for computing the inverse of the Vandermonde matrix. We show that the inverse of a ( n + 1 ) × ( n + 1 ) Vandermonde matrix can be computed recursively using the inverse of a reduced size n × n Vandermonde matrix. The size of n × n Vandermonde matrix can be further reduced until we reach to a size small enough that the inverse can be computed easily. One of the advantages of the proposed recursive algorithm is its capability to update the inverse matrix when the size of the matrix increases due to observing new incoming entries. The entries of the Vandermonde matrix can be considered as a sequence such that by observing each new entry, the size of the matrix increases by one. In the real-world applications, we may confront situations where a complete set of the entries is not available in advance and the matrix entries are observed sequentially (Aliyari Ghassabeh & Abrishami Moghaddam, Citation2013; Aliyari Ghassabeh, Rudzicz, & Abrishami Moghaddam, Citation2015). The proposed techniques in Kaufman (Citation1969), Neagoe (Citation1996), El-Mikkawya (Citation2003), Gohberg and Olshevsky (Citation1997), Eisinberg and Fedele (Citation2006) require to have access to the whole entries in advance to find the inverse matrix. In contrast to the previously mentioned methods, the proposed recursive algorithm has the ability to observe the entries sequentially and update the new (increased size) inverse matrix simultaneously. The computational cost for computing the inverse of ( n + 1 ) × ( n + 1 ) Vandermonde matrix using n × n dimensional inverse matrix after observing the new entry is O ( n 2 ) , which is considerably less than the usual matrix inversion techniques. Furthermore, the proposed recursive algorithm is presented in a matrix form that makes it suitable for hardware implementation and reduces the computational time and complexity.

The paper is organized as follows: Section 2 introduces the new recursive algorithm for computing the inverse of a Vandermonde matrix. The simulation results are given in Section 3, where the running time of the proposed algorithm for computing the inverse of a Vandermonde matrix is compared with the running time of the inverse function implemented in MATLAB. The concluding remarks are given in Section 4.

2. Proposed recursive algorithm

Let A n = A ( a 1 , a 2 , , a n ) , n 1 denote an n × n Vandermonde matrix defined over the set of all complex numbers C , A ( a 1 , a 2 , , a n ) = 1 1 1 a 1 a 2 a n a 1 n - 1 a 2 n - 1 a n n - 1 , a i C , i = 1 , , n .

The determinant of A n Footnote 1 is given by | A | = 1 i < j n ( a j - a i ) (Mirsky, Citation2011). Therefore, the Vandermonde matrix is nonsingular if and only if the parameters a i , i = 1 , , n are distinct.

Definition 1

Let B n = B ( a 1 , a 2 , , a n ) , n 1 be an n × n matrix defined by B n = B ( a 1 , a 2 , , a n ) = a 1 a 2 a n a 1 2 a 2 2 a n 2 a 1 n a 2 n a n n , a i C , i = 1 , , n .

Then we have the following proposition

Proposition 1

   (Horn & Johnson, Citation1990) Suppose that A n and B n are defined as above. Let d 0 denote the determinant of the Vandermonde matrix A n , i.e. | A n | = d . Assume a i 0 , i = 1 , , n , then | B n | = d i = 1 n a i and B n - 1 is given by B n - 1 = 1 d c o f A n 11 a 1 c o f A n 21 a 1 c o f A n n 1 a 1 c o f A n 12 a 2 c o f A n 22 a 2 c o f A n n 2 a 2 c o f A n 1 n a n c o f A n 2 n a n c o f A n n n a n ,

where c o f A n i j denotes the ijth cofactor of the matrix A n .

Therefore, if A n - 1 is available then B n - 1 can be computed with complexity O ( n 2 ) using Proposition 1.Footnote 2 Now we are in a position to prove the following lemma that introduces a recursive algorithm for finding the inverse of a Vandermonde matrix

Lemma 1

Let A n + 1 = A ( a 1 , a 2 , , a n + 1 ) denote an ( n + 1 ) × ( n + 1 ) Vandermonde matrix such that a i 0 , i = 1 , , n + 1 . Let B n = B ( a 1 , a 2 , , a n ) be an n × n matrix according to Definition 1. Let I n denote the identity matrix of order n. Furthermore, let b i j denote the ijth element of B n - 1 . Then the inverse of the Vandermonde matrix A n + 1 is given by the following recursive formula A n + 1 - 1 = E n + 1 C n + 1 D n + 1 ,

where D n + 1 = 1 0 0 0 B n - 1 0 , C n + 1 = 1 - 1 - 1 - 1 - 1 - 1 2 1 1 1 - 1 1 2 1 1 - 1 1 1 1 2 , E n + 1 = 1 c 1 0 0 - c 2 c 1 - c 3 c 1 I n - c n + 1 c 1 ,

where B n - 1 is computed using Proposition 1, C n + 1 is a constant matrix, and c i , i = 1 , , n + 1 are given by(1) c 1 = 1 - a 11 - a 21 - a 31 - a n 1 , c i + 1 = a i 1 - c 1 , i = 1 , , n a i 1 = b i 1 a n + 1 + b i 2 a n + 1 2 + + b i n a n + 1 n , i = 1 , , n . (1)

Remark 1

(a)

The last two equations in 1 can be rewritten in matrix form as follows (2) a 11 a 21 a n 1 = B n - 1 a n + 1 a n + 1 2 a n + 1 n , c 2 c 3 c n + 1 = B n - 1 a n + 1 a n + 1 2 a n + 1 n - c 1 1 1 1 . (2)

(b)

For a two dimensional Vandermonde matrix A 2 = A ( a 1 , a 2 ) , where a 1 and a 2 are nonzero and distinct, D 2 , C 2 , and E 2 are given as followsFootnote 3 (3) D 2 = 1 0 0 1 a 2 , C 2 = 1 - 1 - 1 2 , E 2 = a 2 a 2 - a 1 0 - 2 a 1 + a 2 a 2 - a 1 1 . (3) By multiplying these three matrices, we obtain E 2 C 2 D 2 = 1 a 2 - a 1 a 2 - 1 - a 1 1 = A 2 - 1 , which is the well-known formula for the inverse of a 2 × 2 Vandermonde matrix 1 1 a 1 a 2 .

(c)

To compute E n , it appears that c 1 cannot be zero. The following lemma guarantees that c 1 is always non-zero.

Lemma 2

The coefficient c 1 defined in (1) is always a non-zero number.

Expressing the equations in a matrix form reduces the running time and makes the hardware implementation easier. Note that the coefficients c i , i = 2 , , n + 1 are needed to construct E n + 1 and using (2) they can be computed in O ( n 2 ) . The complexity of computing the matrix product C n + 1 D n + 1 is O ( n 2 ) and the complexity of multiplying the result by E n + 1 is O(n), therefore the inverse of a ( n + 1 ) × ( n + 1 ) Vandermonde matrix A n + 1 - 1 can be found using A n - 1 in O ( n 2 ) . The required steps for finding A n + 1 - 1 = A - 1 ( a 1 , a 2 , , a n + 1 ) from A n - 1 = A - 1 ( a 1 , a 2 , , a n ) is given as follows

(a)

Compute B n - 1 using Proposition 1.

(b)

Construct C n + 1 , D n + 1 , and E n + 1 using (1) and (2).

(C)

The inverse matrix is the product of the matrices in step (b), i.e. A n + 1 - 1 = E n + 1 C n + 1 D n + 1 .

If the inverse of A n is not available, we can represent A n - 1 as a function of A n - 1 - 1 and continue the recursion procedure. The recursion continues until we reach a Vandermonde matrix with a known inverse matrix.

The recursive algorithm for finding the inverse of an n × n Vandermonde matrix, A n - 1 , can be summarized as follows

Note that we also can start with a 2 × 2 Vandermonde matrix A 2 , increase the dimension of the Vandermonde matrix by one in each iteration and compute its inverse using Lemma 1. The iterations stop until the Vandermonde matrix has the desired size. In the next section, we compare the running time of the proposed algorithm with the running time of the matrix inversion function implemented in MATLAB.

3. Simulation results

In the following simulations we compute the running time of the proposed algorithm to find the inverse of a Vandermonde matrix in an adaptive manner for polynomial interpolation. The running time is compared with the inverse function implemented in MATLAB. The algorithm is implemented in MATLAB and the simulations run on a PC with Intel Pentium 4, 2.6 GHZ CPU, and 2048 Mb RAM.

Given a set of n observations ( x i , y i ) , i = 0 , 1 , , n - 1 , where x i s are distinct and x i 0 , i = 0 , 1 , n - 1 , we can find a unique polynomial p of degree n - 1 such that p ( x i ) = y i , i = 0 , 1 , , n - 1 (Phillips, Citation2003). Suppose that the interpolation polynomial p is given by p ( x ) = a 0 + a 1 x + a 2 x + a n - 1 x n - 1 . Then the problem can be written in the following matrix form(4) 1 x 0 x 0 n - 2 x 0 n - 1 1 x 1 x 1 n - 2 x 1 n - 1 1 x n - 1 x n - 1 n - 2 x n - 1 n - 1 a 0 a 1 a n - 1 = y 0 y 1 y n - 1 . (4)

It is a system of n linear equations and the unknown coefficients a i , i = 0 , 1 , , n - 1 are given by(5) a 0 a 1 a n - 1 = ( A n - 1 ) t y 0 y 1 y n - 1 , (5)

where A n = A ( x 0 , x 1 , , x n - 1 ) is an n × n Vandermonde matrix. It is clear from (5) that finding the unknown coefficients involves computing the inverse of the associated Vandermonde matrix and multiplying it by [ y 0 , y 1 , , y n - 1 ] t .

In the following simulations we assume that the observations are made sequentially and the observed data are used to find the polynomial p that passes through the given points. We assume that the input data are observed in an increasing order, i.e. 0 < x 0 < x 1 < < x n - 1 . As the number of observed data increases the degree of the polynomial p also linearly increases. For example, if we have k pairs of input data, the degree of the polynomial p is k - 1 and by observing the next pair of the data the degree of the polynomial increases to k. In other words, upon arrival of each observation the size of the associated Vandermonde matrix increases by one and we are required to compute the inverse of the new increased size Vandermonde matrix. After each observation, the current matrix inversion algorithms need the whole data set to compute the inverse of the Vandermonde matrix (e.g. the matrix inverse function in MATLAB requires the whole observations to find the inverse matrix). But the proposed algorithm uses only the current observation and the previous inverse matrix to update the inverse of the higher order Vandermonde matrix. As mentioned before, the computational cost for updating the Vandermonde matrix using the proposed algorithm is O ( n 2 ) , which is much less than most regular matrix inversion techniques.

Figure 1. Comparison between the running times of the proposed algorithm and the matrix inversion function in MATLAB. The size of the Vandermonde matrix increases from 10 to 80, and in each iteration the time required to find the inverse function is computed

Figure 1. Comparison between the running times of the proposed algorithm and the matrix inversion function in MATLAB. The size of the Vandermonde matrix increases from 10 to 80, and in each iteration the time required to find the inverse function is computed

We compute the required time for updating the Vandermonde matrix using the proposed algorithm as a function of the size of the matrix. The results are compared with the running time of the function implemented in MATLAB for computing the inverse matrix. For each matrix size, we repeated the experiment 100 times and the average running time was found. The initial size of the Vandermonde matrix is 10 and by adding new entries (observing new data) it gradually increases to 80. For the sake of simplicity, the input elements of a k × k Vandermonde matrix are assumed to be 1 , 2 , , k , i.e., A k = A ( 1 , 2 , , k ) . As mentioned before, for finding the polynomial p with degree n - 1 we need n pairs of observations ( x i , y i ) , i = 1 , 2 , , n . Since our goal here is to test the performance of the proposed algorithm for finding the inverse of the Vandermonde matrix, we just need the first element of each observation. Figure compares the running times of the proposed algorithm with the MATLAB inverse function for finding the inverse of a Vandermonde matrix as a function of the matrix size. The x-axis in Figure is the size of the Vandermonde matrix, and y-axis is the requires time (second) to find the inverse matrix. It can be observed from Figure that the proposed algorithm is faster than the matrix inversion function in MATLAB.Footnote 4

4. Conclusion

Computing the inverse of a Vandermonde matrix arises in many applications such as polynomial interpolation, curve fitting, and signal processing. In this paper, we proposed a fast recursive algorithm to find the inverse of a Vandermonde matrix. The proposed algorithm in each iteration uses the new entry and the previous inverse matrix to compute the inverse of the increased size Vandermonde matrix. The proposed algorithm can be implemented as a recursive function to find the inverse of a Vandermonde matrix recursively. The running times of the proposed algorithm to find the inverse of a Vandermonde matrix are compared with the inverse function implemented in MATLAB and the simulation results showed that for a sequential data the proposed algorithm is faster than the inverse function in MATLAB.

Additional information

Funding

The author received no direct funding for this research.

Notes on contributors

Youness Aliyari Ghassabeh

Youness Aliyari Ghassabeh received the BS degree in electrical engineering from the University of Tehran, Tehran, Iran, in 2004, the MS degree from K.N. Toosi University of Technology, Tehran, in 2006, and the PhD degree in mathematics and engineering from Queen’s University, Kingston, ON, Canada, in 2013. He was a postdoctoral fellow at the Toronto Rehabilitation Institute, University Health Network, Toronto, ON between 2013 to 2014. He is currently a manager in Operational Risk Methodology group at Bank of Montreal, Toronto, ON, Canada. His main research interests include machine learning, statistical pattern recognition, probability theory and stochastic processes, image/signal processing, source coding and information theory.

Notes

1 For the sake of simplicity, hereafter we use A n to refer to an n × n Vandermonde matrix A ( a 1 , a 2 , , a n ) .

2 Note that c o f A n i j / d , i , j = 1 , , n is jith element of A n - 1 (Horn & Johnson, Citation1990).

3 We assume that the new entries are added from the left to the Vandermonde matrix. So, for a 2 × 2 Vandermonde matrix A 2 = A ( a 1 , a 2 ) , we have A 1 = 1 and B 1 = a 2 . For the general n × n case, see the Proof of Lemma 1 in Appendix.

4 Note that for the proposed algorithm, the required time for finding the inverse matrix using the previous inverse matrix and the new entry is reported.

5 Otherwise, the determinant of the Vandermonde matrix is zero and the inverse does not exist

References

  • Aliyari Ghassabeh, Y. , & Abrishami Moghaddam, H. (2013). Adaptive linear discriminant analysis for online feature extraction. Machine Vision Applications , 24 , 777–794.
  • Aliyari Ghassabeh, Y. , Rudzicz, F. , & Abrishami Moghaddam, H. (2015). Fast incremental LDA feature extraction. Pattern Recognition , 31 , 1999–2012.
  • Barker, H. A. , Tan, A. H. , & Godfrey, K. R. (2004). Optimal levels of perturbation signals for nonlinear system identification. IEEE Transactions on Automatic Conrol , 49 , 1404–1407.
  • El-Mikkawya, M. E. A. (2003). Inversion of a generalized Vandermonde matrix. International Journal of Computer Mathematics , 80 , 759–765.
  • Eisinberg, A. , & Fedele, G. (2006). On the inversion of the Vandermonde matrix. Applied Mathematics and Computation , 174 , 1384–1397.
  • Gautschi, W. , & Inglese, G. (1988). Lower bounds for the condition number of Vandermonde matrix. Numerische Mathematik , 52 , 241–250.
  • Gohberg, I. , & Olshevsky, V. (1997). The fast generalized Parker--Traub algorithm for inversion of Vandermonde and related matrices. Journal of Complexity , 13 , 208–234.
  • Horn, R. A. , & Johnson, C. R. (1990). Matrix analysis . Cambridge: Cambridge University Press.
  • Kaufman, I. (1969). The inversion of the Vandermonde matrix and transformation to the Jordan canonical form. IEEE Transactions on Automatic Control , 14 , 774–777.
  • Mirsky, L. (2011). An introduction to linear algebra . New York, NY: Dover Publications.
  • Neagoe, V. (1996). Inversion of the Van der Monde matrix. IEEE Signal Processing Letters , 3 , 119–120.
  • Olkkonen, H. , & Olkkonen, J. T. (2010). Sampling and reconstruction of transient signals by parallel exponential filters. IEEE Transactions on Circuits and Systems - Part II: Express Briefs , 57 , 426–429.
  • Phillips, G. M. (2003). Interpolation and approximation by polynomials . New York, NY: Springer Verlag.
  • Ryan, O. , & Debbah, M. (2009). Asymptotic behaviour of random Vandermonde matrices with entries on the unit circle. IEEE Transaction on Information Theory , 55 , 3115–3147.
  • Respondek, J. S. (2016). Incremental numerical recipes for the high efficient inversion of the confluent Vandermonde matrices. Computers & Mathematics with Applications , 71 , 489–502.
  • Wertz, H. (1965). On the numerical inversion of a recurrent problem: The Vandermonde matrix. IEEE Transactions on Automatic Control , 10 , 492.
  • Wang, Z. , Scaglione, A. , Giannakis, G. , & Barbarossa, S. (1999). Vandermonde--Lagrange mutually orthogonal flexible transceivers for blind CDMA in unknown multipath. In Proceedings of 2nd IEEE Workshop on Signal Processing Advances in Wireless Communications (pp. 42–45). Minneapolis, MN.
  • Wilf, H. S. (1958). Curve-fitting matrices. The American Mathematical Monthly , 65 , 272–274.
  • Tou, J. (1964). Determination of the inverse Vandermonde matrix. IEEE Transactions on Automatic Control , 9 , 314.

Appendix 1

Proof of Lemma 1

Proof

We show that E n + 1 C n + 1 D n + 1 A n + 1 = I n + 1 , where I n + 1 is a ( n + 1 ) × ( n + 1 ) identity matrix. To achieve this goal, we start by computing the product of D n + 1 and A n + 1 , D n + 1 A n + 1 = 1 0 0 0 B n - 1 0 1 1 1 1 a n + 1 a 1 a 2 a n a n + 1 n a 1 n a 2 n a n n .

It is straightforward to show that D n + 1 A n + 1 = 1 1 1 1 a 11 1 0 0 a 21 0 1 0 a n 1 0 0 1 = 1 1 1 1 a 11 a 21 I n a n 1 ,

where(A1) a 11 = b 11 a n + 1 + b 12 a n + 1 2 + + b 1 n a n + 1 n , a 21 = b 21 a n + 1 + b 22 a n + 1 2 + + b 2 n a n + 1 n , = a n 1 = b n 1 a n + 1 + b n 2 a n + 1 2 + + b n n a n + 1 n . (A1)

The above equations can be written in the following compact matrix form(A2) a 11 a 21 a n 1 = B n - 1 a n + 1 a n + 1 2 a n + 1 n . (A2)

Now, we evaluate C n + 1 D n + 1 A n + 1 , C n + 1 D n + 1 A n + 1 = 1 - 1 - 1 - 1 - 1 - 1 - 1 2 1 1 1 1 - 1 1 2 1 1 1 - 1 1 1 1 1 2 1 1 1 1 a 11 1 0 0 a 21 0 1 0 a n 1 0 0 1 .

It is simple to show that(A3) C n + 1 D n + 1 A n + 1 = c 1 0 0 0 0 c 2 1 0 0 0 c 3 0 1 0 0 c n 0 0 1 0 c n + 1 0 0 0 1 = c 1 0 0 c 2 I n c n + 1 , (A3)

where(A4) c 1 = 1 - a 11 - a 21 - a 31 - a n 1 , (A4) (A5) c 2 = - 1 + 2 a 11 + a 21 + a 31 a n 1 = a 11 - c 1 , c 3 = - 1 + a 11 + 2 a 21 + a 31 a n 1 = a 21 - c 1 , c 4 = - 1 + a 11 + a 21 + 2 a 31 a n 1 = a 31 - c 1 , = = c n + 1 = - 1 + a 11 + a 21 + a 31 2 a n 1 = a n 1 - c 1 . (A5)

The above equalities can be written in the following matrix(A6) c 2 c 3 c n + 1 = a 11 a 21 a n 1 - c 1 1 1 1 . (A6)

It remains to show that E n + 1 C n + 1 D n + 1 A n + 1 = I n + 1 . Using Equation (8), we obtain(A7) E n + 1 C n + 1 D n + 1 A n + 1 = 1 c 1 0 0 - c 2 c 1 - c 3 c 1 I n - c n + 1 c 1 E n + 1 c 1 0 0 c 2 I n c n + 1 C n + 1 D n + 1 A n + 1 = I n + 1 . (A7)

Therefore, E n + 1 C n + 1 D n + 1 = A n + 1 - 1 .

Appendix 2

Proof of Lemma 2

Proof

We use contradiction to show c 1 0 . Assume the coefficient c 1 is zero, i.e. c 1 = 0 . Then using Equations (1) and (2), we have(B1) a 11 + a 21 + + a n 1 = 1 (B1) (B2) B n a 11 a 21 a n 1 = a n + 1 a n + 1 2 a n + 1 n . (B2)

By combining Equations (9) and (10), we obtain(B3) 1 1 1 1 a 1 a 2 a n a n + 1 a 1 2 a 2 2 a n 2 a n + 1 2 a 1 n a 2 n a n n a n + 1 n a 11 a 21 a n 1 - 1 = 0 0 0 0 . (B3)

The first matrix in (15) is a ( n + 1 ) × ( n + 1 ) Vandermonde matrix A n + 1 . Since a i , i = 1 , , n + 1 are distinct,Footnote 5 therefore A n + 1 is a nonsingular matrix and its inverse exists. By multiplying both sides of Equation (15) by A n + 1 - 1 , we obtain(B4) a 11 a 21 a n 1 - 1 = 0 0 0 0 . (B4)

The above equality implies that our assumption, c 1 = 0 , is not correct. Therefore, c 1 0 .