Improving the condition number of estimated covariance matrices

Figures & data

Table 1 Change in standard deviation for the SOAR covariance matrix for both methods of reconditioning.

κmax	$\sigma$	${\sigma }_{RR}$	α corr. RR	${\sigma }_{ME}$	α corr. ME
1000	2.23606	2.26471	1.013	2.25439	1.008
500	2.23606	2.29340	1.026	2.27599	1.018
100	2.23606	2.51306	1.124	2.45737	1.099

Columns 4 and 6 show α, the multiplicative inflation factor corresponding to the values for ${\sigma }_{RR}$ and ${\sigma }_{ME},$ respectively.

Fig. 1. Changes to correlations between the original SOAR matrix and the reconditioned matrices for ${\kappa }_{\mathit{max}}=100.$ (a) shows $\mathbf{C}(100,:)={\mathbf{C}}_{\mathit{MVI}}(100,:)$ (black solid), ${\mathbf{C}}_{RR}(100,:)$ (red dashed), ${\mathbf{C}}_{ME}(100,:)$ (blue dot-dashed) (b) shows $100\times \frac{\mathbf{C}(100,:)-{\mathbf{C}}_{RR}(100,:)}{\mathbf{C}(100,:)}$ (red dashed) and $100\times \frac{\mathbf{C}(100,:)-{\mathbf{C}}_{ME}(100,:)}{\mathbf{C}(100,:)}$ (blue dot-dashed). As the SOAR matrix is symmetric, we only plot the first 100 entries for (b).

Fig. 2. Standard deviations for the IASI covariance matrix $\mathbf{\Sigma }$ (black solid), ${\mathbf{\Sigma }}_{RR}$ (red dashed), ${\mathbf{\Sigma }}_{ME}$ (blue dot-dashed) for ${\kappa }_{\mathit{max}}=100.$

Fig. 3. Difference in correlations for IASI (a) $(\mathbf{C}-{\mathbf{C}}_{RR})°\mathit{sign}(\mathbf{C}),$ (b) $(\mathbf{C}-{\mathbf{C}}_{ME})°\mathit{sign}(\mathbf{C}),$ and (c) $({\mathbf{C}}_{ME}-{\mathbf{C}}_{RR})°\mathit{sign}(\mathbf{C}),$ where $°$ denotes the Hadamard product. Red indicates that the absolute correlation is decreased by reconditioning and blue indicates the absolute correlation is increased. The colourscale is the same for (a) and (b) but different for (c). Condition numbers of the corresponding covariance matrices are given by $\kappa (\mathbf{R})=2005.98,\kappa ({\mathbf{R}}_{RR})=100$ and $\kappa ({\mathbf{R}}_{ME})=100.$

Fig. 4. Change in pointwise difference of discrete Fourier transform (DFT) from ${\mathbf{x}}_{\mathit{est}}$ to ${\mathbf{x}}_{\mathit{mod}}$ where a_est denotes the vector of coefficients of the imaginary part of $\mathit{DFT}({\mathbf{x}}_{\mathit{est}}).$ A positive (negative) value indicates that ${\mathbf{x}}_{\mathit{mod}}$ is closer to (further from) ${\mathbf{x}}_{\mathit{true}}$ than ${\mathbf{x}}_{\mathit{est}}$ and the amplitude shows how large this change is. Vertical dashed lines show the locations of non-zero values for the true signal.

Fig. 5. Difference between a_true and a_RR for different choices of κ_max. Vertical dashed lines show the locations of non-zero values for the true signal.

Table 2 Changes to convergence of RR, MI and MVI for different values of κ_max.

κmax	10, 000	1, 000	100	50	10
RR	245	244	170	141	73
ME	240	239	193	145	76
Infl RR	244	244	238	233	199

For all choices of κ_max, convergence for ${\mathbf{R}}_{\mathit{true}}$ occurs in 17 iterations and ${\mathbf{R}}_{\mathit{est}}$ occurs in 244 iterations.