Duality Mapping for Schatten Matrix Norms

Abstract

In this paper, we fully characterize the duality mapping over the space of matrices that are equipped with Schatten norms. Our approach is based on the analysis of the saturation of the Hölder inequality for Schatten norms. We prove in our main result that, for $p\in (1,\infty ),$ the duality mapping over the space of real-valued matrices with Schatten-p norm is a continuous and single-valued function and provide an explicit form for its computation. For the special case p = 1, the mapping is set-valued; by adding a rank constraint, we show that it can be reduced to a Borel-measurable single-valued function for which we also provide a closed-form expression.

Keywords:

• Banach spaces
• duality mapping
• Hölder inequality
• Schatten norm
• singular value decomposition

1. Introduction

In linear algebra and matrix analysis, Schatten norms are a family of spectral matrix norms that are defined via the singular-value decomposition [1]. They have appeared in many applications such as image reconstruction [2, 3], image denoising [4], and tensor decomposition [5], to name a few.

Generally, the Schatten-p norm of a matrix is the ${\ell }_p$ norm of its singular values [6]. The family contains some well-known matrix norms: The Frobenius and the spectral (operator) norms are special cases in the family, with p = 2 and $p=\infty ,$ respectively. The case p = 1 (trace or nuclear norm) is of particular interest for applications as it can be used to recover low-rank matrices [7]. This is the current paradigm in matrix completion, where the goal is to recover an unknown matrix given some of its entries [8]. Prominent examples of applications that can be reduced to low-rank matrix-recovery problems are phase retrieval [9], sensor-array processing [10], system identification [11], and index coding [12, 13].

In addition to their many applications in data science, Schatten norms have been extensively studied from a theoretical point of view. Various inequalities concerning Schatten norms have been proven [14–22]; sharp bounds for commutators in Schatten spaces have been given [23, 24]; moreover, facial structure [25], Fréchet differentiablity [26], and various other aspects [27, 28] have been studied already.

Our objective in this paper is to investigate the duality mapping in spaces of matrices that are equipped with Schatten norms. The duality mapping is a powerful tool to understand the topological structure of Banach spaces [29, 30]. It has been used to derive powerful characterizations of the solution of variational problems in function spaces [31, 32] and also to determine generalized linear inverse operators [33]. Here, we prove that the duality mapping over Schatten-p spaces with $p\in (1,+\infty )$ is a single-valued and continuous function which, in fact, highlights the strict convexity of these spaces. Although the provided characterization is intuitive, we could not find it in the literature and this is, to the best of our knowledge, the first work which provides a direct way of computing this mapping in this case. For the special case p = 1, the mapping is set-valued. However, we prove that, by adding a rank constraint, it reduces to a single-valued Borel-measurable function. In both cases, we also derive closed-form expressions that allow one to compute them explicitly.

The paper is organized as follows: In Section 2, we present relevant mathematical tools and concepts that are used in this paper. We study the duality mapping of Schatten spaces and propose our main result in Section 3. We provide further discussions regarding the introduced mappings in Section 4.

2. Preliminaries

2.1. Dual norms, Hölder inequality, and duality mapping

Let V be a finite-dimensional vector space that is equipped with an inner-product $〈·,·〉:V\times V\to \mathbb{R}$ and let $\left|\right|·|{|}_X:V\to {\mathbb{R}}_{\ge 0}$ be an arbitrary norm on V. We then denote by X the space V equipped with $\left|\right|·|{|}_X.$ Clearly, X is a Banach space, because all finite-dimensional normed spaces are complete. The dual norm of X, denoted by $\left|\right|·|{|}_{X\prime }:V\to {\mathbb{R}}_{\ge 0},$ is defined as (1) $$\left|\right|\mathbf{v}|{|}_{X\prime }=\underset{\mathbf{u}\in V\backslash \left\{\mathbf{0}\right\}}{\sup }\frac{〈\mathbf{v},\mathbf{u}〉}{\left|\right|\mathbf{u}|{|}_X},$$(1) for any $\mathbf{v}\in V.$ Following this definition, one would directly obtain the generic duality bound (2) $$〈\mathbf{v},\mathbf{u}〉\le \left|\right|\mathbf{v}\left|{|}_{X\prime }\right|\left|\mathbf{u}\right|{|}_X,$$(2) for any $\mathbf{v},\mathbf{u}\in V.$ Saturation of Inequality (2(2) $$〈\mathbf{v},\mathbf{u}〉\le \left|\right|\mathbf{v}\left|{|}_{X\prime }\right|\left|\mathbf{u}\right|{|}_X,$$(2) ) is the key concept of dual conjugates that is formulated in the following definition.

Definition 1.

Let V be a finite-dimensional vector space and let $(||·|{|}_X,\left|\right|·|{|}_{X\prime })$ be a pair of dual norms that are defined over V. The pair $(\mathbf{u},\mathbf{v})\in V\times V$ is said to be a $(X,X\prime )$-conjugate, if

• $〈\mathbf{v},\mathbf{u}〉=\left|\right|\mathbf{v}\left|{|}_{X\prime }\right|\left|\mathbf{u}\right|{|}_X,$

• $\left|\right|\mathbf{v}|{|}_{X\prime }=|\left|\mathbf{u}\right|{|}_X.$

For any $\mathbf{u}\in V,$ the set of all elements $\mathbf{v}\in V$ such that $(\mathbf{u},\mathbf{v})$ forms an $(X,X\prime )$-conjugate is denoted by ${\mathcal{J}}_X(\mathbf{u})\subseteq V.$ We refer to the set-valued mapping ${\mathcal{J}}_X:V\to 2^V$ as the duality mapping. If, for all $\mathbf{u}\in V,$ the set ${\mathcal{J}}_X(\mathbf{u})$ is a singleton, then we indicate the duality mapping for the X-norm via the single-valued function ${\mathrm{J}}_X:V\to V$ with ${\mathcal{J}}_X(\mathbf{u})=\{{\mathrm{J}}_X(\mathbf{u})\}.$

It is worth mentioning that, for any $\mathbf{u}\in V,$ the set ${\mathcal{J}}_X(\mathbf{u})$ is nonempty. In fact, the closed ball $B=\{\mathbf{v}\in V:|\left|\mathbf{v}\right|{|}_{X\prime }=\left|\right|\mathbf{u}\left|{|}_X\right\}$ is compact and, hence, the function $\mathbf{v}\mapsto 〈\mathbf{v},\mathbf{u}〉$ attains its maximum value at some ${\mathbf{v}}^{*}\in B.$ Now, following Definition 1, one readily verifies that $(\mathbf{u},{\mathbf{v}}^{*})$ is an $(X,X\prime )$-conjugate.

We conclude this part by providing a classical and illustrative example. Let $V={\mathbb{R}}^n$ for some $n\in \mathbb{N}.$ For any $p\in [1,+\infty ],$ the ${\ell }_p$-norm of a vector $\mathbf{u}=(u_i)\in {\mathbb{R}}^n$ is defined as (3) $$\left|\right|\mathbf{u}|{|}_p=\{\begin{array}{cc}{\left(\sum _{i=1}^n|u_i{|}^p\right)}^{\frac{1}{p}},\hfill & p<+\infty \hfill \\ {\max }_i\left|u_i\right|,\hfill & p=+\infty .\hfill \end{array}$$(3)

It is widely known that the dual norm of ${\ell }_p$ is the ${\ell }_q$-norm, where (p, q) are Hölder conjugates (i.e., $1/p+1/q=1$) [34]. This stems from the Hölder inequality which states that (4) $$〈\mathbf{v},\mathbf{u}〉\le \left|\right|\mathbf{u}\left|{|}_p\right|\left|\mathbf{v}\right|{|}_q,$$(4) for all $\mathbf{u}=(u_i),\mathbf{v}=(v_i)\in {\mathbb{R}}^n.$ In the sequel, we exclude the trivial cases $\mathbf{u}=\mathbf{0}$ and $\mathbf{v}=\mathbf{0}$ to avoid unnecessary complexities in our statements.

When $1<p<+\infty ,$ Inequality (4(4) $$〈\mathbf{v},\mathbf{u}〉\le \left|\right|\mathbf{u}\left|{|}_p\right|\left|\mathbf{v}\right|{|}_q,$$(4) ) is saturated if and only if $u_i{v}_i\ge 0$ for $i=1,\dots ,n$ and there exists a constant c > 0 such that $|\mathbf{u}{|}^p=c|\mathbf{v}{|}^q,$ where $|\mathbf{u}{|}^p=(|u_i{|}^p).$ This ensures that the duality mapping is single-valued and also yields the map (5) $${\mathrm{J}}_p(\mathbf{u})=\text{sign}(\mathbf{u})\frac{|\mathbf{u}{|}^{p-1}}{\left|\right|\mathbf{u}|{|}_p^{p-2}}.$$(5)

For p = 1, one can verify that the equality happens if and only if, for any index $i=1,\dots ,n$ with $u_i\ne 0,$ one has that (6) $$v_i=\text{sign}(u_i)\left|\right|\mathbf{v}|{|}_{\infty }.$$(6)

In other words, the vector $\mathbf{v}$ should attain its extreme values at places where $\mathbf{u}$ has nonzero values, with the sign being determined by the corresponding element in $\mathbf{u}.$

Due to (6(6) $$v_i=\text{sign}(u_i)\left|\right|\mathbf{v}|{|}_{\infty }.$$(6) ), the set ${\mathcal{J}}_1(\mathbf{u})$ is not necessarily a singleton. However, if we add an additional sparsity constraint, then the mapping becomes single-valued. This leads us to introduce the new notion of sparse duality mapping in Definition 2.

Definition 2.

Let V be a finite-dimensional vector space and let $s_0:V\to \mathbb{N}$ be an integer-valued function that acts as a sparsity measure. Assuming a pair $(||·|{|}_X,\left|\right|·|{|}_{X\prime })$ of dual norms over V, we call the pair $(\mathbf{u},\mathbf{v})\in V\times V$ a sparse $(X,X\prime )$-conjugate if

• $(\mathbf{u},\mathbf{v})$ forms an $(X,X\prime )$-conjugate pair. In other words, $\mathbf{v}\in \mathcal{J}(\mathbf{u}).$

• The quantity $s_0(\mathbf{v})$ attains its minimal value over the set $\mathcal{J}(\mathbf{u}).$

We denote the set of sparse conjugates of $\mathbf{u}$ by ${\mathcal{J}}_{X,s_0}(\mathbf{u}).$ Whenever ${\mathcal{J}}_{X,s_0}(\mathbf{u})$ is a singleton for any $\mathbf{u}\in V,$ we refer to the single-valued function ${\mathrm{J}}_{X,s_0}:V\to V$ with ${\mathcal{J}}_{X,s_0}(\mathbf{u})=\{{\mathrm{J}}_{X,s_0}(\mathbf{u})\}$ as the sparse duality mapping.

Following Definition 2, if we use the ${\ell }_0$-norm as the sparsity measure, that is $s_0(\mathbf{u})=\left|\right|\mathbf{u}|{|}_0=\text{Card}(\{i:u_i\ne 0\})$¹, then we have the sparse duality mapping (7) $$\begin{array}{c}{\mathrm{J}}_{1,0}:{\mathbb{R}}^n\to {\mathbb{R}}^n:\mathbf{u}=(u_i)\mapsto \mathbf{v}=(v_i)={\mathrm{J}}_{1,0}(\mathbf{u}),\\ v_i=\{\begin{array}{cc}\text{sign}(u_i)\left|\right|\mathbf{u}|{|}_1,\hfill & u_i\ne 0\hfill \\ 0,\hfill & u_i=0.\hfill \end{array}\end{array}$$(7)

Finally, we mention that, for p = + ∞, the reduced set ${\mathcal{J}}_{\infty ,0}$ is not single-valued. Indeed, let us define $I_{\max }(\mathbf{u})=\{i:|u_i|=|\left|\mathbf{u}\right|{|}_{\infty }\}\subseteq \{1,\dots ,n\}.$ We readily deduce from (6) that $\mathbf{v}=(v_1,\dots ,v_n)\in {\mathcal{J}}_{\infty }(\mathbf{u})$ if and only if v_i = 0 whenever $i\notin I_{\max }(\mathbf{u})$ and $\text{sign}(v_i)=\text{sign}(u_i)$ for $i\in I_{\max }(\mathbf{u})$ with $\sum _{i\in I_{\max }(\mathbf{u})}\left|v_i\right|=\left|\right|\mathbf{u}|{|}_{\infty }.$ This shows that ${\mathcal{J}}_{\infty }(\mathbf{u})$ is a convex set with ${\mathcal{J}}_{\infty ,0}(\mathbf{u})$ being its extreme points, where ${\mathcal{J}}_{\infty ,0}(\mathbf{u})=\{u_i{\mathbf{e}}_i:i\in I_{\max }(\mathbf{u})\}.$

2.2. Schatten p-norm

It is widely known that any matrix $\mathbf{A}\in {\mathbb{R}}^{m\times n}$ can be decomposed as (8) $$\mathbf{A}=\mathbf{US}{\mathbf{V}}^T,$$(8) where $\mathbf{U}\in {\mathbb{R}}^{m\times m}$ and $\mathbf{V}\in {\mathbb{R}}^{n\times n}$ are orthogonal matrices and $\mathbf{S}$ is an m by n rectangular diagonal matrix with nonnegative real entries ${\sigma }_1\ge {\sigma }_2\ge \cdots \ge {\sigma }_{\min (m,n)}\ge 0$ sorted in descending order [35]. In the literature, (8) is known as the singular-value decomposition (SVD) and the entries σ_i are the singular values of $\mathbf{A}.$ In general, the SVD of a matrix $\mathbf{A}$ is not unique. However, the diagonal matrix $\mathbf{S}$ and, consequently, its entries, are fully determined from $\mathbf{A}.$ In other words, the values of σ_i are invariant to a specific choice of decomposition. This is why one can refer to the diagonal entries of $\mathbf{S}$ as the “singular values” of $\mathbf{A}.$

When $\mathbf{A}$ is not full rank, one can obtain a reduced version of (8). Indeed, if we denote the rank of $\mathbf{A}$ by r, then we have that (9) $$\mathbf{A}={\mathbf{U}}_r{\mathbf{S}}_r{\mathbf{V}}_r^T,$$(9) where ${\mathbf{U}}_r\in {\mathbb{R}}^{m\times r}$ and ${\mathbf{V}}_r\in {\mathbb{R}}^{n\times r}$ are (sub)-orthogonal matrices such that ${\mathbf{U}}_r^T{\mathbf{U}}_r={\mathbf{V}}_r^T{\mathbf{V}}_r={\mathbf{I}}_r$ and ${\mathbf{S}}_r=\text{diag}(\mathit{\sigma })$ is a diagonal matrix that contains positive singular values $\mathit{\sigma }=({\sigma }_1,\dots ,{\sigma }_r)\in {\mathbb{R}}^r$ of $\mathbf{A}.$

For any $p\in [1,+\infty ],$ the Schatten-p norm of $\mathbf{A}$ is defined as (10) $$\left|\right|\mathbf{A}|{|}_{S_p}=\{\begin{array}{cc}{\left(\sum _{i=1}^r{\sigma }_i^p\right)}^{\frac{1}{p}},\hfill & p<+\infty \hfill \\ {\sigma }_1,\hfill & p=+\infty .\hfill \end{array}$$(10)

We remark that (10(10) $$\left|\right|\mathbf{A}|{|}_{S_p}=\{\begin{array}{cc}{\left(\sum _{i=1}^r{\sigma }_i^p\right)}^{\frac{1}{p}},\hfill & p<+\infty \hfill \\ {\sigma }_1,\hfill & p=+\infty .\hfill \end{array}$$(10) ) defines a family of quasi norms for $p\in (0,1).$ In the extreme case p = 0, the Schatten-0 norm actually coincides with the rank of the matrix, i.e. $\left|\right|\mathbf{A}|{|}_{S_0}=\text{rank}(\mathbf{A}).$ The Schatten quasi norms have also been studied in the literature from both theoretical and practical point of views (see, [36–39], and references therein).

3. Duality mapping in Schatten spaces

For any $p\in [1,\infty ],$ the dual of the Schatten-p norm is the Schatten-q norm, where $q\in [1,\infty ]$ is such that $\frac{1}{p}+\frac{1}{q}=1$ [1]. This is due to the generalized version of Hölder’s inequality for Schatten norms, as stated in Proposition 1. While this is a known result (see, for example, [40]), it is also the basis for the present work, which is the reason why we provide a proof in A.

Proposition 1.

For any pair $(p,q)\in {[1,+\infty ]}^2$ of Hölder conjugates with $\frac{1}{p}+\frac{1}{q}=1$ and any pair of matrices $\mathbf{A},\mathbf{B}\in {\mathbb{R}}^{m\times n}$, we have that (11) $$〈\mathbf{A},\mathbf{B}〉=\text{Tr}\left({\mathbf{A}}^T\mathbf{B}\right)\le \left|\right|\mathbf{A}\left|{|}_{S_p}\right|\left|\mathbf{B}\right|{|}_{S_q}.$$(11)

In Proposition 2, we investigate the case where the Hölder inequality is saturated, in the sense that (12) $$\text{Tr}\left({\mathbf{A}}^T\mathbf{B}\right)=\left|\right|\mathbf{A}\left|{|}_{S_p}\right|\left|\mathbf{B}\right|{|}_{S_q}.$$(12)

This saturation is central to our work, as it is tightly linked to the notion of duality mapping.

Proposition 2.

Let (p, q) be a pair of Hölder conjugates and let $\mathbf{A},\mathbf{B}\in {\mathbb{R}}^{m\times n}$ be a pair of nonzero matrices with reduced SVDs of the form (13) $$\mathbf{A}={\mathbf{U}}_r\text{diag}(\mathit{\sigma }){\mathbf{V}}_r^T,\hspace{1em}\mathbf{B}={\tilde{\mathbf{U}}}_{\tilde{r}}\text{diag}(\tilde{\mathit{\sigma }}){\tilde{\mathbf{V}}}_{\tilde{r}}^T.$$(13)

• If $p\in (1,\infty )$, then the Hölder inequality is saturated if and only if we have that (14) $$\mathbf{B}=c{\mathbf{U}}_r\text{diag}({\mathrm{J}}_p(\mathit{\sigma })){\mathbf{V}}_r^T$$(14) or, equivalently, (15) $$\mathbf{A}=c^{-1}{\tilde{\mathbf{U}}}_{\tilde{r}}\text{diag}({\mathrm{J}}_q(\tilde{\mathit{\sigma }})){\tilde{\mathbf{V}}}_{\tilde{r}}^T,$$(15)

  where $c=\frac{\left|\right|\mathbf{B}|{|}_{S_q}}{\left|\right|\mathbf{A}|{|}_{S_p}}$ and ${\mathrm{J}}_p(·)$ and ${\mathrm{J}}_q(·)$ are the duality mappings for the ${\ell }_p$ and ${\ell }_q$ norms, respectively (see (5(5) $${\mathrm{J}}_p(\mathbf{u})=\text{sign}(\mathbf{u})\frac{|\mathbf{u}{|}^{p-1}}{\left|\right|\mathbf{u}|{|}_p^{p-2}}.$$(5) )).

• If p = 1, then a necessary condition for the saturation of the Hölder inequality is that (16) $$\text{rank}(\mathbf{A})\le r_1\le \text{rank}(\mathbf{B}),$$(16)

where $r_1=\text{Card}(\{i:{\tilde{\sigma }}_i={\tilde{\sigma }}_1\})$ is the multiplicity of the first singular value of $\mathbf{B}$. Moreover, if we denote the first r₁ singular vectors of $\mathbf{B}$ in (13) by ${\tilde{\mathbf{U}}}_1\in {\mathbb{R}}^{m\times r_1}$ and ${\tilde{\mathbf{V}}}_1\in {\mathbb{R}}^{n\times r_1}$, then the Hölder inequality is saturated if and only if there exists a symmetric matrix $\mathbf{X}\in {\mathbb{R}}^{r_1\times r_1}$ such that (17) $$\mathbf{A}={\tilde{\mathbf{U}}}_1\mathbf{X}{\tilde{\mathbf{V}}}_1^T.$$(17)

Finally in the rank-equality case $\text{rank}(\mathbf{A})=\text{rank}(\mathbf{B})$, we have saturation if and only if (18) $$\mathbf{B}=c{\mathbf{U}}_r{\mathbf{V}}_r^T,$$(18)

where $c=\left|\right|\mathbf{B}|{|}_{S_{\infty }}$ and the matrices U_r and V_r are defined in (13).

Remark 1.

Note that even though the reduced SVD is not unique (i.e. there are multiple choices for the sub-orthogonal matrices in (13(13) $$\mathbf{A}={\mathbf{U}}_r\text{diag}(\mathit{\sigma }){\mathbf{V}}_r^T,\hspace{1em}\mathbf{B}={\tilde{\mathbf{U}}}_{\tilde{r}}\text{diag}(\tilde{\mathit{\sigma }}){\tilde{\mathbf{V}}}_{\tilde{r}}^T.$$(13) )), the parametric forms given in Proposition 2 do not depend on a specific decomposition and the results are invariant to any arbitrary choice of these reduced SVDs, primarily due to the “only if” parts of the statements.

The proof of Proposition 2 can be found in B. We observe that, in the case $p\in (1,\infty ),$ the saturation of Hölder inequality provides a very tight link between the two matrices: If we know one of them, then the other lies in a one-dimensional ray that is parameterized by the constant c > 0. However, in the special case p = 1, the identification is not as simple. There again, for a given matrix $\mathbf{B},$ one can fully characterize the set of admissible matrices $\mathbf{A}.$ However, for the reverse direction, an additional rank-equality constraint is essential to reduce the set of admissible matrices $\mathbf{B}$ to just one ray.

Inspired from Proposition 2, we now propose our main result in Theorem 1, where we explicitly characterize the duality mapping for the Schatten p-norms. The proof of Theorem 1 can be found in C.

Theorem 1.

Let $p,q\in [1,+\infty ]$ be a pair of Hölder conjugates with $\frac{1}{p}+\frac{1}{q}=1$ and $\mathbf{A}\in {\mathbb{R}}^{m\times n}$ a matrix whose reduced SVD is specified in (9(9) $$\mathbf{A}={\mathbf{U}}_r{\mathbf{S}}_r{\mathbf{V}}_r^T,$$(9) ).

• If $1<p<+\infty$, then the single-valued duality mapping ${\mathrm{J}}_{S_p}:{\mathbb{R}}^{m\times n}\to {\mathbb{R}}^{m\times n}$ is well-defined and can be expressed as (19) $${\mathrm{J}}_{S_p}:\mathbf{A}={\mathbf{U}}_r\text{diag}(\mathit{\sigma }){\mathbf{V}}_r^T\mapsto {\mathbf{A}}^{*}={\mathbf{U}}_r\text{diag}({\mathrm{J}}_p(\mathit{\sigma })){\mathbf{V}}_r^T.$$(19)

• If p = 1 and if we consider the rank function as the sparsity measure in Definition 2, then the sparse duality mapping ${\mathrm{J}}_{S_1,\text{rank}}:{\mathbb{R}}^{m\times n}\to {\mathbb{R}}^{m\times n}$ is well-defined (singleton) and is given as (20) $${\mathrm{J}}_{S_1,\text{rank}}:\mathbf{A}={\mathbf{U}}_r\text{diag}(\mathit{\sigma }){\mathbf{V}}_r^T\mapsto {\mathbf{A}}^{*}=\left|\right|\mathit{\sigma }|{|}_1{\mathbf{U}}_r{\mathbf{V}}_r^T.$$(20)

• If p = + ∞, then the set-valued mapping ${\mathcal{J}}_{S_{\infty }}(·)$ can be described as (21) $${\mathcal{J}}_{S_{\infty }}(\mathbf{A})=\left\{{\sigma }_1{\mathbf{U}}_1\mathbf{X}{\mathbf{V}}_1^T:\mathbf{X}\in {\mathbb{R}}^{r_1\times r_1}\mathrm{ }\text{is}\mathrm{ }\text{symmetric}\mathrm{ }\text{and}\mathrm{ }\left|\right|\mathbf{X}|{|}_{S_1}=1\right\},$$(21)

where r₁ denotes the multiplicity of the first singular value σ₁ of $\mathbf{A}$ and ${\mathbf{U}}_1,{\mathbf{V}}_1$ are singular vectors that correspond to σ₁ in (9(9) $$\mathbf{A}={\mathbf{U}}_r{\mathbf{S}}_r{\mathbf{V}}_r^T,$$(9) ). Finally, the set of sparse dual conjugates is the collection of rank-1 elements of ${\mathcal{J}}_{S_{\infty }}(\mathbf{A})$ which can be characterized as (22) $${\mathcal{J}}_{S_{\infty },\text{rank}}(\mathbf{A})=\{{\sigma }_1{\mathbf{U}}_1\mathbf{p}{\mathbf{p}}^T{\mathbf{V}}_1^T:\mathbf{p}\in {\mathbb{R}}^{r_1},|\left|\mathbf{p}\right|{|}_2=1\}.$$(22)

4. Discussion

Theorem 1 provides an interesting characterization of the duality mapping in three scenarios: The first case is $1<p<+\infty$ which is the most straightforward one. Theorem 1 tells us that the mapping is single-valued and also gives a formula to compute the dual conjugate ${\mathbf{A}}^{*}$ of any matrix $\mathbf{A}\in {\mathbb{R}}^{m\times n}.$ We use this result to deduce the continuity of the duality mapping as well as the strict convexity of the Schatten space in this case (see Corollary 1). In the second case, with p = 1, the mapping is not single-valued. However, there is a unique element in the set of dual conjugates with the minimal rank (that is equal to the rank of $\mathbf{A}$) and, hence, we can construct a single-valued sparse duality mapping. Finally, we showed in the third case, characterized by p = + ∞, that neither the set of dual conjugates nor the ones with the minimal rank are unique.

In Corollary 1, we highlight some consequences of Theorem 1 concerning the strict convexity of Schatten spaces and the continuity of the duality mapping.

Corollary 1.

The Banach space of m by n matrices equipped with the Schatten-p norm is strictly convex, if and only if $p\in (1,+\infty )$. In this case, the function ${\mathrm{J}}_{S_p}:{\mathbb{R}}^{m\times n}\to {\mathbb{R}}^{m\times n}$ is continuous.

Proof.

For $p\in (1,+\infty ),$ we know from Theorem 1 that the duality mapping ${\mathrm{J}}_{S_p}$ is bijective. Moreover, it is known that all finite-dimensional Banach spaces are reflexive. Now, following [41], we deduce the strict convexity of the space of m by n matrices with Schatten-p norm.

For p = 1 and p = + ∞, we can readily verify that $$\begin{array}{c}{\left|\left|\alpha \left(\begin{array}{ccc}1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & 0\end{array}\right)+(1-\alpha )\left(\begin{array}{ccc}0& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & 1\end{array}\right)\right|\right|}_{S_1}={\left|\left|\left(\begin{array}{ccc}\alpha & \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & (1-\alpha )\end{array}\right)\right|\right|}_{S_1}=1,\\ {\left|\left|\alpha \left(\begin{array}{ccc}1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & 0\end{array}\right)+(1-\alpha )\left(\begin{array}{ccc}1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & 1\end{array}\right)\right|\right|}_{S_{\infty }}={\left|\left|\left(\begin{array}{ccc}1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & (1-\alpha )\end{array}\right)\right|\right|}_{S_{\infty }}=1,\end{array}$$ for all $\alpha \in (0,1),$ which shows that the Schatten space is not strictly convex for $p=1,+\infty .$

Finally, the Schatten-p norm is known to be Fréchet differentiable for $p\in (1,+\infty )$ [26]. Moreover, the duality mapping of any Banach space with Fréchet-differentiable norms is guaranteed to be continuous [42, 43]. Combining the two statements, we deduce the continuity of the duality mapping in this case. □

By contrast, the sparse duality mapping ${\mathrm{J}}_{S_1,\text{rank}}(·)$ is not continuous. This is best explained by providing a counterexample. Specifically, let us consider the sequence of 2 by 2 matrices $${\mathbf{S}}_k=\left(\begin{array}{cc}1& 0\\ 0& \frac{1}{k}\end{array}\right),\hspace{1em}k\in \mathbb{N}.$$

It is clear that ${\mathbf{S}}_k\to {\mathbf{S}}_{\infty }=\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right).$ However, we have that $$\forall k\in \mathbb{N}:{\mathrm{J}}_{S_1,\text{rank}}({\mathbf{S}}_k)=\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right),\hspace{1em}\text{while}\hspace{1em}{\mathrm{J}}_{S_1,\text{rank}}({\mathbf{S}}_{\infty })=\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right),$$ which shows the discontinuity of ${\mathrm{J}}_{S_1,\text{rank}}$ in the space of 2 by 2 matrices. This can be generalized to space of matrices with arbitrary dimensions $m,n\in \mathbb{N}.$

Although ${\mathrm{J}}_{S_1,\text{rank}}$ is not continuous, we now show that it is Borel-measurable and, hence, that it can be approximated with arbitrary precision by a continuous mapping due to Lusin’s theorem [34].

Proposition 3.

For any $m,n\in \mathbb{N}$, the sparse duality mapping ${\mathrm{J}}_{S_1,\text{rank}}$ is a Borel-measurable matrix-valued function over the space of m by n matrices.

Before going into the proof of Proposition 3, we present a preliminary result.

Lemma 1.

The set ${\mathcal{R}}_r\subseteq {\mathbb{R}}^{m\times n}$ of m by n matrices of rank r is Borel-measurable.

Proof.

First note that $${\mathcal{R}}_1=\{\mathbf{u}{\mathbf{v}}^T:\mathbf{u}\in {\mathbb{R}}^m,\mathbf{v}\in {\mathbb{R}}^n\}.$$

The set ${\mathcal{R}}_1$ is the image of the continuous mapping ${\mathbb{R}}^m\times {\mathbb{R}}^n\to {\mathbb{R}}^{m\times n}:(\mathbf{u},\mathbf{v})\mapsto \mathbf{u}{\mathbf{v}}^T$ and, hence, is Borel-measurable.

Now, denote by ${\mathcal{R}}_{\le r}\subseteq {\mathbb{R}}^{m\times n},$ the set of matrices with rank no more than r. Using the identity $${\mathcal{R}}_{\le r}={\mathcal{R}}_1+\cdots +{\mathcal{R}}_1,\hspace{1em}\text{(r }\text{times}\mathrm{)},$$ we deduce that ${\mathcal{R}}_{\le r}$ and, consequently, ${\mathcal{R}}_r={\mathcal{R}}_{\le r}\backslash {\mathcal{R}}_{\le (r-1)}$ are also Borel-measurable sets. □

Proof of Proposition 3.

Consider a Borel-measurable set $\mathcal{B}\subseteq {\mathbb{R}}^{m\times n}.$ We show that ${\mathcal{B}}_{\text{inv}}={\mathrm{J}}_{S_1,\text{rank}}^{-1}(\mathcal{B})$ is also Borel-measurable. By defining ${\mathcal{B}}_{\text{inv},r}={\mathcal{B}}_{\text{inv}}\cap {\mathcal{R}}_r,$ we can partition ${\mathcal{B}}_{\text{inv}}$ as $${\mathcal{B}}_{\text{inv}}=\underset{r=1}{\overset{\min (m,n)}{\cup }}{\mathcal{B}}_{\text{inv},r}.$$

Hence, it is sufficient to show that each partition ${\mathcal{B}}_{\text{inv},r}$ is Borel-measurable.

Define the set ${\mathcal{P}}_r\subseteq {\mathcal{R}}_r^2$ as $${\mathcal{P}}_r=\{(\mathbf{A},\mathbf{B})\in {\mathcal{R}}_r\times \mathcal{B}:\text{Tr}({\mathbf{A}}^T\mathbf{B})=|\left|\mathbf{A}\right|{|}_{S_1}\left|\right|\mathbf{B}|{|}_{S_{\infty }},\hspace{1em}|\left|\mathbf{A}\right|{|}_{S_1}=\left|\right|\mathbf{B}\left|{|}_{S_{\infty }}\right\}.$$

The set ${\mathcal{P}}_r$ introduces a relation over ${\mathcal{R}}_r$ whose domain is ${\mathcal{B}}_{\text{inv},r}.$ In other words, we have that $${\mathcal{B}}_{\text{inv},r}=\{\mathbf{A}\in {\mathcal{R}}_r:\exists \mathbf{B}\in \mathcal{B},(\mathbf{A},\mathbf{B})\in {\mathcal{P}}_r\}.$$

Since the trace and norm are continuous (and, consequently, Borel-measurable) functions and ${\mathcal{R}}_r\times \mathcal{B}$ is a Borel-measurable set (using Lemma 1), we deduce that the relation induced from ${\mathcal{P}}_r$ is Borel-measurable as well. Finally, we use [44, Proposition 2.1] to show that its domain is Borel-measurable. □

5. Conclusion

In this paper, we studied the duality mapping in finite-dimensional Schatten spaces. Based on a careful investigation of the cases where the Hölder inequality saturates, we provided an explicit form for this mapping when $p\in (1,+\infty ).$ Furthermore, by adding a rank constraint, we proved that the mapping becomes single-valued for the special case p = 1. As for p = + ∞, we showed that the mapping yields a convex set whose elements are explicitly characterized. Finally, we discussed our theorem and studied the continuity of the introduced mappings as well as the strict convexity of the Schatten spaces. A possible future direction of research is to extend the results of this paper to infinite-dimensional Schatten spaces and even, in full generality, to linear operators over Hilbert spaces.

A. Proof of Proposition 1

Proof.

Let us recall the reduced SVD of the matrix $\mathbf{A}$ as (23) $$\mathbf{A}={\mathbf{U}}_r{\mathbf{S}}_r{\mathbf{V}}_r^T,$$(23) where $r=\text{rank}(\mathbf{A}),{\mathbf{U}}_r=[{\mathbf{u}}_1\cdots {\mathbf{u}}_r]\in {\mathbb{R}}^{m\times r},{\mathbf{V}}_r=[{\mathbf{v}}_1\cdots {\mathbf{v}}_r]\in {\mathbb{R}}^{n\times r},$ and $\mathbf{S}=\text{diag}({\sigma }_1,\dots ,{\sigma }_r).$ Similarly, for the matrix $\mathbf{B},$ we have that (24) $$\mathbf{B}={\tilde{\mathbf{U}}}_{\tilde{r}}{\tilde{\mathbf{S}}}_{\tilde{r}}{\tilde{\mathbf{V}}}_{\tilde{r}}^T,$$(24) where $\tilde{r}=\text{rank}(\mathbf{A}),{\tilde{\mathbf{U}}}_{\tilde{r}}=[{\tilde{\mathbf{u}}}_1\cdots {\tilde{\mathbf{u}}}_{\tilde{r}}]\in {\mathbb{R}}^{m\times \tilde{r}},{\tilde{\mathbf{V}}}_{\tilde{r}}=[{\tilde{\mathbf{v}}}_1\cdots {\tilde{\mathbf{v}}}_r]\in {\mathbb{R}}^{n\times \tilde{r}},$ and $\tilde{\mathbf{S}}=\text{diag}({\tilde{\sigma }}_1,\dots ,{\tilde{\sigma }}_{\tilde{r}}).$ A direct computation then reveals that (25) $$\text{Tr}\left({\mathbf{A}}^T\mathbf{B}\right)=\sum _{i=1}^r\sum _{j=1}^{\tilde{r}}{\sigma }_i{\tilde{\sigma }}_j{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j.$$(25)

By using the weighted Hölder inequality for vectors [45], we obtain for $p\ne 1$ that (26) $$\sum _{i=1}^r\sum _{j=1}^{\tilde{r}}{\sigma }_i{\tilde{\sigma }}_j{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\le {\left(\sum _{i=1}^r{\sigma }_i^p\sum _{j=1}^{\tilde{r}}\left|{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\right|\right)}^{\frac{1}{p}}{\left(\sum _{j=1}^{\tilde{r}}{\sigma }_j^p\sum _{i=1}^r\left|{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\right|\right)}^{\frac{1}{q}}$$(26) and for p = 1 that (27) $$\sum _{i=1}^r\sum _{j=1}^{\tilde{r}}{\sigma }_i{\tilde{\sigma }}_j{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\le \left(\sum _{i=1}^r{\sigma }_i\sum _{j=1}^{\tilde{r}}\left|{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\right|\right)\left|\right|\tilde{\mathit{\sigma }}|{|}_{\infty }.$$(27)

Finally, by invoking Cauchy-Schwartz and the orthonormality of the matrices ${\mathbf{U}}_r,{\mathbf{V}}_r,{\tilde{\mathbf{U}}}_{\tilde{r}},{\tilde{\mathbf{V}}}_{\tilde{r}},$ we deduce for $i=1,\dots ,r$ that (28) $$\sum _{j=1}^{\tilde{r}}\left|{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\right|\le {\left(\sum _{j=1}^{\tilde{r}}{({\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j)}^2\right)}^{\frac{1}{2}}{\left(\sum _{j=1}^{\tilde{r}}{({\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j)}^2\right)}^{\frac{1}{2}}\le \left|\right|{\mathbf{u}}_i\left|{|}_2\right|\left|{\mathbf{v}}_i\right|{|}_2=1,$$(28)

For $j=1,\dots ,\tilde{r},$ we deduce that (29) $$\sum _{i=1}^r\left|{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\right|\le {\left(\sum _{i=1}^r{({\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j)}^2\right)}^{\frac{1}{2}}{\left(\sum _{i=1}^r{({\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j)}^2\right)}^{\frac{1}{2}}\le \left|\right|{\tilde{\mathbf{u}}}_i\left|{|}_2\right|\left|{\tilde{\mathbf{v}}}_i\right|{|}_2=1.$$(29)

The combination of these inequalities completes the proof. □

B. Proof of Proposition 2

Proof.

We separate the two cases and analyze each one independently.

Case 1: $1<p<+\infty .$ We prove (14(14) $$\mathbf{B}=c{\mathbf{U}}_r\text{diag}({\mathrm{J}}_p(\mathit{\sigma })){\mathbf{V}}_r^T$$(14) ) and deduce (15(15) $$\mathbf{A}=c^{-1}{\tilde{\mathbf{U}}}_{\tilde{r}}\text{diag}({\mathrm{J}}_q(\tilde{\mathit{\sigma }})){\tilde{\mathbf{V}}}_{\tilde{r}}^T,$$(15) ) by symmetry. Following the proof of Proposition 2 and considering the reduced SVD of the matrices $\mathbf{A}$ and $\mathbf{B}$ given in (23(23) $$\mathbf{A}={\mathbf{U}}_r{\mathbf{S}}_r{\mathbf{V}}_r^T,$$(23) ) and (24(24) $$\mathbf{B}={\tilde{\mathbf{U}}}_{\tilde{r}}{\tilde{\mathbf{S}}}_{\tilde{r}}{\tilde{\mathbf{V}}}_{\tilde{r}}^T,$$(24) ), we immediately see that the inequalities (26(26) $$\sum _{i=1}^r\sum _{j=1}^{\tilde{r}}{\sigma }_i{\tilde{\sigma }}_j{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\le {\left(\sum _{i=1}^r{\sigma }_i^p\sum _{j=1}^{\tilde{r}}\left|{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\right|\right)}^{\frac{1}{p}}{\left(\sum _{j=1}^{\tilde{r}}{\sigma }_j^p\sum _{i=1}^r\left|{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\right|\right)}^{\frac{1}{q}}$$(26) ), (28(28) $$\sum _{j=1}^{\tilde{r}}\left|{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\right|\le {\left(\sum _{j=1}^{\tilde{r}}{({\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j)}^2\right)}^{\frac{1}{2}}{\left(\sum _{j=1}^{\tilde{r}}{({\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j)}^2\right)}^{\frac{1}{2}}\le \left|\right|{\mathbf{u}}_i\left|{|}_2\right|\left|{\mathbf{v}}_i\right|{|}_2=1,$$(28) ), and (29(29) $$\sum _{i=1}^r\left|{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\right|\le {\left(\sum _{i=1}^r{({\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j)}^2\right)}^{\frac{1}{2}}{\left(\sum _{i=1}^r{({\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j)}^2\right)}^{\frac{1}{2}}\le \left|\right|{\tilde{\mathbf{u}}}_i\left|{|}_2\right|\left|{\tilde{\mathbf{v}}}_i\right|{|}_2=1.$$(29) ) should all be saturated. The equality condition of the weighted Hölder implies the existence of a positive constant $\alpha >0$ such that, for all $(i,j)\in \{1,\dots ,r\}\times \{1,\dots ,\tilde{r}\},$ we have one of the following conditions: (30) $${\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j=0,\hspace{1em}\text{or}$$(30) (31) $${\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j>0\hspace{1em}\text{and}\hspace{1em}{\tilde{\sigma }}_j^q=\alpha {\sigma }_i^p.$$(31)

Moreover, the saturation of (28(28) $$\sum _{j=1}^{\tilde{r}}\left|{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\right|\le {\left(\sum _{j=1}^{\tilde{r}}{({\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j)}^2\right)}^{\frac{1}{2}}{\left(\sum _{j=1}^{\tilde{r}}{({\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j)}^2\right)}^{\frac{1}{2}}\le \left|\right|{\mathbf{u}}_i\left|{|}_2\right|\left|{\mathbf{v}}_i\right|{|}_2=1,$$(28) ) implies that (32) $${\mathbf{u}}_i\in \text{Range}({\tilde{\mathbf{U}}}_{\tilde{r}}),\hspace{1em}{\mathbf{v}}_i\in \text{Range}({\tilde{\mathbf{V}}}_{\tilde{r}})\mathrm{\hspace{1em}}\mathrm{\hspace{1em}}\forall i=1,\dots ,r$$(32) and also that there exists a positive constant ${\beta }_i>0$ (positivity follows from (31(31) $${\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j>0\hspace{1em}\text{and}\hspace{1em}{\tilde{\sigma }}_j^q=\alpha {\sigma }_i^p.$$(31) ) and (30(30) $${\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j=0,\hspace{1em}\text{or}$$(30) )) such that (33) $${\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j={\beta }_i{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j,\hspace{1em}\forall j=1,\dots ,\tilde{r}.$$(33)

However, from the normality of u_i and (32(32) $${\mathbf{u}}_i\in \text{Range}({\tilde{\mathbf{U}}}_{\tilde{r}}),\hspace{1em}{\mathbf{v}}_i\in \text{Range}({\tilde{\mathbf{V}}}_{\tilde{r}})\mathrm{\hspace{1em}}\mathrm{\hspace{1em}}\forall i=1,\dots ,r$$(32) ), we have that (34) $$1=\left|\right|{\mathbf{u}}_i|{|}_2^2=\sum _{j=1}^{\tilde{r}}|{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{|}^2={\beta }_i^2\sum _{j=1}^{\tilde{r}}|{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j{|}^2={\beta }_i^2|\left|\mathbf{v}\right|{|}_2^2={\beta }_i^2$$(34) which, together with the positivity of β_i, leads to the conclusion that ${\beta }_i=1$ for $i=1,\dots ,r.$ Using this, we rewrite (33(33) $${\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j={\beta }_i{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j,\hspace{1em}\forall j=1,\dots ,\tilde{r}.$$(33) ) in matrix form as (35) $${\mathbf{U}}_r^T{\tilde{\mathbf{U}}}_{\tilde{r}}={\mathbf{V}}_r^T{\tilde{\mathbf{V}}}_{\tilde{r}}.$$(35)

Similarly, the saturation of (29) implies that (36) $${\tilde{\mathbf{u}}}_j\in \text{Range}({\mathbf{U}}_r),\hspace{1em}{\tilde{\mathbf{v}}}_i\in \text{Range}({\mathbf{V}}_r),$$(36) for all $j=1,\dots ,\tilde{r}.$ Putting together (32(32) $${\mathbf{u}}_i\in \text{Range}({\tilde{\mathbf{U}}}_{\tilde{r}}),\hspace{1em}{\mathbf{v}}_i\in \text{Range}({\tilde{\mathbf{V}}}_{\tilde{r}})\mathrm{\hspace{1em}}\mathrm{\hspace{1em}}\forall i=1,\dots ,r$$(32) ) and (36(36) $${\tilde{\mathbf{u}}}_j\in \text{Range}({\mathbf{U}}_r),\hspace{1em}{\tilde{\mathbf{v}}}_i\in \text{Range}({\mathbf{V}}_r),$$(36) ), we deduce that $r=\tilde{r}$ and (37) $$\text{Range}({\mathbf{U}}_r)=\text{Range}({\tilde{\mathbf{U}}}_{\tilde{r}}),\hspace{1em}\text{Range}({\mathbf{V}}_r)=\text{Range}({\tilde{\mathbf{V}}}_{\tilde{r}}).$$(37)

This implies the existence of two orthogonal matrices $\mathbf{P},\mathbf{Q}\in {\mathbb{R}}^{r\times r}$ such that (38) $${\tilde{\mathbf{U}}}_{\tilde{r}}={\mathbf{U}}_r\mathbf{P},\hspace{1em}{\tilde{\mathbf{V}}}_{\tilde{r}}={\mathbf{V}}_r\mathbf{Q}.$$(38)

However, replacing (38(38) $${\tilde{\mathbf{U}}}_{\tilde{r}}={\mathbf{U}}_r\mathbf{P},\hspace{1em}{\tilde{\mathbf{V}}}_{\tilde{r}}={\mathbf{V}}_r\mathbf{Q}.$$(38) ) in (35(35) $${\mathbf{U}}_r^T{\tilde{\mathbf{U}}}_{\tilde{r}}={\mathbf{V}}_r^T{\tilde{\mathbf{V}}}_{\tilde{r}}.$$(35) ), we conclude that (39) $$\mathbf{P}={\mathbf{U}}_r^T{\mathbf{U}}_r\mathbf{P}={\mathbf{U}}_r^T{\tilde{\mathbf{U}}}_{\tilde{r}}={\mathbf{V}}_r^T{\tilde{\mathbf{V}}}_{\tilde{r}}={\mathbf{V}}_r^T{\mathbf{V}}_r\mathbf{Q}=\mathbf{Q}.$$(39)

This implies that the matrix $\mathbf{B}$ can be represented as (40) $$\mathbf{B}={\mathbf{U}}_r\mathbf{P}{\tilde{\mathbf{S}}}_{\tilde{r}}{\mathbf{P}}^T{\mathbf{V}}_r^T={\mathbf{U}}_r{\mathbf{S}}_0{\mathbf{V}}_r^T,$$(40) where ${\mathbf{S}}_0=\mathbf{P}{\tilde{\mathbf{S}}}_{\tilde{r}}{\mathbf{P}}^T.$ We now show that ${\mathbf{S}}_0$ is a diagonal matrix. Indeed, by denoting the (i, j)-th entry of $\mathbf{P}$ as $p_{i,j}$ such that $\mathbf{P}=[{\mathbf{p}}_1\cdots {\mathbf{p}}_r]=[p_{i,j}],$ we rewrite (30(30) $${\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j=0,\hspace{1em}\text{or}$$(30) ) and (31(31) $${\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j>0\hspace{1em}\text{and}\hspace{1em}{\tilde{\sigma }}_j^q=\alpha {\sigma }_i^p.$$(31) ) as (41) $$p_{i,j}=0,\hspace{1em}\text{or}$$(41) (42) $$p_{i,j}>0\hspace{1em}\text{and}\hspace{1em}{\tilde{\sigma }}_j^q=\alpha {\sigma }_i^p,$$(42) for all $(i,j)\in {\{1,\dots ,r\}}^2.$ Moreover, by expanding the (i, j)-th entry of the matrix ${\mathbf{S}}_0,$ we have that $$\begin{array}{c}{\left[{\mathbf{S}}_0\right]}_{i,j}={\left[\mathbf{P}{\tilde{\mathbf{S}}}_{\tilde{r}}{\mathbf{P}}^T\right]}_{i,j}=\sum _{k=1}^r{p}_{i,k}{\tilde{\sigma }}_k{p}_{j,k}=\sum _{k=1}^r{p}_{i,k}{\sigma }_i^{\frac{p}{q}}{\alpha }^{\frac{1}{q}}{p}_{j,k}\\ ={\sigma }_i^{\frac{p}{q}}{\alpha }^{\frac{1}{q}}{\mathbf{p}}_i^T{\mathbf{p}}_j={[{\mathrm{J}}_p(\mathit{\sigma })]}_i{c}_{\mathbf{B}}\delta [i-j],\end{array}$$ where $\delta [·]$ denotes the Kronecker delta and $c_{\mathbf{B}}={\alpha }^{\frac{1}{q}}>0$ is a positive constant. Finally, we obtain the announced expression in (14(14) $$\mathbf{B}=c{\mathbf{U}}_r\text{diag}({\mathrm{J}}_p(\mathit{\sigma })){\mathbf{V}}_r^T$$(14) ) by replacing the above characterization of ${\mathbf{S}}_0$ in (40(40) $$\mathbf{B}={\mathbf{U}}_r\mathbf{P}{\tilde{\mathbf{S}}}_{\tilde{r}}{\mathbf{P}}^T{\mathbf{V}}_r^T={\mathbf{U}}_r{\mathbf{S}}_0{\mathbf{V}}_r^T,$$(40) ).

For the converse, we note that, if the matrix $\mathbf{B}$ is in the form of (14(14) $$\mathbf{B}=c{\mathbf{U}}_r\text{diag}({\mathrm{J}}_p(\mathit{\sigma })){\mathbf{V}}_r^T$$(14) ), then we have that $$\begin{array}{l}\text{Tr}\left({\mathbf{A}}^T\mathbf{B}\right)=\text{Tr}\left({\mathbf{U}}_r\text{diag}(\mathit{\sigma }){\mathbf{V}}_r^T{\left({\mathbf{U}}_r\text{diag}({\mathrm{J}}_p(\mathit{\sigma })){\mathbf{V}}_r^T\right)}^T\right)\\ =c_{\mathbf{B}}\text{Tr}\left(\text{diag}(\mathit{\sigma }){\mathbf{V}}_r^T{\mathbf{V}}_r\text{diag}({\mathrm{J}}_p(\mathit{\sigma })){\mathbf{U}}_r^T{\mathbf{U}}_r\right)\\ =c_{\mathbf{B}}{\mathit{\sigma }}^T{\mathrm{J}}_p(\mathit{\sigma })\\ =c_{\mathbf{B}}\left|\right|\mathit{\sigma }\left|{|}_p\right||{\mathrm{J}}_p(\mathit{\sigma })|{|}_q=\left|\right|\mathbf{A}\left|{|}_{S_p}\right|\left|\mathbf{B}\right|{|}_{S_q},\end{array}$$ which shows that the equality is indeed saturated in this case.

Case 2: p = 1. In this case, the saturation of the weighted Hölder inequality implies that, for all $(i,j)\in \{1,\dots ,r\}\times \{1,\dots ,\tilde{r}\},$ we have that (43) $${\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j=0,\hspace{1em}\text{or}$$(43) (44) $${\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j>0\hspace{1em}\text{and}\hspace{1em}{\tilde{\sigma }}_j={\tilde{\sigma }}_1.$$(44)

For equality, we also need to have the saturation of (28(28) $$\sum _{j=1}^{\tilde{r}}\left|{\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j\right|\le {\left(\sum _{j=1}^{\tilde{r}}{({\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j)}^2\right)}^{\frac{1}{2}}{\left(\sum _{j=1}^{\tilde{r}}{({\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j)}^2\right)}^{\frac{1}{2}}\le \left|\right|{\mathbf{u}}_i\left|{|}_2\right|\left|{\mathbf{v}}_i\right|{|}_2=1,$$(28) ), which we showed to be equivalent to (32(32) $${\mathbf{u}}_i\in \text{Range}({\tilde{\mathbf{U}}}_{\tilde{r}}),\hspace{1em}{\mathbf{v}}_i\in \text{Range}({\tilde{\mathbf{V}}}_{\tilde{r}})\mathrm{\hspace{1em}}\mathrm{\hspace{1em}}\forall i=1,\dots ,r$$(32) ) and (35(35) $${\mathbf{U}}_r^T{\tilde{\mathbf{U}}}_{\tilde{r}}={\mathbf{V}}_r^T{\tilde{\mathbf{V}}}_{\tilde{r}}.$$(35) ). From (32(32) $${\mathbf{u}}_i\in \text{Range}({\tilde{\mathbf{U}}}_{\tilde{r}}),\hspace{1em}{\mathbf{v}}_i\in \text{Range}({\tilde{\mathbf{V}}}_{\tilde{r}})\mathrm{\hspace{1em}}\mathrm{\hspace{1em}}\forall i=1,\dots ,r$$(32) ), we deduce the existence of matrices ${\mathbf{P}}_1,{\mathbf{P}}_2\in {\mathbb{R}}^{\tilde{r}\times r}$ such that (45) $${\mathbf{U}}_r={\tilde{\mathbf{U}}}_{\tilde{r}}{\mathbf{P}}_1,\mathrm{\hspace{1em}}\mathrm{\hspace{1em}}{\mathbf{V}}_r={\tilde{\mathbf{V}}}_{\tilde{r}}{\mathbf{P}}_2.$$(45)

The replacement of these in (35(35) $${\mathbf{U}}_r^T{\tilde{\mathbf{U}}}_{\tilde{r}}={\mathbf{V}}_r^T{\tilde{\mathbf{V}}}_{\tilde{r}}.$$(35) ) implies that (46) $${\mathbf{P}}_1^T={\mathbf{P}}_1^T{\tilde{\mathbf{U}}}_{\tilde{r}}^T{\tilde{\mathbf{U}}}_{\tilde{r}}={\mathbf{U}}_r^T{\tilde{\mathbf{U}}}_{\tilde{r}}={\mathbf{V}}_r^T{\tilde{\mathbf{V}}}_{\tilde{r}}={\mathbf{P}}_2^T{\tilde{\mathbf{V}}}_{\tilde{r}}^T{\tilde{\mathbf{V}}}_{\tilde{r}}={\mathbf{P}}_2^T,$$(46) and, hence, that ${\mathbf{P}}_1={\mathbf{P}}_2=[p_{i,j}]\in {\mathbb{R}}^{\tilde{r}\times r}.$ Now, one can rewrite the conditions (43(43) $${\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j=0,\hspace{1em}\text{or}$$(43) ) and (44(44) $${\mathbf{u}}_i^T{\tilde{\mathbf{u}}}_j{\mathbf{v}}_i^T{\tilde{\mathbf{v}}}_j>0\hspace{1em}\text{and}\hspace{1em}{\tilde{\sigma }}_j={\tilde{\sigma }}_1.$$(44) ) and deduce that, for any $j=1,\dots ,\tilde{r},$ we have that (47) $$p_{j,i}=0,\hspace{1em}\forall i=1,\dots ,r\hspace{1em}\text{or}\hspace{1em}{\tilde{\sigma }}_j={\tilde{\sigma }}_1.$$(47)

From Conditions (47(47) $$p_{j,i}=0,\hspace{1em}\forall i=1,\dots ,r\hspace{1em}\text{or}\hspace{1em}{\tilde{\sigma }}_j={\tilde{\sigma }}_1.$$(47) ) and following the definition of r₁ (the multiplicity of the largest singular value), we deduce that (48) $${\mathbf{P}}_1=\left[\begin{array}{c}\mathbf{P}\\ {\mathbf{0}}_{r_{\text{res}}\times r}\end{array}\right],$$(48) where $\mathbf{P}\in {\mathbb{R}}^{r_1\times r}$ and $r_{\text{res}}=(\tilde{r}-r_1).$ Using this form and the definition of ${\tilde{\mathbf{U}}}_1$ and ${\tilde{\mathbf{V}}}_1$ (given in the statement of the proposition), we rewrite (45(45) $${\mathbf{U}}_r={\tilde{\mathbf{U}}}_{\tilde{r}}{\mathbf{P}}_1,\mathrm{\hspace{1em}}\mathrm{\hspace{1em}}{\mathbf{V}}_r={\tilde{\mathbf{V}}}_{\tilde{r}}{\mathbf{P}}_2.$$(45) ) as (49) $${\mathbf{U}}_r={\tilde{\mathbf{U}}}_1\mathbf{P},\mathrm{\hspace{1em}}\mathrm{\hspace{1em}}{\mathbf{V}}_r={\tilde{\mathbf{V}}}_1\mathbf{P}.$$(49)

Therefore, (50) $${\mathbf{I}}_r={\mathbf{U}}_r^T{\mathbf{U}}_r={\mathbf{P}}^T{\tilde{\mathbf{U}}}_1^T{\tilde{\mathbf{U}}}_1\mathbf{P}={\mathbf{P}}^T\mathbf{P}.$$(50)

Hence, $\mathbf{P}$ is a sub-orthogonal matrix and $$\text{rank}(\mathbf{B})=\tilde{r}\ge r_1\ge \text{rank}(\mathbf{P})\ge r=\text{rank}(\mathbf{A}).$$

The replacement of (49(49) $${\mathbf{U}}_r={\tilde{\mathbf{U}}}_1\mathbf{P},\mathrm{\hspace{1em}}\mathrm{\hspace{1em}}{\mathbf{V}}_r={\tilde{\mathbf{V}}}_1\mathbf{P}.$$(49) ) in the reduced SVD of $\mathbf{A}$ yields the announced expression with $\mathbf{X}=\mathbf{PS}{\mathbf{P}}^T.$

Based on the definitions of r₁, ${\tilde{\mathbf{U}}}_1,$ and ${\tilde{\mathbf{V}}}_1,$ we note that one can rewrite the reduced SVD of $\mathbf{B}$ as (51) $$\mathbf{B}={\tilde{\sigma }}_1{\tilde{\mathbf{U}}}_1{\tilde{\mathbf{V}}}_1^T+{\tilde{\mathbf{U}}}_{\text{res}}{\tilde{\mathbf{S}}}_{\text{res}}{\tilde{\mathbf{V}}}_{\text{res}}^T,$$(51)

where ${\tilde{\mathbf{U}}}_{\text{res}}\in {\mathbb{R}}^{m\times r_{\text{res}}},{\tilde{\mathbf{S}}}_{\text{res}}\in {\mathbb{R}}^{r_{\text{res}}\times r_{\text{res}}},$ and ${\tilde{\mathbf{V}}}_{\text{res}}\in {\mathbb{R}}^{n\times r_{\text{res}}}$ are the remaining singular values and vectors such that $$\tilde{\mathbf{U}}=\left[{\tilde{\mathbf{U}}}_1\hspace{1em}{\tilde{\mathbf{U}}}_{\text{res}}\right],\hspace{1em}\tilde{\mathbf{V}}=\left[{\tilde{\mathbf{V}}}_1\hspace{1em}{\tilde{\mathbf{V}}}_{\text{res}}\right],\hspace{1em}\tilde{\mathbf{S}}=\left[\begin{array}{cc}{\tilde{\sigma }}_1{\mathbf{I}}_{r_1}& \mathbf{0}\\ \mathbf{0}& {\tilde{S}}_{\text{res}}\end{array}\right].$$

Now, if $\mathbf{A}$ admits the form (17(17) $$\mathbf{A}={\tilde{\mathbf{U}}}_1\mathbf{X}{\tilde{\mathbf{V}}}_1^T.$$(17) ) and if we consider the SVD of $\mathbf{X}=\mathbf{PS}{\mathbf{P}}^T$ (the assumption that $\mathbf{X}$ is symmetric ensures that is has an orthogonal eigen-decomposition), then $$\begin{array}{l}\text{Tr}({\mathbf{A}}^T\mathbf{B})=\text{Tr}({\tilde{\mathbf{V}}}_1\mathbf{P}\mathbf{S}{\mathbf{P}}^T{\tilde{\mathbf{U}}}_1^T({\tilde{\sigma }}_1{\tilde{\mathbf{U}}}_1{\tilde{\mathbf{V}}}_1^T+{\tilde{\mathbf{U}}}_{\text{res}}{\tilde{\mathbf{S}}}_{\text{res}}{\tilde{\mathbf{V}}}_{\text{res}}^T))\\ ={\tilde{\sigma }}_1\text{Tr}({\tilde{\mathbf{V}}}_1\mathbf{P}\mathbf{S}{\mathbf{P}}^T{\tilde{\mathbf{U}}}_1^T{\tilde{\mathbf{U}}}_1{\tilde{\mathbf{V}}}_1^T)+\text{Tr}({\tilde{\mathbf{V}}}_1\mathbf{P}\mathbf{S}{\mathbf{P}}^T{\tilde{\mathbf{U}}}_1^T{\tilde{\mathbf{U}}}_{\text{res}}{\tilde{\mathbf{S}}}_{\text{res}}{\tilde{\mathbf{V}}}_{\text{res}}^T)\\ ={\tilde{\sigma }}_1\text{Tr}({\tilde{\mathbf{V}}}_1\mathbf{P}\mathbf{S}{\mathbf{P}}^T{\mathbf{I}}_{r_1}{\tilde{\mathbf{V}}}_1^T)+\text{Tr}({\tilde{\mathbf{V}}}_1\mathbf{P}\mathbf{S}{\mathbf{P}}^T{\mathbf{0}}_{r_1\times r_{\text{res}}}{\tilde{\mathbf{S}}}_{\text{res}}{\tilde{\mathbf{V}}}_{\text{res}}^T)\\ ={\tilde{\sigma }}_1\text{Tr}(\mathbf{S}{\mathbf{P}}^T{\tilde{\mathbf{V}}}_1^T{\tilde{\mathbf{V}}}_1\mathbf{P})+0\\ ={\tilde{\sigma }}_1\text{Tr}(\mathbf{S}{\mathbf{P}}^T\mathbf{P})\\ ={\tilde{\sigma }}_1\text{Tr}\left(\mathbf{S}\right)=\left|\right|\mathbf{B}\left|{|}_{S_{\infty }}\right|\left|\mathbf{A}\right|{|}_{S_1},\end{array}$$ which establishes the sufficiency in this case.

Finally, assuming that $r=r_1=\tilde{r},$ we deduce that $\mathbf{P}\in {\mathbb{R}}^{r\times r}$ is an orthogonal matrix and, hence, that ${\mathbf{P}}^{-1}={\mathbf{P}}^T.$ Now, using (49(49) $${\mathbf{U}}_r={\tilde{\mathbf{U}}}_1\mathbf{P},\mathrm{\hspace{1em}}\mathrm{\hspace{1em}}{\mathbf{V}}_r={\tilde{\mathbf{V}}}_1\mathbf{P}.$$(49) ) and the rank assumption, we can simplify the expansion (51(51) $$\mathbf{B}={\tilde{\sigma }}_1{\tilde{\mathbf{U}}}_1{\tilde{\mathbf{V}}}_1^T+{\tilde{\mathbf{U}}}_{\text{res}}{\tilde{\mathbf{S}}}_{\text{res}}{\tilde{\mathbf{V}}}_{\text{res}}^T,$$(51) ) as (52) $$\mathbf{B}={\tilde{\sigma }}_1{\tilde{\mathbf{U}}}_1{\tilde{\mathbf{V}}}_1^T={\tilde{\sigma }}_1{\mathbf{U}}_r{\mathbf{P}}^T{\left({\mathbf{V}}_r{\mathbf{P}}^T\right)}^T={\tilde{\sigma }}_1{\mathbf{U}}_r{\mathbf{P}}^T\mathbf{P}{\mathbf{V}}_r^T={\tilde{\sigma }}_1{\mathbf{U}}_r{\mathbf{V}}_r^T.$$(52)

C. Proof of Theorem 1

Proof.

Case I: $1<p<+\infty .$ Assume that $(\mathbf{A},\mathbf{B})$ forms an (S_p, S_q)-conjugate pair. Hence, we have that $〈\mathbf{A},\mathbf{B}〉=\left|\right|\mathbf{A}\left|{|}_{S_p}\right|\left|\mathbf{B}\right|{|}_{S_q}$ which, together with Proposition 2, implies that $\mathbf{B}$ admits the form $$\mathbf{B}=\frac{\left|\right|\mathbf{B}|{|}_{S_q}}{\left|\right|\mathbf{A}|{|}_{S_p}}{\mathbf{U}}_r\text{diag}({\mathrm{J}}_p(\mathit{\sigma })){\mathbf{V}}_r^T={\mathbf{U}}_r\text{diag}({\mathrm{J}}_p(\mathit{\sigma })){\mathbf{V}}_r^T.$$

Case II: p = 1. Similarly to the previous case, consider $\mathbf{A}\in {\mathbb{R}}^{m\times n}$ and $\mathbf{B}\in {\mathcal{J}}_{S_1,\text{rank}}(\mathbf{A}).$ We have that (53) $$\text{Tr}\left({\mathbf{A}}^T\mathbf{B}\right)=\left|\right|\mathbf{A}\left|{|}_{S_1}\right|\left|\mathbf{B}\right|{|}_{S_{\infty }}$$(53) (54) $$\left|\right|\mathbf{A}|{|}_{S_1}=|\left|\mathbf{B}\right|{|}_{S_{\infty }}$$(54) (55) $$\text{rank}(\mathbf{B})\le \text{rank}(\mathbf{C}),\hspace{1em}\forall \mathbf{C}\in {\mathcal{J}}_{S_1}(\mathbf{A}).$$(55)

From (53(53) $$\text{Tr}\left({\mathbf{A}}^T\mathbf{B}\right)=\left|\right|\mathbf{A}\left|{|}_{S_1}\right|\left|\mathbf{B}\right|{|}_{S_{\infty }}$$(53) ) and using Proposition 2, we deduce that $\text{rank}(\mathbf{B})\ge \text{rank}(\mathbf{A})$ which, together with (55(55) $$\text{rank}(\mathbf{B})\le \text{rank}(\mathbf{C}),\hspace{1em}\forall \mathbf{C}\in {\mathcal{J}}_{S_1}(\mathbf{A}).$$(55) ), implies that $\mathbf{B}$ should be equal to $$\mathbf{B}=\left|\right|\mathbf{B}|{|}_{S_{\infty }}{\mathbf{U}}_r{\mathbf{V}}_r^T=|\left|\mathit{\sigma }\right|{|}_1{\mathbf{U}}_r{\mathbf{V}}_r^T,$$ where the last equality is obtained using (54(54) $$\left|\right|\mathbf{A}|{|}_{S_1}=|\left|\mathbf{B}\right|{|}_{S_{\infty }}$$(54) ).

Case III: p = +∞. Following Proposition 2, any matrix $\mathbf{B}\in {\mathcal{J}}_{S_{\infty }}(\mathbf{A})$ can be expressed as $$\mathbf{B}={\mathbf{U}}_1\tilde{\mathbf{X}}{\mathbf{V}}_1^T,$$ where $\tilde{\mathbf{X}}\in {\mathbb{R}}^{r_1\times r_1}$ is a symmetric matrix. By defining $\mathbf{X}={\sigma }_1^{-1}\tilde{\mathbf{X}},$ one readily verifies that $\mathbf{B}={\sigma }_1{\mathbf{U}}_1\mathbf{X}{\mathbf{V}}_1^T.$ By recalling the normalization constraint $\left|\right|\mathbf{A}|{|}_{S_{\infty }}=|\left|\mathbf{B}\right|{|}_{S_1},$ we therefore obtain that $${\sigma }_1=\left|\right|\mathbf{A}|{|}_{S_{\infty }}=|\left|\mathbf{B}\right|{|}_{S_1}={\sigma }_1\left|\right|\mathbf{X}|{|}_{S_1},$$ which implies that $\left|\right|\mathbf{X}|{|}_{S_1}=1.$ To show that ${\mathcal{J}}_{S_{\infty }}(\mathbf{A})$ is convex, consider two symmetric matrices ${\mathbf{X}}_0$ and ${\mathbf{X}}_1$ in the unit ball of Schatten-1 norm and define $${\mathbf{B}}_{\alpha }={\sigma }_1{\mathbf{U}}_1{\mathbf{X}}_{\alpha }{\mathbf{V}}_1^T,\hspace{1em}{\mathbf{X}}_{\alpha }=\alpha {\mathbf{X}}_0+(1-\alpha ){\mathbf{X}}_1$$ for $\alpha \in [0,1].$ On one hand, from the linearity of traces, we have that $$\text{Tr}({\mathbf{A}}^T{\mathbf{B}}_{\alpha })=\text{Tr}\left({\mathbf{A}}^T\left(\alpha {\mathbf{B}}_1+(1-\alpha ){\mathbf{B}}_0\right)\right)=\alpha \text{Tr}({\mathbf{A}}^T{\mathbf{B}}_1)+(1-\alpha )\text{Tr}({\mathbf{A}}^T{\mathbf{B}}_0).$$

On the other hand, from the definition of ${\mathbf{X}}_0$ and ${\mathbf{X}}_1,$ we deduce that ${\mathbf{B}}_0,{\mathbf{B}}_1\in {\mathcal{J}}_{S_{\infty }}(\mathbf{A}).$ Hence, $$\text{Tr}({\mathbf{A}}^T{\mathbf{B}}_{\alpha })=\alpha \left|\right|\mathbf{A}|{|}_{S_{\infty }}^2+(1-\alpha )|\left|\mathbf{A}\right|{|}_{S_{\infty }}^2=\left|\right|\mathbf{A}|{|}_{S_{\infty }}^2.$$

However, from the Hölder inequality and the convexity of norms, we have that $$\text{Tr}({\mathbf{A}}^T{\mathbf{B}}_{\alpha })\le \left|\right|\mathbf{A}\left|{|}_{S_{\infty }}\right|\left|{\mathbf{B}}_{\alpha }\right|{|}_{S_1}\le \left|\right|\mathbf{A}|{|}_{S_{\infty }}\left(\alpha \left|\right|{\mathbf{B}}_1|{|}_{S_1}+(1-\alpha )|\left|{\mathbf{B}}_0\right|{|}_{S_1}\right)=|\left|\mathbf{A}\right|{|}_{S_{\infty }}.$$

This implies that the Hölder inequality is saturated and also that $\left|\right|{\mathbf{B}}_{\alpha }|{|}_{S_1}=|\left|\mathbf{A}\right|{|}_{S_{\infty }}$ which, altogether, implies that ${\mathbf{B}}_{\alpha }\in {\mathcal{J}}_{S_{\infty }}(\mathbf{A})$ for all $\alpha \in [0,1].$

Finally, we observe that the set ${\mathcal{J}}_{S_{\infty }}(\mathbf{A})$ contains all matrices of the form $\mathbf{B}={\mathbf{U}}_1\mathbf{p}{\mathbf{p}}^T{\mathbf{V}}_1^T$ for any vector $\mathbf{p}\in {\mathbb{R}}^{r_1}$ with $\left|\right|\mathbf{p}|{|}_2=1.$ These are indeed all the rank-1 elements of ${\mathcal{J}}_{S_{\infty }}(\mathbf{A})$ which, due to the Definition 2, forms the set of sparse dual conjugates. □

Additional information

Funding

This work was supported in part by the European Research Council (H2020-ERC Project GlobalBioIm) under Grant 692726 and in part by the Swiss National Science Foundation, Grant 200020_184646/1.

Notes

1 Although this functional does not satisfy the homogeneity property of a norm, it has been widely referred to as the ${\ell }_0$-norm.