On Stewart’s Perturbation Theorem for SVD

In [15], Stewart established a perturbation theory for the singular subspaces associated with the SVD of a matrix $G\in\mathbb{C}^{m\times n}$ slightly perturbed. In the same paper he also investigated the eigenspace of a matrix and the deflating subspace of a regular matrix pencil subject to perturbations. But in this paper, we limit our scope to one of his theorems, namely, [15, Theorem 6.4] on SVD, where both the Frobenius and spectral norms are used to measure perturbations. Our goal is to establish a more general and yet sharper version of [15, Theorem 6.4] in the spectral norm and any unitarily invariant norm. Our new version offers flexibility in its applications, for example, the version with the unitarily invariant norm also set to the spectral norm is more convenient to use in our recent work [18] for analyzing the quality of a reduced order model by approximate balanced truncation [1, 2]. Even in the case of using both the Frobenius and spectral norms, our results are slightly better than Stewart’s [15, Theorem 6.4] in terms of a less stringent condition and yet a sharper bound.

Given $G\in\mathbb{C}^{m\times n}$, let

	$$U\equiv\hbox{}\vbox{\kern 0.86108pt\hbox{$\kern 0.0pt\kern 2.5pt\kern-5.0pt\left[\kern 0.0pt\kern-2.5pt\kern-5.55557pt\vbox{\kern-0.86108pt\vbox{\vbox{ \halign{\kern\arraycolsep\hfil\@arstrut$\kbcolstyle#$\hfil\kern\arraycolsep& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep&& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep\cr 5.0pt\hfil\@arstrut$\scriptstyle$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle r$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle m-r\\$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle U_{1}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle U_{2}$\hfil\kern 5.0pt\crcr}}}}\right]$}}\in\mathbb{C}^{m\times m},\quad V\equiv\hbox{}\vbox{\kern 0.86108pt\hbox{$\kern 0.0pt\kern 2.5pt\kern-5.0pt\left[\kern 0.0pt\kern-2.5pt\kern-5.55557pt\vbox{\kern-0.86108pt\vbox{\vbox{ \halign{\kern\arraycolsep\hfil\@arstrut$\kbcolstyle#$\hfil\kern\arraycolsep& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep&& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep\cr 5.0pt\hfil\@arstrut$\scriptstyle$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle r$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle n-r\\$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle V_{1}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle V_{2}$\hfil\kern 5.0pt\crcr}}}}\right]$}}\in\mathbb{C}^{n\times n}$$		(1.1a)
be two unitary matrices such that $G$ admits the decomposition
	$$U^{\operatorname{H}}GV\equiv\begin{bmatrix}U_{1}^{\operatorname{H}}\\ U_{2}^{\operatorname{H}}\end{bmatrix}G[V_{1},V_{2}]=\hbox{}\vbox{\kern 0.86108pt\hbox{$\kern 0.0pt\kern 2.5pt\kern-5.0pt\left[\kern 0.0pt\kern-2.5pt\kern-5.55557pt\vbox{\kern-0.86108pt\vbox{\vbox{ \halign{\kern\arraycolsep\hfil\@arstrut$\kbcolstyle#$\hfil\kern\arraycolsep& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep&& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep\cr 5.0pt\hfil\@arstrut$\scriptstyle$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle r$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle n-r\\\scriptstyle r$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle G_{1}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 0\\\scriptstyle m-r$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 0$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle G_{2}$\hfil\kern 5.0pt\crcr}}}}\right]$}},$$		(1.1b)

where $1\leq r<\min\{m,n\}$. For example, this can be the SVD of $G$, for which $G_{1}$ is diagonal with nonnegative diagonal entries as some of the singular values of $G$ and $G_{2}$ is leading diagonal by which we mean that only its entries along the main diagonal line starting at the top-left corner may be nonzero and nonnegative and they are some of the singular values of $G$, too. By convention, $G$ has $\min\{m,n\}$ singular values $\sigma_{i}(G)$ arranged in decreasing order:

$$\sigma_{1}(G)\geq\sigma_{2}(G)\geq\cdots\geq\sigma_{\min\{m,n\}}(G).$$

(1.2)

Denote the singular value set and its extended set of $G$ by

$$\operatorname{sv}(G)=\{\sigma_{i}(G)\}_{i=1}^{\min\{m,n\}},\quad\operatorname{sv}_{\operatorname{ext}}(G)=\operatorname{sv}(G)\cup\{\mbox{$|m-n|$ copies of $0$s}\},$$

(1.3)

where the union is meant to be the multiset union.

Consider now that $G$ is perturbed to $\widetilde{G}=G+E\in\mathbb{C}^{m\times n}$, and partition

$$U^{\operatorname{H}}\widetilde{G}V\equiv\begin{bmatrix}U_{1}^{\operatorname{H}}\\ U_{2}^{\operatorname{H}}\end{bmatrix}(G+E)[V_{1},V_{2}]=\hbox{}\vbox{\kern 0.86108pt\hbox{$\kern 0.0pt\kern 2.5pt\kern-5.0pt\left[\kern 0.0pt\kern-2.5pt\kern-5.55557pt\vbox{\kern-0.86108pt\vbox{\vbox{ \halign{\kern\arraycolsep\hfil\@arstrut$\kbcolstyle#$\hfil\kern\arraycolsep& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep&& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep\cr 5.0pt\hfil\@arstrut$\scriptstyle$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle r$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle n-r\\\scriptstyle r$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle G_{1}+E_{11}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle E_{12}\\\scriptstyle m-r$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle E_{21}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle G_{2}+E_{22}$\hfil\kern 5.0pt\crcr}}}}\right]$}}.$$

(1.4)

Naturally, one would ask whether $\widetilde{G}$ admits a decomposition that is “close” to the one for $G$ in (1.1). Stewart [15, Theorem 6.4] provided an answer to that by seeking orthogonal matrices [15, p.760]

	$\displaystyle\check{U}$	$\displaystyle\equiv\hbox{}\vbox{\kern 0.86108pt\hbox{$\kern 0.0pt\kern 2.5pt\kern-5.0pt\left[\kern 0.0pt\kern-2.5pt\kern-5.55557pt\vbox{\kern-0.86108pt\vbox{\vbox{ \halign{\kern\arraycolsep\hfil\@arstrut$\kbcolstyle#$\hfil\kern\arraycolsep& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep&& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep\cr 5.0pt\hfil\@arstrut$\scriptstyle$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle r$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle m-r\\$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\check{U}_{1}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\check{U}_{2}$\hfil\kern 5.0pt\crcr}}}}\right]$}}=[U_{1},U_{2}]\begin{bmatrix}I_{r}&-\Gamma^{\operatorname{H}}\\ \Gamma&I_{m-r}\end{bmatrix}\begin{bmatrix}(I+\Gamma^{\operatorname{H}}\Gamma)^{-1/2}&0\\ 0&(I+\Gamma\Gamma^{\operatorname{H}})^{-1/2}\end{bmatrix},\\\check{V}$	$\displaystyle\equiv\hbox{}\vbox{\kern 0.86108pt\hbox{$\kern 0.0pt\kern 2.5pt\kern-5.0pt\left[\kern 0.0pt\kern-2.5pt\kern-5.55557pt\vbox{\kern-0.86108pt\vbox{\vbox{ \halign{\kern\arraycolsep\hfil\@arstrut$\kbcolstyle#$\hfil\kern\arraycolsep& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep&& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep\cr 5.0pt\hfil\@arstrut$\scriptstyle$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle r$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle n-r\\$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\check{V}_{1}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\check{V}_{2}$\hfil\kern 5.0pt\crcr}}}}\right]$}}=[V_{1},V_{2}]\begin{bmatrix}I_{r}&-\Omega^{\operatorname{H}}\\ \Omega&I_{n-r}\end{bmatrix}\begin{bmatrix}(I+\Omega^{\operatorname{H}}\Omega)^{-1/2}&0\\ 0&(I+\Omega\Omega^{\operatorname{H}})^{-1/2}\end{bmatrix}$		(1.5c)

such that

$$\check{U}^{\operatorname{H}}\widetilde{G}\check{V}\equiv\begin{bmatrix}\check{U}_{1}^{\operatorname{H}}\\ \check{U}_{2}^{\operatorname{H}}\end{bmatrix}(G+E)[\check{V}_{1},\check{V}_{2}]=\hbox{}\vbox{\kern 0.86108pt\hbox{$\kern 0.0pt\kern 2.5pt\kern-5.0pt\left[\kern 0.0pt\kern-2.5pt\kern-5.55557pt\vbox{\kern-0.86108pt\vbox{\vbox{ \halign{\kern\arraycolsep\hfil\@arstrut$\kbcolstyle#$\hfil\kern\arraycolsep& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep&& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep\cr 5.0pt\hfil\@arstrut$\scriptstyle$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle r$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\scriptstyle n-r\\\scriptstyle r$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\check{G}_{1}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 0\\\scriptstyle m-r$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 0$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle\check{G}_{2}$\hfil\kern 5.0pt\crcr}}}}\right]$}}.$$

(1.6)

Theorem 1.1 below is [15, Theorem 6.4] in our notations here.

A number of results can be deduced from this informative theorem, e.g., bounds on $\|U-\check{U}\|_{\operatorname{F}}$ and $\|V-\check{V}\|_{\operatorname{F}}$, the explicit expressions of $\check{G}_{1}$ and $\check{G}_{2}$ in terms of $\Omega$ and $\Gamma$, $G_{1}$, $G_{2}$, and $E_{ij}$, and also the singular values of $\widetilde{G}$ is the multiset union of the singular values of $\check{G}_{1}$ and $\check{G}_{2}$.

Although the spectral norm $\|\cdot\|_{2}$ is used in defining $\underaccent{\bar}{\delta}$, the Frobenius norm $\|\cdot\|_{\operatorname{F}}$ is in full display in defining $\hat{\varepsilon}$ and in bounding $\Omega$ and $\Gamma$. A straightforward version of it, using only the spectral norm can be easily derived in light of the equivalency inequalities

$$\|B\|_{2}\leq\|B\|_{\operatorname{F}}\leq\sqrt{\operatorname{rank}(B)}\,\|B\|_{2}\leq\sqrt{\min\{s,t\}}\,\|B\|_{2}\quad\mbox{for $B\in\mathbb{C}^{s\times t}$}.$$

For example, we can conclude that if $\underaccent{\bar}{\delta}>0$ and

$$\tilde{\varepsilon}:=\sqrt{\min\{m-r,n-r,r\}}\frac{\sqrt{\|E_{12}\|_{2}^{2}+\|E_{21}\|_{2}^{2}}}{\underaccent{\bar}{\delta}}<\frac{1}{2},$$

(1.10)

then there exist $\Omega\in\mathbb{C}^{(n-r)\times r}$ and $\Gamma\in\mathbb{C}^{(m-r)\times r}$ satisfying

$$\sqrt{\|\Gamma\|_{\operatorname{F}}^{2}+\|\Omega\|_{\operatorname{F}}^{2}}\leq\frac{1+\sqrt{1-4(\tilde{\varepsilon}/\underaccent{\bar}{\delta})^{2}}}{1-2(\tilde{\varepsilon}/\underaccent{\bar}{\delta})^{2}+\sqrt{1-4(\tilde{\varepsilon}/\underaccent{\bar}{\delta})^{2}}}\frac{\tilde{\varepsilon}}{\underaccent{\bar}{\delta}}<2\,\frac{\tilde{\varepsilon}}{\underaccent{\bar}{\delta}}$$

(1.11)

such that (1.6) with (1.5) holds. There are two clear drawbacks of such a straightforward version: 1) a much stronger condition in (1.10) than something like

$$\frac{\sqrt{\|E_{12}\|_{2}^{2}+\|E_{21}\|_{2}^{2}}}{\underaccent{\bar}{\delta}}<\frac{1}{2}$$

(1.12)

as one might possibly expect, and 2) a much weaker conclusion in (1.11) than something like

$$\sqrt{\|\Gamma\|_{2}^{2}+\|\Omega\|_{2}^{2}}\leq\sqrt{\|\Gamma\|_{\operatorname{F}}^{2}+\|\Omega\|_{\operatorname{F}}^{2}}<2\,\frac{\sqrt{\|E_{12}\|_{2}^{2}+\|E_{21}\|_{2}^{2}}}{\underaccent{\bar}{\delta}}$$

(1.13)

also as one might possibly expect. The appearances of $m$ and $n$ in (1.10) are particularly unsatisfactory for large (huge) $m$ and usually modest $r$.

Our goal in this paper is to create a version of Theorem 1.1 that uses the spectral norm and any unitarily invariant norm. Our eventual version, Theorem 3.1, when restricted exclusively to the spectral norm, does not exactly coincide with the possible expectations in (1.12) and (1.13), but is very much close to it, namely (1.12) and (1.13) with $\sqrt{\|E_{12}\|_{2}^{2}+\|E_{21}\|_{2}^{2}}$ and $\sqrt{\|\Gamma\|_{2}^{2}+\|\Omega\|_{2}^{2}}$ replaced by $\max\{\|E_{12}\|_{2},\|E_{21}\|_{2}\}$ and $\max\{\|\Gamma\|_{2},\|\Omega\|_{2}\}$, respectively. In particular, $m$ and $n$ disappear altogether.

The rest of this paper is organized as follows. Section 2 states two preliminary results for later use in our proofs. Our main result, a variant of Stewart’s, is stated and proved in Section 3. In Section 4, we compare our main result realized for the spectral norm only and for both the spectral and Frobenius norm with Stewart’s result in Theorem 1.1 and its potential consequences in (1.10) and (1.11). Finally, we draw our conclusions in Section 5.

We will continue to adopt the notation introduced so far in this section such as $\|\cdot\|_{2}$ and $\|\cdot\|_{\operatorname{F}}$. In particular, $G\in\mathbb{C}^{m\times n}$ has $\min\{m,n\}$ singular values denoted by $\sigma_{i}(G)$ in decreasing order as in (1.2), and accordingly two singular value sets $\operatorname{sv}(G)$ and $\operatorname{sv}_{\operatorname{ext}}(G)$ in (1.3), and $\sigma_{\max}(G):=\sigma_{1}(G)$ and $\sigma_{\min}(G):=\sigma_{\min\{m,n\}}(G)$.

In this section, we will state four lemmas that we will need later. The first lemma, Lemma 2.1, is due to Stewart [14, 15], and the second one, Lemma 2.2, summarizes known bounds on the solution to the Sylvester equation with Hermitian coefficient matrices of Davis and Kahan [6] and of Bhatia, Davis, and McIntosh [4]. The third one, Lemma 2.3, establishes bounds on the solution pair to a set of the coupled Sylvester equations and the results are new, except the one for the Frobenius norm. Finally, the fourth lemma, Lemma 2.4, is likely known.

We need the notation of unitarily invariant norm to go forward. A matrix norm $\|\cdot\|_{\operatorname{ui}}$ is called a unitarily invariant norm on $\mathbb{C}^{m\times n}$ if it is a matrix norm and has the following two additional properties [3, 16]:

1. 1.

   $\|U^{\operatorname{H}}BV\|_{\operatorname{ui}}=\|B\|_{\operatorname{ui}}$ for all unitary matrices $U\in\mathbb{C}^{m\times m}$ and $V\in\mathbb{C}^{n\times n}$ and $B\in\mathbb{C}^{m\times n}$;

2. 2.

   $\|B\|_{\operatorname{ui}}=\|B\|_{2}$, the spectral norm of $B$, if $\operatorname{rank}(B)=1$.

Two commonly used unitarily invariant norms are

the spectral norm: $\|B\|_{2}=\max_{j}\sigma_{j}$, the Frobenius norm: $\|B\|_{\operatorname{F}}=\sqrt{\sum_{j}\sigma_{j}^{2}}$ ,

where $\sigma_{1},\sigma_{2},\ldots,\sigma_{\min\{m,n\}}$ are the singular values of $B$. In what follows, $\|\cdot\|_{\operatorname{ui}}$ denotes a general unitarily invariant norm.

In this article, for convenience, any $\|\cdot\|_{\operatorname{ui}}$ we use is generic to matrix sizes in the sense that it applies to matrices of all sizes. Examples include the matrix spectral norm $\|\cdot\|_{2}$, the Frobenius norm $\|\cdot\|_{\operatorname{F}}$, and the trace norm. One important property of unitarily invariant norms is

$$\|XYZ\|_{\operatorname{ui}}\leq\|X\|_{2}\cdot\|Y\|_{\operatorname{ui}}\cdot\|Z\|_{2}$$

for any matrices $X$, $Y$, and $Z$ of compatible sizes.

The next lemma summarizes known bounds on the solution of the Sylvester equation $AX-XB=S$. These bounds have played important roles in eigenspace variations as demonstrated by Davis and Kahan [6, 1970], and Bhatia, Davis, and McIntosh [4, 1983]. For a brief review, see [10].

The results of Lemma 2.2 have an alternative interpretation through a linear operator

$$\boldsymbol{L}\,:\,X\in\mathbb{C}^{s\times r}\,\to\,\boldsymbol{L}(X)=XA-BX\in\mathscr{B},$$

(2.5)

endowed with certain unitarily invariant norm $\|\cdot\|_{\operatorname{ui}}$ on $\mathbb{C}^{s\times r}$, including the spectral and Frobenius norms as special ones, where $A\in\mathbb{R}^{r\times r}$ and $B\in\mathbb{R}^{s\times s}$ are Hermitian. With any given $\|\cdot\|_{\operatorname{ui}}$, there is an induced operator norm $\|\cdot\|$ on the linear operator from $\mathbb{C}^{s\times r}$ to itself. Translating the results of Lemma 2.2 yields the following corollary.

For subspace variation associated with SVD, a set of two coupled Sylvester equations come into play:

$$XA-BY=S,\quad YA^{\operatorname{H}}-B^{\operatorname{H}}X=T,$$

(2.6)

where $A\in\mathbb{C}^{r\times r}$, $B\in\mathbb{C}^{s\times t}$, $S\in\mathbb{C}^{s\times r}$, and $T\in\mathbb{C}^{t\times r}$. Note that $B$ is possibly a nonsquare matrix, in which case we pad some blocks of zeros to $B$, $Y$, and $S$ or $T$ to yield an equivalent set of two coupled Sylvester equations with $B$ being square. Specifically, if $s>t$, let

$$B_{\operatorname{ext}}=[B,0_{s\times(s-t)}],\,Y_{\operatorname{ext}}=\begin{bmatrix}Y\\ 0_{(s-t)\times r}\end{bmatrix},\,S_{\operatorname{ext}}=S,\,\,T_{\operatorname{ext}}=\begin{bmatrix}T\\ 0_{(s-t)\times r}\end{bmatrix},$$

whereas if $s<t$, let

$$B_{\operatorname{ext}}=\begin{bmatrix}B\\ 0_{(t-s)\times t}\end{bmatrix},\,Y_{\operatorname{ext}}=[Y,0_{(t-s)\times r}],\,S_{\operatorname{ext}}=\begin{bmatrix}S\\ 0_{(t-s)\times r}\end{bmatrix},\,\,T_{\operatorname{ext}}=T.$$

Then (2.6) is equivalent to

$$XA-B_{\operatorname{ext}}Y_{\operatorname{ext}}=S_{\operatorname{ext}},\quad Y_{\operatorname{ext}}A^{\operatorname{H}}-B_{\operatorname{ext}}^{\operatorname{H}}X=T_{\operatorname{ext}},$$

(2.7)

which can be merged into one Sylvester equation

$$\begin{bmatrix}0&X\\ Y_{\operatorname{ext}}&0\end{bmatrix}\begin{bmatrix}0&A^{\operatorname{H}}\\ A&0\end{bmatrix}-\begin{bmatrix}0&B_{\operatorname{ext}}\\ B_{\operatorname{ext}}^{\operatorname{H}}&0\end{bmatrix}\begin{bmatrix}0&X\\ Y_{\operatorname{ext}}&0\end{bmatrix}=\begin{bmatrix}S_{\operatorname{ext}}&0\\ 0&T_{\operatorname{ext}}\end{bmatrix}.$$

(2.8)

It can be seen that

$$\operatorname{eig}(\begin{bmatrix}0&A^{\operatorname{H}}\\ A&0\end{bmatrix})=\operatorname{sv}(A)\cup(-\operatorname{sv}(A)),\,\operatorname{eig}(\begin{bmatrix}0&B_{\operatorname{ext}}\\ B_{\operatorname{ext}}^{\operatorname{H}}&0\end{bmatrix})=\operatorname{sv}_{\operatorname{ext}}(B)\cup(-\operatorname{sv}_{\operatorname{ext}}(B)),$$

(2.9)

where negating a set means negating each element of the set, and

$$\left\|\begin{bmatrix}0&X\\ Y_{\operatorname{ext}}&0\end{bmatrix}\right\|_{\operatorname{ui}}=\left\|\begin{bmatrix}0&X\\ Y&0\end{bmatrix}\right\|_{\operatorname{ui}},\quad\left\|\begin{bmatrix}S_{\operatorname{ext}}&0\\ 0&T_{\operatorname{ext}}\end{bmatrix}\right\|_{\operatorname{ui}}=\left\|\begin{bmatrix}S&0\\ 0&T\end{bmatrix}\right\|_{\operatorname{ui}}$$

(2.10)

for any unitary invariant norm. Apply Lemma 2.2 to get

Lemma 2.3 has more than what we need later. In fact, we will only use (2.13) and (2.15) in our later development.

The results of Lemma 2.3 have an alternative interpretation through a linear operator

$$\begin{array}[]{cccc}\boldsymbol{T}\,:&\mathscr{B}:=\mathbb{C}^{s\times r}\times\mathbb{C}^{t\times r}&\,\to&\boldsymbol{T}(X,Y)\in\mathscr{B}\\ &(X,Y)&\,\to&(XA-BY,YA^{\operatorname{H}}-B^{\operatorname{H}}X),\end{array}$$

(2.19)

endowed with certain norm on $\mathscr{B}$ to make it a Banach space, including (cf. those used in Lemma 2.3)

	$$\|(X,Y)\|=\left\|\begin{bmatrix}0&X\\ Y&0\end{bmatrix}\right\|_{\operatorname{ui}}\equiv\left\|\begin{bmatrix}X&0\\ 0&Y\end{bmatrix}\right\|_{\operatorname{ui}}$$		(2.20a)
for any $(X,Y)\in\mathscr{B}$, where $\|\cdot\|_{\operatorname{ui}}$ is any given unitarily invariant norm. Two particular ones are
	$$\|(X,Y)\|=\max\{\|X\|_{2},\|Y\|_{2}\}\quad\mbox{or}\quad\sqrt{\|X\|_{\operatorname{F}}^{2}+\|Y\|_{\operatorname{F}}^{2}},$$		(2.20b)
upon realizing $\|\cdot\|_{\operatorname{ui}}$ in (2.20a) as the spectral norm or the Frobenius norm. Another possible endowed norm on $\mathscr{B}$ is
	$$\|(X,Y)\|=\max\{\|X\|_{\operatorname{ui}},\|Y\|_{\operatorname{ui}}\}.$$		(2.20c)

With each endowed norm, there is an induced operator norm $\|\cdot\|$ on the linear operator from $\mathscr{B}$ to itself. Translating the results of Lemma 2.3 yields the following corollary.

The next lemma is likely known. We state it here with a proof for self-containedness.