Almost Ramanujan Expanders from Arbitrary Expanders via Operator Amplification

Expander graphs are fundamental objects in computer science and mathematics, possessing a variety of applications in both fields [HLW06, Lub12]. Indeed, expanders (and expansion) play a central role in numerous algorithmic advances, cryptographic schemes, circuit and proof complexity lower bounds, derandomization and pseudorandom generators, error correcting codes, … and are central to structural results in group theory, algebra, number theory, geometry, combinatorics.

A central quality measure of expansion of an infinite family of $d$-regular graphs $\{X_{i}\}_{i\in\mathbb{N}}$ is the second largest singular value of its normalized adjacency matrix, which we denote by $\lambda(X_{i})\in[0,1]$. We say that a family $\{X_{i}\}_{i\in\mathbb{N}}$ is $\lambda$-expanding, for some fixed $\lambda<1$, if $\lambda(X_{i})\leq\lambda$ for every member $X_{i}$ of the family. The smaller is the expansion parameter $\lambda$, the more spectrally expanding is the family. (For simplicity, we will sometimes discuss single graphs rather than families, and say that $X$ is a $(d,\lambda)$-expander if it is $d$-regular and satisfies $\lambda(X)\leq\lambda$.)

A random $d$-regular graph with $d\geq 3$ is easily shown [Pin73] to be $.99$-expanding with high probability, giving rise to the existence of expanding families. The quest to explicitly¹¹1See Definition 2.2 for a formal definition. construct bounded degree expanders started with Margulis’ paper [Mar73], and has been an extremely active research area in the past half-century. Today, we have a large arsenal of constructions and tools to establish expansion which are quite different in nature—algebraic, analytic, combinatorial, and mixtures of these (for a short survey of this wealth see [Wig18, Sec 8.7])—and we will discuss a few of them below.

All the different constructions above yield $d$-regular $\lambda$-expanding families with some specific constants $d$ and $\lambda$. Now, a large variety of structural and algorithmic applications call for optimizing both parameters, and understanding the best trade-off between them. One example which is directly related to this paper is the study of random walks on expanders, sometimes used for randomness-efficient error-reduction of probabilistic algorithms, and also in the construction of randomness extractors. The surprising expander Chernoff bound of Gillman [Gil98] informally says that a sequence of highly correlated $k$ vertices along a random walk in a $(d,\lambda)$-expander, is almost as good a sampler as a sequence of $k$ independent vertices, with respect to empirical sums. Saving randomness calls for minimizing the degree $d$ while improving the quality of the sample requires minimizing the expansion parameter $\lambda$.

However, for any choice of degree $d$, the spectral expansion $\lambda$ cannot be made arbitrarily small. The Alon–Boppana bound [Nil91] shows that $\lambda(X_{i})\geq 2\sqrt{d-1}/d-{o_{n}}(1)$. It intuitively says that the infinite $d$-regular tree is the best possible spectral expander, raising the challenge of achieving it by finite graphs. This challenge was first met by the (independent) seminal papers of [LPS88, Mar88]; they constructed optimal spectrally expanding families, dubbed Ramanujan graphs, satisfying the (Ramanujan bound) $\lambda(X_{i})\leq 2\sqrt{d-1}/d$. The investigation of expanding families near or achieving the optimal Ramanujan bound has received much attention. However, since then, only one essentially different construction of Ramanujan graphs was found 30 years later by [MSS15].

A study of almost Ramanujan expanders, in which the bound above is nearly matched, has received much attention as well. Friedman [Fri03] greatly strengthened Pinsker’s bound above [Pin73], showing that with high probability, a random $d$-regular graph $X$ satisfies $\lambda(X)\leq 2\sqrt{d-1}/d+{o_{n}}(1)$. For explicit constructions, one successful approach was the lifting method of Bilu–Linial [BL06], which achieves $d\leq\widetilde{O}(1/\lambda^{2})$, and famously led to the (exact) Ramanujan expanders of [MSS15] mentioned above.

A different paradigm to explicitly construct almost optimal expanders, which is central for this paper, is to start with a weak expander and amplify it to a strong one via combinatorially defined graph products. This originated with the zig-zag product of [RVW00]. They showed that their basic zig-zag construction achieves an explicit family of expanders with $d\leq 1/\lambda^{4}$. They further derandomize the basic zig-zag product to achieve $d\leq 1/\lambda^{3}$. Ben-Aroya and Ta-Shma [BATS08] in their ingenious “s-wide zig-zag product", nearly matched the optimal quadratic bound²²2We call any such bound near-optimal or almost Ramanujan. Of course, reducing the $o(1)$ slack in the exponent is clearly of much interest., achieving $d\leq 1/\lambda^{2+{o_{\lambda}}(1)}$. Their “higher-order” version of zig-zag [BATS08] will be central in our work. Note that these results yield specific constructions but do not provide a generic technique to amplify arbitrary expanders.

While for some applications and structural results, one is free to start with any family of expanders, for many others, the initial graphs are externally given to us (as e.g. is the case for understanding the expansion of Cayley graphs of groups). So it is highly desirable to have generic algorithms for expander construction that can operate within these constraints. Moreover, seeking different constructions and analysis tools has led to surprising applications beyond those intended (e.g., the resolution of the Kadison–Singer conjecture by [MSS14] and the proof of $\mathrm{SL}=\mathrm{L}$ by Reingold [Rei05]). Motivated by this, we study the following question:

When can a family of weak expanders be amplified to a strong (almost-Ramanujan) one?

We show that this is always possible: any expander family can be locally and efficiently converted into an almost Ramanujan family. More precisely, starting from any family of bounded degree expanders, it is possible to obtain, for any desired target expansion $\lambda>0$, a new family of $\lambda$-expanders close to the Ramanujan bound.

Our main result for general families of expander graphs is as follows.

The key insight is to use a result of König that says that the adjacency matrix, say $A_{X}$, of an arbitrary regular graph, can be written as a sum of permutation matrices. Thus we have, $A_{X}=\mathchoice{\underset{s\sim S}{\mathbb{E}}\left[\rho_{\mathrm{def}}(s)\right]}{{\mathbb{E}}_{s\sim S}[\rho_{\mathrm{def}}(s)]}{{\mathbb{E}}_{s\sim S}[\rho_{\mathrm{def}}(s)]}{{\mathbb{E}}_{s\sim S}[\rho_{\mathrm{def}}(s)]}$ where $S$ is a set of permutations, and $\rho_{\mathrm{def}}$ is the defining representation that maps a permutation to the matrix defining it. The task is to construct a set $S^{\prime}$, with $\left\lVert\mathchoice{\underset{s\sim S^{\prime}}{\mathbb{E}}\left[\rho_{\mathrm{def}}(s)\right]}{{\mathbb{E}}_{s\sim S^{\prime}}[\rho_{\mathrm{def}}(s)]}{{\mathbb{E}}_{s\sim S^{\prime}}[\rho_{\mathrm{def}}(s)]}{{\mathbb{E}}_{s\sim S^{\prime}}[\rho_{\mathrm{def}}(s)]}\right\rVert_{\textup{op}}\leq\lambda$. We study a general version of this question with an arbitrary group representation of a finite group (See Definition 2.3) . This is directly connected to the expansion of Cayley graphs.

We will now discuss some applications of this operator amplification technique which allows us to improve other pseudorandom objects. All the "pseudorandom" objects below are expanders (with various structural properties), and for each of these, we amplify their spectral bound to almost Ramanujan. We stress that our amplification preserves the underlying structure and therefore, produces another object with the same properties. Precise definitions of these objects will be given in Section 5.

We first rephrase the technical result of our paper from the perspective of “bias amplification”. This makes it easy to compare to prior work, which we crucially build upon. Let $f:S\rightarrow\symsf{M}_{\ell}({\mathbb{C}})$ be a matrix-valued function. The quantity $\left\lVert\mathchoice{\underset{s\sim S}{\mathbb{E}}\left[\symsf{f}(s)\right]}{{\mathbb{E}}_{s\sim S}[\symsf{f}(s)]}{{\mathbb{E}}_{s\sim S}[\symsf{f}(s)]}{{\mathbb{E}}_{s\sim S}[\symsf{f}(s)]}\right\rVert_{\textup{op}}$ is known as the bias of the operator-valued function $f$ with respect to $S$. The key idea in the prior works (and ours) is to establish an “bias amplification” result of the form,

Such a result is closely related to expansion of Cayley graphs in the following way. For a group $G$, a set $S\subseteq G$ is called an $\varepsilon$-biased set if for every non-trivial irreducible representation, $\rho$, of $G$, we have $\left\lVert\mathchoice{\underset{s\sim S}{\mathbb{E}}\left[\rho(s)\right]}{{\mathbb{E}}_{s\sim S}[\rho(s)]}{{\mathbb{E}}_{s\sim S}[\rho(s)]}{{\mathbb{E}}_{s\sim S}[\rho(s)]}\right\rVert_{\textup{op}}\leq\varepsilon$. Here, a group representation is a homomorphism from $G$ to the general linear group of operators, $\symsf{M}_{\ell}({\mathbb{C}})$. The notion of $\varepsilon$-biased set over $G={\mathbb{Z}}^{k}_{2}$ was introduced in the pioneering work of Naor and Naor [NN90]. These small-bias sets have found numerous applications (e.g., [ABN⁺92, Vad12, TS17]).

A symmetric subset $S\subseteq G$ is an $\varepsilon$-biased set if and only if the $\operatorname{Cay}(G,S)$ is an $\varepsilon$-expander. Therefore if $\operatorname{Cay}(G,S)$ is a given $\lambda_{0}$-expander, we can apply Theorem 1.8 for each irreducible representation $\rho$. This yields that $\operatorname{Cay}(G,S^{\prime})$ is a $\lambda$-expander where $S^{\prime}=\{s_{t}\cdots s_{1}\mid(s_{1},\cdots,s_{t})\in W\}$. The degree of the new graph is $|S^{\prime}|=|W|$. Note that over Abelian groups, the scalar amplification suffices as the irreducible representations of Abelian groups are $1$-dimensional, i.e., scalar-valued functions called characters. However, for general finite groups, one needs the amplification result for operator-valued functions.

The results of this paper have some curious features, which we would like to elaborate on. For most of them, we will use the following "bare bones" description of our main spectral amplification result. Namely, let $S$ be a finite set and ${\mathcal{H}}$ a Hilbert space. Let $\symsf{f}$ be a function mapping elements of $S$ to operators on ${\mathcal{H}}$ of unit norm, such that $\left\lVert{\mathbb{E}}_{s\in S}[\symsf{f}(s)]\right\rVert_{\textup{op}}\leq\lambda_{0}$. For any $\lambda>0$ take $t=c\log(1/\lambda)$ (for appropriate $c$). We extend $\symsf{f}$ from $S$ to $S^{t}$ by defining $\symsf{f}(s_{1},\ldots,s_{t})=\symsf{f}(s_{1})\cdots\symsf{f}(s_{t})$. Clearly, $\left\lVert{\mathbb{E}}_{r\in S^{t}}[\symsf{f}(r)]\right\rVert_{\textup{op}}\leq(\left\lVert{\mathbb{E}}_{s\in S}[\symsf{f}(s)]\right\rVert_{\textup{op}})^{t}\leq\lambda$. Our main result is an explicit construction of a (pseudorandom) subset $S^{\prime}\subseteq S^{t}$, of size only $|S^{\prime}|=O(|S|/\lambda^{2+o(1)})$, with a similar guarantee, namely $\left\lVert{\mathbb{E}}_{s^{\prime}\in S^{\prime}}[\symsf{f}(s^{\prime})]\right\rVert_{\textup{op}}\leq\lambda$.

We start in Section 2 by summarizing basic definitions and the notation used throughout the paper. In Section 3, we generalize the simpler construction of Ta-Shma based on expander walks. Apart from serving as a nice warm-up to the more-involved construction, it will be used as a bootstrap for the more involved construction based on $s$-wide replacement product which is the subject of Section 4. Here, we prove the main amplification result (a formal version of Theorem 1.9 above) and instantiate using known constructions and those obtained from Section 3 which establishes Theorem 1.2. Section 5 discusses the permutation amplification trick and formally completes the proof of Theorem 1.1. It also discusses the other applications in more detail. Finally, Section 6 gives an operator version of the expander mixing lemma which improves the analysis of [CMR13].

Since we deal with various vector spaces and graphs, we will find it useful to establish some convenient notation. While we recall these in the relevant section, the following is a summary for ready reference.

1. $\cdot$

   The main multigraphs we study will be $X$ and $Y$ with vertices $V_{X},V_{Y}$ and normalized adjacency operators $\symsf{A}_{X},\symsf{A}_{Y}$.

2. $\cdot$

   We denote vertices of $X,Y$ by $x,y$ and an ordered tuple of vertices by $\vec{x}=(x_{0},\cdots,x_{t})$.

3. $\cdot$

   We use $u,v,w$ to denote arbitrary vectors in $\mathcal{H}$ and $x,y$ for basis vectors of ${\mathbb{C}}[V_{X}],{\mathbb{C}}[V_{Y}]$ where ${\mathbb{C}}[V_{X}]$ is the complex vector space with the elements of $V_{X}$ being a orthonormal basis.

4. $\cdot$

   The tensored vector spaces have an induced inner product. For $\mathcal{X}_{\mathcal{H}}\coloneqq\mathcal{H}\otimes{\mathbb{C}}[V_{X}]$, it is $\left\langle v\otimes x,w\otimes x^{\prime}\right\rangle=\left\langle v,w\right\rangle_{\mathcal{H}}\left\langle x,x^{\prime}\right\rangle$. Similarly, we have one on $\mathcal{XY}_{\mathcal{H}}\coloneqq\mathcal{X}_{\mathcal{H}}\otimes{\mathbb{C}}[V_{Y}]$.

5. $\cdot$

   Orthogonal decomposition: $\mathcal{X}_{\mathcal{H}}=\mathcal{X}_{\mathcal{H}}^{\parallel}\oplus\mathcal{X}_{\mathcal{H}}^{\perp}$ where $\mathcal{X}_{\mathcal{H}}^{\parallel}\coloneqq\textup{span}\{v\otimes\vec{1}\mid v\in\mathcal{H}\}$. Here, $\vec{1}$ denotes the un-normalized all-ones vector. Similarly, $\mathcal{XY}_{\mathcal{H}}=\mathcal{XY}_{\mathcal{H}}^{\parallel}\oplus\mathcal{XY}_{\mathcal{H}}^{\perp}$, where $\mathcal{XY}_{\mathcal{H}}^{\parallel}\coloneqq\textup{span}\{z\otimes\vec{1}\mid z\in\mathcal{X}_{\mathcal{H}}\}$.

6. $\cdot$

   The operator $\smash{\mathrel{\mathop{\symsf{A}}\limits^{\vbox to0.0pt{\kern-2.0pt\hbox{$\scriptstyle\mathbin{\mathchoice{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}}$}\vss}}}}$ denotes the extension of operator $\symsf{A}$ to a tensor product of spaces where it acts as identity on the other spaces. For example, $\symsf{A}_{X}$ acts on ${\mathbb{C}}[V_{X}]$ and its extension to $\mathcal{X}_{\mathcal{H}}$ is $\mathrel{\mathop{\symsf{A}}\limits^{\vbox to0.0pt{\kern-2.0pt\hbox{$\scriptstyle\mathbin{\mathchoice{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}}$}\vss}}}_{X}=\symsf{I}_{\mathcal{H}}\otimes\symsf{A}_{X}$. However, if we were working on $\mathcal{XY}_{\mathcal{H}}$, it would be $\mathrel{\mathop{\symsf{A}}\limits^{\vbox to0.0pt{\kern-2.0pt\hbox{$\scriptstyle\mathbin{\mathchoice{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}}$}\vss}}}_{X}=\symsf{I}_{\mathcal{H}}\otimes\symsf{A}_{X}\otimes\symsf{I}_{Y}$ instead¹⁰¹⁰10The spaces will be self-evident and the use of the same notation should not be confusing..

7. $\cdot$

   Given an operator valued function $\symsf{f}:V_{X}\rightarrow\mathcal{L}(\mathcal{H})$, the generalized bias operator is defined on the basis as¹¹¹¹11An equivalent matrix definition is $\symsf{\Pi}_{\symsf{f}}\coloneqq\sum_{x\in V_{X}}\symsf{f}(x)\,\otimes\symsf{E}_{x,x}$ where $\symsf{E}_{x,x}\in\mathbb{C}^{V_{X}\times V_{X}}$ is the diagonal matrix with exactly one non-zero entry of value $1$ in the row and column indexed by the vertex $x$.,

   $${\symsf{\Pi}}_{\symsf{f}}\colon\mathcal{X}_{\mathcal{H}}\rightarrow\mathcal{X}_{\mathcal{H}},\;\ v\otimes x\mapsto\symsf{f}(x)\,v\otimes x.$$

• Proof:  

  $${\symsf{\Pi}}_{\symsf{f}}\left(\mathrel{\mathop{\symsf{A}}\limits^{\vbox to0.0pt{\kern-2.0pt\hbox{$\scriptstyle\mathbin{\mathchoice{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}}$}\vss}}}_{X}{\symsf{\Pi}}_{\symsf{f}}\right)^{t}\mathchoice{\underset{x\in V_{X}}{\mathbb{E}}\left[v\otimes x\right]}{{\mathbb{E}}_{x\in V_{X}}[v\otimes x]}{{\mathbb{E}}_{x\in V_{X}}[v\otimes x]}{{\mathbb{E}}_{x\in V_{X}}[v\otimes x]}~{}=~{}\mathchoice{\underset{(x_{t},\cdots,x_{0})\in\mathcal{W}_{t}}{\mathbb{E}}\left[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})\right]}{{\mathbb{E}}_{(x_{t},\cdots,x_{0})\in\mathcal{W}_{t}}[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})]}{{\mathbb{E}}_{(x_{t},\cdots,x_{0})\in\mathcal{W}_{t}}[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})]}{{\mathbb{E}}_{(x_{t},\cdots,x_{0})\in\mathcal{W}_{t}}[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})]}v\otimes x_{t}\,.$$

  (1)

  This can be shown easily via induction on $t$, and we refer to Lemma 4.7 for a formal proof of a more general statement. We use projection and lifting maps to move between the spaces $\mathcal{X}_{\mathcal{H}}$ and $\mathcal{H}$. Define $\symsf{P}_{\mathcal{H}}:\mathcal{X}_{\mathcal{H}}\rightarrow\mathcal{H}$ and $\symsf{L}_{\mathcal{H}}:\mathcal{H}\rightarrow\mathcal{X}_{\mathcal{H}}$, as,

  $\displaystyle\symsf{P}_{\mathcal{H}}(w\otimes x)=w,\;\;\symsf{L}_{\mathcal{H}}(v)=\mathchoice{\underset{x\in V_{X}}{\mathbb{E}}\left[v\otimes x\right]}{{\mathbb{E}}_{x\in V_{X}}[v\otimes x]}{{\mathbb{E}}_{x\in V_{X}}[v\otimes x]}{{\mathbb{E}}_{x\in V_{X}}[v\otimes x]}=\frac{1}{\left\lvert V_{X}\right\rvert}\,v\otimes\vec{1}\,.$

  From the definition, $\left\lVert\symsf{L}_{\mathcal{H}}(v)\right\rVert=\left\lVert v\right\rVert\frac{\left\lVert\vec{1}\right\rVert}{\left\lvert V_{X}\right\rvert}=\frac{\left\lVert v\right\rVert}{\sqrt{\left\lvert V_{X}\right\rvert}}$ and thus, $\smash{\left\lVert\symsf{L}_{\mathcal{H}}\right\rVert_{\textup{op}}=1/\sqrt{\left\lvert V_{X}\right\rvert}}$. We can use Cauchy-Schwarz to get that $\smash{\left\lVert\symsf{P}_{\mathcal{H}}\right\rVert_{\textup{op}}=\sqrt{\left\lvert V_{X}\right\rvert}}$. Now, we put this together to obtain a simple expression on the quantity we need to bound

  	$\displaystyle\left\lVert\mathchoice{\underset{(x_{0},\ldots x_{t})\in\mathcal{W}_{t}}{\mathbb{E}}\left[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})\right]}{{\mathbb{E}}_{(x_{0},\ldots x_{t})\in\mathcal{W}_{t}}[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})]}{{\mathbb{E}}_{(x_{0},\ldots x_{t})\in\mathcal{W}_{t}}[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})]}{{\mathbb{E}}_{(x_{0},\ldots x_{t})\in\mathcal{W}_{t}}[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})]}\right\rVert_{\textup{op}}$	$\displaystyle~{}=~{}\sup_{\left\lVert v\right\rVert=1}\left\lVert\mathchoice{\underset{(x_{0},\cdots,x_{t})\in\mathcal{W}_{t}}{\mathbb{E}}\left[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})\right]}{{\mathbb{E}}_{(x_{0},\cdots,x_{t})\in\mathcal{W}_{t}}[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})]}{{\mathbb{E}}_{(x_{0},\cdots,x_{t})\in\mathcal{W}_{t}}[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})]}{{\mathbb{E}}_{(x_{0},\cdots,x_{t})\in\mathcal{W}_{t}}[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})]}v\right\rVert_{2}$
  		$\displaystyle~{}=~{}\sup_{\left\lVert v\right\rVert=1}\left\lVert\symsf{P}_{\mathcal{H}}\,\left(\mathchoice{\underset{(x_{0},\cdots,s_{t})\in\mathcal{W}_{t}}{\mathbb{E}}\left[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})\right]}{{\mathbb{E}}_{(x_{0},\cdots,s_{t})\in\mathcal{W}_{t}}[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})]}{{\mathbb{E}}_{(x_{0},\cdots,s_{t})\in\mathcal{W}_{t}}[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})]}{{\mathbb{E}}_{(x_{0},\cdots,s_{t})\in\mathcal{W}_{t}}[\symsf{f}(x_{t})\cdots\symsf{f}(x_{0})]}v\otimes x_{t}\right)\right\rVert_{2}$
  		$\displaystyle~{}=~{}\sup_{\left\lVert v\right\rVert=1}\left\lVert\symsf{P}_{\mathcal{H}}{\symsf{\Pi}}_{\symsf{f}}\left(\mathrel{\mathop{\symsf{A}}\limits^{\vbox to0.0pt{\kern-2.0pt\hbox{$\scriptstyle\mathbin{\mathchoice{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}}$}\vss}}}_{X}{\symsf{\Pi}}_{\symsf{f}}\right)^{t}\mathchoice{\underset{x\in V_{X}}{\mathbb{E}}\left[v\otimes x\right]}{{\mathbb{E}}_{x\in V_{X}}[v\otimes x]}{{\mathbb{E}}_{x\in V_{X}}[v\otimes x]}{{\mathbb{E}}_{x\in V_{X}}[v\otimes x]}\right\rVert_{2}$
  		$\displaystyle~{}=~{}\sup_{\left\lVert v\right\rVert=1}\left\lVert\symsf{P}_{\mathcal{H}}{\symsf{\Pi}}_{\symsf{f}}\left(\mathrel{\mathop{\symsf{A}}\limits^{\vbox to0.0pt{\kern-2.0pt\hbox{$\scriptstyle\mathbin{\mathchoice{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}}$}\vss}}}_{X}{\symsf{\Pi}}_{\symsf{f}}\right)^{t}\symsf{L}_{\mathcal{H}}\,v\right\rVert_{2}$
  		$\displaystyle~{}\leq~{}\left\lVert{\symsf{\Pi}}_{\symsf{f}}\left(\mathrel{\mathop{\symsf{A}}\limits^{\vbox to0.0pt{\kern-2.0pt\hbox{$\scriptstyle\mathbin{\mathchoice{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}}$}\vss}}}_{X}{\symsf{\Pi}}_{\symsf{f}}\right)^{t}\right\rVert_{\textup{op}}\left\lVert\symsf{P}_{\mathcal{H}}\right\rVert_{\textup{op}}\left\lVert\symsf{L}_{\mathcal{H}}\right\rVert_{\textup{op}}$
  		$\displaystyle~{}\leq~{}\left\lVert{\symsf{\Pi}}_{\symsf{f}}\left(\mathrel{\mathop{\symsf{A}}\limits^{\vbox to0.0pt{\kern-2.0pt\hbox{$\scriptstyle\mathbin{\mathchoice{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}{\vbox{\hbox{$\scriptscriptstyle\circ$}}}}$}\vss}}}_{X}{\symsf{\Pi}}_{\symsf{f}}\right)^{t}\right\rVert_{\textup{op}}\,.$

  The last inequality follows from submultiplicativity of the operator norm and the observation that $\left\lVert{\symsf{\Pi}}_{\symsf{f}}\right\rVert_{\textup{op}}=\left\lVert f\right\rVert_{\infty}\leq 1$.      

Now that we have reduced the problem to studying the operator norm, we will study how the norm decays as we take walks. We use the decomposition, $\mathcal{X}_{\mathcal{H}}=\mathcal{X}_{\mathcal{H}}^{\parallel}\oplus\mathcal{X}_{\mathcal{H}}^{\perp}$ where $\mathcal{X}_{\mathcal{H}}^{\parallel}\coloneqq\textup{span}\{v\otimes\vec{1}\mid v\in\mathcal{H}\}$. The decay comes from two sources. For $z\in\mathcal{X}_{\mathcal{H}}^{\perp}$, we get a decay by $\lambda(X)$ by the definition of $X$ being an expander. 3.4 shows that for $z\in\mathcal{X}_{\mathcal{H}}^{\parallel}$, we get a decay from ${\symsf{\Pi}}_{\symsf{f}}$, equal to the initial bias. We put this together in Theorem 3.1 to obtain the desired exponential decay.

• Proof:   From definition of $\mathcal{X}_{\mathcal{H}}^{\parallel}$, we can assume that $z=u\otimes\vec{1}$. Computing we have,

  	$\displaystyle\left\lVert\left({\symsf{\Pi}}_{\symsf{f}}\left(u\otimes\vec{1}\right)\right)^{\parallel}\right\rVert_{2}~{}$	$\displaystyle=~{}\sup_{w\in\mathcal{H}\colon\left\lVert w\otimes\vec{1}\right\rVert_{2}=1}\left\lvert\left\langle w\otimes\vec{1},{\symsf{\Pi}}_{\symsf{f}}\left(u\otimes\vec{1}\right)\right\rangle\right\rvert$
  		$\displaystyle=~{}\sup_{w\in\mathcal{H}\colon\left\lVert w\otimes\vec{1}\right\rVert_{2}=1}\left\lvert\left\langle w\otimes\vec{1},{\symsf{\Pi}}_{\symsf{f}}\left(u\otimes\sum_{x\in V_{X}}x\right)\right\rangle\right\rvert$
  		$\displaystyle=~{}\sup_{w\in\mathcal{H}\colon\left\lVert w\otimes\vec{1}\right\rVert_{2}=1}\left\lvert\left\langle w\otimes\vec{1},\sum_{x\in V_{X}}\left(\symsf{f}(x)\,u\otimes x\right)\right\rangle\right\rvert$
  		$\displaystyle=~{}\sup_{w\in\mathcal{H}\colon\left\lVert w\otimes\vec{1}\right\rVert_{2}=1}\left\lvert\sum_{x\in V_{X}}\left\langle w,\symsf{f}(x)\,u\right\rangle\left\langle\vec{1},x\right\rangle\right\rvert$
  		$\displaystyle=~{}\sup_{w\in\mathcal{H}\colon\left\lVert w\otimes\vec{1}\right\rVert_{2}=1}\left\lvert\left\langle w,\left\lvert V_{X}\right\rvert\left(\mathchoice{\underset{x\in V_{X}}{\mathbb{E}}\left[\symsf{f}(x)\,\right]}{{\mathbb{E}}_{x\in V_{X}}[\symsf{f}(x)\,]}{{\mathbb{E}}_{x\in V_{X}}[\symsf{f}(x)\,]}{{\mathbb{E}}_{x\in V_{X}}[\symsf{f}(x)\,]}\right)u\right\rangle\right\rvert$
  		$\displaystyle~{}\leq~{}\left\lVert\mathchoice{\underset{x\in V_{X}}{\mathbb{E}}\left[\symsf{f}(x)\,\right]}{{\mathbb{E}}_{x\in V_{X}}[\symsf{f}(x)\,]}{{\mathbb{E}}_{x\in V_{X}}[\symsf{f}(x)\,]}{{\mathbb{E}}_{x\in V_{X}}[\symsf{f}(x)\,]}\right\rVert_{\textup{op}}\left\lvert V_{X}\right\rvert\left\lVert w\right\rVert\left\lVert u\right\rVert=\left\lVert\mathchoice{\underset{x\in V_{X}}{\mathbb{E}}\left[\symsf{f}(x)\,\right]}{{\mathbb{E}}_{x\in V_{X}}[\symsf{f}(x)\,]}{{\mathbb{E}}_{x\in V_{X}}[\symsf{f}(x)\,]}{{\mathbb{E}}_{x\in V_{X}}[\symsf{f}(x)\,]}\right\rVert_{\textup{op}}\left\lVert z\right\rVert_{2}\,.$

  The last line follows as $\left\lVert z\right\rVert_{2}=\left\lVert u\right\rVert_{2}\sqrt{\left\lvert V_{x}\right\rvert}$ and $\left\lVert w\right\rVert=\frac{1}{\sqrt{\left\lvert V_{x}\right\rvert}}$.      

We show that for every two¹²¹²12This is the source of loss of a factor of $2$ in the exponent (which leads to degree $O(\left\lvert S\right\rvert/\lambda^{4+o(1)})$ rather than the desired degree of $O(\left\lvert S\right\rvert/\lambda^{2+o(1)})$ we will later achieve. Note that the same loss occurs in the original zig-zag analysis of [RVW00], which was later remedied by the $s$-wise zig-zag of [BATS08]. steps of the walk, the norm of the (associated) operator decays as follows.