\documentclass[11pt]{article}
\include{lecture}
\begin{document}
\lecture{12}{2/25/2010}{Randomness Continued}{Beth Skubak \& Nathan Collins}

% ****** DRAFT **********
%\draft % put this here only if you want to indicate this is a draft

% ******* ********* ********* ********* ********* ********* ********* ********* |

In the last lecture we introduced randomized computation in terms of
machines that have access to a source of random bits and that return
correct answers more than $\frac{1}{2}$ of the time, and discussed several
problems where algorithms involving randomization are useful.  Today, we will
study more closely the complexity classes related to randomized machines, and
show how they are related to previous classes we have studied, including 
deterministic and non-deterministic classes.  We also will show a relationship to
non-uniform complexity classes by proving that
$\BPP$ has polynomial-size circuits.  In fact, the conjecture in the community
is that $\BPP = \P$, though currently, all that is known is that 
$\BPP \subseteq \Sigma^{p}_{2} \cap \Pi^{p}_{2}$, as we will prove today.

\section{Model}
We model randomness by equipping a standard Turing machine with
a random bit tape, to which the finite control has one-way read
access.  We assume this tape is filled with random bits from a perfectly
uniform distribution before computation begins. (The plausibility of 
true randomness in this assumption will be discussed in later lectures.)

The machine is given one-way read-only access because the random bit
tape is not counted as used space.  To reuse a random value it must
be stored on some work tape, while using a random value takes a
single computational step to read an entry from the random bit tape.

\subsection{Time and Space}
The complexity classes $\BPTIME(t)$ and $\BPSPACE(s)$ 
(bounded-error probabilistic time/space) are analogous 
to their deterministic counterparts.  They represent the classes
of problems solvable by a randomized machine with 2-sided error 
under some time or space bound, with the requirement that:

\begin{equation*}
\label{12:eqn:bptime}
\text{Pr[error]} \leq \frac{1}{3}.
\end{equation*}
This bound is standard but somewhat arbitrary; recall that any error bounded
below and away from $\frac{1}{2}$ can be made small by running the machine
multiple times and taking the majority vote.
In particular, we will consider the classes $\BPP$ (bounded-error probabilistic
polynomial time) and $\BPL$ (log space).

Complexity classes also exist for 1-sided and 0-sided machines.
The classes $\RTIME(t)$ and $\RSPACE(s)$, and the specific classes R (or $\RP$) and
$\RL$, refer to problems solvable by randomized machines with 1-sided 
error, i.e., which meet the following criteria:
\begin{align*}
&\text{Pr[error} | x \in L \text{]}
  \leq \frac{1}{2} \\
&\text{Pr[error} | x \in \bar L \text{]}
  = 0.
\end{align*}
Error in machines in 1-sided machines can be reduced by running the machine
a fixed number of times and accepting if a positive result is ever given,
since the machine never returns false positives.

The classes $\ZPTIME(t)$ and $\ZPSPACE(s)$, and specifically the classes $\ZPP$
and $\ZPL$, refer to problems solvable by 0-sided randomized machines.
These machines must meet these criteria:

\begin{align*}
\label{12:eqn:zptime}
&\text{Pr[error]} = 0 \\
&\text{Pr[``Unknown'']}  \leq \frac{1}{2}
\end{align*}
We may similarly reduce error in a 0-sided machine by running it at most a fixed
number of times and taking the first (if any) non-"Unknown" answer that is given.

\begin{claim}
The class $\ZPP$ is equivalent to the class $\ZPP2$ of problems solvable
by randomized machines, also with no chance of error, that run in
\emph{expected} polynomial time.
\end{claim}
\begin{proof}
A $\ZPP$ machine can be run repeatedly until it outputs a definite
answer.  By the definition of ZPP each run halts in polynomial time,
and outputs an answer with probability $\geq \frac{1}{2}$.  If $N^c$ is
the running time of our $\ZPP$ algorithm, then the expected running time 
of our modified algorithm is:
\begin{align}
\text{E(run time)} &\leq \sum_{i=1}^{\infty} (N^c \cdot i)\cdot
                     (\frac{1}{2})^i \\
&= N^c \sum_{i=1}^{\infty} i \cdot (\frac{1}{2})^i = 2 N^c
\end{align}
This new machine has expected running time polynomial and thus satisfies
the definition of $\ZPP2$.

Similarly a $\ZPP2$ machine can be run on a clock for $t \cdot$ E(run time),
then terminated with output ``Unknown''.  This modified algorithm outputs
``Unknown'' with probability at most:
\begin{align}
&\text{Pr}[ x \geq t \cdot E(x)] \leq \frac{1}{t} \text{ (Markov's inequality)}.\end{align}
To ensure the probability of outputting ``Unknown'' is at most 1/2, we 
set $t=2$.  As this new algorithm satisfies the error criterion of $\ZPP$ and
runs in polynomial time, it is a $\ZPP$ algorithm for the language.
\end{proof}


\begin{claim}
$\ZPP = \RP \cap \coRP$
\end{claim}
The proof of this claim is left as an exercise.

% ******* ********* ********* ********* ********* ********* ********* ********* |

\subsection{Space Complications}
In a deterministic or even non-deterministic setting, 
any log-space algorithm terminates
in polynomial time (if it terminates at all), due to the polynomial
bound on the number of possible machine configurations.  Obviously a repeated
configuration signifies a loop, which occurs if and only if
the machine does not halt.

Things become more complicated when randomness is involved.  A repeated
configuration does not necessarily signify an infinite loop, since a
random bit may allow the machine to take a different path rather than
repeating.  \emph{When discussing
space bounds in this setting we assume machines that always halt.}  This 
restriction gives us the nice property that a randomized log-space
machine runs in polynomial time (and in general, a space $s$ machine
runs in $2^{O(s)}$ time).

Note that machines of this type are distinct from those which halt with
probability 1.  We define a separate class, $\lfloor \ZPL \rceil$, for that
type of machine.  An example of the addition power this model has over the 
one stated above is given by the following claim, which we leave as an 
exercise.  While we do not expect that $\ZPP \subseteq \NP$ or that
$\BPL \subseteq \NL$, we know that
\begin{claim}
$\lfloor \ZPL \rceil = \NL$.
\end{claim}
For more on how these definitional issues affect the power of randomness in 
the log-space setting, see the survey \cite{Saks}.

% **************************
% SECTION
% **************************

\section{Relation to Deterministic Classes}
Relating these randomized classes to the known deterministic classes
provides a measure of their computational power.  One immediate result
is that
\[\RTIME(t) \subseteq \NTIME(t).\]
This follows from the fact that we may view any randomized machine $M$ with
1-sided error as a non-deterministic machine $M'$.  If there exists a sequence
of random bits so that $M$ accepts, we say that $M'$ accepts; note that $M'$
runs simultaneously the paths determined by all possible such strings of random bits.
We also have the inclusions
\begin{align*}
&\RP \subseteq \NP \\
&\P \subseteq \ZPP
   \subseteq  \RP
   \subseteq \BPP
   \subseteq \PSPACE
   \subseteq \EXP
\end{align*}
All but one inclusion are by definition.  The result that $\BPP \subseteq \PSPACE$ follows
from the fact that we can compute
the probability of acceptance by a randomized machine, by exhaustively
generating all possible strings of results of coin flips, of which there are
only polynomially many, and running the machine on each.  The space needed for each 
run is only polynomial, and we need only keep a tally of the acceptance or rejection
from each run.

It is an open question whether any of these containments is proper, though at
least one must be since $\P \subsetneq \EXP$.

Considering space complexity, we have the inclusions
\begin{equation}
\L \subseteq \ZPL
   \subseteq \RL
   \subseteq \BPL
   \subseteq \text{Uniform} \NC^2
   \subseteq \DSPACE(\log^2{N}).
\end{equation}
The result that Uniform $\NC^2 \subseteq \DSPACE(\log^2{N})$ follows by a proof similar to 
that of $\NC^1\subseteq \L$ given in the last lecture, in which we evaluate each path
through a $\log^2$-depth circuit, and so need only the space necessary to store one such path.
The fact that $\BPL \subseteq$ Uniform $\NC^2$ takes a bit more work.

\begin{claim}
$BPL \subseteq$ Uniform $\NC^2$
\end{claim}
\begin{proof}
A $\BPL$ computation, can be viewed as a Markov chain of machine
configurations.  For a log-space machine the size of the set of
configurations is polynomial.  The Markov chain can be represented
as a matrix, with each entry $M_{ij}$ representing the probability
of transitioning from state $j$ to state $i$.  (Note that these 
probabilities are time-independent, as needed for a Markov process.)
Exponentiating this
matrix $N$ times gives the probability matrix for
state transitions taking exactly $N$ steps.  This can be used to
determine the probability of reaching an accepting configuration.
We multiply $M^N$ by the probability vector representing the fact
that the machine begins in the first configuration:
\begin{equation}
\left[ \begin{array}{cccc}
p_{11} & p_{12} & \cdots & p_{1m} \\
p_{21} & p_{22} & \cdots & p_{2m} \\
\vdots & \vdots & \ddots & \vdots \\
p_{m1} & p_{m2} & \cdots & p_{mm} \end{array} \right]^N
\left[ \begin{array}{c}
1 \\
0 \\
\vdots \\
0 \end{array} \right]
\end{equation}
This is iterated matrix multiplication, which as discussed last time
is in Uniform $\NC^2$.  The result is a probability vector, with one
entry giving the probability of reaching the accepting
state in $N$ steps.
\end{proof}

In fact there is a tighter bound known for the class BPL:
\begin{claim}
$\BPL \subseteq \DSPACE{(\log^{3/2}{N})}$
\end{claim}
The proof for this claim will be presented in a future lecture, along
with evidence for the conjectures $\BPP = \P$ and $\BPL = \L$.  If these
conjectures hold, it would show that randomness does not truly provide
any additional computational power.  Even then, however, randomness would
still be very useful in practice to reduce computation time for several
problems, included those mentioned earlier in this lecture.  The speed-up 
for any problem, however, could only be polynomial.  Of course, for now
these conjectures remain unproven.

The following facts are also known (proving the second is the final
problem on Homework 1).

\begin{proposition}
\begin{align*}
&\BPP^{\BPP} = \BPP \\
&\NP \subseteq \BPP \implies \PH \subseteq \BPP \\
&\NP \subseteq \BPP \implies \NP = \RP
\end{align*}
\end{proposition}

The results given above relate randomized complexity classes among each other
and with deterministic and nondeterministic complexity classes.  The following
result relates the randomized class BPP to the non-uniform class 
$P/poly$, showing that in general randomness can be replaced by non-uniformity.
\begin{theorem}
$\BPP \subseteq$ P/poly and $\BPL \subseteq$ L/poly
\end{theorem}
\begin{proof}
Consider a $\BPP$ algorithm.  We first make the probability of error smaller
than the number of inputs of a given length $N$: Pr[error] $< 
\frac{1}{2^N}$.  Recall that running a BPP 
algorithm with error $1/2-\delta$ for $k$ times and taking the majority
vote results in a BPP algorithm computing the same language but now with
error at most $e^{-2k\delta^2}$.  We assume original error at most 1/3, 
so $\delta\geq 1/6$.  We need to pick $k$ large enough so that
$e^{-2k(1/6)^2}<1/2^N$.  Then a large enough $k=\Theta(N)$ suffices, meaning
the resulting BPP algorithm still runs in polynomial time.

Because we have reduced the error to less than $1/2^N$,
given any distinct setting
for the random bit tape, the probability that there exists some
input $x$ of length $N$ on which the machine makes an error is
less than one.  Therefore there exists at least one coin flip
sequence for which the machine gives the correct result on all
inputs of length $N$.  This sequence of coin flips is the advice
given.  As the amount of randomness used is some polynomial, this is a 
polynomial amount of advice.

The proof of the second claim uses the same process and the same advice;
note that it may be completed in log space since each run of the original
$\BPL$ machine takes only log space by definition, and we need only keep
a tally of the polynomially many votes to take the majority, which requires
only a logarithmic number of bits.
\end{proof}
We stated above that it is conjectured that BPP=P, though we have been unable 
to prove this.  In fact, while $R \subseteq \NP$, we cannot even prove that
$\BPP\subseteq \NP$.  We will show shortly, however,
that $\BPP$ lies within the polynomial hierarchy.


 \section{Hierarchy Results for $\BPP$}

Generally, whenever we introduce a new computational
 model, we look for hierarchy results: Does the class of problems we
 can solve using the model in question depend on the resources we are
 allowed?  Most hierarchy results use a computable enumeration of all
 machines of the type in question, but the following can be shown.
 % I left this as an exercise because he said ``I'm going to leave it
 % as a exercise that''
 \begin{exercise}
   There does not exist a computable enumeration of the randomized
   machines with error probability bounded away from $\frac{1}{2}$.
 \end{exercise}
 \noindent Therefore, typical hierarchy arguments fail for randomized
 machines.  The typical hierarchy arguments can be tailored to prove a
 hierarchy theorem for a modified model of randomized computation
 called promise-$\BPP$.  See HW 1.2.  However, no hierarchy result is
 known for $\BPP$.

\section{$\BPP$ and the Poly Time Hierarchy}

Although we don't know if $\BPP = \P$, or even if $\BPP \subseteq
\NP$, we do know that $\BPP \subseteq \PH$:

\begin{theorem}\label{12.thm.BPPInSecondLevelOfPH}
  $\BPP \subseteq \Sigma^{p}_{2} \cap \Pi^{p}_{2}$
\end{theorem}
\begin{proof}
  Fix $M$ a randomized polytime machine that accepts $L \in \BPP$, and
  let $r$ be the number of random bits $M$ uses when running on an
  input $x$ of size $n = |x|$.  We will now show that $\BPP \subseteq
  \Sigma^p_2$.  Since $\BPP = co\BPP$ it follows that $\BPP
  \subseteq co\Sigma^p_2 = \Pi^p_2$, completing the proof.

  \begin{figure} [h]
    \centering
    $x \in L$
    \input{rej-acc-ellipse.pstex_t}
    \quad   
    \input{acc-rej-ellipse.pstex_t}
    $x \notin L$
    % This caption was causing a latex error, but the DVI was being
    % generated just fine.  I eventually discovered that problem was
    % due to \notin, which I use above with no trouble.  Replacing
    % \notin with \not\in fixed the problem.  I'm quite confused ...
    \caption{If the error rate $\varepsilon$ of $M$ is small, then
      $Acc(x)$ will be much larger than $Rej(x)$ when $x \in L$, and
      $Acc(x)$ will be much smaller than $Rej(x)$ when $x \not\in
      L$. }
    \label{12:fig:acceptReject}
  \end{figure}

  We need to remove randomness from the computation.  For a given
  input $x$, the space $\{0,1\}^r$ of $r$-bit strings gets partitioned
  into two pieces: $Acc(x)$, the set of random strings on which $M$
  accepts $x$, and $Rej(x)$, the set of strings on which $M$ rejects
  $x$.  If the error rate $\varepsilon$ of $M$ is small, then $Acc(x)$
  will be much larger than $Rej(x)$ when $x \in L$, and $Acc(x)$ will
  be much smaller than $Rej(x)$ when $x \not\in L$. See Figure
  \ref{12:fig:acceptReject}. If our random computation has
  a small error rate then it will be correct on most random bit
  strings.  We'll turn ``most'' into ``all.'' The idea is that when $x
  \in L$ a few ``shifts'' of $Acc(x)$ will cover the whole space
  $\{0,1\}^r$ of $r$-bit random sequences, while the same few shifts
  of $Acc(x)$ will fail to cover the whole space when $x \not\in L$.

  % I've left the footnote in since I, for one, wouldn't otherwise know
  % what \oplus was denoting.
  If $S \subseteq \{0,1\}^r$ and $\sigma \in \{0,1\}^r$, then we define
  $S \oplus
  \sigma = \{ s \oplus \sigma | s \in S \}$, the \emph{shift} of $S$ by
  $\sigma$.\footnote{The symbol ``$\oplus$'' denotes XOR, or,
    equivalently, addition in $\mathbb{Z}^r_2$} Since shifting is
  invertible (in fact each shift is its own inverse)
   we see that $|S \oplus \sigma| = |S|$.

  We will use shifts to give a $\Sigma_2$-predicate using the
  intuition discussed above.  Namely, consider
  \begin{equation}\label{12.eqn.AccXShiftCoversWholeSpace}
    x \in L \iff \exists \sigma_1, \dots, \sigma_t \forall \rho \in
    \{0,1\}^r [ \rho \in \bigcup_{i=1}^t Acc(x) \oplus \sigma_i] 
  \end{equation}
  where the universal quantifier is
  over all possible strings $\rho$ of random bits.
   Now, $r$ is polynomial in $n$, and so if we can pick $t$ polynomial
    in $n$ as
  well, then the above will be a $\Sigma^p_2$-predicate, provided that
  we can verify the membership of $\rho$ in $\bigcup_{i=1}^t Acc(x)
  \oplus \sigma_i$ in time poly in $n$.  This membership check is no
  problem since we can equivalently check that $\rho \oplus \sigma_i
  \in Acc(x)$ for some $i$.  We can do this computation in
   polynomial time since it corresponds to
  running $M$ on $x$ with the random bit string $\rho \oplus
  \sigma_i$, for polynomially many $\sigma_i$.

  We show that we can pick a suitable polynomial $t$ to make
  \eqref{12.eqn.AccXShiftCoversWholeSpace} true by showing that we can
  choose the shifts $\sigma_i$ randomly with a high rate of success.  There
  are two cases to consider:
  \begin{enumerate}
  \item $x \in L$: For $\rho$ fixed, for a given shift $\sigma$, the probability
  that $\rho$ is not in $Acc(x) \oplus \sigma$ is given by
  \[
    \Pr[ \rho \notin 
      Acc(x) \oplus \sigma] = \Pr[\rho \oplus \sigma \notin Acc(x)] 
    \le \varepsilon,
    \]
  since $|Acc(x)|$ is a fixed size at least $1-\varepsilon$.
  Then for $t$ shifts chosen uniformly and independently, we have
    \[
    \Pr_{\sigma_1,\dots,\sigma_t}[ \rho \notin 
      \bigcup_{i=1}^t Acc(x) \oplus \sigma_i]
    \le \varepsilon^t.
    \]
     So, by a
    union bound, the probability that all random bit strings are not
    covered by the $t$ shifts of $Acc(x)$ is
    \[
    \Pr[ \{0,1\}^r \not\subseteq \bigcup_{i=1}^t Acc(x) \oplus
    \sigma_i] \le |\{0,1\}^r|\varepsilon^t = 2^r\varepsilon^t.
    \]
    Hence, if $2^r\varepsilon^t < 1$, some choice of the shifts
    $\sigma_i$ must cover all the random bit strings.
    
  \item $x \notin L$: We need to be sure that for any choice of the
  shifts
    $\sigma_i$, some bit string $\rho$ makes $M$ reject $x$.  But
    \[
    \mu(\bigcup_{i=1}^t Acc(x) \oplus \sigma_i) \le \sum_{i=1}^t \mu(Acc(x)
    \oplus \sigma_i) = t\varepsilon,
    \]
    since $\mu(Acc(x)) \le \varepsilon$ when $x \not\in L$. Hence in this case
    we need $t\varepsilon < 1$.
  \end{enumerate}
  That is, we need to choose $t$ so that both $t\varepsilon$ and
  $2^r\varepsilon^t$ are less than $1$.  After a short search, we
  see that $t = r$ will work, provided
  that $\varepsilon < \frac{1}{r}$.  From the definition of $\BPP$ we
  only know that $\varepsilon < \frac{1}{3}$, but, using the
  ``majority vote'' trick introduced during the last lecture, we can
  in $k$ runs reduce the error to $e^{-2k\delta^2}$, where $\delta =
  \frac{1}{2} - \varepsilon > 0$.  Of course, this increases the 
  number of random bits $r$ that we will need, but since we can 
  exponentially decrease the error with only a linear increase in bits,
  we see that this is clearly possible to choose such a $k$.
    Since $k$ runs will need $kr$
  random bits, it suffices to choose a polynomial $k$ large
  enough so that $e^{-2k\delta^2} < \frac{1}{rk}$.  In fact, $k = O(\log
  r)$ is sufficient, and an examination of the above shows that the time
  complexity is $O(T^2\log T)$ if the original run-time was $T$.
 
 Therefore, our choice of $t$ is sufficient to make 
 \eqref{12.eqn.AccXShiftCoversWholeSpace} a $\Sigma^p_2$ formula as
 needed.
\end{proof}

%\section{Expanders}
%
%The proof of Theorem \ref{12.thm.BPPInSecondLevelOfPH} used the
%``majority vote'' trick to get an exponential in $k$ increase in
%accuracy using a linear in $k$ increase in random bit usage.  Expander
%graphs, which are introduced here, lead to an ``amplification''
%technique that gives an exponential in $k$ accuracy improvement using
%only a constant increase in random bit usage ($rk$ versus $r + k$).
%Expanders have many other uses, some of which we mention in this
%lecture, and others we may see in later lectures.

%% \begin{definition} [$(k,c)$-expanding]
%%   A graph $G = (V,E)$ is \emph{$(k,c)$-expanding} if $S \subset V$
%%   with $|S| \le k$ implies that $|\Gamma(S)| \ge c|S|$, where
%%   $\Gamma(S) = \{ v \in V | \exists s \in S, (s,v) \in E \}$ is the
%%   \emph{neighborhood} of $S$ in $G$.
%% \end{definition}

%% Notice that any graph is trivially $(k,c)$-expanding for all $k$ for
%% all $c \le 1$.

%% \begin{definition} [Expander family] \label{12.def.expanderFamily}
%%   An \emph{expander family} is an infinite sequence of graphs
%%   $G_1,G_2,\dots$ and fixed numbers $c > 1$ and $d$ such that each $G_n$
%%   has degree $\le d$ and is $(\frac{N}{2},c)$-expanding, where $N =
%%   |V_n|$ is the number of vertices in $G_n$.
%% \end{definition}

%% Intuitively, expander graphs are ``very connected'' in that the number
%% of vertices reachable from a given subset of vertices is proportional
%% to the size of that subset, at least when the subsets aren't so large
%% as to make this impossible.

%% \subsection{Graph Theoretic Properties}

%% From now on we'll assume all our graphs are $d$-regular.  We describe
%% a $d$-regular graph using a normalized adjacency matrix.  This view of 
%% a expander graphs proves useful in the analysis of their properties and 
%% randomized algorithms that make use of them.

%% \begin{definition} [Normalized adjacency matrix]
%%   Given a $d$-regular $G = (V,E)$ its \emph{normalized adjacency
%%     matrix} $A$ is defined by
%%   \[
%%   A_{ij} =
%%   \begin{cases}
%%     \frac{1}{d}, & (i,j) \in E\\
%%     0, & \text{otherwise}
%%   \end{cases}
%%   \]
%% \end{definition}

%% The normalized adjacency matrix describes the Markov chain of a random
%% walk on $G$:
%% \[
%% A_{ij} = \Pr[\text{go to state $i$ from state $j$}],
%% \]
%% and if $p$ is a column vector with $p_i$ the probability of being at
%% vertex $i$, then $(Ap)_i$ gives the probability of being at vertex $i$ 
%% after one random step in $G$.  

%% The normalized adjacency matrix has a number of properties that can be 
%% proved using basic linear algebra.  We will use the following properties, but 
%% we do not prove them here.
%% \begin{proposition}
%%   $A$ is real-symmetric and so: all eigenvalues of $A$ are real, and
%%   $A$ has a basis of orthogonal eigenvectors.
%% \end{proposition}

%% It turns out that the eigenvalues of the normalized adjacency matrix of an 
%% expander are closely connected to the expanders graph properties.  The 
%% following are some basic properties, whose proofs we omit.

%% \begin{proposition}
%%   Let $A$ be a normalized adjacency matrix of a graph on $N$ vertices.
%%   \begin{enumerate}
%%   \item Each eigenvalue $\lambda$ of $A$ satisfies $|\lambda| \le 1$.
%%   \item 1 is an eigenvalue of $A$, with corresponding eigenvector given 
%%     by the uniform distribution on $(1/N,1/N,...,1/N)$.
%%   \item The multiplicity of the eigenvalue $1$ is greater than $1$ iff
%%     $G$ is disconnected.
%%   \item $A$ has $-1$ for an eigenvalue iff $G$ is
%%     bipartite.
%%   \end{enumerate}
%% \end{proposition}

%% As the uniform distribution is always an eigenvector, we look only at the 
%% remaining eigenvector/values.  It turns out that the second largest 
%% eigenvalue in absolute value is often useful to work with.

%% \begin{definition} [$\lambda(G)$]
%%   Let $\Lambda = \{ \lambda | \exists e_\lambda [ Ae_\lambda = \lambda
%%   e_\lambda \wedge \langle e_\lambda, u \rangle = 0 ] \}$. Then
%%   \[
%%   \lambda(G) = \max_{\lambda \in \Lambda} |\lambda|
%%   \]
%%   is the largest eigenvalue corresponding to an eigenvector
%%   orthogonal to $u$.
%% \end{definition}

%% So, if $G$ is connected and not bipartite then $\lambda(G) < 1$. The
%% next two theorems will not be proven here.  See the notes for CS 880
%% Spring 2006 for proofs.

%% \begin{theorem}\label{12.thm.expandingImpliesSpectralGap}
%%   If $G$ is $(\frac{N}{2}, c)$-expanding then $\lambda(G) < f(c,d)$
%%   where $f(c,d) < 1$ if $c > 1$ and is a function whose value depends only
%%   on $c$ and $d$.
%% \end{theorem}

%% \begin{corollary}
%%   Expanders have positive spectral gaps and expander families have
%%   spectral gaps bounded away from 0, where the spectral gap of a 
%%   graph $G$ is $1-\lambda(G)$.
%% \end{corollary}

%% \begin{theorem}\label{12.thm.spectralGapImpliesExpanding}
%%   If $\lambda(G) \le \lambda < 1$ then $G$ is $(\frac{N}{2},
%%   g(\lambda))$-expanding where $g(\lambda) > 1$ when $\lambda < 1$ and is
%%   a function whose value depends only on $\lambda$.
%% \end{theorem}

%% In light of Theorems \ref{12.thm.expandingImpliesSpectralGap} and
%% \ref{12.thm.spectralGapImpliesExpanding} we see that we could just
%% as well have defined expander families in Definition
%% \ref{12.def.expanderFamily} as $d$-regular families with positive
%% spectral gaps.

%% So, for expanders, the uniform distribution $u$ is the only fixed
%% point.  The following Proposition tells us that the larger the
%% spectral gap the faster an arbitrary probability distribution
%% converges to the uniform distribution via a random walk on
%% $G$.

%% \begin{lemma} \label{12.lem.spectralConvergenceRelationship}
%%   For any probability distribution vector $p$
%%   \[
%%   \| A^t p - u \|_1 \le \sqrt{N}\lambda^t,
%%   \]
%%   where $\| v \|_q = \big[\sum_i |v_i|^q \big] ^{1/q}$ is the $q$-norm
%%   of $v$ and $\lambda = \lambda(G)$.
%% \end{lemma}
%% \begin{proof}
%%   Since $Au = u$ we have $A^t p - u = A^t (p - u)$.  Now, $(p - u)
%%   \bot u$:
%%   \[
%%   \langle p - u, u \rangle = \langle p, u \rangle - \langle u, u \rangle
%%   = \sum_{i=1}^N p_i/N - \sum_{i=1}^N 1/N^2 = 1/N - 1/N,
%%   \]
%%   since $p$ is a probability distribution.

%%   Write $p - u = \sum_i a_i e_i$, where the $\{e_i\}$ form an
%%   orthogonal eigen basis for $A$ and $A e_i = \lambda_i e_i$.  Then,
%%   since the $u$-component of $p - u$ is $0$, we have
%%   \[
%%   \| A^t (p - u) \|_2 = \|A^t \sum_i a_i e_i \|_2 = \| \sum_i \lambda^t_i a_i e_i \|_2 \le
%%   \lambda^t \| \sum_i a_i e_i \|_2 = \lambda^t \| p - u \|_2
%%   \]

%%   Now
%%   \[
%%   \| p - u \|^2_2 + \| u \|^2_2 = \| p \|^2_2 \le \| p \|^2_1 = 1,
%%   \]
%%   since $(p - u) \bot u$, and $\| v \|^2_2 \le \| v \|^2_1$ for all
%%   vectors $v$. By the Cauchy-Schwartz inequality, we have
%%   \begin{align*}
%%     \|A^t (p - u)\|_1 &= | \langle
%%     (-1^{\sigma_1},\dots,-1^{\sigma_N}),
%%     A^t (p - u) \rangle |\\
%%     &\le \|(-1^{\sigma_1},\dots,-1^{\sigma_N})\|_2 \| A^t (p - u) \|_2\\
%%     &= \sqrt{N} \|A^t (p - u) \|_2 ,
%%   \end{align*}
%%   where $\sigma_k =
%%   \begin{cases}
%%     0, & (A^t (p - u))_k \ge 0\\
%%     1, & \text{otherwise}
%%   \end{cases}$.  Combining all of the above we get
%%   \begin{align*}
%%     \|A^t (p - u) \|_1 &\le \sqrt{N} \|A^t (p - u) \|_2\\
%%     &\le \sqrt{N} \lambda^t \| (p - u) \|_2\\
%%     &\le \sqrt{N} \lambda^t,
%%   \end{align*}
%%   completing the proof.
%% \end{proof}

%% This lemma can be used to prove that the random walk algorithm for the 
%% undirected path problem needs only polynomially many steps, i.e. that 
%% PATH $\in\BPL$.  The proof uses Lemma 
%% \ref{12.lem.spectralConvergenceRelationship} and the following
%% exercise.

%% \begin{exercise} \label{12.claim.spectralBound} If $G$ is connected and
%%   not bipartite then $\lambda(G) \le 1 - \frac{1}{dN^2}$.
%% \end{exercise}



%% \subsection{Constructions}
%% To prove that expanders exist one can argue that a randomly chosen
%% $d$-regular graph $G$ has a high probability of being an expander.
%% However, we want to use expanders to reduce the amount of randomness
%% needed in random algorithms, so using randomness to construct
%% expanders won't help us.  We want explicit constructions for which
%% given a vertex $v$ and index $i$ we can compute $v$'s $i$th neighbor
%% in time $\poly(|v|,|i|)$.

%% \begin{example}
%%   For any integer $m \ge 2$, we can get an expander $G$ on vertices $V =
%%   \mathbb{Z}_m \times \mathbb{Z}_m$ with edges given by the relations
%%   \[
%%   \Gamma(\{(x,y)\}) = \{ (x, y \pm x), (x, y \pm (x+1)), (x \pm y, y), (x
%%   \pm (y+1), y) \}.
%%   \]
%%   If $m \ge 4$ then the graph has degree $8$.  The proof that this
%%   construction works is non-trivial.  See the notes for CS 880 for a
%%   partial proof using harmonic analysis.
%% \end{example}

%% There are other efficient constructions of expanders, but the aforementioned
%% expander suffices for our needs.  See the notes for CS880 Spring 2006 for
%% other constructions.


%\subsection{Applications}
%
%We'll use expanders for two purposes:
%\begin{enumerate}
%\item \label{12.enum.deterministicAmplification} \emph{Deterministic
%    amplification}: Given a randomized algorithm $R$ that uses $r$
%  random bits we reduce the error to be less than some arbitrary
%  $\varepsilon$, without using any additional random bits.  This
%  requires running $R$ $\poly(1/\varepsilon)$ times.
%\item \label{12.enum.randomnessEfficientAmplification}\emph{Randomness
%    efficient amplification}: With $R$ and $r$ as
%  above we reduce the error to be less than some arbitrary
%  $\varepsilon$ using $r + O(\log(1/\varepsilon))$ additional random
%  bits and running $R$ $O(\log(1/\varepsilon))$ times.
%\end{enumerate}

%% To accomplish the above amplifications we use an expander graph
%% $G$ whose vertices are in one-to-one correspondence with the bit strings in
%% $\{0,1\}^r$.  For application \ref{12.enum.deterministicAmplification} we run
%% $R$, look at the vertex $v$ in $G$ corresponding to the random bits
%% used by $R$'s run, and then run $R$ once for each neighbor $v'$ of
%% $v$, with random bits corresponding to $v'$.  For
%% application \ref{12.enum.randomnessEfficientAmplification} we use the same $G$ and
%% $v$.  Then, starting from $v$, we perform a random walk in $G$
%% starting from $v$ of length $t = \log(1/\varepsilon)$.  This gives $t$
%% vertices $v_1,\dots,v_t$.  We then run $R$ once for each vertex $v_i$
%% with random bits corresponding to $v_i$.  In both cases, we finish by
%% taking the ``majority vote'' of $R$'s runs.  We discuss these applications
%% and prove their correctness in the next lecture.

%% \section{Next Time}

%% In the next lecture we prove the correctness of the algorithms given in the 
%% applications above.  To accomplish this, we also prove
%% \begin{lemma} [Expander mixing lemma]
%%   Given a $d$-regular expander graph $G = (V,E)$ and $S,T \subseteq V$
%%   \[
%%   \left| \frac{|E(S,T)|}{dN} - \mu(S)\mu(T) \right| \le
%%   \lambda(G)\sqrt{\mu(S)(1 - \mu(S))}\sqrt{\mu(T)(1 - \mu(T))}
%%   \]
%%   where $E(S,T)$ is the set of edges connecting $S$ to $T$.
%% \end{lemma}
%% Notice that $dN$ is the number of ways to choose an edge in $G$: $N$
%% choices of starting vertex and then degree $d$ choices for ending
%% vertex.  Also, $\mu(S)\mu(T)$ is the probability that, given two
%% randomly chosen vertices, one lies in $S$ and the other in $T$.


\begin{thebibliography}{}

\bibitem{Saks}
Michael Saks.
\newblock {Randomization and Derandomization in Space-Bounded Computation}.
\newblock {\em Annual Conference on Structure in Complexity Theory}, 1996.

\end{thebibliography}


\end{document}

% LocalWords:  ZPP Markov's BPL ij Exponentiating cccc BPP polytime Acc Rej kr
% LocalWords:  rk Expanders expanders Derandomization
