\documentclass[11pt]{article}
\usepackage[fleqn]{amsmath}
\usepackage{lecture}

\begin{document}
\sloppy
\lecture{15}{3/13/2013}{Pseudorandomness for Logarithmic Space}{Gautam 
  Prakriya}

\draft % put this here only if you want to indicate this is a draft

In the last lecture, we discussed a recursive construction of a 
pseudorandom generator that is based on expanders: the INW generator.  
In this lecture we will show that the INW-generator with a suitable 
instantiation of an expander family produces pseudorandomness for 
machines that run in logarithmic space.
We also introduce \emph{Nisan's generator}, which is a pseudorandom 
generator that has a recursive structure that is slightly different 
from, but related to, the recursive structure of the INW-generator, 
and which employs hash families instead of expanders.
We begin with a description of the class of algorithms that we want to 
produce pseudorandomness for.

\section{Logspace Turing Machines}
One way to model bounded space computation uses the Turing machine 
model and restricts the length of the work tapes.
A logspace machine $M$ is a Turing machine with a read-only input tape 
that contains the input~$x$, a work tape of size $O(\log |x|)$ that 
corresponds to the memory, and a write-only output tape.
We usually equate access to randomness with the ability to flip a fair 
coin, and we model this situation by allowing the machine to access an 
additional infinite read-once tape that is initialized with bits which 
are sampled independently and uniformly at random.
Notice that the read-once restriction captures the fact that the 
outcome of a coin toss must be stored in the configuration of the 
machine if it is to be used later in the computation.
As usual, we denote the random string used by the machine as $\rho$ 
and its length by $r$.

For notational simplicity we will only talk about decision problems, 
though much of what we discuss here is easily extended to more general 
settings.
We further restrict ourselves to randomized logspace machines with 
bounded, one-sided error, i.e., machines that may falsely reject a 
YES-instance, but that never falsely accept a NO-instances.
This corresponds to $\RL$-machines as formally defined below.
\begin{definition}
  A logspace-machine $M$ is an $\RL{}$-machine for a language $L$ if, 
  for all $x$, we have
  \begin{itemize}
    \item if $x\in L$, then $\Pr_{\rho} [M(x,\rho) \text{ accepts} ] 
      \geq 1/2$,
    \item if $x\not\in L$, then $\Pr_{\rho} [M(x,\rho) \text{ 
        accepts}] \leq 0$, and
    \item
      $M$ halts after at most $\poly(|x|)$ steps.
  \end{itemize}
\end{definition}
The third condition states that the worst-case running time of the 
algorithm should be polynomial.
Notice that any deterministic logspace machine that halts, halts in 
polynomially many steps.  This is not the case for a randomized 
machine, which is why we need the termination condition in the 
definition.
Consider for example the machine that loops until it finds a~$1$ in 
the random string~$\rho$ and then accepts.
With probability one, this machine halts and accepts, but if the 
random string happens to be the infinite $0$-string, it never halts, 
so the worst-case running time is unbounded.

\subsection{Branching Programs}
Branching Programs are the non-uniform analogues of bounded-space 
Turing machines.
For our purposes, we think of a branching program as a layered 
directed acyclic graph in which each layer has the same number of 
nodes (see Figure~\ref{figure: branching program}).
Every node except the ones in last layer have two outgoing edges, 
labeled with~$0$ and~$1$, leading to nodes in the next layer.
More formally, a branching program $B$ of length $r$ and width $w$ is 
a function $B:[r-1]\times [w] \times\zo \to [r]\times [w]$ so that 
$B(i,*;*)=(i+1,*)$ holds for all $i<r$, where $[r]=\{1,\dots,r\}$.
Note that branching programs essentially constitute a special case of 
finite state machines.
\begin{figure}[tbp]
\begin{center}%
 \includegraphics[trim=0 6.5cm 0 6cm , clip , scale=0.8]{BP.pdf}%
\end{center}%
\caption{\label{figure: branching program}%
  Pictorial representation of a branching program of width $w=2^S$ and 
  accepting set $A$.
  Every non-final vertex has two outgoing edges to the next layer, and 
  in each layer, a particular bit~$\rho_i$ of the random string is 
  read.}
\end{figure}


A branching program~$B$ can be viewed as a model of computation by 
picking a start node $s=(1,*)$ in the first layer and a set of 
accepting nodes $A\subseteq\{r\}\times [w]$ in the last layer.
Then a string $\rho$ is \emph{accepted} by~$B$ if there is a path from 
$s$ to $A$ such that the edge labels in the path correspond to the 
bits in the string $\rho$.

An $\RL$-machine~$M$ can be translated to a branching 
program~$B=B_{M,x}$ as soon as the input~$x$ is known:
We let~$w$ be the total number of configurations that the logspace 
machine~$M$ can be in when $x$ is fixed.
Since~$M$ has $S=O(\log |x|)$ space and $O(1)$ internal states, the 
total number of distinct configurations of~$M$ is upper bounded by 
$O(2^{S}) = \poly(|x|)$.
In each layer, the set of nodes $(i,*)$ corresponds to the set of 
configurations that~$M$ can be in right before step $i$.
Let $s$ correspond to the initial configuration of~$M$, that is, the 
configuration that $M$ on input $x$ is in right before it reads the 
first random bit $\rho_1 \in\zo$.
Let $C_b$ be the configuration of $M$ right before reading $\rho_2$ in 
case $\rho_1=b$.
Then we define $B(s,\rho_1)=(2,C_{\rho_1})$.
In general, every time a new random-bit is read and depending on the 
value of the bit, the machine~$M$ moves to a new configuration, and 
this transition is modeled by $B$.
Notice that we only model changes in the configuration of the machine 
when a random bit is read; the deterministic part of the computation 
and its dependency on the actual input~$x$ is hidden in the 
non-uniformity of the model.
The length of the branching program is equal to the number of random 
bits~$r$ read by the machine and therefore upper bounded by the 
running time~$\poly(|x|)$ of $M$.
Finally, the accepting states~$A$ in the final layer correspond to the 
accepting configurations of~$M$.

Let us introduce some notation to talk about specific runs of $B$ when 
the original input is fixed and the randomness input is given.
On (randomness) input~${\rho\in\zo^{r'}}$, we overload our notation of 
$B$ to a function $B:[r-1]\times[w]\times\zo^{r'}\to [r]\times[w]$ 
defined recursively as $B(i,j; \rho_1 \rho_2 \dots \rho_\ell)= B( 
B(i,j;\rho_1); \rho_2\dots\rho_\ell)$.
Finally, we may also write $B : \zo^r \to \{0,1\}$ for the function 
computed by the branching program~$B$, that is,
\begin{align*}
  B(\rho) \doteq
  \begin{cases}
    1 & \text{if $B(s;\rho) \in A$, and}\\
    0 & \text{otherwise}\,.
  \end{cases}
\end{align*}
Since $M$ was an $\RL$-machine for a language $L$, the program 
$B=B_{M,x}$ satisfies
$\Pr_\rho ( B(\rho)=1 ) = 0$ if $x\not\in L$, and it satisfies 
$\Pr_\rho ( B(\rho)=1 ) \geq 1/2$ if $x \in L$.
In either case, we expressed the acceptance probability of $M$ on 
input~$x$ in terms of the probability that $B$ outputs one.
Since $B$ can be seen as a directed graph with out-degree two, the 
latter is equal to the probability that a random walk in $B$, started 
at $s$, will end in $A$ after $r$ steps.

\section{Pseudorandomness for Branching Programs}
In this section we prove that the INW-generator instantiated with a 
suitable expander family fools read-once branching programs.
After that, we introduce another recursive construction of a PRG for 
space bounded computation known as Nisan's generator.

For convenience, let us discuss what it means to be pseudorandom for 
branching programs.
We say that a generator $G : \zo^\ell \to \zo^r$ is 
$\epsilon$-pseudorandom for branching programs of length~$r$ and 
width~$w$ if, for all such branching programs~$B$, we have
\begin{align}\label{pseudorandom}
  \left|
    \Pr_{\rho \sim U_r} \big(B(\rho) = 1\big)
    -
    \Pr_{\rho \sim G(U_\ell)} \big(B(\rho) = 1\big)
  \right|
  \leq \epsilon\,.
\end{align}
\subsection{INW-Generator}

We recall the definition of the INW-generator, 
Definition~\ref{lect14:definition: INW} from Lecture 14, but we 
specialize the definition to the case in which the output of the 
generator is a binary string; this is what we ultimately need as the 
input of the branching program. There are several ways of doing this 
specialization, and we choose to simply restrict the initial symbol 
set $[D]$ to be binary.
\begin{definition}[Impagliazzo--Nisan--Wigderson Generator (binary 
  version)]\mbox{}\\
  Let $H_1,H_2,\dots$ be a sequence of directed outregular graphs such 
  that $H_i$ has out-degree $d_i$, vertex set $\zo \times [d_1] 
  \times\dots\times[d_{i-1}]$ and neighbor function 
  $\Gamma_{H_i}:V(H_i)\times [d_i]\to V(H_i)$.
  Then the INW-generator is the family of functions
  $G_i: \zo\times [d_1]\times\dots\times[d_i] \rightarrow \zo^{2^i}$ 
such that
  \begin{align*}
    G_0(h_0) &\doteq h_0\\
    G_{i+1}(h_0\ldots h_{i+1})
    &\doteq
    G_i\Big(h_0\ldots h_i\Big)\circ G_i\Big( \Gamma_{H_{i+1}}( 
    h_0\ldots h_{i};h_{i+1})\Big)\,,
  \end{align*}
  where $h_0\in [D]$ and $h_j \in [d_j]$ for all $j>0$.
\end{definition}
We let $H_1, H_2,\dots$ be a family regular expanders of degree at 
most~$d$ each and with some sufficiently good spectral expansion at 
most~$\lambda$ to be chosen later.
We have seen before that $d\sim 1/\lambda^2$ can be achieved.
Unfortunately, it is technically tricky to use the fully explicit 
family that we constructed based on the replacement product, because 
we have to worry about the exact degrees and number of vertices that 
we need in the construction of the INW-generator.
We ignore these issues here and assure the reader that these 
difficulties can be overcome.

Note that the first $d-1$ graphs above, that is, $H_1,\dots,H_{d-1}$, 
will be chosen to be complete graphs, that is, $H_1$ is the complete 
graph on two vertices and all self-loops, and $H_{d-1}$ is the 
complete graph with $d$ vertices and all self-loops.
Only after that point will we employ a $d$-regular family of expanders 
with second-largest eigenvalue at most~$\lambda$.

To transform the INW-generator $G_k$ to a generator $G_k:\zo^\ell \to 
\zo^r$ that takes a binary input, we represent the elements of 
$\zo\times [d_1]\times\dots[d_k]$ in binary, which can be done with 
strings of length $\ell \doteq
1+\sum_{i=1}^k \log d_i \leq k \cdot \log d$.
Since $r=2^k$ and $d\sim 1/\lambda^2$, we achieve a seed length of 
$\ell \sim \log r \cdot \log(1/\lambda)$.
To get the error down to $\epsilon>0$ for general branching programs, 
we will require $\lambda=\poly\left(\frac{\epsilon}{rw}\right)$ in the 
analysis below, for which reason the seed length of the INW-generator 
is
\begin{align*}
  \ell \sim \log r \cdot \Big(\log r + \log w + \log 
  (1/\epsilon)\Big)\,.
\end{align*}
More succinctly, in the most interesting case where the width $w$ is 
polynomial in $r$ and the error $\epsilon$ is a constant, say $1/2$, 
the seed length simplifies to $\ell \sim \log^2 r$.

We now prove that the INW-generator is pseudorandom for branching 
programs and analyze the error.
Unfortunately, we don't know how to analyze the error using the 
derandomized squaring of expanders as we did for undirected 
connectivity in Lecture~14.
The analysis below appears to be different, and relies directly on the 
quasirandomness property of expanders that we proved in the expander 
mixing lemma (Lemma~\ref{lect10:lemma: expander mixing} of 
Lecture~10).
\begin{lemma}\label{lemma: INW}
  For all $k\in\Nat$, the INW-generator $G_k$ instantiated with 
  expanders of second-largest eigenvalue at most~$\lambda$ is $(3^k 
  \cdot w\cdot \lambda)$-pseudorandom for branching programs of 
  length~$r=2^k$ and width~$w$.
\end{lemma}
\begin{proof}
  Let $B$ be a branching program length $r=2^k$ and width $w$.
  We need to prove \eqref{pseudorandom}.
  To this end, we will actually prove a stronger condition so that we 
  can use induction.
  The stronger condition is that the statistical distance between the 
  distributions of the state that we reach when reading a uniform 
  input and the one when reading a pseudorandom input is small.
  Formally, the claim is that, for all $i\in\Nat$ and all branching 
  programs~$B$ of length $2^i$, the following holds for all 
  states~$s\in\{1\}\times[w]$:
  \begin{align}\label{strong pseudorandom}
    \sdist\Big(
      B\big(s;U_{2^i}\big)
      ,
      B\big(s;G_i ( U ) \big)
    \Big)
    \leq \epsilon_i \doteq 3^i w \lambda\,.
  \end{align}
  Since we want to avoid specifying the seed length of $G_i$ here, we 
  let the distribution $U$ be uniform on all valid inputs of $G_i$.
  Recall that $B(s;\rho)$ denotes the \emph{state} in $[r]\times[w]$ 
  that we reach when starting~$B$ at $s$ and reading~$\rho$; thus, 
  $B(s; D)$ is a distribution on the states that we reach when the 
  randomness comes from a distribution~$D$.

  To see that \eqref{strong pseudorandom} implies 
  \eqref{pseudorandom}, note that the statistical distance in 
  \eqref{strong pseudorandom} is by definition equal to the maximum of 
  the left-hand side of \eqref{pseudorandom} taken over all sets 
  $A\subseteq [r]\times[w]$.
  
  The proof of \eqref{strong pseudorandom} proceeds by induction on 
  $i$.
  For $i=0$, we have $r=1$ and both $U_r$ and $G_i(U)$ are just a 
  single uniform bit; so the distributions are the same and the 
  statistical distance in \eqref{strong pseudorandom} is equal to 
  zero.
  For the induction step $i\to i+1$, we essentially split the 
  computation of the branching program in the middle and use the 
  induction hypothesis on both parts.
  We use the triangle inequality to show an upper bound on the error 
  of the entire execution in terms of the errors of the two parts.

  So first consider the input to be truly random.
  Then we can rewrite the output distribution under truly random input 
  as
  $B\big(s;U_{2^{i+1}}\big) =
  B\big(B(s;U_{2^i});U_{2^{i}}\big) =
  B\big(M_U;U_{2^{i}}\big)$
  where $M_U$ is the random variable $B(s;U_{2^i})$ that holds the 
  state we reach in the middle layer after reading the first half of 
  the truly random input, and $B\big(M_U;U_{2^{i}}\big)$ is the state 
  that we reach after reading the second half of the truly random 
  input provided that we started at a state sampled from $M_U$.

  Because of the recursive structure of the INW-generator, we can do a 
  similar decomposition for the pseudorandom distribution.
  Namely, we split the uniform input distribution $U$ for $G_{i+1}$ 
  into two parts $X$ and $Y$, so that $X$ corresponds to the uniform 
  distribution on the edge labels $h_0\dots h_i$ and $Y$ corresponds 
  to the uniform distribution on the edge labels $h_{i+1}$.
  Then we can write
  \begin{align}\label{INW simple recursion}
    G_{i+1}(XY) = G_i (X) \circ G_i \big( \Gamma(X;Y) \big)\,,
  \end{align}
  where $\Gamma=\Gamma_{H_{i+1}}$ is the function from the definition 
  of the INW-generator.
  As above, we can then write
  \begin{align*}
    B\big(s;G_{i+1}(U)\big) =
    B\Big(B(s;G_i(X));G_i(\Gamma(X;Y))\Big) =
    B\big(M_P; G_i(\Gamma(X;Y))\big)\,,
  \end{align*}
  where $M_P=B(s;G_i(X))$ is the random variable that holds the state 
  that we reach in the middle layer after reading a random string 
  sampled from the pseudorandom distribution $G_i(X)$.

  Our goal is to show~\ref{strong pseudorandom}, namely that 
  $B\big(s;U_{2^{i+1}}\big)$ and $B\big(s;G_{i+1}(U)\big)$
  are close in statistical distance.
  We are now in position to use the triangle inequality to break up 
  the computation in the middle:
  \begin{align}
    \sdist\Big(
      B\big(s;U_{2^{i+1}}\big)
      ,
      B\big(s;G_{i+1} ( U ) \big)
    \Big)
    &=
    \sdist\Big(
    B\big(M_U; U_{2^i}\big)\;,\;
    B\big(M_P; G_i(\Gamma(X;Y))\big)
    \Big)\nonumber\\
    &\leq
    \sdist\Big(
    B\big(M_U; U_{2^i}\big)\;,\;
    B\big(M_P; U_{2^i}\big)
    \Big)\label{induction1}\\
    &+
    \sdist\Big(
    B\big(M_P; U_{2^i}\big)\;,\;
    B\big(M_P; G_i(U)\big)
    \Big)\label{induction2}\\
    &+
    \sdist\Big(
    B\big(M_P; G_i(U)\big)\;,\;
    B\big(M_P; G_i(\Gamma(X;Y))\big)
    \Big)\,.\label{mixing}
  \end{align}
  In the following claims, we will use the induction hypothesis and 
  the expansion properties to bound this sum by $\epsilon_i + 
  \epsilon_i + w\cdot\lambda$; by the definition in \eqref{strong 
    pseudorandom}, this is equal to $2\cdot 3^i w \lambda + w \lambda 
  \leq 3^{i+1} w \lambda = \epsilon_{i+1}$, which then concludes the 
  induction step.
  Let us now bound the three terms separately.

  To bound the first term of the sum, we use the induction hypothesis 
  on the first half of the branching program.
  \begin{claim}
    $\eqref{induction1} \leq \epsilon_i$\,.
  \end{claim}
  \begin{proof}
    We get $\sdist(M_U,M_P)\leq \epsilon_i$ since $M_U$ and $M_P$ are 
    the output distributions of a branching program of length $2^i$ 
    under uniform input~$U_{2^i}$ and under pseudorandom input sampled 
    from~$G_i(U)$, respectively.
    Furthermore, the process $F : S \mapsto F(S)\doteq B(S;U_{2^i})$ 
    is a (randomized) procedure applied to the (outcome of) the random 
    variable $S$.
    It is a general fact about the statistical distance that no 
    process can increase the statistical distance between two random 
    variables, that is, 
    $\eqref{induction1}=\sdist(F(M_U),F(M_P))\leq\sdist(M_U,M_P)$ 
    holds, which establishes the lemma.
  \end{proof}

  To bound the second term of the sum, we use the induction hypothesis 
  on the second half of the branching program.
  \begin{claim}
    $\eqref{induction2} \leq \epsilon_i$\,.
  \end{claim}
  \begin{proof}
    By the triangle inequality, we get
    \begin{align*}
      \eqref{induction2}
      \leq \Exp_{m\sim M_P}
      \Big[\sdist\Big(B(m;U_{2^i}),B(m;G_i(U))\Big)\Big]
      \leq \max_{m\in\text{middle layer}}
      \sdist\Big(B(s;U_{2^i}),B(s;G_i(U))\Big)\,.
    \end{align*}
    The second half of~$B$ started at any vertex $m$ from the middle 
    layer of~$B$ is a branching program of length $2^i$.
    Hence the induction hypothesis applies and we get an upper bound 
    of $\epsilon_i$.
  \end{proof}
  The following claim is the crux of the INW-generator and the only 
  part of the proof that uses the facts that the branching program has 
  width at most~$w$ and that the function $\Gamma$ in the definition 
  of the INW-generator corresponds to the neighbor function in an 
  expander graph with second-largest eigenvalue at most $\lambda$.
  \begin{claim}
    $\eqref{mixing} \leq w\cdot \lambda$\,.
  \end{claim}
  \begin{proof}
    We will again upper bound the statistical distance by the 
    worst-case state $m$ that we might reach in the middle layer as we 
    did in the proof of the previous claim.
    However, there is a complication since the distribution $X$ may 
    change if we condition on reaching a certain state~$m$ in the 
    middle layer.
    This is because the distributions $M_P$ and $G_i(\Gamma(X;Y))$ are 
    not independent since~$M_P$ and~$X$ may be correlated.
    However, since $M_P$ is a distribution on at most $w$ states, the 
    amount of dependence between the two distributions is actually 
    fairly small.
    Since $Y$ and $M_P$ are fully independent, this limited dependence 
    allows us to use $Y$ to reshuffle $X$ enough for the second half 
    of the branching program not to notice any significant correlation 
    of its input to the input that the first half received.

    Formally, we define the random variable $X_m \doteq (X\vert 
    M_P=m)$ for each possible outcome of the middle state $m$.
    That is, for all $x$ in the support of $X$, we define $\Pr ( X_m = 
    x ) \doteq \Pr ( X = x | M_P = m )$.
    Note that, because~$X$ was uniform, the new distribution~$X_m$ is 
    uniform on its support~$\text{supp}(X_m)$.

    We use the $1$-norm characterization of the statistical distance:
    \begin{align*}
      \eqref{mixing}
      &=
      \frac{1}{2}
      \sum_{f\in\text{final layer}}
      \Big|
      \Pr\big(B(M_P;G_i(U))=f\big) - 
      \Pr\big(B(M_P;G_i(\Gamma(X;Y)))=f\big)
      \Big|
      \\
      &=
      \frac{1}{2}
      \sum_{f\in\text{final layer}}
      \Big|
      \Exp_{m\sim M_P} \Pr\big(B(m;G_i(U))=f\big)
      -
      \Exp_{m\sim M_P} \Pr\big(B(m;G_i(\Gamma(X_m;Y)))=f\big)
      \Big|\\
      &\leq
      w \cdot
        \max_{\substack{
            m\in\text{middle layer}\\
            f\in\text{final layer}}
        }
        \Big|
        \Pr\big(B(m;G_i(U))=f\big)
        -
        \Pr\big(B(m;G_i(\Gamma(X_m;Y)))=f\big)
        \Big|\,.
    \end{align*}
    The inequality above follows from the triangle inequality and the 
    fact that the final layer has at most~$w$ states~$f$.
    It remains to bound the absolute value for all $m$ and $f$, which 
    we will do using the quasirandomness property of the 
    expander~$H_{i+1}$, whose neighbor function 
    is~$\Gamma=\Gamma_{H_{i+1}}$.
    Recall that the vertex set of $H_{i+1}$ is the support of~$X$, and 
    the labels used on the edges is the support of~$Y$.
    Now we define the sets $S \doteq\text{supp}(X_m)=\{x \;|\; 
    B(s;G_i(x))=m\}$ and $T \doteq\{x' \;|\; B(m;G_i(x'))=f\}$.
    Note that $\Pr\big(B(m;G_i(U))=f\big) =
    \frac{|S|\cdot |T|}{|V(H_{i+1})|^2}$
    and $\Pr\big(B(m;G_i(\Gamma(X_m;y)))=f\big) =
    \frac{|E(S,T)|}{|E(H_{i+1})|}$.
    Since the second-largest eigenvalue of~$H_{i+1}$ is at 
    most~$\lambda$, the expander mixing lemma 
    (Lemma~\ref{lect10:lemma: expander mixing} of Lecture~10) implies
    \begin{align*}
      \eqref{mixing}
      &\leq w \cdot
      \max_{m,f}
      \left|\;
        \frac{|S|\cdot |T|}{|V(H_{i+1})|^2}
        \;-\;
        \frac{|E(S,T)|}{|E(H_{i+1})|}
      \;\right|
      \leq w \cdot \lambda\,.
    \end{align*}
  \end{proof}
  We completed the analysis of the pseudorandomness of the 
  INW-generator, and therefore the proof of Lemma~\ref{lemma: INW}.
\end{proof}

We already stated the resulting seed length of the generator, but for 
reference, we restate it as a theorem.
\begin{theorem}[\cite{INW}]
  For all~$r,w\in\Nat$ and $\epsilon>0$, the INW-generator~$G_{\log r} 
  : \zo^\ell \to \zo^r$ with seed length
  \begin{align*}
    \ell \sim \log r \cdot \Big(\log r + \log w + \log 
    (1/\epsilon)\Big)
  \end{align*}
  is $\epsilon$-pseudorandom for branching programs of width~$w$ and 
  length~$r$.
  Furthermore, the generator can be computed in space $O(\ell)$.
\end{theorem}
\begin{proof}
  Let $H_1,H_2,\dots$ be an expander family with $\lambda < 
  \epsilon/(r^2 w)$ and let $k=\log r$.
  Then, by Lemma~\ref{lemma: INW}, the INW-generator~$G_k$ 
  is~$\epsilon$-pseudorandom for branching programs of length~$r$ and 
  width~$w$ since $3^k w \lambda\leq \epsilon$.
  Furthermore, the seed length is $\ell\leq k\cdot \log d$, where $d$ 
  is an upper bound on the degree of the expanders.
  Since $d\sim 1/\lambda^2$ can be achieved, we have $\ell\leq C\cdot
  \big(\log r (\log r+\log w+\log(1/\epsilon))\big)$ for some absolute 
  constant $C>0$ independent from $r$, $w$, and $\epsilon$.

  Furthermore, $G_k$ can be computed in space $O(\ell)$ because of 
  Fact~\ref{lect14:fact:INW-space} in Lecture~14 and the fact that the 
  expander family is fully explicit.
\end{proof}

Finally, we consider the consequences of this theorem for the 
derandomization of randomized logspace~\RL{}.
Recall that \L{} denotes the complexity class for deterministic 
logspace, and we write $\L^2=\text{DSpace}(\log^2 n)$ for the class of 
problems that can be decided by machines that use space~$O(\log^2 n)$.
\begin{corollary}
  \label{corollary: L2}
  $\RL \subseteq \L^2$.
\end{corollary}
\begin{proof}
  Let $L\in\RL$ and let $M$ be an $\RL$-machine that decides~$L$, and 
  let $r=\poly(n)$ be an upper bound on the number of random bits used 
  by $M$ on an input of length~$n$ and $w=\poly(n)$ is an upper bound 
  on the size of $M$'s configuration graph.
  We construct an $\L^2$-machine~$M'$ for~$L$.
  On input~$x$ of length~$n$, we will use the INW-generator~$G$ from 
  Lemma~\ref{lemma: INW} with $\epsilon=1/3$.
  Then $M'$ works as follows:
  \begin{enumerate}
    \item Cycle through all seeds $\sigma\in\zo^\ell$, and
      \begin{enumerate}
        \item Simulate $M$ on input $x$
        \item Keep track of the position $j\in[r]$ in $M$'s random 
          tape
        \item Whenever $M$ requests a random bit, provide 
          $\rho_j\doteq (G(\sigma))_j$ and increment~$j$.
      \end{enumerate}
    \item $M'$ accepts if and only if at least one run of~$M$ above accepts.
  \end{enumerate}
  This algorithm uses space $\Theta(\log^2 n)$ because $\ell\sim\log^2 
  n$.
  For the correctness of the algorithm, let~$x$ be some input.
  If $x\not\in L$, then $\Pr_\rho(M(x;\rho)=1)=0$ and hence no run of 
  $M$ can accept.
  On the other hand, if $x\in L$, then $\Pr_\rho(M(x;\rho)=1)\geq 
  1/2$.
  For the analysis in this case, let $B=B_{M,x}$ be the corresponding 
  branching program of width~$w$ and length~$r$ so that 
  $M(x;\rho)=B(\rho)$ for all $\rho$.
  By Lemma~\ref{lemma: INW}, we have
  $\big|\Pr(B(U_r)=1) - \Pr(B(G(U_\ell))=1)\big| \leq \epsilon=1/3$, 
  and therefore, $\Pr(B(G(U))=1)=\Pr_\sigma(M(x;G(\sigma))=1) \geq 
  \frac{1}{2}-\frac{1}{3} > 0$.
  This implies that the algorithm~$M'$ above finds at least one 
  seed~$\sigma$ that leads to acceptance.
\end{proof}
Note that Corollary~\ref{corollary: L2} can be proved \emph{without} 
using pseudorandom generators.
In fact, $\RL\subseteq\L^2$ as well as the more general result 
$\NL\subseteq\L^2$ about non-deterministic logspace were known long 
before and essentially work in the same manner as the repeated 
squaring of matrices that we discussed in the beginning of Lecture~14.
However the proof above provides a ``black-box derandomization'' in 
that the algorithm can be seen as a black box and only the randomness 
is chosen from a pseudorandom distribution with small support.
This is structurally more elegant, and many people believe that a 
possible proof of $\RL=\L$ would make use of pseudorandom generators.
We will see in Lecture~16 that a pseudorandom generator for $\RL$ can 
indeed be used to prove $\RL\subseteq\L^{1.5}$, which constitutes a 
strict improvement on Corollary~\ref{corollary: L2}.

\subsection{Nisan's generator}


%For every automaton, we can define the Markov chain transition matrix 
%- if you start at state $u$, read $x$, the probability that you end 
%%up at $v$ is $(M(Q))_{v,u}$:
%
%$$ (M(Q))_{v,u} = \Pr_{x \in \Sigma} [Q(u,x)=v] $$
%

We now have a look at Nisan's generator, a different pseudorandom 
generator for branching programs.
It came historically before the INW-generator and is related to it, 
but it is based on pairwise independent hash functions instead of 
expanders.
Recall from Definition~\ref{lect05:definition: hash function} in 
Lecture~5 that a family $\mathcal H$ of pairwise uniform hash 
functions $h:\zo^m\to\zo^m$ is a family that satisfies $\Pr_{h\in 
  \mathcal H}(h(x)=x' \wedge h(y)=y')=2^{-2m}$ for all 
$x,y,x,y'\in\zo^m$.
We also noted in Lecture~5 that the family $\mathcal H=\{h_{a,b}\}$ of 
all linear functions $h_{a,b}(x) = a\cdot x + b$ over~$\GF{2^m}$ is 
such a family.
Once hash functions~$h_1,\dots,h_i$ from the family are chosen, 
Nisan's generator is defined as follows.
\begin{definition}\label{definition: Nisan}
  Let $m$ be a positive integer and let $h_1,\dots,h_i :\zo^m\to\zo^m$ 
  be a sequence of (hash) functions.
  Then we define the Nisan's generator
  $G_{h_1...h_i} : \zo^m \rightarrow \left(\zo^m\right)^{2^i}$
  as follows for all seeds~$\sigma\in\zo^m$:
  \begin{align*}
    G_{\emptyset}(\sigma) &\doteq \sigma \\
    G_{h_1..h_i}(\sigma) &\doteq
    G_{h_1..h_{i-1}}(\sigma) \circ G_{h_1..h_{i-1}}(h_i(\sigma))
  \end{align*}
\end{definition}


For example, if we only have a single hash function~$h$, this yields 
$G_{h}(\sigma) =\sigma \circ h(\sigma)$.
The interpretation in the case of branching programs is that we first 
divide the branching program into \emph{blocks} of length~$m$ each.
Then a random walk of length~$2m$ in the branching program is 
specified by the assignment~$\sigma_1\circ\sigma_2$ to the first two 
blocks.
In a uniform walk, these two assignments are independent and uniform; 
however, in a pseudorandom walk, the first assignment~$\sigma_1$ is 
chosen uniformly at random, but the second step is taken to be 
$\sigma_2=h(\sigma_1)$ in Nisan's generator.
The intuition is that, if $h$ was drawn from a family of pairwise 
uniform hash functions, then the pseudorandom string looks random to 
any branching program.

Let us point out why we need to choose $m\geq \Omega(\log w)$ where 
$w$ is the width of the branching program:
If $\log w$ is larger than $6m$, then there is a simple branching 
program that can detect whether its input is an output of Nisan's 
generator.
For simplicity, we consider the case of three hash functions 
$h_1,h_2,h_3$; here we have
\begin{align*}
 G_{h_1,h_2,h_3}(\sigma)
 &=
 \sigma \circ h_1(\sigma) \circ h_2(\sigma) \circ h_2(h_1(\sigma)) 
 \circ h_3(\sigma) \circ h_3(h_1(\sigma)) \circ h_3(h_2(\sigma)) \circ 
 h_3(h_2(h_1(\sigma)))
 \,.
\end{align*}
Since $\log w\geq 6m$, the branching program can hold six blocks 
entirely in its memory and make arbitrary decisions based on their 
value.
Let $a,b,c,d,e,f,g,h$ be the eight blocks above.
Then the linear equations $e=h_3(a)$ and $f=h_3(b)$ can be used to 
determine the linear function~$h_3$ since~$a$ and~$b$ are different 
with high probability.
The branching program will then claim that the sequence is not random 
if $g=h_3(c)$ holds.
Thus, in order for Nisan's generator to give something useful, the 
block length~$m$ must be $\Omega(\log w)$.

Nisan's generator is $\epsilon$-pseudorandom for length~$r$ and 
width~$w$ branching programs in the following sense:
When $h_1,\dots,h_k$ for $k=\log r$ and $\sigma\in\zo^m$ for 
$m=\Theta(\log r + \log w + \log(1/\epsilon))$ are drawn uniformly and 
independently from $\mathcal H$ and $U_m$, respectively, then 
replacing the branching programs' uniform random input with the output 
of Nisan's generator incurs an error of at most~$\epsilon$ in 
statistical distance.
This result suffices to prove Corollary~\ref{corollary: L2}, but we 
will actually need a stronger version in Lecture~16 when we  
prove~$\RL\subseteq\L^{1.5}$.

A careful analysis of Nisan's generator yields that we can actually 
divide the seed into two parts: In a first phase, the ``outer'' seed 
$h_1,\dots,h_k$ is chosen uniformly at random.
Then the function defined in Definition~\ref{definition: Nisan} is a 
procedure that stretches an ``inner'' seed $\sigma$ of $m\sim\log r$ 
bits to $r \sim m\cdot 2^k$ pseudorandom bits; in order to compute it, 
only $O(m+k)=O(\log r)$ additional work space is required (where we do 
not count the work space required to store~$h_1,\dots,h_k$).
The stronger pseudorandomness statement is that, for all branching 
programs~$B$, we have a very high probability that a uniformly sampled 
outer seed will instantiate the generator~$G_{h_1,\dots,h_k}$ such 
that it is $\epsilon$-pseudorandom for~$B$.
We make this formal below.
\begin{theorem}[\cite{Nisan}]\label{theorem: strong Nisan}
  Let $B$ be any branching program of length~$r=2^k$ and width~$w$ and 
  let $\epsilon>0$.
  Then there is an $m \sim \log r + \log w + \log(1/\epsilon)$ such 
  that
  \begin{align*}
    \Pr_{h_1,\dots,h_k\sim\mathcal H}
    \Big[\;
    \text{$G_{h_1,\dots,h_k}$ is $\epsilon$-pseudorandom for $B$}\;
    \Big] \leq \frac{1}{\poly(r)}\,.
  \end{align*}
  Moreover, the following slightly stronger statement is true as well:
  \begin{align}\label{eq: Nisan concentration}
    \Pr_{h_1,\dots,h_k\sim\mathcal H}
    \Big[\;
    \sdist \Big( B(s;G_{h_1,\dots,h_k}(U_m)) \;,\; B(s;U_{2^k}) \Big) 
    \leq \epsilon\;
    \Big] \leq \frac{1}{\poly(r)}\,.
  \end{align}
\end{theorem}
Recall that $B(s;D)$ refers to the distribution on the states that~$B$ 
reaches when its input is sampled from~$D$.
Furthermore, let us remark that $G_{h_1,\dots,h_k}$ outputs $m2^k$ 
bits, but we actually only use $2^k$ of them; this is an insignificant 
technicality.

We leave it as an exercise for the reader to prove 
Theorem~\ref{theorem: strong Nisan}, it follows along the lines of the 
proof of Lemma~\ref{lemma: INW}; the only important difference is the 
bound on the corresponding third term~\eqref{mixing}.
Instead of the expander mixing lemma, we use a mixing lemma for 
pairwise uniform hash functions.
This is also the step that we get the concentration result from.
We state this lemma and provide a proof sketch.
\begin{lemma}
  Let $m$ be a positive integer, and let $\mathcal H$ be a family of 
  pairwise uniform hash functions $h:\zo^m\to\zo^m$.
  Furthermore, let $S,T\subseteq \zo^m$.
  Then we have
  \begin{align*}
    \Pr_{h \sim \mathcal H}
    \left[\;
    \left|
      \frac{|S| |T|}{2^{2m}}
      -
      \frac{|\{s\in S \;|\; h(s)\in T\}|}{2^m}
    \right|
    <\epsilon
  \;\right]
    > 1-1/(\epsilon^2 \cdot 2^{m})\,.
  \end{align*}
\end{lemma}
\begin{proof}
  Let $X$ be the random variable $|\{s\in S|h(s)\in T\}|/2^m$.
  We now compute its expected value using the $0/1$-indicator variable 
  $X_s=I[h(s)\in T]$:
  \begin{align*}
    \Exp_{h\sim \mathcal H} [X]
    &=
    \Exp_{h\sim \mathcal H} \Big[\sum_{s\in S} X_s/2^m\Big]
    =
    \sum_{s\in S} \Exp_{h\sim \mathcal H} [X_s]/2^m
    =
    \sum_{s\in S} |T| / 2^m = |S|\cdot |T| / 2^{2m}
    \,.
  \end{align*}
  To prove the theorem, we need to prove concentration.
  Below, we first apply Chebyshev's inequality 
  (Proposition~\ref{lect05:chebyshev} in Lecture~5), which requires us 
  to bound the variance of $X$.
  Since $X=\sum_{s\in S} X_s/2^m$ holds and the~$X_s$ are pairwise 
  independent, we can apply 
  Proposition~\ref{lect05:prop:variance:pairwise} in Lecture~5.
  We get:
  \begin{align*}
    \Pr_{h\sim\mathcal H}
    \Big[ \big|\Exp[X]- X\big| \geq \epsilon \Big]
    &\leq \frac{\Var[X]}{\epsilon^2}
    = \frac{\sum_{s\in S} \Var[X_s/2^m]}{\epsilon^2}
    = \frac{\sum_{s\in S} \Var[X_s]/2^{2m}}{\epsilon^2}
    \\
    &\leq \frac{\sum_{s\in S} \Exp[X_s]/2^{2m}}{\epsilon^2}
    = \frac{\Exp[X]/2^{m}}{\epsilon^2}
    \leq \frac{1}{\epsilon^2 2^m}
    \,.
  \end{align*}
  \end{proof}

\bibliographystyle{alpha}
\bibliography{references}

\end{document}
