\documentclass[11pt]{article}

\usepackage{lecture}

\newcommand{\hr}{{\color{black} \rule{\textwidth}{0.2mm}}}

% color commands
\def\red#1{{{\color{red} #1}}}
\def\blue#1{{{\color{blue} #1}}}
\def\green#1{{{\color{green} #1}}}
\def\yellow#1{{{\color{yellow} #1}}}

% input for a generator or an extractor source
\def\source{x}

% seed of an extractor
\def\seed{y}

% random string as input for a branching program
\def\random{\rho}

% input for a communication protocol
\def\cinput{z}

\DeclareMathOperator{\NOF}{NOF}

\begin{document}

\lecture{29}{5/8/2013}{Space Generators from Communication Complexity}{Xi Wu}

\draft

In previous lectures we saw two pseudorandom generators for 
bounded-space machines:
Nisan's generator and the Impagliazzo--Nisan--Wigderson generator.
Both generators are of the form $G_k: \zo^{(k+1)s} \mapsto \zo^{2^k}$ such that
\begin{align}
  \label{eq:framework}
  G_k\Big( \source , \seed \Big) = G_{k-1}\Big(\source\Big) \;,\; 
  G_{k-1}\Big(\Gamma_k(\source, \seed)\Big)
\end{align}
where $\source \in \zo^{ks}$, $\seed \in \zo^s$, and $\Gamma_k: 
\zo^{ks} \times \zo^s \to \zo^{ks}$ is some function that can
``recycle the randomness in $\source$''.
In the INW-generator, $\Gamma$ is the neighbor function of a suitable 
expander graph or it is a suitable extractor.
Nisan's generator can also be cast in the framework 
of~\eqref{eq:framework}, but the definition is a bit subtle.
Here the string $\source$ is of length~$(2k-1)\cdot s$ and of the form 
$\source=\sigma,h_1,\dots,h_{k-1}$ and $\seed$ is of the form 
$\seed=h_k$.
The string $\sigma$ is of length~$s$ and corresponds to Nisan's 
``inner seed'' as discussed in Lecture~15.
The ``outer seed'' consists of the~$k$ strings $h_1,\dots,h_k$ of 
length~$2s$ each, and they determine~$k$ pair-wise uniform hash 
functions from $\zo^{s}\to\zo^{s}$.
Then we obtain Nisan's generator~$G_k$ when $\Gamma_k$ is of the form
\begin{align*}
  \Gamma_k(\sigma,h_1,\dots,h_{k-1};h_{k})
  &=
  h_k(\sigma),h_1,\dots,h_{k-1}
  \,.
\end{align*}

One way to look at generators of type~\eqref{eq:framework} is that 
they exploit the bounded amount of communication that is passed from 
one party to another:
Two players Alice and Bob want to compute a function~$f(X,Y)$ where 
Alice has an input $X$ and Bob has an input $Y$ that is independent 
from~$X$.
In the communication model, we require Alice to do some computation 
on~$X$ and to then send a message to Bob.
Bob receives Alice's message, continues the computation on~$Y$, and 
outputs the result.

Given such a communication protocol, we want to replace the uniform 
distributions~$X$ and~$Y$ with distributions that require less 
randomness, while still approximating the overall behavior of the 
protocol under uniform input distributions.
If the communication between Alice and Bob is limited, then 
intuitively we should be able to ``recycle'' the randomness that is 
used to create Alice's input and pass it on to Bob.
In the case of read-once branching programs (ROBP) of small width, 
Nisan's generator and the INW-generator exploit the communication 
communication bottleneck in the following way:
The branching program is split into half.
Alice looks at the input of the first half of the program and computes 
the state in the middle layer of the program that is reached on the 
given input.
She communicates this state to Bob, who can use it as the start state 
of the second half of the program and compute the final state of the 
program that is reached when reading the second half of the input.
Since the number of bits used in the communication is only logarithmic 
in the width of the program, all but a logarithmic number of random 
bits from the first half can be reused for Bob's input, which leads to 
the framework~\eqref{eq:framework}.
The program has a highly hierarchical structure since each half of the 
program is again an oblivious read-once branching program, and thus, 
the generator can be applied recursively.

Note that a similar intuition applies to the shrinkage-based 
pseudorandom generator that we discussed in the last few lectures.
In this situation, the branching programs can be non-oblivious in that 
the order in which the input is read is not known to the generator and 
each variable can be read multiple times.
The intuition for the pseudorandom generator was as follows:
We divide the input into two parts: the first half is determined by a 
pseudorandom restriction~$X$ and the second half~$Y$ consists of the 
variables not set by the random restriction.
Alice computes the restricted circuit~$C|_X$ and simplifies it as much 
as possible; by the arguments in the last lecture, the circuit size 
shrinks significantly compared to the original circuit.
Alice then communicates the shrunk circuit to Bob, who evaluates the 
received circuit on its input variables~$Y$.
Again, if we condition on the communication between Alice and Bob, we 
can recycle most of the randomness from~$X$ to generate~$Y$.

\section{Number-on-Forehead model of communication}
In this lecture, we will see a pseudorandom generator for 
bounded-space machines that exploits communication protocols such as 
the above, except that we allow for more than two players, i.e., we 
consider \emph{multiparty} communication.
In particular, the pseudorandom generator will be based on a function 
that requires a large amount of communication between the players in a 
certain model of multiparty communication, namely one-way unicast 
communication.
This is a more restrictive model than the \emph{number-on-forehead} 
model, which we will introduce first.
\begin{definition}[Number-on-Forehead (NOF) Model]
  An NOF-protocol~$P$ that computes a function $f:\zo^{np} \mapsto 
  \zo$ consists of the following parts:
  \begin{enumerate}
    \item
      \emph{Players}: There are $p$ players, denoted by $1,\dots, p$.
    \item
      \emph{Input}: The overall input is $\cinput_1,\dots,\cinput_p 
      \in \zo^n$.
      Each player $j\in[p]$ gets access to all input strings 
      \emph{except for} $\cinput_j$.
    \item
      \emph{Communication channel}: The players share a write-once 
      blackboard to communicate and every player can see what is 
      written on the blackboard.
    \item
      \emph{Goal}: Compute $f(\cinput_1,\dots,\cinput_p)$.
  \end{enumerate}
\end{definition}
The input string $\cinput_i$ can be thought of a being written on the 
forehead of player~$i$.
That is, player~$i$ can see everyone else's input but not their own.
The number of bits written down on the blackboard in an execution of 
the protocol is the amount of communication used in an execution.
The \emph{communication complexity} of a function~$f$ in the NOF-model 
is the minimum amount of communication that an NOF-protocol requires 
to compute $f$ correctly on all inputs of length~$np$.
A trivial upper bound on the communication complexity of any function 
is~$n$:
Player~$1$ can write the string~$\cinput_2$ on the blackboard so that 
player~$2$ knows all input strings and can compute 
$f(\cinput_1,\dots,\cinput_p)$.

We write $\NOF_P(f)=C$ to denote that~$P$ computes~$f$ correctly on 
all inputs with communication cost at most~$C$.
More generally, we say that $P$ computes $f$ with advantage 
$\varepsilon$ if
\begin{align*}
  \Pr_{\cinput_1 \cdots \cinput_p \sim 
    U_{np}}\bigg[P(\cinput_1,\dots,\cinput_p) = 
  f(\cinput_1,\dots,\cinput_p)\bigg] \geqslant \frac{1}{2} + 
  \varepsilon
\end{align*}
In this case, we write $\NOF_P^\varepsilon(f)=C$ to denote the fact 
that~$P$ computes~$f$ with advantage~$\varepsilon$ and cost at 
most~$C$.
Finally we define $\NOF^\varepsilon(f)=\min_P \NOF_P^\varepsilon(f)$ 
as the minimum worst-case communication complexity over all 
NOF-protocols~$P$ that compute~$f$ with advantage~$\varepsilon$.

Let us consider an example of a function that is hard in the 
NOF-model.
We view each~$\cinput_i$ as the characteristic vector of some set $S_i 
\subseteq [n]$.
That is, for every $j \in [n]$, we have $\cinput_i[j] = 1$ if and only 
if $j \in S_i$.
Define
\begin{align*}
  f(\cinput_1,\dots,\cinput_p) =
  \begin{cases}
    1 & \text{if there is a $j\in[n]$ such that } j\in\bigcap_{k=1}^p 
    S_k\,, \\
    0 & \text{otherwise.}
  \end{cases}
\end{align*}
This function is called the \emph{set intersection problem}, and it 
requires $\Omega\left(n^{1/(p+1)}/2^{2p}\right)$ bits of communication 
in the $p$-player NOF-model \cite{}.
The \emph{generalized inner product} is related to the set 
intersection, but it asks the players to compute the \emph{parity} of 
the number of elements in their shared intersection, that is,
\begin{align*}
  f(\cinput_1,\dots,\cinput_p) =
  \Big|\bigcap_{j=1}^p S_j\Big| \bmod 2\,.
\end{align*}
The generalized inner product is one of the hardest problems known in 
the NOF-model, and it requires $\Omega\left(n/2^{2p}\right)$ bits of 
communication \cite{}.
On the other hand, there is also a corresponding upper bound of 
$O\left(pn/2^p\right)$ \cite{}.
It is open whether there is a problem that requires 
$\Omega\left(n/\poly(p)\right)$ bits of communication; Raz~\cite{} 
gives a candidate for such a problem, and a proof implies circuit 
lower bounds for ACC$^0$ \cite{}.

\section{Construction of the BNS-Generator}

In this section, we will use a function~$f$ that~$p$ players want to 
compute using an NOF-protocol to construct a pseudorandom generator.
If the function~$f$ is hard in a suitable model of communication, we 
will later show that the generator is pseudorandom for branching 
programs of bounded width.
We remark that set intersection or generalized inner product could be 
used as a hard function here, but it would still give significantly 
worse seed length than Nisan's generator and the INW-generator.
We will later introduce a function that is more suitable in our 
setting and gives the same asymptotic seed length as the best known 
generators.

\begin{construction}\textbf{(Babai, Nisan, and 
    Szegedy~\cite{BNS92})}\\
  Given a function $f: \zo^{np} \mapsto \zo$ and an enumeration 
  $S_1,\dots,S_{\binom{t}{p}}$ of all subsets of $[t]$ that have 
  size~$p$,
  the \emph{BNS-generator} is the function $G: \zo^{nt} \mapsto 
  \zo^{\binom{t}{p}}$ with
  \begin{align*}
    G(\source)
    &\doteq
    f\Big(\source|_{S_1}\Big)
    \circ
    f\Big(\source|_{S_2}\Big)
    \circ \cdots \circ
    f\Big(\source|_{S_{{\binom{t}{p}}}}\Big)\,.
  \end{align*}
  Here, for 
  $\source=(\source_1,\dots,\source_t)\in\left(\zo^{n}\right)^t$, we 
  let $\source|_S$ denote the string $(\source_i)_{i\in S}$ of length 
  $np$.
\end{construction}
To get a seed length of $O(\log^2 r)$ for $r=\binom{t}{p}$, we will 
later choose $p,t,n\in\Theta(\log r)$ with $p \ll t \ll n$.
Note that the construction above depends not only on the hard 
function~$f$ but also on the ordering of all subsets of $[t]$ that 
have size~$p$.
On a high level, we have two requirements on the ordering and the 
function~$f$:
\begin{enumerate}[(i)]
  \item\label{eq:ordering space}
    Since we want to use~$G$ to derandomize small-space machines, $G$ 
    itself should be computable in space $O(|\source|)=O(nt)$.
  \item\label{eq:ordering distinguisher}
    The ordering should be such that we can use the hardness of~$f$ to 
    prove that the output of $G$ looks pseudorandom to small-width 
    branching programs.
    In particular, the following property will suffice: if there is a 
    small-width branching program that can distinguish the output 
    of~$G$ from the uniform distribution with non-trivial probability, 
    then we can use it to construct a low-cost NOF-protocol that 
    computes~$f$ with non-trivial advantage.
\end{enumerate}
Satisfying~\ref{eq:ordering space} is straightforward since the 
function~$f$ that we will choose is computable in space~$O(|\source|)$ 
and the order will be so that~$S_{i+1}$ can be computed in 
space~$O(|\source|)$ from~$S_i$.
It is~\ref{eq:ordering distinguisher} that requires a bit more 
insight.
In the next section, we will sketch the idea of how to use a 
distinguisher for~$G$ to obtain a low-cost protocol for~$f$, and we 
will pick a suitable ordering along the way.

\section{From Distinguisher to NOF-protocol}

In the last section, we introduced a candidate generator~$G$ based on 
a hard function~$f$.
Now we want to show how a small-space machine~$D$ that can distinguish 
the output of~$G$ from the uniform distribution can be used to 
construct a fairly good NOF-protocol~$\Pi$ for $f$.
If $f$ is \emph{hard} to compute by NOF-protocols, this hardness 
transfers to the inability to distinguish~$G$ from uniform by 
small-space machines.

\begin{lemma}[\cite{BNS92}]\label{lemma: protocol}
  Let $p,t,n\in\Nat$, $f:\zo^{np}\to\zo$, and $G:\zo^{nt}\to\zo^r$ be 
  the BNS-generator.
  If $G(U_{nt})$ is not $\epsilon$-pseudorandom for space~$S$ 
  machines, then $f$ has a $p$-player NOF-protocol that computes it 
  with advantage $\epsilon/r$ and with communication cost at 
  most~$pS$.
\end{lemma}
\begin{proof}
  If $G$ is not $\epsilon$-pseudorandom, then it has a distinguisher.
  Recall that a distinguisher~$D$ is a space-$S$ machine that takes as 
  input a string of length $r=\binom{t}{p}$ and outputs a single bit 
  such that
  \begin{align*}
    \left|
      \Pr \Big[ D(G(U_{nt}))=1 \Big]
      -
      \Pr \Big[ D(U_{r}))=1 \Big]
    \right|
    &> \frac{1}{2}+\varepsilon\,.
  \end{align*}
  In Lecture~23, we used a \emph{hybrid argument} to convert any 
  distinguisher to a \emph{predictor} $P$ with slightly worse 
  parameters.
  In particular, a predictor is a machine that takes a string of 
  length~$\ell<r$ as input and outputs a single bit such that
  \begin{align}\label{eq:predictor}
    \Pr_{\random\sim G(U_{nt})}
    \Big[ P \big( \random_{\leq\ell} \big) = \random_{\ell+1} \Big]
    &> \frac{1}{2}+\varepsilon/r\,.
  \end{align}
  Here, $\random_{\leq\ell}$ denotes the prefix of~$\random$ of 
  length~$\ell$.
  It is this predictor that we are going to construct the protocol~$\Pi$ 
  from.
  For this, let us first realize that $\random_i \doteq ( G(\source) 
  )_i = f\left(\source|_{S_i}\right)$ holds for $\source\in\zo^{nt}$.
  In particular, the bit~$\random_{\ell+1}$ depends only on the 
  $pn$-length substring of $\source$ selected by $S_{\ell+1}$.
  Therefore, we can fix some setting for the rest of~$\source$ 
  in~\eqref{eq:predictor}.
  More precisely, there is a setting 
  for~$\source|_{\overline{S_{\ell+1}}} \in\zo^{(t-p)n}$ such that
  \begin{align}\label{eq:restricted predictor}
    \Pr_{\source|_{S_{\ell+1}}\sim U_{np}}
    \Big[ P \big( \random_{\leq\ell} \big) = \random_{\ell+1} \Big]
    &> \frac{1}{2}+\varepsilon/r\,.
  \end{align}
  Since $\random_{\leq\ell}=\random_{\leq\ell}(\cinput)$ is a function 
  of $\cinput\doteq \source|_{S_{\ell+1}}$ and 
  $\source|_{\overline{S_{\ell+1}}}$ is fixed to a constant, we can 
  rewrite~\eqref{eq:restricted predictor} as
  \begin{align}\label{eq:final predictor}
    \Pr_{\cinput\sim U_{np}}
    \Big[ P \big( \random_{\leq\ell}(\cinput) \big) = f(\cinput) \Big]
    &> \frac{1}{2}+\varepsilon/r\,.
  \end{align}
  Note that, while $\random_{\leq\ell}=\random_{\leq\ell}(\cinput)$ is 
  a function of~$\cinput$, it is not clear that it can be computed in 
  space~$O(S)$ since we do not in general require~$f$ to be computable 
  in that space.
  However, $\random_i$ for $i\leq\ell$ does not depend on \emph{all} 
  $\cinput_j$ for $j\in[p]$.
  This is because $S_i$ is different from $S_{\ell+1}$ for all 
  $i\neq\ell+1$.
  So, while $\random_{\leq\ell}(\cinput)$ may not be computable in 
  small space, an NOF-protocol can simulate~$P$ on input 
  $\random_{\leq \ell}$ since the players themselves are 
  computationally unbounded, and player~$j$ can compute all 
  $\random_i$ that do not depend on~$\cinput_j$.
  The black-board communication then consists of the state in the 
  branching program~$P$, and there is always \emph{some} player who 
  can continue the computation of~$P$ and produce the next state.
  Formally, we get the following framework for a $p$-player 
  NOF-protocol~$\Pi$:
  \begin{enumerate}
    \item[]
      \emph{Input:} Player $j$ gets as input $\cinput_1\dots 
      \cinput_{j-1}\cinput_{j+1}\dots \cinput_p$.
    \item[]
      \emph{Last item written on the blackboard:} A state $(i,s)$ of 
      the branching program~$P$ (initially the start state of~$P$).
    \item[]
      Some player~$j$ who can compute $\random_{i}$ from its input 
      (i.e., $\random_i=\random_i(z)$ does not depend on~$\cinput_j$) 
      goes next and simulates~$P$:
      the branching program is started at~$(i,s)$ and fed the input 
      $\random_{i}$; the player continues to feed 
      subsequent~$\random_i$'s for as long as she can compute them 
      without knowing~$\cinput_j$; finally, player~$j$ writes the 
      state that $P$ ends up in on the blackboard.
  \end{enumerate}
  The fact that $\Pi$ computes~$f$ with advantage $\epsilon/r$ follows 
  from~\eqref{eq:final predictor}.

  To analyze the communication cost, note that $\Pi$ only ever writes 
  down a state of the branching program if the players switch.
  In general, it could be that $\ell=r-1$ and the sequence  
  $S_1,\dots,S_r$ is such that the players end up switching $\sim r/p$ 
  times, which is too large for our purposes.
  Therefore, it is natural to choose the order of the subsets so that 
  each player can simulate $P$ for as long as possible.

  If we imagine for a moment that we knew the positions of 
  $\cinput_1\dots \cinput_p$ inside of $x$ in advance, we could 
  construct such an ordering greedily as follows: we let 
  $S_1,\dots,S_{r/2}$ be all sets that do \emph{not} contain 
  $\cinput_1$, in an arbitrary order.
  Then player~$1$ can simulate $P$ until the middle layer~$r/2$ is 
  reached.
  Furthermore, it is natural to order the remaining~$r/2$ sets so that 
  the first half $S_{r/2+1},\dots,S_{r/2+r/4}$ of the remaining 
  sequence does not contain~$\cinput_2$; then player~$2$ can simulate 
  $P$ until layer $r/2 + r/4$.
  This continues in $p$ iterations until all of the sequence has been 
  constructed and the entire branching program can be simulated by 
  having the players take turns, and each player goes only once.

  The problem with the greedy approach above is that the positions of 
  $\cinput_1\dots \cinput_p$ correspond to the set $S_{\ell+1}$, which 
  depends on the parameter $\ell$ that comes out of the hybrid 
  argument, and we have no control over it.
  However, there is a simple ordering of the subsets that looks like 
  one of the greedy ones above for \emph{all} choices of $S_{\ell+1}$: 
  the lexicographical ordering.
  We simply order the sets such that their corresponding indicator 
  vectors~$\chi_{S_i}\in\zo^t$ are sorted in lexicographical order.
  For clarity, we give the following inductive definition in terms of 
  the sets directly.
  \begin{definition}[Lexicographical ordering]
    Let $U\subset\Nat$ be a non-empty set.
    The sequence of distinct sets $S_1,\dots,S_r$ in $\binom{U}{p}$ 
    with $r=\binom{|U|}{p}$ and $p\leq |U|$ is the lexicographical 
    ordering of $\binom{U}{p}$ if $r = 1$ or if, for $u=\min U$, we 
    have:
    \begin{itemize}
      \item $u \not\in S_1,\dots,S_{r/2}$ and $u \in 
        S_{r/2+1},\dots,S_{r}$, and
      \item the sequences $S_1,\dots,S_{r/2}$ and $S_{r/2+1}\setminus 
        u,\dots,S_r\setminus u$ are both equal to the lexicographical 
        ordering of $\binom{U\setminus u}{p-1}$.
    \end{itemize}
  \end{definition}

  If $G$ is constructed with the above order of the subsets, it is 
  clear that every player in the protocol~$\Pi$ only has to go once.
  Furthermore, the layer~$i$ of the communicated state $(i,s)$ becomes 
  redundant because each player can easily deduce it from the number 
  of states communicated thus far.
  So the only information that the players have to write down is~$s$, 
  the position in the layer, which requires~$S$ bits since that is the 
  space required to run~$P$.
  The total communication on the blackboard is~$p\cdot S$.
\end{proof}

Lemma~\ref{lemma: protocol} says that communication hardness of~$f$ in 
the NOF-model transfers to pseudorandomness of~$G$ for branching 
programs.
Let us formulate this contrapositive explicitly below.
\begin{corollary}\label{corollary: contrapositive NOF}
  Let~$f:\zo^{np}\to\zo$ be a function that does not have a $p$-player 
  NOF-protocol that computes it with advantage $\geq\epsilon$ and 
  communication at most $C$.
  Then the BNS-generator $G:\zo^{nt}\to\zo^r$ constructed from $f$ is 
  $\epsilon r$-pseudorandom for space-$C/p$ machines.
\end{corollary}
The problem with Corollary~\ref{corollary: contrapositive NOF} is that 
we do not not of any function~$f$ that is sufficiently hard in the 
NOF-model.
The hardest functions we know in this model only give a seed length 
that is not even poly-logarithmic in $r$.
We therefore need a more restrictive model of communication if we want 
to use the general framework of Lemma~\ref{lemma: protocol}.

\section{Conservative One-way Unicast Model}

We will now inspect the proof of Lemma~\ref{lemma: protocol} more 
closely; it constructs an NOF-protocol~$\Pi$ that has a very special 
structure, and we will formalize what this structure is to define a 
model of communication that is more restrictive than the NOF-model.
Later we will construct a function that we can prove is very hard in 
the new model.

The protocol in the proof of Lemma~\ref{lemma: protocol} has the 
following properties:
\begin{enumerate}[label=(\arabic*)]
\item Each player acts only once and they go in a fixed order, so the 
  protocol is \emph{one-way}.
\item Each player sends a message only to the next player in the fixed 
  order (and not to all players), so the protocol is \emph{unicast} 
  (as opposed to multicast).
\end{enumerate}
The third and last property is more subtle and used to reduce the 
dependencies between the players: The player $j \in [p]$ does not need 
to know the input 
$\cinput_1,\dots,\cinput_{j-1},\cinput_{j+1},\dots,\cinput_p$ in its 
entirety.
To understand what exactly we mean by this, consider what player $j$ 
is doing:
She picks up the simulation of the branching program $P$ in some 
state $(i,s)$ and will end the simulation in some state $(i',s')$.
For this, she clearly needs to know the branching program $P$ and 
receive the state $s$ from player $j-1$; as we already discussed she 
can deduce the layer~$i$ from her position~$j$ in the order of 
players.
Recall that the players simulate $P$ on input $\random_{\leq\ell}$.
She therefore also needs access to the string 
$\random_i\dots\random_{i'}$ used as the input to $P$ when started 
at state $(i,s)$; she does \emph{not} need access to the string 
$\random_1\dots\random_{i-1}$ or the string 
$\random_{i'+1}\dots\random_{\ell}$.
Since $\random_k=f(\source\vert_{S_k})$ and all coordinates of $x$ 
are constant except for the ones that correspond to 
$\cinput=\source\vert_{S_{\ell+1}}$, this means that $\random'_j\doteq 
\random_i\dots\random_{i'}$ is a function of~$\cinput$.
So player $j$ needs access to the positions $A\doteq 
(S_{i}\cup\dots\cup S_{\ell})\cap S_{\ell+1}$ of $\source$, which 
correspond to positions of $\cinput$.
The choice of the sequence $S_k$ guarantees that $\cinput_j$ does not 
occur in $A$ and player $j$ does not need access to this part of the 
input.
Thus 
$\random'_j=\random'_j(\cinput_1,\dots,\cinput_{j-1},\cinput_{j+1},\dots,\cinput_p)$
is a function of~$\cinput$ but independent from $\cinput_j$.
Unfortunately, we have that 
$\cinput_1,\dots,\cinput_{j-1},\cinput_{j+1},\dots,\cinput_p\in A$ may 
hold in general, so player $j$ needs information from the entire input 
to determine $\random'_j$.
However, the way this information is accessed is very special:
We actually have that $\cinput_1,\dots,\cinput_{j-1}\in S_k$ holds 
\emph{for all} $k\in[i,i']$, whereas the appearance of the 
$\cinput_{j+1},\dots,\cinput_p$ varies in the sets $S_k$ for 
$k\in[i,i']$.
That is, for $k\in[i,i']$, we have 
$f(\source\vert_{S_k})=f(\tilde\cinput_1,\dots,\tilde\cinput_{p})$
where $\tilde\cinput_1=\cinput_1$ up to 
$\tilde\cinput_{j-1}=\cinput_{j-1}$ holds because of the 
lexicographical ordering of the $S_k$, and furthermore, the 
$\tilde\cinput_{j'}$ for $j'\geq j$ are either some constant part of 
$x$, or they are contained in the set 
$\{\cinput_{j+1},\dots,\cinput_p\}$.
Therefore, the protocol constructed in Lemma~\ref{lemma: protocol} 
satisfies the following additional restriction of the NOF-model.
\begin{enumerate}[label=(\arabic*),resume]
\item
  Each player $j$'s access to $\cinput$ can be divided into the 
  following two components:
  \begin{enumerate}
    \item
      Player $j$ has \emph{direct access} to the string 
      $\cinput_{j+1}\dots \cinput_p$
    \item
      Player $j$ has \emph{oracle access} to the function
      $f_j: (\zo^n)^{p-j+1} \to \zo$ with
      \begin{align*}
        f_j\Big(\tilde\cinput_{j},\dots,\tilde\cinput_p\Big)
        &\doteq 
        f\Big(\cinput_1,\dots,\cinput_{j-1},\tilde\cinput_j,\tilde\cinput_{j+1},\dots,\tilde\cinput_p\Big)\,.
      \end{align*}
  \end{enumerate}
  A communication protocol with this type of access is called 
  \emph{conservative}.
\end{enumerate}
The reason why the latter restriction on the communication model makes 
it less difficult to prove the hardness of a function~$f$ in the 
restricted model is that the oracle $f_j$ will in general not leak too 
much information about the string $\cinput_1 \dots \cinput_{j-1}$.

Any NOF-protocol for a function $f$ that also satisfies the conditions 
(1)--(3) is called a \emph{conservative one-way unicast (COWU) 
  communication protocol}.
When talking about the COWU communication model, we make a small 
adjustment to how we measure the amount of communication used by a 
COWU-protocol:
We say that a protocol uses~$C$ bits of communication if the longest 
message that was sent from one player to the next has length~$C$.
Then the following lemma connects the pseudorandomness of the 
BNS-generator~$G$ with the communication complexity of~$f$ in the 
COWU-model.
As discussed, the proof is essentially the same as the one of 
Lemma~\ref{lemma: protocol}.
\begin{corollary}\label{corollary: contrapositive COWU}
  Let~$f:\zo^{np}\to\zo$ be a function that does not have a $p$-player 
  COWU-protocol that computes it with advantage $\geq\epsilon$ and 
  communication at most $C$ per player.
  Then the BNS-generator $G:\zo^{nt}\to\zo^r$ constructed from $f$ is 
  $\epsilon r$-pseudorandom for space-$C$ machines.
\end{corollary}

\section{Parameters in the Logspace Setting}

By Corollary~\ref{corollary: contrapositive COWU}, we know that a hard 
function in the COWU-model gives rise to a pseudorandom generator for 
space-bounded computation.
Let us consider the case of logspace-machines, i.e., machines whose 
space bound $C=O(\log r)$ is at most logarithmic its input length~$r$.
Then we need to choose $\epsilon\leq O(1/r)$ in 
Corollary~\ref{corollary: contrapositive COWU} in order to get a 
non-trivial result.
The construction of the BNS-generator furthermore requires that 
$\binom{t}{p}\geq r$, which can be achieved by setting $p\geq\log r$ 
and $t\geq 2 p$.
This is because $\binom{t}{p}\geq (t/p)^p\geq 2^{p}\geq r$.
If furthermore $p\leq O(\log r)$, then the condition on $\epsilon$ can 
be rewritten as $\epsilon\leq O(2^{-p})$ and the condition on $C$ as 
$C\leq O(p)$.
The function $f$ can only be hard if $n>C$ and, as we will see in the 
rest of this document, is possible to achieve with $n\leq O(p)$.
Overall, we get $nt \sim np \sim \log^2 r$ as the seed length of the 
BNS-generator.

\section{Construction of the Hard Function}

We construct a function~$f:\zo^{np}\to\zo$ that requires a lot of 
communication in the COWU model to compute or to approximate.
The construction is based on an extractor ${\rm 
  Ext}:\zo^n\times\zo^k\to\zo^k$ that is applied iteratively $p$ 
times.
The extractor is going to be a \emph{strong} extractor that is able to 
extract $2k$ random bits when given access to a source of 
\emph{average min-entropy} at least $k$ and a uniform seed of 
length~$k$.
The precise extractor property that we will need is as follows:
\begin{definition}\label{def:our-extractors}
   $\Ext:\zo^n\times\zo^k\to\zo^k$ is a 
   \emph{$(k,\epsilon)$-extractor} if, for all random variables $X$ on 
   $\zo^n$, $Y=U_k$, and $Z$ on $\zo^*$ with $H_\infty(X|Z)\geq k$, we 
   have
   \begin{align}
     \sdist\Big(Z\ Y\ \Ext(X,Y) \;,\; Z\ Y\ U_k\Big) \leq \epsilon\,.
   \end{align}
\end{definition}
Here $H_\infty(X|Z)$ denotes the \emph{conditional average 
  min-entropy}, the definition of which we leave at an intuitive level 
until later.

We define the functions $E^j : \zo^k \times \zo^{nj}\to\zo^k$ for 
$j\in[p]$ as follows:
\begin{align*}
  E^1(\seed; \source_1)
  &\doteq \Ext(\source_1,\seed) \\
  E^2(\seed; \source_1,\source_2)
  &\doteq \Ext(\source_2, E^1(\seed;x_1)) \\
  \vdots\\
  E^j(\seed;\source_1,\dots,\source_j)
  &\doteq \Ext\Big(\source_j, 
  E^{j-1}(\seed;\source_1,\dots,\source_{j-1})\Big)\,.
\end{align*}
That is, $E^1$ takes a seed $\seed$ and $\source_1$ to produce a new 
seed $\seed'=\Ext(\source_1,\seed)$.
Then $E^2$ takes the seed $\seed'$ and $\source_2$ to produce a new 
seed $\seed''=\Ext(\source_2,\seed')$ and so on.
Finally, we let $f$ be the first bit of the output of $E^p$.
Note that there is a small mismatch in the input length because of the 
global seed $\seed$.

We extend the COWU model so that there can be an additional 
input~$\seed$ that \emph{no} player has direct access to, and that is 
only revealed indirectly by oracle access to the function 
$f_j(\tilde\cinput_j,\dots,\tilde\cinput_p)=f(\seed;\source_1,\dots,\source_{j-1},\tilde\cinput_j,\dots,\tilde\cinput_p)$ 
for player~$j$.
The following theorem then captures the hardness result for $f$.
\begin{theorem}[Ganor and Raz~\cite{GR13}]
  \label{thm:hardness}
  Let $E^p:\zo^{k+np}\to\zo^k$ be the function above, constructed from 
  a $(k,\epsilon)$-extractor~$\Ext:\zo^n\times\zo^k\to\zo^k$ in the 
  sense of Definition~\ref{def:our-extractors}, and let $f=(E^p)_1$ be 
  the first bit of this function.
  Then every COWU-protocol~$\Pi$ for $f$ that uses at most $n-k$ bits 
  of communication per player satisfies
  \begin{align}\label{eq:unpredictable}
    \Pr_{yx \sim U_{k+np}}\bigg[\Pi(y,x) = f(y,x)\bigg] \leq 
    \frac{1}{2} + 2^p \epsilon
  \end{align}
\end{theorem}

\subsection{Slight modification of the BNS-generator}
For the above hard function with a hidden input $y$ to lead to a 
pseudorandom generator, we need to add this seed to the BNS-generator 
$G$ as well.
That is, we redefine $G:\zo^k\times\zo^{nt}\to\zo^{\binom{t}{p}}$ as
\begin{align*}
  G(y;\source)
  &\doteq
  f\Big(y;\source|_{S_1}\Big)
  \circ
  f\Big(y;\source|_{S_2}\Big)
  \circ \cdots \circ
  f\Big(y;\source|_{S_{{\binom{t}{p}}}}\Big)\,.
\end{align*}
As long as $k\leq O(nt)$, this has no adverse effect on the seed 
length of the generator, and the proof of Corollary~\ref{corollary: 
  contrapositive COWU} goes through as before.

\section{Proof of Theorem~\ref{thm:hardness}}
The intuition for why the function $f$ is hard is as follows:
Player $1$ does not have direct access to $\seed$ or $\source_1$, but 
only to the function table of~$f_1$.
This function table does not reveal \emph{any} information about 
$\source_1$ and little about~$\seed$, for which reason it will be very 
difficult for player~$1$ to compute $\seed'=E^1(\seed;\source_1)$ with 
any significant advantage.
That is, $\seed'$ is close to uniform and player~$1$ can send only 
very little information about $\seed'$ to player~$2$.
Furthermore, if the communication is bounded, player $1$ can also not 
send a lot of information about $\source_2$ to player~$2$.
That is, even conditioned on the information that player~$2$ receives 
about~$\source_2$ from player~$1$, the min-entropy of $\source_2$ will 
still be pretty large.
Large enough in fact so that the almost uniform seed $\seed'$ can make 
the output of $E^2(\seed;\source_2)$ look almost uniform again, and 
player~$2$ has little chance of computing that value.
This idea for the analysis goes through till player~$p$ and we get 
that the first bit of $E^p$ cannot be predicted with any significant 
advantage.
Theorem~\ref{thm:hardness} follows immediately from the following 
lemma.
\begin{lemma}
  \label{lemma:hardness}
  Let $\E^j:\zo^{k+nj}\to\zo^k$ be the sequence of functions above, 
  constructed from a 
  $(k,\epsilon)$-extractor~$\Ext:\zo^n\times\zo^k\to\zo^k$ in the 
  sense of Definition~\ref{def:our-extractors}.
  Let $\Pi$ be a COWU-protocol for $f$ that uses at most $n-k$ bits of 
  communication per player.
  Furthermore, let $Y=U_k$ and $X_j=U_n$ for $j\in[p]$ be independent 
  distributions, and let $M_j=M_j(Y,X)$ be the distribution of 
  messages sent by player~$j$ on input~$Y,X$.
  Then
  \begin{align}\label{eq:dist-claim}
    \sdist\Big(
      M_j\ X_{\geq j+1}\ \E^j(Y,X_{\leq j})
      \;,\;
      M_j\ X_{\geq j+1}\ U_{k} \Big) \leq (2^j-1)\epsilon
    \,.
  \end{align}
\end{lemma}
In particular, for $j=p$, we have
\begin{align*}
  \sdist\Big( M_p\ \E^p(Y,X) \;,\; M_p\ U_{k} \Big) \leq 
  (2^p-1)\epsilon
  \,,
\end{align*}
which means that $M_p$ (and hence the output of the protocol~$\Pi$) is 
fairly uncorrelated with $E^p$ (and hence $f$).
In particular, it implies~\eqref{eq:unpredictable}.

\begin{proof}[of Lemma~\ref{lemma:hardness}]
  We prove the claim by induction on $j$.
  For convenience, we define the case $j=0$ as $\E^0:\zo^k\to\zo^k$ 
  with $\E^0(y)=y$ and we define $M_0$ as the empty string.
  Then the two distributions in \eqref{eq:dist-claim} are identical, 
  and we have for $j=0$:
  \begin{align*}
    \sdist\Big( M_0\ X\ \E^0(Y) \;,\; M_0\ X\ U_{k} \Big) = 0
    \,.
  \end{align*}

  Now let $j\geq 1$.
  We want to rewrite~\eqref{eq:dist-claim} so that we can apply the 
  induction hypothesis.
  For this, we express $M_j$ in terms of $M_{j-1}$ as follows:
  By definition, $M_j$ is the message that player~$j$ passes to player 
  $j+1$ when
  \begin{itemize}
    \item receiving message $M_{j-1}$ from player $j-1$,
    \item reading the string $X_{\geq j+1}$ it has access to, and
    \item inspecting the truth table of $f_j(Y;X_{\leq j-1},\tilde 
      z_j,\dots,\tilde z_p)$ that it has access to.
  \end{itemize}
  In particular, since each player in the protocol is internally 
  deterministic, $M_j$ is a deterministic function 
  $M_j=g(M_{j-1},X_{\geq j+1},f_j)$ depending on the three items 
  above.
  A crucial observation is now that, because of the recursive 
  structure of $E^p$, the function table of $f_j$ can be fully 
  reconstructed from $f$ and the string $\E^{j-1}=\E^{j-1}(X_{\leq 
    j-1}, Y)$.
  Therefore, $M_j$ can be written as $M_j=g(M_{j-1},X_{\geq 
    j+1},\E^{j-1})$.
  Thus, we can rewrite the statistical distance in 
  \eqref{eq:dist-claim} as
  \begin{align*}
    \sdist\Big(
      &g(M_{j-1},X_{\geq j+1},\E^{j-1})\ X_{\geq j+1}\ \E^j(Y,X_{\leq 
        j})
      \;,\;\\
      &g(M_{j-1},X_{\geq j+1},\E^{j-1})\ X_{\geq j+1}\ U_{k} \Big)\\
      \leq \sdist\Big(
      &M_{j-1}\ X_{\geq j+1}\ \E^{j-1}\ \E^j(Y,X_{\leq j})
      \;,\;\\
      &M_{j-1}\ X_{\geq j+1}\ \E^{j-1}\ U_{k} \Big)\,.\\
      = \sdist\Big(
      &M_{j-1}\ X_{\geq j+1}\ \E^{j-1}\ \Ext(X_j,\E^{j-1})
      \;,\;\\
      &M_{j-1}\ X_{\geq j+1}\ \E^{j-1}\ U_{k}
      \Big)
      \\
      \leq
      \sdist\Big(
      &M_{j-1}\ X_{\geq j+1}\ \E^{j-1}\ \Ext(X_j,\E^{j-1})
      \;,\;
      M_{j-1}\ X_{\geq j+1}\ U'_k\ \Ext(X_j,U'_k)
      \Big)
      \\+
      \sdist\Big(
      &M_{j-1}\ X_{\geq j+1}\ U'_k\ \Ext(X_j,U'_k)
      \;,\;
      M_{j-1}\ X_{\geq j+1}\ U'_k\ U_{k}
      \Big)
      \\+
      \sdist\Big(
      &M_{j-1}\ X_{\geq j+1}\ U'_k\ U_{k}
      \;,\;
      M_{j-1}\ X_{\geq j+1}\ E^{j-1}\ U_{k}
      \Big)
      =: \delta_1+\delta_2+\delta_3\,.
  \end{align*}
  The first inequality follows because of the general fact that 
  applying the same deterministic function to two random variables can 
  only decrease their statistical distance
  (we remark that this also works if the function~$g$ is a randomized 
  mapping and the players have access to private randomness).
  The equality follows by definition of $E^j$.
  And the second inequality is the triangle inequality.

  Now we can bound $\delta_1
      \leq
      \sdist\Big(
      M_{j-1}\ X_{\geq j}\ \E^{j-1}
      \;,\;
      M_{j-1}\ X_{\geq j}\ U'_k
      \Big)
      \leq (2^{j-1}-1)\epsilon$
  and $\delta_3
      \leq
      (2^{j-1}-1)\epsilon$
      by induction hypothesis.
  Furthemore, $\delta_2\leq\epsilon$ follows because $\Ext$ is a 
  $(k,\epsilon)$-extractor in the sense of 
  Definition~\ref{def:our-extractors}, which we apply with $X=X_j$, 
  $Y=U'_k$, and $Z=M_{j-1}\ X_{\geq j+1}$.
  This is possible since even conditioned on $M_{j-1}$, which is a 
  random variable with support at most $2^{n-k}$, the random source 
  $X_j$ has enough average min-entropy left, that is,
  $H_\infty(X|Z)=H_\infty(X_j|M_{j-1}\ X_{\geq j+1})\geq 
  H_\infty(X_j|X_{\geq j+1}) - (n-k)=n-(n-k)=k$ and the extractor 
  works for the given average $k$-source and produces a distribution 
  that is $\epsilon$-close to the uniform distribution in the sense 
  above.

  Overall, we get $\delta_1+\delta_2+\delta_3\leq (2^j-2) \epsilon + 
  \epsilon=(2^j-1)\epsilon$.
\end{proof}


\bibliographystyle{alpha}
\bibliography{references}

\appendix
\section{Some Basics of Min-Entropy}

Our construction of hard function in COWU model requires the use of strong extractors.
In this section we list some important basic knowledge about min-entropy, which is central to the understanding of extractors.
As usual, given a random variable $X$, its min-entropy is defined to be ${\rm H}_\infty(X) = -\log\max_x\Pr[X=x]$.
It satisfies the following properties, as we would expect for a quantity that measures randomness.
\begin{fact}
  \begin{enumerate}
  \item (Joint distribution always has more randomness) ${\rm H}_\infty(X,Y) \geqslant {\rm H}_\infty(X)$
    for any random variables $X,Y$.
  \item (Randomness is additive if independent) if $X,Y$ are independent then ${\rm H}_\infty(X,Y) = {\rm H}_\infty(X) + {\rm H}_\infty(Y)$.
  \end{enumerate}
\end{fact}
Similarly to conditional Shannon entropy we define conditional min-entropy:
Given two random variables $X,Y$
\begin{align*}
  {\rm H}_{\infty}(X|Y) &= -\log\E_{y \sim Y}\bigg[\max_x\Pr[X=x | Y=y]\bigg] \\
  &= -\log\E_{y \sim Y}\bigg[2^{-{\rm H}_\infty(X|Y=y)}\bigg]
\end{align*}
Compare this to the definition of conditional Shannon entropy ${\rm H}(X|Y) = \E_{y \sim Y}[{\rm H}(X|Y=y)]$.
For conditional Shannon entropy we have the nice equality ${\rm H}(X|Y) = {\rm H}(X,Y) - {\rm H}(Y)$,
which is known as the \emph{chain rule for Shannon entropy}.
If $X,Y$ are independent, then for both notions we have
\begin{fact}
  If $X,Y$ are independent, ${\rm H}_\infty(X|Y) = {\rm H}_\infty(X)$,
  and ${\rm H}(X|Y) = {\rm H}(X)$.
\end{fact}
Either of these two equalities can be verified directly from corresponding definition.
But for Shannon entropy, it also follows from chain rule and that ${\rm H}(X,Y) = {\rm H}(X) + {\rm H}(Y)$ for independent $X,Y$.
Note that in this case, we still have for min-entropy ${\rm H}_\infty(X|Y) = {\rm H}_\infty(X,Y) - {\rm H}_\infty(Y)$.

However, when $X,Y$ are \emph{not} independent, the relationship between ${\rm H}_\infty(X|Y)$ and ${\rm H}_\infty(X,Y)$ become more subtle.
One could construct a joint distribution $X,Y$ such that ${\rm H}_\infty(X,Y) \geqslant n$, ${\rm H}_\infty(Y) = O(1)$,
but ${\rm H}_\infty(X|Y) = O(1)$. One such construction is via the following characterization of conditional entropy,
\begin{proposition}
  Given a joint distribution $X,Y$ over ${\cal X} \times {\cal Y}$,
  average min-entropy ${\rm H}_\infty(X|Y)$ measures the predictability of $X$ from $Y$:
  \begin{align*}
    {\rm H}_\infty(X|Y) = -\log\max_{P:{\cal Y} \mapsto {\cal X}}\Pr_{(x,y) \sim (X,Y)}\Big[P(y) = x\Big]
  \end{align*}
\end{proposition}

Fortunately, it turns out that if the support of $Y$ is small,
then we could still have something meaningful.
\begin{fact}
  If $|{\rm supp}(Y)| \leqslant 2^\ell$, then ${\rm H}_\infty(X|Y) \geqslant {\rm H}_\infty(X,Y) - \ell$.
\end{fact}
A direct averaging argument gives the following,
\begin{fact}
  If ${\rm H}_\infty(X|Y) \geqslant k$,
  then with probability at least $1-\varepsilon$ over $y \sim Y$,
  $${\rm H}_\infty(X|Y=y) \geqslant k - \log\frac{1}{\varepsilon}$$
\end{fact}
Combining these two facts we can deduce the \emph{chain rule for min-entropy}.
\begin{fact}
  Suppose that $|{\rm Y}| \leqslant 2^\ell$.
  Then with probability at least $1-\varepsilon$ over $y \sim Y$,
  $${\rm H}_\infty(X|Y=y) \geqslant {\rm H}_\infty(X,Y) - \ell - \log\frac{1}{\varepsilon}
  \geqslant {\rm H}_\infty(X) - \ell - \log\frac{1}{\varepsilon}$$
\end{fact}
Note that in this chain-rule, we condition on specific $y \sim Y$,
and an additional $\log(1/\varepsilon)$ min-entropy is lost.
Finally, we will also work with multiple conditioning.
\begin{proposition}
  \label{lec29:prop:conditional-min-entrpy-multi-conditioning}
  Suppose that $|{\rm supp}(Z)| \leqslant 2^\ell$.
  ${\rm H}_\infty(X|Y,Z) \geqslant {\rm H}_\infty(X,Z|Y) - \ell \geqslant {\rm H}_\infty(X|Y) - \ell$.
\end{proposition}
This follows from that double expectation is a double summation
\begin{align*}
  \E_{y,z}\bigg[\max_x\Pr[X=x|Y=y,Z=z]\bigg]
  &= \E_y\Bigg[\E_z\bigg[\max_x\Pr\Big[(X=x | Z=z) \Big| Y=y\Big]\bigg]\Bigg] \\
  &= \E_y\Bigg[2^{- {\rm H}_\infty\big((X|Z) \big| Y=y\big)} \Bigg] \\
  &\leqslant \E_y\bigg[ 2^{-\bigg( {\rm H}_\infty\Big((X, Z) \Big| Y=y\Big) - \ell \bigg)} \bigg] \\
  &= 2^\ell\E_y\bigg[ 2^{-{\rm H}_\infty\Big((X, Z) \Big| Y=y\Big)} \bigg]
\end{align*}
Now take $-\log$ on both sides (note $\leqslant$ becomes $\geqslant$) and we are done.

\section{Hardness in the COWU Model}

We are left to analyze the communication complexity of $\E^p$ we 
constructed in the last section.
For this we need strong average min-entropy extractor, defined as follows,

\begin{definition}
  A $(k, \varepsilon)$-\emph{average min-entropy extractor} is a function ${\rm Ext}: \zo^n \times \zo^d \mapsto \zo^m$
  such that for any joint random variable $(X,Y)$ with $H(X|Y) \geqslant k$,
  we have that $({\rm Ext}(X, U_d), Y)$ is $\varepsilon$-close to $(U_m, Y)$.
  Similar to usual extractor, an average min-entropy extractor is \emph{strong} if
  $(U_d, {\rm Ext}(X, U_d), Y)$ is $\varepsilon$-close to $(U_d, U_m, Y)$.
\end{definition}

\begin{proposition}
  A $(k,\varepsilon)$-extractor is a $(k, 4\sqrt{\varepsilon})$-average min-entropy extractor.
\end{proposition}
\begin{proof}
  In the high level, the proof involves two steps.
  \begin{enumerate}
  \item We show that for any $k \leqslant n-1$, and any $t > 0$,
    a $(k,\varepsilon)$-extractor is also a $(k-t, 2^{t+2}\varepsilon)$-extractor.
  \item We choose a proper $t$ so that $(k,\varepsilon)$-extractor gives a $(k, 4\sqrt{\varepsilon})$-average min-entropy extractor.
  \end{enumerate}
  Assume step 1 for a while, we show how to do step 2. Suppose that ${\rm H}_\infty(X|Y) \geqslant k$.
  Then with probability at least $(1-\delta)$ over $y \sim Y$, ${\rm H}_\infty(X|Y=y) \geqslant k-\log(1/\delta)$.
  Now step 1 says that applying a $(k,\varepsilon)$-extractor gives error $2^{\log(1/\delta) + 2}\varepsilon$.
  Therefore the total error is
  \begin{align*}
    \delta + \frac{4\varepsilon}{\delta}
  \end{align*}
  Set $\delta = 2\sqrt{\varepsilon}$ gives the required bound.

  It is left to complete step 1. We first make an observation about extractor.
  By definition, ${\rm Ext}$ is a $(k,\varepsilon)$-extractor means
  \begin{align*}
    \max_{X:k\text{-source}}\; \max_{T \subseteq \zo^m}\Big|\Pr[{\rm Ext}(X,U_d) \in T] - \mu(T)\Big| \leqslant \varepsilon
  \end{align*}
  Therefore we can swap the order the quantifier, which is
  \begin{align*}
    \max_{T \subseteq \zo^m}\;\max_{X:k\text{-source}}\Big|\Pr[{\rm Ext}(X,U_d) \in T] - \mu(T)\Big| \leqslant \varepsilon
  \end{align*}
  Assume for contradiction that ${\rm Ext}$ is not a $(k-t, 2^{t+2}\varepsilon)$-extractor.
  Therefore for some $T \subseteq \zo^m$ and $(k-t)$-source $Y$,
  \begin{align*}
    \Big|\Pr[{\rm Ext}(Y,U_d) \in T] - \mu(T)\Big| > 2^{t+2}\varepsilon
  \end{align*}
  Without loss of generality, suppose that
  \begin{align*}
    \Pr[{\rm Ext}(Y,U_d) \in T] - \mu(T) > 2^{t+2}\varepsilon
  \end{align*}
  Our plan now has two steps:
  \begin{enumerate}
  \item Construct a $k$-source $X$, such that $X = \lambda Y + (1-\lambda) Z$,
    where $Z$ is also a $k$-source.
  \item Show that $\Pr[{\rm Ext}(X,U_d) \in T] - \mu(T) > \varepsilon$.
  \end{enumerate}
  Let $p_X,p_Y,p_Z$ be the probability mass functions of $X,Y$ and $Z$.
  In order to do step 1, we need that for every $x \in \zo^n$,
  \begin{align*}
    p_X(x) = \lambda p_Y(x) + (1-\lambda)p_Z(x) \leqslant 2^{-k}
  \end{align*}
  Because $Y$ is a $(k-t)$-source, so it suffices that
  \begin{align*}
    &\lambda 2^{-(k-t)} + (1-\lambda)p_Z(x) \leqslant 2^{-k} \\
    \Leftrightarrow&\ (1-\lambda)p_Z(x) \leqslant (1-2^t\lambda)2^{-k} \\
    \Leftarrow&\ p_Z(x) \leqslant (1-2^t\lambda)2^{-k}
  \end{align*}
  Therefore it suffices to set $2^t\lambda < 1$, and so $Z$ exists.

  To do step 2, we have
  \begin{align*}
    &\Pr[{\rm Ext}(X, U_d) \in T] - \mu(T)\\
    =& \Pr[{\rm Ext}(\lambda Y + (1-\lambda)Z, U_d) \in T] - \mu(T) \\
    =& \lambda \Pr[{\rm Ext}(Y, U_d) \in T] + (1-\lambda)\Pr[{\rm Ext}(Z, U_d) \in T] - \mu(T) \\
    >& \lambda\Big(\mu(T) + 2^{t+2}\varepsilon\Big) + (1-\lambda)\Big(\mu(T)\pm\varepsilon\Big) -\mu(T) \\
    =& \lambda 2^{t+2}\varepsilon \pm (1-\lambda)\varepsilon
  \end{align*}
  Therefore it suffices to set that $\lambda 2^{t+2}\varepsilon - (1-\lambda)\varepsilon \geqslant \varepsilon$,
  which is that $\lambda \geqslant 2/(2^{t+2}+1)$. Set $\lambda = 1/2^{t+1}$,
  note that $2^t\lambda < 1$ is satisfied.
  Now $p_Z(x) \leqslant (1- 2^t\lambda)2^{-k} = (1/2)2^{-k} = 2^{-(k+1)}$.
  Because $k \leqslant n-1$, $k+1 \leqslant n$, so $D$ exists.
  The proof is complete.
\end{proof}
\red{How to show $3\varepsilon$? This is problem 6.8 in Vadhan's monograph.}

As we will see the constant $3$ is not that important in our analysis, so I will drop it,
and pretend that we have a $(k, \varepsilon)$-average min-entropy extractor.
Note that the reduction used in this proposition preserves strongness of extractors.
Therefore
\begin{corollary}
  A $(k,\varepsilon)$-strong extractor is a $(k, 3\varepsilon)$-strong average min-entropy extractor.
\end{corollary}

\begin{proposition}
  \label{prop:2}
  Let $X$ be a random variable over $\cal U$ and $Y,Z$ be random variables over $\cal V$.
  If $(X,Y)$ and $(X,Z)$ has statistical distance at most $\varepsilon$,
  then for any deterministic function $P: {\cal U} \mapsto {\cal V}$:
  \begin{align*}
    \bigg| \Pr_{(u,v) \sim (X,Y)}\bigg[ P(u) = v \bigg] - \Pr_{(u,v) \sim (X,Z)}\bigg[ P(u) = v \bigg] \bigg| \leqslant \varepsilon
  \end{align*}
\end{proposition}
\begin{proof}
  Let $T$ be the event of all pairs $(u,v)$ such that $P(u) = v$.
\end{proof}


\end{document}
