\documentclass[11pt]{article}

\usepackage{lecture}

\begin{document}

\lecture{24}{4/22/2013}{Hardness Amplification}{Xi Wu}

\draft

In the last two lectures we show how to translate average-case hardness and further worst-case hardness for \emph{circuits},
via Nisan-Wigderson generator, to pseudorandomness.
Let's first review the parameters involved.
In a PRG $G: \zo^d \mapsto \zo^r$, $d$ is a function of $r$ and error $\varepsilon$.
Assume a function $g: \zo^m \mapsto \zo$ in $\E$ of average hardness ${\sf H}_g(m) = s$,
and an $(r,a)$-design $S_1,\dots,S_r$ ($|S_i| = m$ and $|S_i \cap S_j| \leqslant a$).
Given a seed $\sigma \in \zo^d$, Nisan-Wigderson generator outputs
\begin{align*}
  g(\sigma|_{S_1}) \circ g(\sigma|_{S_2}) \circ \cdots \circ g(\sigma|_{S_r})
\end{align*}
By an unpredictability argument, we show that this generator fools circuits of size $s - r2^a$
with error $\varepsilon$ satisfying $\varepsilon/r = 1/s$.
Now, with a design where $a = O(\log r)$ and $d = O(m^2/a)$, we can bound seed length $d$ using $r,\varepsilon$ as follows,
\begin{align*}
  & m = {\sf H}_g^{-1}(s) = {\sf H}_g^{-1}(r/\varepsilon) && \text{by average hardness}\\
  & d = O\Bigg( \frac{{\sf H}_g^{-1}(r/\varepsilon)^2}{\log r} \Bigg) && \text{by combinatorial design}
\end{align*}
Let's consider a simple instantiation: suppose ${\sf H}_g(m) = 2^m$ and $\varepsilon = 1/r$,
then ${\sf H}_g^{-1}(r/\varepsilon) = \log r/\varepsilon = O(\log r)$, and the seed length $d = O(\log r)$,
which is optimal in $r$ up to constant.

For worst-case hardness, we start with $f: \zo^n \mapsto \zo$ of \emph{worst-case hardness} ${\sf C}_f(n) = s$.
The idea is to encode $\chi_f$ using a binary code ${\rm Enc}: \zo^{N} \mapsto \zo^M$~\footnote{As usual, capital letter denotes powering: $N = 2^n$, etc.}
so that the resulting binary sequence gives the characteristic sequence of a function $g$ that is \emph{average-case hard},
and then plug $g$ into Nisan-Wigderson generator.
The intuition here is that an efficient decoding algorithm,
with oracle access to a small circuit $h: \zo^m \mapsto \zo$ that computes ${\rm Enc}(\chi_f)$ somewhat well on average,
enables reconstructing $f$ correctly everywhere.

As discussed in the last lecture, the notion of decoding we need is \emph{locally list-decoding}:
given decoding radius $1/2 - \delta$ (note that this can be very close to the limit of decoding radius of binary code).
we need that for every $n$, there is a list of circuits $D_1, D_2, \dots$ of size $s'$ so that
for any $f: \zo^n \mapsto \zo$, if ${\rm agr}({\rm Enc}(\chi_f), \chi_h) \geqslant 1/2 + \delta$,
then there is a circuit $D_i$ so that $D_i^h$ computes $f$ correctly everywhere,
and this is a contradiction if the size of $D_i^h$ is less than $s$.
Note that $|D_i^h| = s' \cdot |h|$ where $|h|$ denotes the size of $h$.
Therefore, this argument translates worst-case hardness of size $s$
to $(s/s', \delta)$-average-case hardness (generally, a function $f$ at length $n$ is $(s, \varepsilon)$-average-case hard
if any circuit of size $s$ can correctly compute at most $1/2 + \varepsilon$.
In the argument above, we are interested in $(s,1/s)$-average-case hard).

This procedure of amplifying worst-case hardness to average-case hardness is known as \emph{hardness amplification} in literature.
In our setting, we are amplifying the hardness against non-uniform complexity class: namely circuits.
In this lecture we elaborate the details of this amplification with a focus on how the local list-decoding works.

\section{Hardness Amplification}

In this section, we prove the following quantitative version of the above discussion
\begin{theorem}
  There exists ${\rm Enc}_{n,s}: \zo^N \mapsto \zo^{M}$ where $m$ is a function of $n$ and $s$
  such that
  \begin{enumerate}
  \item [(1)] ${\rm Enc}_{n,s}$ is computable in time $\poly(N,s)$.
  \item [(2)] For every $n,s$ there exists a list of circuits $D_1,D_2,\dots$ of size $\poly(n,s)$
    such that for any $h: \zo^m \mapsto \zo$ and $f: \zo^n \mapsto \zo$ such that if
    \begin{align*}
      \delta_H\bigg( {\rm Enc}(\chi_f), \chi_h \bigg) \leqslant \frac{1}{2} - \frac{1}{s}
    \end{align*}
    then there exists $i$ such that $D_i^h$ computes $f$ correctly everywhere.
  \end{enumerate}
\end{theorem}

\subsection{High Level Ideas}

Our main construction is to concatenate an outer code with distance $1 - O(1/s)$ and is locally list decodable up to distance $1 - O(1/s)$
with Hadamard code as inner code. Let's focus on the outer code.
A natural first try is Reed-Solomon code. Consider Reed-Solomon code of degree $d$ over $\GF{q}$.
We identify every information word in $\GF{q}^{d+1}$ as the values of univariate polynomial $P$ of degree at most $d$ in $d+1$ distinct positions.
The encoding of $P$ is its values at all positions.
However, Reed-Solomon is not locally list decodable.
The rough reason is that, given a received word of size $2^{O(n)}$, we can only afford to look at $\polylog(n)$ positions,
however these few positions give little information for local decoding.
For example, suppose that we want to decode at $x \in \zo^n$,
looking at any $d$ positions leave $x$ to be equally likely to be any symbol in $\GF{q}$, which is useless.

Instead we use \emph{low-degree extension} and Reed-Muller codes (how it overcomes the local list decoding issue will come out later).
Precisely we identify a $d$-variate polynomial $P$ of degree $\ell-1$ in each variable
by its evaluation at each point of a $\ell^d$ (which we set to be the length of the information word, that is $N = \ell^d$) cube.
We claim that the values on this cube uniquely determines the polynomial {\color{red} give the inductive proof}.
Note that first we extend an information word of length $\ell^d$ to $q^d$,
and second, the total degree is bounded by $d(\ell - 1)$ so the relative minimum distance,
by Schwartz-Zippel Lemma is at least $1 - \frac{d\ell}{q}$, and we set $d\ell/q = \Theta(1/s)$.

The way we decode is where the improvement comes. Precisely, in order to decode at $x$,
we randomly restrict the received word to a line $L$ that passes through $x$.
We do this by uniformly pick a point $y$, form line $L(t): \GF{q} \mapsto \GF{q}^d$ by $x + ty$.
It is important to observe that restricting to $L$ gives a univariate polynomial in $t$ of degree at most $d\ell$.

Now consider decoding at radius $1 - O(d\ell/q)$. Because each point on the line is uniformly distributed in the cube $\GF{q}^d$,
the expected fraction of points that agree with $P|_L$ is $\Omega(d\ell/q)$.
Because points on a line are pairwise-independent {\color{red} explain this},
so we can use Chebyshev bound to show that actually this holds with high probability.
Therefore decoding restricting to this line suffices! But now this is exactly list decoding Reed-Solomon of degree at most $\ell d$,
and we only need to query about $d\ell$ position, which could be much smaller than $\ell^d$.

\subsection{List-Decoding of Reed-Solomon Code}

In this section we give a list-decoding algorithm for Reed-Solomon code which is first discovered by Guruswami and Sudan.
We identify each information word $f \in \GF{q}^{d+1}$ as evaluations of a polynomial of degree at most $d$ at $d+1$ points.
The encoding is the evaluation of $f$ at every point. On input $r \in \GF{q}^q$,
we first find a nonzero bivariate polynomial $Q(Y,X)$ of degree $d_Y$ in $Y$ and degree $d_X$ in $X$,
so that for every $y \in \GF{q}$ (so we have in total $q$ equations)
\begin{align*}
  Q(y, r(y)) = 0
\end{align*}
There are $(d_Y+1)(d_X+1)$ coefficients in $Q$, so nonzero $Q$ exists provided that
\begin{align*}
  (d_Y+1)(d_X+1) > q \tag{1}
\end{align*}
Now we argue that for any polynomial $g$ of degree at most $d$,
if ${\rm agr}({\rm Enc}(f), g) \geqslant \varepsilon$, then $Q(Y, g(Y)) \equiv 0$.
This is true provided that the number of inputs we vanish ($\varepsilon q$) is larger than the degree of $Q(Y, g(Y))$,
so gives condition
\begin{align*}
  \varepsilon q > d_Y + d \cdot d_X \tag{2}
\end{align*}
Therefore now if we consider univariate polynomial $Q^*(X)$ by viewing $Q$ as polynomial in $(\GF{q}[Y])[X]$,
then the $g$'s we are seeking for are all in the factorization of $Q^*$.
The size of the list is bounded by the degree of $Q^*$, which is at most $d_X$

Now we set parameters, we are interested in maximizing the decoding radius $1-\varepsilon$,
while minimizing the list size $d_X$. This gives that $\varepsilon, d_Y, d_X$ as function of $d,q$.
Set $d_Y = \lceil \sqrt{dq} \rceil$ and $d_X = \lceil \sqrt{q/d} \rceil$, we have that (1) is satisfied,
and to satisfy (2),
\begin{align*}
  \varepsilon q > 2\sqrt{qd}
\end{align*}
Therefore set $\varepsilon \approx 2\sqrt{d/q}$.
Note that the minimum distance of Reed-Solomon code is $\delta = d/q$,
where $R = d/q$ is the rate of the code.
therefore our decoding radius is $1 - O(\sqrt{R})$.
Going beyond $1 - \sqrt{R}$ for $R < 1/16$ is a long-standing open problem (which is known as the Guruswami-Sudan radius),
which is finally resolved by Parvaresh-Vardy codes (as we saw in previous lectures).

\subsection{Local List-Decoding of Reed-M\"{u}Der codes}

Let's delve into the parameters.
{\color{red} elaborate the details}.

Setting parameters we have
\begin{align*}
  & d = \Theta\Bigg(\frac{\log N}{\log\log N}\Bigg) \\
  & \ell = \Theta(d \cdot s) \\
  & q = \Theta((ds)^2)
\end{align*}

\section{Generalization to Other Models}

Our argument for general circuits (both average-case hardness and worst-case hardness arguments) carry through for branching programs.
We leave this as an exercise to read to verify.
One immediate instantiation of this hardness-randomness tradeoff is that
if there is a function $f_m \in \DSPACE(O(m))$ requires branching programs of linear exponential size,
then $\BPL = \L$. In fact we get that $\BPL$ with two-way access to the random tape equals $L$.

\section{On Fooling Constant-depth Circuits}

Our argument for general circuits also applies to constant-depth circuits.
However, in this case, we only know that the average-case hardness argument could carry through
because we do not know whether the list decoding algorithm is computable by small depth circuits.
Fortunately, average-case hardness for these circuits are known,
and so we have unconditional pseudorandom generator for constant-depth circuits.

\begin{theorem}
  If $f$ can be computed by a circuit of depth $d$ and size $s$,
  for every $\Delta > 0$, there exists a multivariate polynomial $P: \zo^m \mapsto {\mathbb R}$ of total degree at most $\Delta$ such that
  \begin{align*}
    \Exp_{x \leftarrow U_n}\bigg[ \bigg| f(x) - P(x) \bigg| ^2\bigg] \leqslant 2 \cdot s \cdot 2^{-\Delta^{1/2d}/20}
  \end{align*}
\end{theorem}

\begin{theorem}
  Any $k$-wise uniform distribution with $k = (\log s/\varepsilon)^{O(d^2)}$ is $\varepsilon$-pseudorandom
  for circuit of depth $d$ and size $s$.
\end{theorem}

The first important observation towards proving this result is the following,
for any polynomial $P: \zo^n \mapsto {\mathbb R}$ of degree at most $k$,
\begin{align*}
  \Exp[P(D)] = \Exp[P(U)]
\end{align*}
if $D$ is poly-logarithmic-wise independent.

\end{document}
