\documentclass[11pt]{article}
\include{lecture}
\begin{document}
\lecture{26}{4/24/2010}{Probabilistically Checkable Proofs}{Tom Watson and Andrew Bolanowski}

\draft

In this lecture we continue our discussion of interactive proof systems.
We develop a nonrelativizing proof technique, encapsulated in the so-called
\emph{sumcheck protocol}, and use it to prove that $\P^\SHARPP\subseteq\IP$.
Building on this technique, we strengthen the result and show that
$\IP=\PSPACE$, i.e., polynomial space exactly captures the power of
interactive proof systems. Then we introduce the notion of \emph{multiple
prover interactive proof systems} and relate them to \emph{probabilistically
checkable proofs}, in which the interaction between prover and verifier
is eliminated but the proof is allowed to be very long. There are some
extremely profound and involved results in these areas, which we briefly
discuss.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% New section %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Interactive Proof Systems for $\SHARPP$}

Two lectures ago we discussed proof complexity, in which the central question
is whether $\coNP$ statements have short proofs. In the last lecture we
generalized our notion of proof to include both randomness in the verification
and interaction with a prover. This led to the notion of an \emph{interactive
proof system}, in which there are two parties interacting to decide whether a
given input is in a particular language. There is a \emph{prover} who is
computationally unrestricted, and therefore knows whether the input is in the
language, but it is the (skeptical) probabilistic \emph{verifier} who gets the
final say. We can think of the prover as trying to get the verifier to accept,
and we want it to be the case that if the input is in the language, then some
prover can, in fact, convince the verifier to accept with probability
at least some value $c$ (informally, we can prove everything that's true,
the \emph{completeness} property) and that if the input is not in the
language, then \emph{no} prover, not even a devious one, can convince the
verifier to accept with probability greater than some value $s$ (informally,
we cannot prove anything that's false, the \emph{soundness} property).

We gave examples of interactive proof systems for the Graph Nonisomorphism
problem, which is a problem for which we don't know of short, efficiently
verifiable classical proofs (although under certain reasonable complexity
theoretic assumptions, they are known to exist). We now demonstrate that
not only do all $\coNP$ statements have short proofs in the present sense, but
(presumably) many more languages have short proofs. We establish the exact
power of interactive proof systems by showing that $\IP=\PSPACE$.  We point out
that the proof is nonrelativizing. As a step toward
proving this result, we now develop an interactive proof for the problem
of deciding whether the number of satisfying assignments to a given CNF
formula is exactly some number $k$. This allows us to establish that
$\P^\SHARPP\subseteq\IP$.

Our interactive proof system involves two main ingredients. The first is
arithmetization of the given formula $\phi$, as discussed in the last lecture.
This is a simple procedure that produces a multivariate polynomial
$\widetilde{\phi}(x_1,\ldots,x_n)$ that agrees with $\phi$ on the Boolean
cube. This is the step that prevents our proof from relativizing; we leave
it as an exercise to argue formally why this is the case. This polynomial has
two very important properties: every variable has degree at most $m$ in it,
where $m$ is the number of clauses in $\phi$, and it can be efficiently
evaluated (despite possibly having exponentially many monomials when expanded).
These are the only two properties we need in order to apply the second main
ingredient, the sumcheck protocol, which we describe next.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% New section %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Sumcheck Protocol}

Our goal is to design a verifier to decide whether the number of satisfying
assignments to $\phi$ is equal to some number $k_0$. This is equivalent
to checking the following identity:
\begin{equation}
\sum_{x_1=0}^1\sum_{x_2=0}^1\cdots\sum_{x_n=0}^1\widetilde{\phi}
(x_1,x_2,\ldots,x_n)=k_0.
\label{24:eqn:sumcheck}
\end{equation}
We don't have time to evaluate $\widetilde{\phi}$ for all $2^n$ settings of
$x_1,\ldots,x_n$, so we enlist the help of the prover. The sumcheck protocol
is a method for doing this that is secure against cheating provers.

The key idea is to consider the univariate polynomial $g(x)$ defined by
\[g(x)=\sum_{x_2=0}^1\cdots\sum_{x_n=0}^1\widetilde{\phi}(x,x_2,\ldots,x_n).\]
This is the same as the expression in Equation (\ref{24:eqn:sumcheck}) except
that we don't sum over $x_1$. Since $x_1$ has degree at most $m$ in
$\widetilde{\phi}$, $g(x)$ has degree at most $m$. If we knew $g(x)$, then we
would be able to check Equation (\ref{24:eqn:sumcheck}) by checking
$g(0)+g(1)=k_0$, using the observation that
\[g(0)+g(1)=\sum_{x_1=0}^1\sum_{x_2=0}^1\cdots\sum_{x_n=0}^1\widetilde{\phi}
(x_1,x_2,\ldots,x_n).\]
We don't have $g(x)$, but we can ask the prover to send it to us. We know
that $g(x)$ has at most $m+1$ coefficients, which is manageable, but it's
conceivable that these coefficients are prohibitively huge numbers.
This turns out not to be an issue, but we postpone the discussion of this.

Now the prover sends us some polynomial $g'(x)$, which may not be equal to
$g(x)$ if the prover is cheating. We check that $g'(x)$ has degree at
most $m$ and that $g'(0)+g'(1)=k_0$ and reject if either does not hold.
Now if Equation (\ref{24:eqn:sumcheck}) is true, then the prover can just
send us the correct polynomial $g'(x)=g(x)$ and everything will be fine. If
Equation (\ref{24:eqn:sumcheck}) is not true, then if the prover wants to
have a prayer of getting us to accept, then it's forced to send an
incorrect polynomial $g'(x)\neq g(x)$. Since the two polynomials agree in at
most $m$ places, we can pick a random number $\xi_1$ from a reasonably small
set $I$ and with high probability, $g'(\xi_1)\neq g(\xi_1)$. If we had $g(x)$
then we could just evaluate $g(\xi_1)$ and $g'(\xi_1)$ and with high
probability catch the cheating prover. Of course, the whole point is that we
don't have $g(x)$, but we have now reduced our original problem, verifying
Equation (\ref{24:eqn:sumcheck}), to a simpler instance of the same problem,
namely verifying
\begin{equation}
\sum_{x_2=0}^1\cdots\sum_{x_n=0}^1\widetilde{\phi}(\xi_1,x_2,\ldots,x_n)=k_1
\label{24:eqn:sumcheck2}
\end{equation}
where $k_1=g'(\xi_1)$. By sending us $g'(x)$, the prover has implicitly claimed
that Equation (\ref{24:eqn:sumcheck2}) holds. It had no way of knowing what
number $\xi_1$ we would pick, though, which gives us an edge. If Equation
(\ref{24:eqn:sumcheck}) holds, then Equation (\ref{24:eqn:sumcheck2}) holds
when the prover just sends us the correct polynomial. If Equation
(\ref{24:eqn:sumcheck}) does not hold, then with high probability Equation
(\ref{24:eqn:sumcheck2}) does not hold, regardless of the prover's behavior.
In either case we are back to where we started, except that now there are
only $n-1$ variables being summed over. We iterate this process until
our task is reduced to the problem of verifying
\begin{equation}
\widetilde{\phi}(\xi_1,\xi_2,\ldots,\xi_n)=k_n
\label{24:eqn:sumcheck3}
\end{equation}
for some numbers $\xi_1,\ldots,\xi_n\in I$ and $k_n$. Now we use the
other property required of $\widetilde{\phi}$, namely that is can be evaluated
efficiently, to explicitly check that Equation (\ref{24:eqn:sumcheck3})
holds. We accept if it holds and reject otherwise. This is the only
time in the protocol when we are willing to accept.

\begin{theorem}
  The sumcheck protocol is a public-coin interactive proof system that
  verifies Equation (\ref{24:eqn:sumcheck}) with perfect completeness and
  polynomially small soundness, provided $\widetilde{\phi}$ has degree at most
  $m$ in every variable and can be evaluated in polynomial time.
  \label{24:thm:sumcheckworks}
\end{theorem}

\begin{proof}
  That the protocol is public-coin is clear --- in each step the verifier
  uses its randomness only to pick the number $\xi_i$, and then it sends
  $\xi_i$ to the prover. That the protocol has perfect completeness is also
  straightforward to see --- if Equation (\ref{24:eqn:sumcheck}) holds then the
  prover can just send us the correct polynomial $g'(x)=g(x)$ in every phase
  and the consistency check $g'(0)+g'(1)=k_i$ will always pass and all the
  implicit claims of the form of Equation (\ref{24:eqn:sumcheck2}) will be
  true, so the final check of Equation (\ref{24:eqn:sumcheck3}) will also
  pass, leading to acceptance with probability $1$.
  
  Now we argue the soundness. Suppose Equation (\ref{24:eqn:sumcheck})
  does not hold, but nevertheless we accept. In the last phase, the claim
  we're checking, Equation (\ref{24:eqn:sumcheck3}), is true (otherwise we
  would reject), but in the first phase, the claim we're checking, Equation
  (\ref{24:eqn:sumcheck}), is false. Thus there must be some phase where
  the current claim transitions from being false to being true. In this phase,
  the prover sends us some $g'(x)\neq g(x)$ (since otherwise we would reject
  after seeing that the consistency check $g'(0)+g'(1)=k_i$ fails). For the
  claim generated by this phase to be true, it must be the case that
  $g'(\xi_{i+1})=g(\xi_{i+1})$, which happens with probability at most
  $m/|I|$. The probability that we accept is at most the probability
  that this happens in some phase, which by a union bound is at most
  $mn/|I|$. We can choose $|I|$ to be polynomially large to make the soundness
  polynomially small.

\begin{equation}
Pr[V \Leftrightarrow P Accepts | the result the prover gives \neq]  \le   mn/|I|
\label{24:eqn:sumcheckprob}
\end{equation}
  
  The only thing left to verify is that the protocol runs in polynomial time.
  The final check can be done in polynomial time by hypothesis, so the
  only issue is that the coefficients of the $g(x)$ polynomials might become
  too large. One can argue that the coefficients do not become too large.
  We can avoid that analysis by letting $I$ be the integers mod $p$ for a
  small prime $p$, and letting all computations take place in the field of
  integers mod $p$. Now we need to worry about the soundness, though; it
  could be the case that the originally claimed value $k_0$ is congruent
  to the true value mod $p$, and in that case the prover can make the verifier
  always accept. However, the difference between $k_0$ and the true value
  cannot have too many prime factors (certainly fewer than $n$). The density
  of primes is great enough that picking $p$ at random from the set of
  primes of polynomial magnitude (and hence logarithmic bit length) affects
  the soundness by only a polynomially small amount. Thus the overall
  soundness is still polynomially small, and computations mod $p$ can
  certainly be done efficiently. Selecting $p$ at random is not a problem
  since the verifier can enumerate all primes in the desired range by
  brute force.
\end{proof}

\begin{corollary}
  Every language in $\P^\SHARPP$ has a public-coin interactive proof system
  with perfect completeness, and in particular $\P^\SHARPP\subseteq\IP$.
\end{corollary}

\begin{proof}
  We can simulate a $\P^\SHARPP$ machine, and whenever it makes an oracle
  query, reduce the query to a \#SAT instance, have our prover tell us
  how many satisfying assignments it has, and run the sumcheck protocol to
  verify this. The resulting protocol still runs in polynomial time. If
  the input is in the language, then the prover can follow the protocol,
  leading to acceptance with probability $1$. If the input is not in the
  language, then for us to accept, the prover must cheat on at least
  one query, and this slips by us with exponentially small probability per
  query. By a union bound, the soundness of this protocol is still
  exponentially small.
\end{proof}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% New section %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Interactive Proof Systems for $\PSPACE$}
\label{24:sec:pspace}

Using the sumcheck protocol, we can now characterize the power of interactive
proof systems.

\begin{theorem}
  Every language in $\PSPACE$ has a public-coin interactive proof system
  with perfect completeness, and in particular $\PSPACE\subseteq\IP$.
  \label{24:thm:pspaceinip}
\end{theorem}

\begin{corollary}
  $\IP=\PSPACE$.
  \label{24:cor:ipequalspspace}
\end{corollary}

\begin{proof}
  The inclusion $\PSPACE\subseteq\IP$ follows from Theorem
  \ref{24:thm:pspaceinip}. The inclusion $\IP\subseteq\PSPACE$ is left as
  an exercise; it just involves a space-efficient simulation of an interactive
  proof system.
\end{proof}

In the last lecture we mentioned that every interactive proof system can
be converted to one with perfect completeness without increasing the number
of rounds, and also to one with both perfect completeness and public coins with
only an increase of two in the number of rounds. In particular, requiring
perfect completeness and public coins does not decrease the power of $\IP$.
Theorem \ref{24:thm:pspaceinip} and Corollary \ref{24:cor:ipequalspspace}
provide an alternative proof of the latter fact: every language in $\IP$
is in $\PSPACE$ by Corollary \ref{24:cor:ipequalspspace}, and every
language in $\PSPACE$ has a public-coin interactive proof system with
perfect completeness by Theorem \ref{24:thm:pspaceinip}.

We now prove the main result of this section.

\begin{proof}[Proof of Theorem \ref{24:thm:pspaceinip}]
  One idea is to show that $TQBF\in\IP$ by arithmetizing quantified
  formulas and using a protocol similar to the sumcheck protocol.
  Arithmetizing a universal quantifier, however, results in a squaring of
  the degree, which could lead to very high degree polynomials. This can
  be remedied by using an extra step that reduces the degree of the polynomial
  after arithmetizing a universal quantifier. This approach does lead to a
  proof that $\PSPACE\subseteq\IP$, but we go a different route, employing
  the divide-and-conquer strategy used in our proof that
  $\NSPACE(s)\subseteq\DSPACE(s^2)$.
  
  In the sumcheck protocol, our goal was to count the number of satisfying
  assignments to a CNF formula, or equivalently the number of accepting
  computation paths of a nondeterministic machine. Now we are given a
  deterministic machine running in space $s$ and wish to determine the
  number of accepting computation paths, which is either $0$ or $1$.
  The computation paths, however, could be exponentially long, so we can't
  work directly with a formula expressing the whole computation. We
  instead work with polynomials derived from the divide-and-conquer approach
  of verifying a computation tableau. This involves another application
  of the main idea of the sumcheck protocol --- in a divide-and-conquer step,
  we do not just randomly pick a half of the tableau to recurse on, because
  this only gives us a probability $1/2$ of catching a cheating prover.
  Instead we capture this with a polynomial where plugging in $0$ corresponds
  to checking one half of the tableau and plugging in $1$ corresponds to
  checking the other half, and we plug in a random value from a much larger
  domain. In this way we sort of ``handle both halves at once'', and catch
  a cheating prover with high probability.
  
  We now pursue this intuition formally. We assume for simplicity of notation
  that the configurations of the machine $M$ we wish to simulate are bit
  strings of length $s$, and that the machine makes exactly $2^s$ transitions,
  where $2^s$ is a power of $2$ (thus the computation tableau has $2^s+1$
  rows). This last assumption can be justified by assuming that once $M$
  reaches a halting configuration, it just keeps transitioning to that same
  configuration forever. We also assume there is a unique accepting
  configuration. For $\ell=0,\ldots,s$ we define the following predicate on
  pairs of configurations:
  \[P_\ell(C_0,C_1)=\begin{cases}1&\text{if $C_0\vdash_M^{2^\ell}C_1$}\\
  0&\text{otherwise}\end{cases}.\]
  That is, $P_\ell(C_0,C_1)$ indicates whether executing $2^\ell$
  transitions of $M$ starting from configuration $C_0$ leads to configuration
  $C_1$. These predicates satisfy the following recurrence: $P_0$ is
  just specified by the transition function of $M$, and for $\ell>0$,
  \begin{equation}
  P_\ell(C_0,C_1)=\sum_{C_{1/2}\in\{0,1\}^s}P_{\ell-1}
  (C_0,C_{1/2})P_{\ell-1}(C_{1/2},C_1).
  \label{24:eqn:recurrence}
  \end{equation}
  Thus, the claim that we need to verify is
  \begin{equation}
  P_s(C_0,C_1)=k_s,
  \end{equation}
  where $C_0$ is the unique initial configuration determined by the input,
  $C_1$ is the unique accepting configuration, and $k_s=1$.
  
  Our first step is to arithmetize these predicates to obtain multivariate
  polynomials (in fact, polynomials in exactly $2s$ variables)
  $\widetilde{P}_\ell(C_0,C_1)$ such that $\widetilde{P}_\ell$ agrees with
  $P_\ell$ on the Boolean cube. We have the following fact.
  
  \begin{claim}
    Under an appropriate encoding of configurations, there exists a
    $2s$-variate polynomial $\widetilde{P}_0(C_0,C_1)$ that agrees with
    $P_0(C_0,C_1)$ on the Boolean cube, can be evaluated in polynomial time,
    and such that the degree of every variable is $O(1)$.
    \label{24:clm:transition}
  \end{claim}
  
  We leave the proof of Claim \ref{24:clm:transition} as an exercise.
  It just involves summing over all possible tape head positions and
  transitions, and expressing that the contents of the configurations
  are consistent with that transition by multiplying together many simple
  checks.
  
  The polynomial $\widetilde{P}_0$ and Equation (\ref{24:eqn:recurrence})
  immediately specify all the polynomials $\widetilde{P}_\ell$. Moreover, in
  each $\widetilde{P}_\ell$, every variable has degree $O(1)$. This can be
  argued by induction, using the fact that in Equation
  (\ref{24:eqn:recurrence}), the only variables whose degrees increase are
  $C_{1/2}$, but these are summed out. The constant degree property is not
  critical, though; it turns out that our argument would go through if the
  degrees of the variables were larger.

  The claim we need to verify is now
  \begin{equation}
  \widetilde{P}_s(C_0,C_1)=k_s.
  \label{24:eqn:mainclaim}
  \end{equation}
  Now looking at Equation (\ref{24:eqn:recurrence}), we see that if
  $\widetilde{P}_{s-1}$ were easy to evaluate, then we would be in exactly a
  position to use the sumcheck protocol to verify our claim and be done. We
  don't know how to evaluate $\widetilde{P}_{s-1}$ efficiently, but we observe
  that actually, the sumcheck protocol reduces the problem of verifying
  \[\sum_{C_{1/2}\in\{0,1\}^s}\widetilde{P}_{s-1}
  (C_0,C_{1/2})\widetilde{P}_{s-1}(C_{1/2},C_1)=k_s\]
  to the problem of verifying
  \begin{equation}
  \widetilde{P}_{s-1}(C_0,\gamma)\widetilde{P}_{s-1}(\gamma,C_1)=\kappa_s
  \label{24:eqn:secondclaim}
  \end{equation}
  for some particular values $\gamma$ and $\kappa_s$.
  
  It looks as if we've almost reduced our original problem, checking
  Equation (\ref{24:eqn:mainclaim}), to a simpler instance of the same
  problem. However, Equation (\ref{24:eqn:secondclaim}) involves the
  product of two evaluations of $\widetilde{P}_{s-1}$, so it is not quite the
  same as our original problem. The two factors correspond to the two
  halves of the computation tableau, and if we were to recurse on both
  halves, then we would be no better off than just simulating the entire
  computation from start to finish. Also, we can't pick just one of the branches to simulation, as that would destroy the soundness. The following is the key idea that leads
  to an exponential savings in running time.
  
  Define $h(x)$ to be a univariate polynomial such that $h(0)=(C_0,\gamma)$
  and $h(1)=(\gamma,C_1)$. More precisely, $h$ is a $2s$-tuple of
  univariate polynomials, one for each bit of the output. Each bit is a
  simple linear function of $x$ (since $C_0$, $\gamma$, and $C_1$ are fixed).
  Now we're interested in checking
  \[\widetilde{P}_{s-1}(h(0))\widetilde{P}_{s-1}(h(1))=\kappa_s,\]
  which seems similar to the consistency check $g'(0)+g'(1)=k_0$ in the
  sumcheck protocol. Here we set $g(x)=\widetilde{P}_{s-1}(h(x))$. Since each
  of the $2s$ components of $h$ has degree at most $1$, and each of the $2s$
  variables in $\widetilde{P}_{s-1}$ has degree $O(1)$, $g(x)$ is a univariate
  polynomial of degree $O(s)$. If we had $g(x)$ then we could just check that
  $g(0)g(1)=\kappa_s$ and be done. Like in the sumcheck protocol, we ask
  the prover for the coefficients of $g(x)$ and it gives us some $g'(x)$ of
  degree $O(s)$. We check that $g'(0)g'(1)=\kappa_s$ and reject if this does
  not hold. In the event that $g(0)g(1)\neq\kappa_s$, this forces the prover
  to give us an incorrect polynomial $g'(x)\neq g(x)$. In order to catch a
  cheating prover, we pick a random $\xi\in I$, and now with high probability,
  $g'(\xi)\neq g(\xi)$. We can't detect this immediately since we can't
  evaluate $g(\xi)$, but we have reduced our original problem to a simpler
  instance of the same problem. By sending $g'(x)$, the prover has implicitly
  claimed that
  \begin{equation}
  \widetilde{P}_{s-1}(h(\xi))=k_{s-1},
  \label{24:eqn:thirdclaim}
  \end{equation}
  where $k_{s-1}=g'(\xi)$. Now $k_s$ has been replaced by $k_{s-1}$,
  $(C_0,C_1)$ has been replaced by $h(\xi)$, and $\ell$ has been reduced
  from $s$ to $s-1$. We can iterate this process until we've reduced the
  problem to checking 
  \begin{equation}
  \widetilde{P}_0(\alpha)=k_0,
  \label{24:eqn:fourthclaim}
  \end{equation}
  where $\alpha$ is some $2s$-tuple of numbers (not necessarily bits) and
  $k_0$ is some number. By Claim \ref{24:clm:transition}, this can be done
  efficiently. We accept if Equation (\ref{24:eqn:fourthclaim}) holds and
  reject otherwise. This is the only time in the protocol when we are
  willing to accept.
  
  As in the case of the sumcheck protocol, we need to worry about
  whether the coefficients become too large. In this case, they can become
  very large, which necessitates performing all computations modulo some
  small prime. Since the number of accepting computation paths is either $0$
  or $1$, the soundness is unaffected, regardless of which prime we use.
  
  We now recap the protocol and argue its correctness. The main outline
  consists of $s$ iterations, each of which reduces the problem of verifying
  the value of $\widetilde{P}_\ell$ on some input to the problem of verifying
  the value of $\widetilde{P}_{\ell-1}$ on some other input, and then a final
  verification of the value of $\widetilde{P}_0$ on some input, which can be
  done efficiently. Each iteration consists of running the sumcheck
  protocol with the identity (\ref{24:eqn:recurrence}) to reduce the problem
  of verifying the value of $\widetilde{P}_\ell$ on some input to the problem
  of verifying the value of the product of $\widetilde{P}_{\ell-1}$ on two
  other inputs. This is then reduced to the problem of verifying the value of
  $\widetilde{P}_{\ell-1}$ on a single input by interpolating through the
  two inputs with a univariate polynomial $h(x)$ and setting the single
  input to be $h(\xi)$ for a randomly chosen $\xi$.
  
  Like the sumcheck protocol, this protocol is public-coin and has perfect
  completeness. Now if the original claim, Equation (\ref{24:eqn:mainclaim}),
  is false, then in order for us to accept, the current claim must transition
  from being false to being true in some iteration. By a union bound,
  the soundness is at most $s$ times the probability that this happens in
  any given iteration. Within an iteration, there are two steps: we obtain
  an intermediate claim via the sumcheck protocol, and then we obtain the
  claim for the next iteration using the interpolating polynomial $h(x)$
  evaluated at a random $\xi$. The probability that the intermediate claim
  becomes true is at most $O(s/|I|)$ by Theorem {\ref{24:thm:sumcheckworks}}
  (since there are $O(s)$ variables and each has constant degree). The
  probability that the claim transitions from false to true in the second
  step is at most $O(s/|I|)$ since the $g(x)$ has degree $O(s)$.
  By a union bound, the probability that the claim transitions from false
  to true in a given iteration is also $O(s/|I|)$, and so the
  soundness of the protocol is $O(s^2/|I|)$. Since $s$ is only
  polynomial, $|I|$ doesn't have to be too large to make the soundness small,
  and thus we don't have to pick the prime $p$ to be very large. We can find
  an appropriate $p$ by brute force.
\end{proof}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% New section %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Probabilistically Checkable Proofs}

\subsection{Multiple Prover Interactive Proof Systems}
\label{24:sec:mips}

We bridge the gap between interactive proof systems and probabilistically
checkable proofs by introducing \emph{multiple prover interactive proof
systems}, in which there are multiple provers that the verifier may
query throughout the protocol, and they are allowed to cooperatively
decide on any strategy before the protocol begins, \emph{but they may
not communicate during the protocol}. A multiple prover interactive proof
system is said to decide a language $L$ with completeness $c$ and soundness
$s$ if for all $x\in L$ there exist provers that make the verifier accept
with probability at least $c$, and for all $x\not\in L$, no provers can
make the verifier accept with probability more than $s$. Define the class
$\MIP$ to be the set of languages decided by polynomial-time multiple prover
interactive proof systems with any number of provers (though the verifier can
clearly only query polynomially many) and say completeness $2/3$ and soundness
$1/3$.

The additional power afforded by having multiple provers is similar to a
strategy used by the police to question suspected accomplices:
interrogating the suspects separately and checking the consistency of
their stories improves the chances of being able to catch them lying. However, it turns out that after we add the second prover, any additional provers provide no extra power.
Just as we quantified the exact power of interactive proof systems in
Corollary \ref{24:cor:ipequalspspace}, one can quantify the exact power
of multiple prover interactive proof systems as follows.

\begin{theorem}
  $\MIP=\NEXP$.
  \label{24:thm:mipequalsnexp}
\end{theorem}

We omit the proof of Theorem \ref{24:thm:mipequalsnexp}, which uses similar
ingredients as our proof of Theorem \ref{24:thm:pspaceinip} but is more
involved. In particular we note that arithmetization does not relative, but we can use the prover to supply the oracle. Of course, then we have to verify that oracle. We note, however, that this theorem and the fact that
$\NP\subsetneq\NEXP$ (which follows from the nondeterministic time hierarchy)
imply that we can decide \emph{provably} more languages with multiple
prover interactive proof systems than with classical proofs (i.e., $\NP$).

Since we believe that $\PSPACE\subsetneq\NEXP$, it seems that multiple prover
interactive proof systems are strictly more powerful than single prover
interactive proof systems. Does each additional prover yield more power?
The following theorem, which we prove shortly, shows that the answer is no.
Define $\MIP[k]$ to be the set of languages decided by multiple prover
interactive proof systems with $k$ provers.

\begin{theorem}
  $\MIP=\MIP[2]$.
  \label{24:thm:mipequalsmip2}
\end{theorem}

We are now ready to study the very influential notion of probabilistically
checkable proofs, in which the proof is entirely written down and
the element of interaction is removed, but the proofs are in principle
allowed to be very long.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% New section %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Probabilistically Checkable Proofs}

\begin{definition}
  A \emph{probabilistically checkable proof system} with completeness $c$
  and soundness $s<c$ for a language $L$ is a polynomial-time probabilistic
  oracle Turing machine $V$ such that if $x\in L$ then there exists a proof
  $\Pi$ such that \[\Pr[V^\Pi(x)\text{ accepts}]\geq c,\] and if $x\not\in L$
  then for all proofs $\Pi$, \[\Pr[V^\Pi(x)\text{ accepts}]\leq s.\]
\end{definition}

Typical settings of the completeness and soundness parameters are $c=1$,
$s=1/2$.

The key difference between a probabilistically checkable proof system and an
interactive proof system is that in the former, the prover is a fixed oracle
--- its responses to all possible queries must be fixed for a given input and
are not allowed to change after the verifier starts computing. With
probabilistically checkable proofs, we are usually interested in the amount
of randomness and number of queries needed by the verifier. This motivates
the following definition.

\begin{definition}
  The class $\PCP(r(n),q(n))$ is the set of languages decidable by a
  probabilistically checkable proof system where the verifier uses
  at most $r(n)$ random bits and queries at most $q(n)$ bits of the proof.
\end{definition}

\begin{theorem}
  $\PCP(r(n),q(n))\subseteq\NTIME(2^{O(r(n)+q(n))}n^{O(1)})$.
  \label{24:thm:pcpntime}
\end{theorem}

\begin{proof}
  Consider a probabilistically checkable proof system where the verifier
  uses $r(n)$ random bits and makes $q(n)$ queries. For each random
  bit sequence, the number of locations of the proof that could ever get
  queried is $2^{q(n)}$ (due to the possibly adaptive nature of the
  verifier). Thus over all proofs and all random strings, there are at
  most $2^{r(n)+q(n)}$ locations that ever get queried (for a given
  input). A nondeterministic machine can guess the addresses and answers to
  these queries in time $2^{r(n)+q(n)}n^{O(1)}$. It can then run over
  all random bit sequences, simulate the verifier for each one (looking up
  the answers to queries in the guessed table), and explicitly compute
  the probability of acceptance of the verifier for the guessed proof. This
  takes time $2^{r(n)}n^{O(1)}$. If the probability of acceptance is at least
  the completeness of the probabilistically checkable proof system then the
  machine accepts, and otherwise it rejects.
\end{proof}

We now draw the connection between multiple prover interactive proof systems
and probabilistically checkable proofs. Consider the connection with 2 provers. The verifier looks at 1 prover and goes through the entire process with that prover. Then the verifier asks the second prover a random question asked of the first prover. If the first prover was changing his answers based on the previous questions asked of him, then the second prover has a chance of catching him. We verify only a small part of the proof given by the first prover. This helps the idea that only a small number of queries are needed to verify the proof. Note that the following result
immediately implies Theorem \ref{24:thm:mipequalsmip2} and shows that
$\MIP=\PCP(poly(n),poly(n))$.

\begin{theorem}
  $\MIP\subseteq\PCP(poly(n),poly(n))\subseteq\MIP[2]$.
  \label{24:thm:mippcp}
\end{theorem}

\begin{proof}
  We first argue the inclusion $\MIP\subseteq\PCP(poly(n),poly(n))$.
  For a given multiple prover interactive proof system, we construct
  a probabilistically checkable proof system where the proof is interpreted
  as a complete specification of the provers' strategies, i.e., a list
  of their responses for every possible message history. The verifier can
  just simulate the multiple prover interactive proof system, looking up
  the provers' responses by querying the provided proof. The completeness
  and soundness are the same as for the multiple prover interactive
  proof system.
  
  For the inclusion $\PCP(poly(n),poly(n))\subseteq\MIP[2]$, we can have the
  verifier just simulate the given probabilistically checkable proof system,
  using one of its provers to answer queries to the proof. The only problem is
  that this prover is allowed to be adaptive, whereas in the probabilistically
  checkable proof system, the entire proof is written down from the beginning.
  We use a second prover to try to catch the first prover if it behaves
  nonadaptively. However, the second prover is also allowed to be adaptive, so
  we're only safe asking it one question. Specifically, when the simulation is
  over, if the verifier decides to accept then before doing so it picks one of
  the queries it made to the first prover uniformly at random, makes the same
  query to the second prover, and only proceeds to accept if their answers
  agree. Since the provers cannot communicate during the protocol, the
  second prover's possible responses form a nonadaptive strategy.
  
  Now if the input is in the language, then the two provers can just respond
  according to the proof specified by the probabilistically checkable proof
  system, leading to acceptance to probability at least the completeness of
  that system. If the input is not in the language, then we can compute the
  probability of acceptance as follows, where $q$ is the number of queries
  made and $A$ denotes the event that for at least one query asked of the
  first prover, its response differs from the response we would get if we
  asked the second prover.
  \begin{align*}
  \Pr\big(\text{accept}\big)&=\Pr\big(\text{accept}~\big|~A\big)
  \cdot\Pr\big(A\big)+\Pr\big(\text{accept}~\big|~\overline{A}\big)
  \cdot\Pr\big(\overline{A}\big)\\
  &\leq\Pr\big(\text{accept}~\big|~A\big)+
  \Pr\big(\text{accept}~\big|~\overline{A}\big)\\
  &\leq\Big(1-\frac{1}{q}\Big)+s.
  \end{align*}
  In the last inequality, $\Pr\big(\text{accept}~\big|~A\big)\leq 1-1/q$
  follows from the fact that we catch the provers' inconsistency with
  probability at least $1/q$, and
  $\Pr\big(\text{accept}~\big|~\overline{A}\big)\leq s$ follows from the
  soundness property applied to the proof defined by the second prover's
  responses.
  
  Without care, $s$ could remain large so the final inequality is useless. Fortunately, the original probabilistically checkable proof system can be amplified so
  that $s$ is exponentially small and $c$ is exponentially close to $1$. Then
  since $q$ is only polynomially large, the soundness $1-1/q+s$ of this
  multiple prover interactive proof system is non-negligibly less than its
  completeness $c$. We can repeat the whole protocol a few times and see
  whether the average number of accepted runs is less than or at least the
  midpoint between $1-1/q+s$ and $c$. Intuitively, there shouldn't be
  issues with adaptivity when we rerun the protocol from scratch since the
  queries made in previous runs give the provers no information about what will
  happen in future rounds. More formally, each possible execution of the
  first $k-1$ runs defines a proof for the original probabilistically
  checkable proof system, namely, the proof specified by the responses of
  the second prover in the $k$th run given the execution of the first $k-1$
  runs. The soundness and completeness properties apply to each of these
  exponentially many proofs corresponding to the different executions of the
  first $k-1$ runs. Now suppose the input is not in the language. Then the
  probability of acceptance on the $k$th run given any particular execution of
  the first $k-1$ runs is at most $1-1/q+s$ and hence the random variable
  indicating acceptance on the $k$th run is at most some indicator random
  variable that is $1$ with probability exactly $1-1/q+s$ conditioned on each
  execution of the first $k-1$ runs. It follows that the indicator random
  variables thus defined for each of the runs are fully independent and all
  have expectation $1-1/q+s$, so by a Chernoff bound the soundness is
  driven down exponentially. Similarly, the completeness is improved.
\end{proof}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% New section %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{The PCP Theorem}

By the nondeterministic time hierarchy theorem, we know that there exist
languages in $\NEXP$ that do not have short, efficiently verifiable proofs.
However, Theorems \ref{24:thm:mipequalsnexp} and \ref{24:thm:mippcp} imply
that all languages in $\NEXP$ have efficiently verifiable proofs, where we
allow the verifier to randomly spot check a few locations of the proof.
Can we scale this result down to get a similar result for $\NP$? That is,
can we replace classical proofs with ones where it is only necessary to
spot check a few random locations? This is the content of the famous
PCP Theorem, which is possibly the most celebrated and involved result
in all of theory of computing.

\begin{theorem}[{\bf The PCP Theorem}]
  $\NP=\PCP(O(\log{n}),O(1))$.
  \label{24:thm:pcp}
\end{theorem}

It is possible to get the $O(1)$ down to 3. The inclusion $\PCP(O(\log{n}),O(1))\subseteq\NP$ follows immediately from
Theorem \ref{24:thm:pcpntime}. The other direction is much harder. We do not
prove this result in this course.

\begin{theorem}
  $\NP\subseteq\PCP(poly(n),O(1))$.
  \label{24:thm:pcpweak}
\end{theorem}

In the probabilistically checkable proof system we design for Theorem
\ref{24:thm:pcpweak}, the proof is an exponentially long encoding of a
classical proof. Nevertheless, this result captures what is perhaps the most
surprising aspect of the PCP Theorem, namely that only a constant number of
bits of the proof need to be queried to verify its correctness.


\begin{theorem}
  $\NEXP\subseteq\PCP(poly(n), poly(n))$.
  \label{24:thm:pcpnexp}
\end{theorem}

The proof of this theorem involves using the PCP theorem, scaling $\NEXP$ to $\NP$ and scaling back.

\subsection{Next Time}

One of the major applications of the PCP Theorem is hardness of approximation
results. Many important combinatorial optimization problems are $\NP$-hard
to solve exactly, and for many of these problems the PCP Theorem implies
that it is $\NP$-hard even to approximate them within certain factors.
In fact, for some problems we have tight results, i.e., approximation
algorithms and hardness results with essentially matching approximation
factors. Thus the PCP Theorem has allowed us to characterize the
approximability of these problems. The basic reason the PCP Theorem leads
to hardness of approximation results is that the prover is trying to solve
an optimization problem --- that of maximizing the probability that the
verifier accepts --- and this theorem introduces a non-negligible gap in
this probability between the cases where the input is in the language and
not in the language.

In fact we have some bounds on some problems, where we can get within a factor $k$ of the optimal result, but if we can solve the problem for any smaller $k$ in polynomial time, then $\P = \NP$!

In the next lecture, we will see concrete examples of how the PCP Theorem
leads to hardness of approximation results. We will also prove Theorem
\ref{24:thm:pcpweak}. The proof involves encoding classical proofs
with the Hadamard code and using harmonic analysis to analyze the properties
of the resulting probabilistically checkable proof system.


\end{document}
