\documentclass[11pt]{article}
\include{lecture}
\usepackage{hyperref}

\begin{document}

% ********************
% LECTURE
% ********************
\lecture{11}{2/23/2010}{Randomness}{Brian Rice \& Jake Rosin}

% ****** DRAFT **********
%\draft % put this here only if you want to indicate this is a draft

% ******* ********* ********* ********* ********* ********* ********* ********* |



Last time we discussed parallelism as an extension of our existing model of
computation.  Today we complete that discussion. Then, we will introduce
randomness into our computational model.  We will show the qualitative 
usefulness of randomness through efficient probabilistic algorithms for a 
few difficult problems, and then quantify its power in comparison to 
existing computational classes.

\section{Last Bits of Parallellism}

Last time we left off the last step in proving the equivalence of several
computational classes.

\begin{proposition}Log-depth formulas can be simulated by bounded--width 
branching programs of polynomial size.\end{proposition}

\begin{proof}
The BP $M$ we construct has the following properties:
\begin{itemize}
\item The width of $M$ is 5.
\item The label of each node depends only on its layer, that is, $M$ is an 
\emph{oblivious BP}.
\item Between any two levels, all of the 0--branches have distinct end states 
and all of the 1--branches have distinct end states. Since the width of $M$ is 
constant, this makes $M$ a \emph{permutation BP}.
\item Overall, the effect of $M$ is the identity permutation $e$ if its input 
is rejected, or a given 5-cycle $\pi$ if its input is accepted.  We say that 
such a BP is \emph{$\pi$--accepting}.
\end{itemize}

We will construct $M$ recursively.  That is, the proof is by induction on 
formula size.  The base case is for individual input variables.  This is 
easy, as we just let the 0--branches from the top layer to the second 
(bottom) layer form the identity permutation $e$ and the 1--branches form the 
permuation $\pi$.

We will use the following results to recursively construct a polynomial size
BP of the required type for a given log-depth formula.

\begin{claim} If there exists a $\pi$--accepting BP for the formula $\phi$ of 
size $s$, for \emph{some} 5-cycle $\pi$, then there exists a 
$\sigma$--accepting BP of size $s$ for $\phi$ for \emph{any} 5-cycle 
$\sigma \ne e$.\end{claim}
\begin{proof}
In the symmetric group, every 5-cycle $\sigma$ is conjugate to $\pi$; that is,
there exists a permutation $\gamma$ such that $\sigma = \gamma^{-1}\pi\gamma$.
So, to construct a $\sigma$--acceptor from a given $\pi$--acceptor, we need 
only topermute the machine's top-layer nodes by $\gamma^{-1}$ and the 
bottom-layer nodes by $\gamma$.  This does not change the size of the program.
\end{proof}
\begin{corollary} If we can decide the formula $\phi$ using a $\pi$--acceptor
of size $s$, then we can decide the formula $\neg\phi$ using a 
$\pi$--acceptor of size $s$.\end{corollary}
\begin{proof}
Build a $\pi^{-1}$--acceptor that decides $\phi$.  We can do this by the
previous claim. Apply $\pi$ to the last row to obtain a $\pi$--acceptor
that decides $\neg\phi$.  Again, this does not change the size $s$.\end{proof}

\begin{claim} If there exists a $\pi$--acceptor of size $s_\phi$ that decides 
the formula $\phi$ and there exists a $\sigma$--acceptor of size $s_\psi$ that
decides the formula $\psi$, and $\tau = \pi^{-1}\sigma^{-1}\pi\sigma \ne e$,
then there exists a $\tau$--acceptor of size $2(s_A + s_B)$ that decides the
formula $\phi \wedge \psi$.  
\end{claim}
\begin{proof} Suppose that $M_\phi$ is the $\pi$--acceptor deciding $\phi$, 
and $M_B$ is the $\sigma$--acceptor deciding $\psi$.  Let $M_{\phi\wedge \psi}$
be the machine formed by concatenating $M_{\phi}^{-1}$, $M_{\psi}^{-1}$, 
$M_\phi$, and $M_\psi$, in that order, start-to-finish. ($M_\phi^{-1}$ is a 
$\pi^{-1}$--acceptor deciding $\phi$, and $M_\psi^{-1}$ is analogous. 
The previous claim shows how to build these.)

Consider the fate of input $x$ fed to $M_{\phi\wedge\psi}$.  If 
$\phi(x)\wedge \psi(x)$ is true, then on input $x$ we obtain the permutation
$\pi^{-1}$ from $M_\phi^{-1}$, followed by $\sigma^{-1}$ from $M_\psi^{-1}$, 
followed by $\pi$ from $M_\phi$, followed by $\sigma$ from $M_\psi$. Because 
$\tau = \pi^{-1}\sigma^{-1}\pi\sigma$, the net permutation that $x$ undergoes
is $\tau$.

If, instead, $\psi(x)$ is false, then $M_\psi^{-1}$ and $M_\psi$ perform the 
identity permutation on input $x$. Therefore, the net permutation experienced 
on input $x$ is $\pi^{-1}\pi = e$ if $\phi(x)$ is true, and just $e$ if not.
In any case $x$ is rejected. The case where $\phi(x)$ is false is similar.
So, the machine $M_{\phi\wedge\psi}$ accepts precisely those $x$ such that  
$\phi(x)\wedge\psi(x)$ is true -- that is, it decides $\phi\wedge\psi$,
and the size of the machine is $2(s_\phi + s_\psi)$.  \end{proof}

There exist 5-cycles $\pi$ and $\sigma$ so that $\tau \ne e$, for example, 
let $\pi = (1 2 3 4 5), \sigma = (1 3 5 4 2), $ and $\tau = (1 2 5 3 4)$. Thus,
we can combine branching programs to simulate AND gates; and since 
we've already seen how to simulate NOT gates, we can simulate 
OR gates by De Morgan's laws.

So, consider the standard post-order formula traversal that these constructions 
suggest. At each level, our branching program may increase by no more than a 
factor of two. If the given formula has depth $d$, then the branching program 
we construct has size $O(2^d)$.  Since $d$ is $O(\log n)$, the size of the 
branching program is polynomial in $n$.
\end{proof}

% *******************
% SECTION
% *******************
\section{Motivation for Randomness}
Randomness appears to have great value as a computational tool.  It simplifies
problems in a large number of settings.  The use of randomness also seems 
essential to cryptography, where the use of a deterministic method to hide a 
secret means that it could then be deterministically found by an adversary.
Randomness in a cryptographic setting will be covered in upcoming lectures.

Here we deal with randomness in the standard setting: that of realizing a mapping
from inputs to outputs; for example, decision problems.  Intuitively randomness
may not seem useful in such a setting, but there are certain problems which may
be solved more efficiently using randomness than by any known deterministic
algorithm.  However, it is not known whether randomness truly provides
additional computing power, as we will see.

\section{Concept}
\label{rand:section:2}
To make use of randomness we allow a Turing machine to flip coins and base 
decisions on the outcome of those flips.  The configuration of the machine 
at a given time therefore becomes a random variable based on coin flips.  
As the outcome is dictated by the final configuration of the machine, it 
too becomes a random variable.  In the context of decision problems, this 
introduces the possibility of an incorrect result.

Our aim is that the probability of an error be made negligible, but useful 
results can be easily generated by a machine which errs with probability 
nontrivially less than $\frac{1}{2}$ by running the machine repeatedly and 
taking the majority vote of its outcomes.  Given a machine which produces 
an erroneous result with probability $\epsilon = \frac{1}{2} - \delta$, 
we can take the majority vote of the results from a number of independent runs.
This allows us to ensure that the error is as small as we wish.  For $k$
independent runs, we have

\begin{equation}
\label{11:eqn:prerror}
\text{Pr[Majority vote is wrong]}
 = \sum_{i=\frac{k}{2}}^k 
     {k \choose i} \epsilon^i(1 - \epsilon)^{k-i}
 \leq 2^k(\epsilon(1 - \epsilon))^{\frac{k}{2}}
 \leq (1 - 4\delta)^{\frac{k}{2}}
 \leq e^{-2k\delta^2}
\end{equation}

This shows it is possible to produce an exponentially small probability of error
through majority vote of a polynomial number of runs, provided that each run
has probability of error bounded away from $\frac{1}{2}$.

There are different ways to allow error in a randomized computation, and which
type of error is allowed may affect the class of languages that can be 
computed.  
\begin{itemize}
\item 2-sided error: our algorithm can have both false positives (invalid input
accepted) and false negatives (valid input is rejected).
\item 1-sided error: error possible on only one side; typically we have false 
negatives but not false positives. Invalid input is always rejected, while 
valid input is rejected with small probability.
\item 0-sided error: never provides an incorrect answer, but may answer 
``unknown'' with small probability.
\end{itemize}

The computation above, using majority vote to make the error exponentially 
small, works for algorithms with 2-sided error.

Randomized quick sort is an example of a  0-sided algorithm.  Quick sort is even
guaranteed to produce the correct result: it will never answer ``unknown.''
The behavior of quick sort is affected by randomness, however, as the time, 
space, and other aspects of program behavior become random variables.

% *****************************************
% SECTION
% *****************************************

\section{Randomized Algorithms}
The goal when randomness is introduced is to obtain solutions with more efficient
algorithms than are known deterministically, with efficiency measured in time or
space.

\subsection{Sequential Time Efficiency}
\subsubsection{Polynomial Identity Testing}
\emph{Polynomial identity testing} is the following problem.
Given an arithmetic formula $\varphi$ composed of addition, subtraction,
multiplication, brackets and variables, the problem is to determine if 
$\varphi \equiv 0$.  Solving this problem deterministically appears difficult;
one method involves expanding all terms and comparing monomials, but the number
of monomials may be exponential in the length of the formula.  In fact, all 
known deterministic algorithms run in exponential time.

A randomized solution is formed by choosing values $\bar{x}$ for all variables
at random and evaluating the formula.  If the result $\phi(\bar{x}) \neq 0$ the 
formula is rejected; if $\phi(\bar{x})= 0$, then we guess $\phi \equiv 0$, but
may be in error with some probability.  A bound on the probability of an error
can be determined as below.  Given that values for each variable $x_i$ are 
chosen uniformly at random from some set $I$, with $d$ 
being the degree of $\varphi$, we have the conditional probability:

\begin{equation}
\label{11:eqn:prpolyident}
\text{Pr[}
  \varphi(x_1,\ldots x_n) = 0 | \varphi \not\equiv 0
\text{]}
  \leq \frac{d}{|I|}
\end{equation}

Provided that the presentation of our polynomial is given as a formula (rather
than as an arithmetic circuit; in that case, see the next example), the degree
of a formula is bounded by its length, so $d \leq N$.  The interval $I$ should
be chosen such that $\frac{d}{|I|}$ is sufficiently small.  
We choose $I$ so that $|I|$ is some polynomial in $N$, thus ensuring
that $\frac{d}{|I|}$ is small, but that each value in $I$ is still specified
with $O(\log N)$ bits. Evaluating the formula at worst
raises the variables to the power $N$, meaning the resulting numbers have 
$O(N\log{N})$ bits.  All the arithmetic operations involved may be performed in
polynomial time in the bit length of these numbers.  This forms a 1-sided
randomized algorithm for polynomial identity testing, which errs on 
nonmembers with small probability (producing false positives).

As we noted, no efficient determininstic algorithm for this problem is known.
In fact, a subexponential deterministic algorithm would imply nontrivial lower
bounds for circuit complexity.

If our polynomial is presented via an arithmetic circuit, much of the above
can be made to work, but it is not easy to determine whether
$\phi(\bar{x}) = 0$, since $\phi$ may have degree exponential in the length
of the input.  Thus, in order to solve the problem in that case, we need
an efficient algorithm for the arithmetic circuit value problem.

% ******* ********* ********* ********* ********* ********* ********* *********
\subsubsection{Arithmetic Circuit Value}

\emph{Arithmetic circuit value} is the following problem: given an arithmetic
circuit and inputs, does it evaluate to zero?  
Because the degree of the corresponding polynomial can be exponential in the 
depth of the circuit, exact calculation uses $2^{\poly{N}}$ bits, which
prevents evaluation in polynomial time.  The solution is to perform all 
calculations modulo some random number $m$ of at most $\poly{N}$ bits.  
This allows evaluation in polynomial time, since we need only work with a
number of bits equal to the number of bits in $m$ but introduces another 
source of error; namely, when our result is zero modulo $m$, but not in
fact equal to zero.

If the value is in fact nonzero, there are polynomimally many prime numbers
$p$ which produce a zero incorrectly when the computation is performed 
modulo $p$.  These are the prime factors of the actual value of the circuit.
In order to reduce the probability of an error, we would therefore like to
test modulo polynomially many different primes $p$.  This could easily be
achieved if we had some way of choosing random primes; however, to do this
is not an easy problem.  Instead, since working modulo composite numbers
still cannot give us false negatives (that is, if the value is zero, it must
be zero modulo every $m$), we simply choose enough random numbers to ensure
that, with high probability, sufficiently many of them are primes.  We choose
numbers with number of bits polynomial in $N$ at random.  From the prime number
theorem, we have that the prime counting function 
$\pi(x) := \{p<x\ :\ p\ \mathrm{prime}\}$ satisfies 
\begin{equation}
\label{11:eqn:pnt}
\pi(x) \sim \frac{x}{\ln{x}}.
\end{equation}
This tells us that a randomly chosen $M$-bit number is prime with probability
approaching $\frac{1}{M}$.  Hence we can choose polynomially many numbers with
$\poly(N)$ bits at random and, with high probability, have polynomially many of
them be prime (for appropriate choices of the polynomials).  Thus, if the
value of our arithmetic circuit is zero modulo all of our $m$, then it is
equal to zero with high probability.

The result is a 1-sided algorithm with Pr[correct]$ = \frac{1}{\poly}$
which errs on nonmembers.  Running the algorithm multiple times can increase the
confidence in the result.  As was the case for polynomial identity testing, 
the best known deterministic algorithm for arithmetic circuit testing runs
in exponential time.


\subsection{Parallel Time Efficiency}

Some problems have polynomial time determinisitve algorithms, but these
algorithms are inherently sequential, and there may be no known efficient
parallel algorithms for solving the problems.  In these cases, we can
sometimes improve the efficiency by finding an efficient parallel algorithm
using randomness.

The problem of determining the \emph{existence of a perfect matching in 
bipartite graphs} is such a problem.  An inherently sequential polynomial
time algorithm exists, but there is no known efficient deterministic
parallel algorithm.  However, we can find one using randomness.

A bipartite graph $G$ can be represented as an $N \times N$ adjacency matrix.
We then replace every `1' in the adjacency matrix with a random variable 
unique to its location.  For example:

\begin{equation}
\label{11:eqn:adjacency}
M = \left[ \begin{array}{ccccc}
x_{11} & 0 & 0 & x_{14} & \cdots \\
0 & x_{22} & 0 & 0 & \cdots \\
\vdots & \vdots & \vdots & \vdots & \ddots \end{array} \right]
\end{equation}

\begin{claim}
$G$ has a perfect matching iff Det($M$)$ \not \equiv 0$.
\end{claim}
\begin{proof}
One term in the determinant exists for every possible permutation, and 
permutations are in a 1-to-1 correspondence with potential perfect matchings.
Thus the determinant contains exactly one term for every potential perfect
matching; the coefficient of that term is zero if that matching
is not realizable in the graph, and $1$ if it is.  Different perfect matchings
lead to different monomials, which if they have a non-zero coefficient
are realizable for the graph.  Since all the monomials are distinct (and so no
cancellation can occur, the polynomial is not identically zero if and only if
$G$ has a perfect matching.
\end{proof}

The determinant is a multi-variate polynomial of degree at most $N$.
Checking for a perfect matching can be performed by checking if
that polynomial is $\equiv 0$, which can be accomplished
using the randomized algorithm above.  Unfortunately, this would not be a 
parallel algorithm.  However, this is not just any polynomial; its values
are the values of the determinants of the matrices obtained by substituting
in numbers for the variables in the matrix.  Computing the determinant
is a linear algebra problem, which, as we noted in the last lecture, is in
$\NC^2$. We use this fact to devise an $\NC^2$ algorithm:

\begin{itemize}
\item Replace all `1's in the adjacency matrix with a random value in a 
  suitable interval $I=[0,\poly(N)]$.
\item Compute the determinant in parallel.
\end{itemize}

If the result is non-zero, then a perfect matching exists.  If it is zero, the
algorithm can be repeated until we become sufficiently confident in the result.

\subsection{Space Efficiency}
The \emph{undirected path problem} was until recently most efficiently
solved by a randomized algorithm.  We now have an efficient
deterministic $\L$ solution, but the problem is included as a
well-known example.  Note that the related directed path problem
is $\NL$-complete.

Given an undirected graph $G$ and two vertices $s$ and $t$, the 
problem is to determine if a path exists from $s$ to $t$.  The
randomized algorithm is a random walk beginning from $s$.  This
walk ends after a certain number of steps, or immediately if
$t$ is reached.  Performing a polynomial number of steps
reduces to a small probability the chance of not reaching $t$
if $s$ and $t$ are connected.

This algorithm can be performed in logarithmic space, storing
the following:
\begin{itemize}
\item Current location
\item Destination $t$
\item The number of steps taken - at most polynomially large, and so
representable in logarithmic bits.
\end{itemize}

This gives us a logarithmic space randomized algorithm with 1-sided error.

The efficient deterministic solution to this problem uses expanders, which
will be discussed in a future lecture.

\section{Next Lecture}
In the next lecture we will continue our examination of randomness by 
formally expanding our Turing machine model, defining randomized complexity 
classes, and examining their relation to deterministic and nondeterministic 
complexity classes.


\end{document}