\documentclass[11pt]{article}
\include{lecture}
\begin{document}
% *************
% LECTURE
% *************
\lecture{1}{9/6/2011}{Model of Computation}{Jeff Kinne}
% ******** DRAFT *********
%\draft % put this here only if you want to indicate this is a draft
Today we begin a review of basic Complexity Theory material
that is typically covered in an undergraduate theory of computing course.
In this first lecture, we give a brief course overview and
define the model of computation we
use to formalize our intuitive notion of a computer.
% *******************
% SECTION
% *******************
\section{Course Overview}
This course provides a graduate-level introduction to computational complexity
theory, the study of the power and limitations of efficient computation.
In the first part of the course we focus on the standard setting, in which one
tries to realize a given mapping of inputs to outputs in a time- and
space-efficient way. We develop models of computation that represent the
various capabilities of digital computing devices, including parallelism,
randomness, quantum effects, and non-uniformity. We also introduce models based
on the notions of nondeterminism, alternation, and counting, which precisely
capture the power needed to efficiently compute important types of mappings.
The meat of this part of the course consists of intricate relationships between
these models, as well as some separation results.
In the second part we study other computational processes that arise in diverse
areas of computer science, each with their own relevant efficiency measures.
Specific topics include:
\begin{itemize}
\item proof complexity, interactive proofs, and probabilistically checkable
proofs -- motivated by verification,
\item pseudorandomness and zero-knowledge -- motivated by cryptography and
security,
\item computational learning theory -- motivated by artificial intelligence,
\item communication complexity -- motivated by distributed computing,
\item query complexity -- motivated by databases.
\end{itemize}
All of these topics have grown into substantial research areas in their own
right. We will cover the main concepts and key results from each one.
% *******************
% SECTION
% *******************
\section{The Standard Setting}
\subsection{Machine Model}
The deterministic Turing machine is the model of computation we use to capture
our intuitive notion of a computer. A Turing machine is depicted in Figure
\ref{01:fig:TM}. The finite control has a finite number states that it
can be in at any time, and can read from and/or write to the various memory
tapes. Based on the current state and the contents of the tapes, the finite
control changes its state and alters the contents of the tapes.
\begin{figure}[h]
\centering
\input{01.TM.pstex_t}
\caption{An illustration of a deterministic Turing machine. The finite
control has read-only access to the input tape, write-only access to a
one-way output tape, and read-write access to a constant $k$ many work
tapes.}
\label{01:fig:TM}
\end{figure}
\begin{definition}[sequential access Turing machine]
A \emph{sequential access deterministic Turing machine}
$M$ is defined by a tuple:
$M=(Q,\Sigma,\Gamma,q_{start},q_{halt},\delta)$, where $Q$ is a finite set of
possible states of the finite control, $\Sigma$ is a finite input and
output alphabet,
$\Gamma$ is a finite work-tape alphabet, $q_{start}$ is the start
state, $q_{halt}$ is the halt state, and $\delta$ is the finite control's
transition function.
The transition function has the form
$$\delta:Q\setminus \{q_{halt}\}\times \Sigma \times \Gamma^k \rightarrow
Q\times \{\Sigma \cup \{\epsilon\}\}\times \Gamma^k \times
\{L,R\} \times \{L,R\}^k.$$ The input to $\delta$ represents the current
state, the current symbol being scanned on the input tape, and the current
symbol being scanned on each work tape. The output represents
the next state of the finite control, a symbol to write to the
output tape (possibly empty), symbols to write to each work tape, and
which direction to move the head on the input and work tapes.
Both $\Sigma$ and $\Gamma$ contain a blank symbol, which is used to
denote empty tape cells. We often use the binary alphabet, which in
addition consists of the symbols 0 and 1.
\label{def:TM}
\end{definition}
We think of the use of a Turing machine as consisting of three steps. First,
the machine is initialized as follows: place input on input tape with tape
head on first symbol of input, work tapes are empty with their tape heads on
the left-most cell of the tape, the output tape is empty, and the finite
control is set to $q_{start}$. Second, the machine is allowed to run one step
at a time by repeatedly applying the transition function $\delta$.
Third, if the
machine ever halts by entering $q_{halt}$, the computation is finished and
we can read the output from the output tape. We use $M(x)$ to denote the
output of $M$ on input $x$ when this computation halts.
At first sight, the definition of a Turing machine may seem to be too
restrictive to correspond to our intuitive notion of computing.
However, the Turing machine has been shown to be
just as powerful and roughly as efficient as traditional computers (we
discuss this more later in this lecture).
The model of Turing machine we use in this course is a modification of the
definition given above. The definition above is a bit too restrictive as
it does not allow indirect memory addressing, which real computers rely on.
\begin{definition}[random access Turing machine]
A \emph{random access Turing machine}
is a Turing machine that functions as in Definition \ref{def:TM} with
regards to its output and sequential access work tapes. The input
tape and any fixed number of work tapes may be \emph{random access tapes}
rather than sequential access tapes.
Each random access tape has an associated sequential access
\emph{index work tape}.
The tape head of a random access tape is moved by a special jump operation
that moves its tape head to the location specified by its index tape and
re-initializes the index tape.
The tape head position of a random access tape is not altered by the
transition
function, but the contents of the memory cell are read and written by
the transition function.
\end{definition}
Notice that each tape is either a sequential tape or a random access
tape but not both. We think of the sequential access tapes as performing
operations that are usually performed in registers on modern computers such
as arithmetic, while the random access tapes are used for storage. We
choose the random access Turing machine as our basic model of computation for
this course as it more closely models modern computers than sequential access
Turing machines.
% *******************
% SECTION
% *******************
\subsection{Computing a Relation}
The main setting in the first part of the course is the Turing machine's
ability to compute various relations.
\begin{definition}
Given a \emph{relation} $R\subset \Sigma^*\times\Sigma^*$, we say that a
Turing machine $M$ \emph{computes} $R$ if the following holds.
For all inputs $x\in\Sigma^*$, $M$ on input
$x$ halts and outputs a $y\in\Sigma^*$ such that $(x,y)\in R$ if such a $y$
exists and indicates ``no'' if no such $y$ exists.
\end{definition}
Indicating no can be accomplished by some encoding scheme on the output or
by having two different halt states.
As an example relation, consider the shortest path problem. For this relation,
we wish to compute a shortest path between two vertices in a graph. Here,
the input is a description of the graph and source and destination vertices;
the output is the description of a path through the graph. Notice that
a function is a special case of a relation, where for a given $x$ there is
at most one $y$ such that $(x,y)\in R$. Factoring is an example, where
the input is an integer $n$, and the output is the unique prime factorization
of $n$ listing the prime factors from smallest to largest.
We are often interested in a particular kind of relation, called
a decision problem.
\begin{definition}[decision problem]
A \emph{decision problem}
is a relation where the output is always either 1 or 0.
In this case, an output of 1 indicates that the input has some property, and
0 indicates the input does not have the property. We refer to the
set of inputs with corresponding output 1 as a \emph{language}.
\end{definition}
As an example, consider the problem of determining if a string is a
palindrome.
We define the language PALINDROMES as the set of strings that read the same
front to back as they do from back to front. The corresponding relation
assigns 1 to each palindrome and 0 to each non-palindrome.
It is often the
case that the complexity of computing a relation is captured by the complexity
of computing a related decision problem. For example, we can turn the
factoring problem into a decision problem by trying to compute the
$i^{th}$ bit of the output. That is, on input $\langle n,i\rangle$, we
try to compute the $i^{th}$ bit of the description of the prime factorization
of $n$.
% *******************
% SECTION
% *******************
\section{Time and Space Complexity}
The Turing machine was originally defined and used in the theory of
computability (also known as recursion theory). In this setting, the goal
is to determine which relations are computable.
For example, a famous early result is that the Halting
Problem is not computable by any Turing machine. It may seem surprising at
first that there are uncomputable relations. A closer inspection makes this
fact obvious: there are only countably many Turing machines, while
there are uncountably many different relations.
In contrast to the setting of computability, complexity theory is concerned
with the efficiency with which a relation is computable. To this end, we
consider the amount of resources a Turing machine uses during its computation.
The two standard resources to consider are time and space.
\begin{definition}
Let $M$ be a Turing machine and $x$ an input to $M$. Then
$$
\begin{array}{l}
\textrm{$t_M(x)$ = the number of steps until $M$ halts on input $x$,}\\
t_M(n) = \max(t_M(x) | x\in \Sigma^n), \\
\textrm{$s_M(x)$ = the sum over all work tapes of the maximum cell
touched until $M$ halts,}\\
s_M(n) = \max(s_M(x) | x\in \Sigma^n).
\end{array}
$$
\end{definition}
$t_M(x)$ corresponds to the time used by $M$ on input $x$, while
$s_M(x)$ corresponds to the amount of memory used. A possible alternative
to the definition we have given for $s_M(x)$ would be to only count the
number of work-tape cells that are touched. Our definition counts all cells
that are left of some work-tape cell that is touched, and hence counts
even unused cells that are left of some used cell.
Our definition is
more natural from the perspective that if you want to run the
algorithm on a computer, you'll need one with that much memory. If an
algorithm uses 10K space with the alternative definition, it may not run on a
machine with 10K of RAM, at least not without modification; it does with
the our definition. Also notice that the configuration of a machine
(the positions of its tapes, contents of its work tapes, and its internal
state) can be described using $O(s_M + \log(|x|))$ space, while this is not
true with the alternative definition. For most
purposes, the choice in definition does not effect the power or efficiency
of the model.
We are often interested
in the worst-case complexity of a Turing machine, and hence have defined the
worst-case measures $t_M(n)$ and $s_M(n)$. These correspond to the worst
performance of $M$ on any input of length $n$.
Notice in the definition of $s_M(x)$ that we do not count the input tape
or output tape memory cells that are used. This choice in definition is the
reason we distinguished between the different types of tapes in the first
place, and allows us to consider Turing machines that compute non-trivial
relations using sub-linear space. A definition including input tape usage
in $s_M(x)$ would preclude non-trivial sub-linear space algorithms as then
the entire input could not even be read. We will consider algorithms that
use at least logarithmic space as this is the amount required to index
into the input. For time usage, linear is the smallest we consider as we
must at least read the entire input.
\end{document}