\documentclass[11pt]{article}
\include{lecture}
\begin{document}


% *************
% LECTURE
% *************
\lecture{1}{9/6/2011}{Model of Computation}{Jeff Kinne}


% ******** DRAFT *********
%\draft % put this here only if you want to indicate this is a draft


Today we begin a review of basic Complexity Theory material
that is typically covered in an undergraduate theory of computing course.
In this first lecture, we give a brief course overview and 
define the model of computation we 
use to formalize our intuitive notion of a computer.



% *******************
% SECTION
% *******************
\section{Course Overview}
This course provides a graduate-level introduction to computational complexity 
theory, the study of the power and limitations of efficient computation.

In the first part of the course we focus on the standard setting, in which one 
tries to realize a given mapping of inputs to outputs in a time- and 
space-efficient way. We develop models of computation that represent the 
various capabilities of digital computing devices, including parallelism, 
randomness, quantum effects, and non-uniformity. We also introduce models based 
on the notions of nondeterminism, alternation, and counting, which precisely 
capture the power needed to efficiently compute important types of mappings. 
The meat of this part of the course consists of intricate relationships between
these models, as well as some separation results.

In the second part we study other computational processes that arise in diverse
areas of computer science, each with their own relevant efficiency measures. 
Specific topics include:
\begin{itemize}
\item proof complexity, interactive proofs, and probabilistically checkable 
  proofs -- motivated by verification,
\item pseudorandomness and zero-knowledge -- motivated by cryptography and 
  security,
\item computational learning theory -- motivated by artificial intelligence,
\item communication complexity -- motivated by distributed computing,
\item query complexity -- motivated by databases. 
\end{itemize}

All of these topics have grown into substantial research areas in their own 
right. We will cover the main concepts and key results from each one.




% *******************
% SECTION
% *******************
\section{The Standard Setting}

\subsection{Machine Model}
The deterministic Turing machine is the model of computation we use to capture 
our intuitive notion of a computer.  A Turing machine is depicted in Figure
\ref{01:fig:TM}.  The finite control has a finite number states that it 
can be in at any time, and can read from and/or write to the various memory
tapes.  Based on the current state and the contents of the tapes, the finite
control changes its state and alters the contents of the tapes.

\begin{figure}[h]
  \centering
  \input{01.TM.pstex_t}
  \caption{An illustration of a deterministic Turing machine.  The finite
  control has read-only access to the input tape, write-only access to a 
  one-way output tape, and read-write access to a constant $k$ many work 
  tapes.}
  \label{01:fig:TM}
\end{figure}
 
\begin{definition}[sequential access Turing machine]
  A \emph{sequential access deterministic Turing machine}
  $M$ is defined by a tuple: 
  $M=(Q,\Sigma,\Gamma,q_{start},q_{halt},\delta)$, where $Q$ is a finite set of 
  possible states of the finite control, $\Sigma$ is a finite input and 
  output alphabet,
  $\Gamma$ is a finite work-tape alphabet, $q_{start}$ is the start
  state, $q_{halt}$ is the halt state, and $\delta$ is the finite control's
  transition function.

  The transition function has the form 
  $$\delta:Q\setminus \{q_{halt}\}\times \Sigma \times \Gamma^k \rightarrow
  Q\times \{\Sigma \cup \{\epsilon\}\}\times \Gamma^k \times
  \{L,R\} \times \{L,R\}^k.$$  The input to $\delta$ represents the current
  state, the current symbol being scanned on the input tape, and the current
  symbol being scanned on each work tape.  The output represents
  the next state of the finite control, a symbol to write to the 
  output tape (possibly empty), symbols to write to each work tape, and 
  which direction to move the head on the input and work tapes.

  Both $\Sigma$ and $\Gamma$ contain a blank symbol, which is used to
  denote empty tape cells. We often use the binary alphabet, which in 
  addition consists of the symbols 0 and 1.
  \label{def:TM}
\end{definition}

We think of the use of a Turing machine as consisting of three steps.  First,
the machine is initialized as follows: place input on input tape with tape 
head on first symbol of input, work tapes are empty with their tape heads on 
the left-most cell of the tape, the output tape is empty, and the finite 
control is set to $q_{start}$.  Second, the machine is allowed to run one step 
at a time by repeatedly applying the transition function $\delta$.  
Third, if the 
machine ever halts by entering $q_{halt}$, the computation is finished and 
we can read the output from the output tape.  We use $M(x)$ to denote the 
output of $M$ on input $x$ when this computation halts.

At first sight, the definition of a Turing machine may seem to be too 
restrictive to correspond to our intuitive notion of computing.
However, the Turing machine has been shown to be
just as powerful and roughly as efficient as traditional computers (we 
discuss this more later in this lecture).

The model of Turing machine we use in this course is a modification of the 
definition given above.  The definition above is a bit too restrictive as
it does not allow indirect memory addressing, which real computers rely on.
\begin{definition}[random access Turing machine]
  A \emph{random access Turing machine}
  is a Turing machine that functions as in Definition \ref{def:TM} with 
  regards to its output and sequential access work tapes.  The input
  tape and any fixed number of work tapes may be \emph{random access tapes}
  rather than sequential access tapes.
  Each random access tape has an associated sequential access 
  \emph{index work tape}.
  The tape head of a random access tape is moved by a special jump operation
  that moves its tape head to the location specified by its index tape and
  re-initializes the index tape.
  The tape head position of a random access tape is not altered by the 
  transition 
  function, but the contents of the memory cell are read and written by 
  the transition function.
\end{definition}
Notice that each tape is either a sequential tape or a random access
tape but not both.  We think of the sequential access tapes as performing
operations that are usually performed in registers on modern computers such 
as arithmetic, while the random access tapes are used for storage.  We 
choose the random access Turing machine as our basic model of computation for 
this course as it more closely models modern computers than sequential access
Turing machines.


% *******************
% SECTION
% *******************
\subsection{Computing a Relation}
The main setting in the first part of the course is the Turing machine's 
ability to compute various relations.
\begin{definition}
  Given a \emph{relation} $R\subset \Sigma^*\times\Sigma^*$, we say that a 
  Turing machine $M$ \emph{computes} $R$ if the following holds.
  For all inputs $x\in\Sigma^*$, $M$ on input
  $x$ halts and outputs a $y\in\Sigma^*$ such that $(x,y)\in R$ if such a $y$ 
  exists and indicates ``no'' if no such $y$ exists.
\end{definition}

Indicating no can be accomplished by some encoding scheme on the output or 
by having two different halt states.

As an example relation, consider the shortest path problem.  For this relation,
we wish to compute a shortest path between two vertices in a graph.  Here, 
the input is a description of the graph and source and destination vertices;
the output is the description of a path through the graph.  Notice that 
a function is a special case of a relation, where for a given $x$ there is 
at most one $y$ such that $(x,y)\in R$.  Factoring is an example, where 
the input is an integer $n$, and the output is the unique prime factorization
of $n$ listing the prime factors from smallest to largest.

We are often interested in a particular kind of relation, called
a decision problem.
\begin{definition}[decision problem]
  A \emph{decision problem} 
  is a relation where the output is always either 1 or 0.
  In this case, an output of 1 indicates that the input has some property, and
  0 indicates the input does not have the property.   We refer to the 
  set of inputs with corresponding output 1 as a \emph{language}.
\end{definition}

As an example, consider the problem of determining if a string is a 
palindrome.
We define the language PALINDROMES as the set of strings that read the same
front to back as they do from back to front.  The corresponding relation 
assigns 1 to each palindrome and 0 to each non-palindrome.  

It is often the 
case that the complexity of computing a relation is captured by the complexity
of computing a related decision problem.  For example, we can turn the 
factoring problem into a decision problem by trying to compute the 
$i^{th}$ bit of the output.  That is, on input $\langle n,i\rangle$, we 
try to compute the $i^{th}$ bit of the description of the prime factorization
of $n$.



% *******************
% SECTION
% *******************
\section{Time and Space Complexity}
The Turing machine was originally defined and used in the theory of 
computability (also known as recursion theory).  In this setting, the goal 
is to determine which relations are computable.
For example, a famous early result is that the Halting
Problem is not computable by any Turing machine.  It may seem surprising at 
first that there are uncomputable relations.  A closer inspection makes this 
fact obvious: there are only countably many Turing machines, while 
there are uncountably many different relations.

In contrast to the setting of computability, complexity theory is concerned
with the efficiency with which a relation is computable.  To this end, we 
consider the amount of resources a Turing machine uses during its computation.
The two standard resources to consider are time and space.
\begin{definition}
  Let $M$ be a Turing machine and $x$ an input to $M$.  Then 
  $$
  \begin{array}{l}
    \textrm{$t_M(x)$ =  the number of steps until $M$ halts on input $x$,}\\
    t_M(n) = \max(t_M(x) | x\in \Sigma^n), \\
    \textrm{$s_M(x)$ =  the sum over all work tapes of the maximum cell 
      touched until $M$ halts,}\\
    s_M(n) = \max(s_M(x) | x\in \Sigma^n).
  \end{array}
  $$
\end{definition}
$t_M(x)$ corresponds to the time used by $M$ on input $x$, while 
$s_M(x)$ corresponds to the amount of memory used.  A possible alternative
to the definition we have given for $s_M(x)$ would be to only count the 
number of work-tape cells that are touched.  Our definition counts all cells
that are left of some work-tape cell that is touched, and hence counts 
even unused cells that are left of some used cell.  
Our definition is 
more natural from the perspective that if you want to run the
algorithm on a computer, you'll need one with that much memory. If an
algorithm uses 10K space with the alternative definition, it may not run on a
machine with 10K of RAM, at least not without modification; it does with
the our definition.  Also notice that the configuration of a machine 
(the positions of its tapes, contents of its work tapes, and its internal 
state) can be described using $O(s_M + \log(|x|))$ space, while this is not 
true with the alternative definition.  For most
purposes, the choice in definition does not effect the power or efficiency 
of the model.

We are often interested
in the worst-case complexity of a Turing machine, and hence have defined the 
worst-case measures $t_M(n)$ and $s_M(n)$.  These correspond to the worst
performance of $M$ on any input of length $n$.

Notice in the definition of $s_M(x)$ that we do not count the input tape 
or output tape memory cells that are used.  This choice in definition is the 
reason we distinguished between the different types of tapes in the first 
place, and allows us to consider Turing machines that compute non-trivial
relations using sub-linear space.  A definition including input tape usage
in $s_M(x)$ would preclude non-trivial sub-linear space algorithms as then
the entire input could not even be read.  We will consider algorithms that 
use at least logarithmic space as this is the amount required to index
into the input.  For time usage, linear is the smallest we consider as we 
must at least read the entire input.

\end{document}
