---------------------------------------------------------------------
CS 577 (Intro to Algorithms)
Lec 12 (10/12/06) Shuchi Chawla
---------------------------------------------------------------------
All-pairs shortest paths
========================
In today's lecture we saw how to solve the single-source shortest
paths problem using the dynamic programming based Bellman-Ford
algorithm. Bellman is largely credited for inventing dynamic
programming. See Section 6.8 in the book for details of this
algorithm. We will now discuss a generalization of this algorithm.
Say we want to compute the length of the shortest path between *every*
pair of vertices. We could do this by running Bellman-Ford n times,
once for possible start s. This would take O(mn^2) time. Can we do any
better? We will see next an O(n^3 log n) time algorithm for this
problem. When m is large, O(mn^2) is larger than O(n^3 log n).
The key to speeding up the algorithm is to do all the n runs of the
algorithm simultaneously!
Recall that in Bellman-Ford, we keep an array of the length of the
currently best path from any node to the start s. In order to compute
all-pairs shortest paths, we maintain n such arrays simultaneously. In
other words, we maintain an n by n array (or matrix), where the
(i,j)th entry at step k denotes the length of the best path between i
and j using at most k edges.
Given graph G, we can initialize matrix A_1(G) as follows:
- A_1[i,i] = 0 for all i.
- if there is an edge from i to j, then A_1[i,j] = length of that edge.
- otherwise, A_1[i,j] = infinity.
I.e., A_1[i,j] = length of shortest path from i to j using 1 or fewer edges.
This matrix A_1 is also called the "adjacency matrix" of the graph.
Now, following the basic Dynamic Programming idea, can we use this to
produce a new matrix A_2 where A_2[i,j] = length of the shortest path
from i to j using 2 or fewer edges?
Answer: yes. A_2[i,j] = min_k (A_1[i,k] + A_1[k,j])
[think about why this is true]
Note that computing A_2 is much like multiplying A_1 by itself, except
that we change "*" to "+" and we change "+" to "min". It therefore
takes O(n^3) time to compute A_2.
Next, we can compute a matrix A_3 where A_3[i,j] = length of the shortest
path from i to j using 3 or fewer edges, by "multiplying" A_2 by A_1.
I.e., A_3[i,j] = min_k (A_2[i,k] + A_1[k,j]).
Proceeding in this manner for n steps, we obtain a matrix where the
(i,j)th entry specifies the length of the shortest path from i to j
using n or fewer edges, i.e. the length of the shortest path from i to
j. The whole process involves n iterations each taking O(n^3) time, so
the total time complexity currently is O(n^4).
How can we improve this running time?
Note that we can go directly from A_2 to A_4 via the following
operation:
A_4[i,j] = min_k (A_2[i,k] + A_2[k,j])
[think about why this works]
Likewise we can go from A_4 to A_8:
A_8[i,j] = min_k (A_4[i,k] + A_4[k,j])
In this way, by repeated "squaring", we can obtain A_n from A_1 in
only log n iterations instead of n. Therefore, our running time is
O(n^3 log n).
The final algorithm is extremely simple:
Initialize A = adj. matrix of the graph.
Do ceiling(log n) times:
for all i, j, B[i][j] := min_k (A[i][k] + A[k][j])
for all i, j, A[i][j] := B[i][j]
---------------------------------------------------------------------
All-pairs shortest paths via Floyd-Warshall
===========================================
We can in fact shave off another factor of log n from the running
time. Here is an algorithm for the all-pairs shortest paths problem
that runs in time O(n^3). The idea is that instead of increasing the
number of edges in the path, we'll increase the set of vertices we
allow as intermediate nodes in the path. In other words, starting
from the same base case (the shortest path that uses no intermediate
nodes), we'll then go on to the shortest path that's allowed to use
node 1 as an intermediate node. Then the shortest path that's allowed
to use {1,2} as intermediate nodes, and so on.
/* after each iteration of the outside loop, A[i][j] = len of i->j
path that's allowed to use vertices in the set 1..k */
for k = 1 to n do:
for each i,j do:
A[i][j] = min( A[i][j], (A[i][k] + A[k][j]));
i.e., you either go through k or you don't. Total time is O(n^3).
What's amazing here is how compact and simple the code is!
Why is this algorithm correct?
As for other dynamic programming algorithms, we will prove this using
induction. Our invariant is that up to step k, and for all i and j,
the array A contains the correct length of the shortest paths from i
to j using vertices 1..k.
The base case of k=0 is trivial -- the matrix A is the adjacency
matrix of the graph, and each shortest path is either a direct edge
between i and j, or infinite length.
For the inductive step, suppose that the shortest path between i and j
using vertices 1..(k+1) does not go over the vertex k+1. Then this
path is also the shortest path between i and j using vertices 1..k. So
its length gets considered by us as the previous value of A[i][j].
Finally, suppose that the shortest path between i and j using vertices
1..(k+1) does go over the vertex k+1. Then, the path from i to k+1
uses only the vertices 1..k, therefore its length is given by A[i][k],
and likewise, the path from k+1 to j uses only the vertices 1..k,
therefore its length is given by A[k][j]. So, once again, we compute
the correct length for the path.
Given these lengths, how do we figure out the actual paths from some
given i to a given j? We leave this as an exercise for you.
---------------------------------------------------------------------