📗 Search problems solve for a sequence of optimal actions to minimize the total cost of these actions.
📗 Reinforcement learning is similar but the cost is unknown so the agent has the learn the costs (usually in the form of reward) while finding the optimal actions.
➩ Board games.
➩ Robotic control.
➩ Autonomous vehicle control.
➩ Economic models.
➩ Large language models.
📗 The total number of possible action sequences is large, so efficient algorithms are needed to search only the relevant ones.
📗 A search problem is defined by a state space graph (search graph): Wikipedia.
➩ \(V\) is the set of nodes (or vertices) representing the states (outcome from a sequence of actions).
➩ \(E\) is the set of edges (or arcs) representing the possible actions at each state.
➩ \(c\) is the cost (or weights) associated with each edge, assumed to be positive.
📗 The search graph is usually in the form of a tree.
➩ At the beginning of a search, only the initial state is expanded.
➩ The list of states that can be reached from some state \(s\) by performing some action is called successors of \(s\).
➩ Expanding a state means generating the list of successors and adding them to the search tree.
➩ The leafs of the search tree is called the frontier (or the fringe).
➩ The search is over when a goal state is expanded. The path from the initial state to the goal state is called a solution to the search problem.
➩ The solution is optimal if it has the smallest cost.
📗 A search strategy is the order in which the states are expanded.
TopHat Discussion
📗 [1 points] people with one flashlight (torch) want to go across a river. The bridge can hold two people at a time, and they must cross with the flashlight. The time it takes for each person to cross the river:
A
B
C
D
What is the minimum total time required for everyone to cross the river?
📗 BFS (Breadth First Search) uses Queue for the frontier: remove from the front, add to the back: Wikipedia.
📗 BFS is complete.
📗 BFS is optimal when the cost of every action is 1.
📗 Time complexity is \(T = 1 + b + b^{2} + ... + b^{d}\).
➩ This count includes the root and the goal.
📗 Space complexity is \(S = b^{d+1}\).
📗 Bidirectional search does BFS from the initial and goal states at the same time and stops when they meet.
TopHat Quiz
(Past Exam Question) ID:
📗 [4 points] Suppose the states are integers between and . The initial state is , and the goal state is . The successors of a state \(i\) are \(2 i\) and \(2 i + 1\), if exist. How many states are expanded using a ? Include both the initial and goal states.
📗 Note: use the convention used in the lectures, push the states with larger index into the stack first (i.e. expand the states with the smaller index first).
📗 DFS (Depth First Search) uses Stack for the frontier: remove from the front, add to the front: Wikipedia.
📗 DFS is incomplete if \(D = \infty\).
📗 DFS is not optimal.
📗 Time complexity is \(T = 1 + b^{D-d+1} + ... + b^{D-1} + b^{D}\).
➩ This count includes the root and the goal.
📗 Space complexity is \(S = \left(b - 1\right) D + 1\).
TopHat Quiz
(Past Exam Question) ID:
📗 [4 points] Suppose the states are integers between and . The initial state is , and the goal state is . The successors of a state \(i\) are \(2 i\) and \(2 i + 1\), if exist. How many states are expanded using a ? Include both the initial and goal states.
📗 Note: use the convention used in the lectures, push the states with larger index into the stack first (i.e. expand the states with the smaller index first).
➩ DFS and stops if path length is larger than \(1\).
➩ DFS and stops if path length is larger than \(2\).
➩ ...
➩ DFS and stops if path length is larger than \(d\).
📗 IDS is complete.
📗 IDS is optimal when the cost of all actions are 1.
📗 Time complexity is \(T = \left(d + 1\right) + d b + \left(d - 1\right) b^{2} + ... + 3 b^{d-2} + 2 b^{d-1} + 1 b^{d}\).
➩ This count includes the root and the goal, and the first DFS step is only on the root.
📗 Space complexity is \(S = \left(b - 1\right) d + 1\).
TopHat Discussion
📗 What is a good search strategy for the water jugs game (note: there are cycles, so the search tree can be a search graph if the search strategy does not keep track of the visited states): Link.
📗 Additional information can be given in the form of a heuristic cost from any state to the goal state: this is an estimate or guess of the minimum cost from \(s\) to a goal state: Wikipedia.
➩ The cost from the initial state to a state \(s\) is denoted by \(g\left(s\right)\).
➩ The cost from the state \(s\) to the goal state is denoted by \(h^\star\left(s\right)\), which is unknown during the search.
➩ The estimated value of \(h^\star\left(s\right)\) is called the heuristic cost, denoted by \(h\left(s\right)\).
📗 Heuristic costs can be used to speed up the search.
TopHat Discussion
📗 Search for a page containing an image. Describe how you find the image.
📗 UCS (Uniform Cost Search, or Dijkstra's Algorithm) expands the state with the lowest current cost \(g\left(s\right)\): Wikipedia.
📗 It is BFS with a priority queue based on \(g\left(s\right)\), and it is equivalent to BFS if the cost of every action is 1.
📗 UCS is complete.
📗 UCS is optimal.
TopHat Quiz
📗 [4 points] Given initial state \(S\) and goal state \(G\), write down the expansion path (list of expanded nodes) for . In case the diagram is not clear, the edge costs are (heuristic on the diagonal).
📗 (Best First) Greedy Search expands the state with the lowest heuristic cost \(h\left(s\right)\): Wikipedia.
📗 BFGS is not an abbreviation of Greedy Search, since it usually stands for Broyden Fletcher Goldfarb Shanno algorithm (a version of a gradient descent algorithm).
📗 Greedy uses a priority queue based on \(h\left(s\right)\).
📗 Greedy is incomplete.
📗 Greedy is not optimal.
TopHat Quiz
📗 [4 points] Given initial state \(S\) and goal state \(G\), write down the expansion path (list of expanded nodes) for . In case the diagram is not clear, the edge costs are (heuristic on the diagonal).
📗 A Search expands the state with the lowest total cost \(g\left(s\right) + h\left(s\right)\): Wikipedia (for A*).
📗 A Search uses a priority queue based on \(g\left(s\right) + h\left(s\right)\).
📗 A is complete.
📗 A is not optimal.
TopHat Quiz
📗 [4 points] Given initial state \(S\) and goal state \(G\), write down the expansion path (list of expanded nodes) for . In case the diagram is not clear, the edge costs are (heuristic on the diagonal).
📗 Answer: .
TopHat Quiz
📗 [4 points] Given initial state \(S\) and goal state \(G\), write down the expansion path (list of expanded nodes) for . In case the diagram is not clear, the edge costs are (heuristic on the diagonal).
📗 \(h_{2}\) dominates \(h_{1}\) (or \(h_{1}\) is dominated by \(h_{2}\)) if \(h_{1}\left(s\right) \leq h_{2}\left(s\right) \leq h^\star\left(s\right)\) for every state \(s\).
📗 A* with a dominated heuristic is less informed and worse for A*.
📗 If optimality is not required, then a heuristic that is close but slightly over estimates the true cost can be preferred.
TopHat Quiz
(Past Exam Question) ID:
📗 [3 points] Let \(h_{1}\) be an admissible heuristic from a state to the optimal goal, A* search with which ones of the following \(h\) will be admissible?
📗 Iterative Deepening A* (IDA*) expands states with limit on \(g\left(s\right) + h\left(s\right)\) instead of depth in IDS: Link, Wikipedia.
📗 Beam Search keeps a priority queue with a limited size: Wikipedia.
Example
📗 A* search is often used to control robotic arms. States are usually represented by rotations angles instead of positions. The search space is called the configuration space: Link.
Review Note
Review of search algorithms:
📗 Ignore edge cost (equivalently, assume edge costs are all 1)
BFS:
Expand the shallowest node first
Complete, optimal (if cost is all 1)
Time complexity ~ \(b^{d}\)
Space complexity ~ \(b^{d}\)
DFS:
Expand deepest node first
Incomplete (when D is infinite), hence not optimal
Time complexity ~ \(b^{D}\)
Space complexity ~ \(b D\)
IDS:
"Search like BFS, fringe like DFS"
Complete, optimal (if cost is all 1)
Time complexity: ~ \(b^{d}\)
Space complexity: ~ \(b d\)
Preferred algorithms if heuristic is unknown
📗 Consider edge cost
UCS:
(Generalization of BFS)
Expand least-cost node first (first half cost \(g\left(s\right)\))
Complete, optimal
\(g\left(s\right)\): first half cost, known.
\(h^\star\left(s\right)\): second half cost, unknown.
\(h\left(s\right)\): an estimate/guess for second half cost; user-specified
BFG:
Expand node with lowest \(h\left(s\right)\)
Incomplete, not optimal.
Issue: ignores \(g\left(s\right)\)
A Search:
Expand node with lowest \(g\left(s\right) + h\left(s\right)\)
Complete, but not optimal.
Issue: \(h\left(s\right)\) may be an overestimate of \(h^\star\left(s\right)\), preventing us to expand a node with low true \(h^\star\left(s\right)\).
testbt,ltb,ltd,ucs,bfg,as,asf,adq
📗 Notes and code adapted from the course taught by Professors Blerina Gkotse, Jerry Zhu, Yudong Chen, Yingyu Liang, Charles Dyer.
📗 If there is an issue with TopHat during the lectures, please submit your answers on paper (include your Wisc ID and answers) or this Google form Link at the end of the lecture.
📗 Anonymous feedback can be submitted to: Form. Non-anonymous feedback and questions can be posted on Piazza: Link