📗 Search problems solve for a sequence of optimal actions to minimize the total cost of these actions.
📗 Reinforcement learning is similar but the cost is unknown so the agent has the learn the costs (usually in the form of reward) while finding the optimal actions.
➩ Board games.
➩ Robotic control.
➩ Autonomous vehicle control.
➩ Economic models.
➩ Large language models.
📗 The total number of possible action sequences is large, so efficient algorithms are needed to search only the relevant ones.
📗 A search problem is defined by a state space graph (search graph): Wikipedia.
➩ \(V\) is the set of nodes (or vertices) representing the states (outcome from a sequence of actions).
➩ \(E\) is the set of edges (or arcs) representing the possible actions at each state.
➩ \(c\) is the cost (or weights) associated with each edge, assumed to be positive.
📗 The search graph is usually in the form of a tree.
➩ At the beginning of a search, only the initial state is expanded.
➩ The list of states that can be reached from some state \(s\) by performing some action is called successors of \(s\).
➩ Expanding a state means generating the list of successors and adding them to the search tree.
➩ The leafs of the search tree is called the frontier (or the fringe).
➩ The search is over when a goal state is expanded. The path from the initial state to the goal state is called a solution to the search problem.
➩ The solution is optimal if it has the smallest cost.
📗 A search strategy is the order in which the states are expanded.
TopHat Discussion
📗 [1 points] people with one flashlight (torch) want to go across a river. The bridge can hold two people at a time, and they must cross with the flashlight. The time it takes for each person to cross the river:
A
B
C
D
What is the minimum total time required for everyone to cross the river?
📗 BFS (Breadth First Search) uses Queue for the frontier: remove from the front, add to the back: Wikipedia.
📗 BFS is complete.
📗 BFS is optimal when the cost of every action is 1.
📗 Time complexity is \(T = 1 + b + b^{2} + ... + b^{d}\).
➩ This count includes the root and the goal.
📗 Space complexity is \(S = b^{d+1}\).
📗 Bidirectional search does BFS from the initial and goal states at the same time and stops when they meet.
TopHat Quiz
(Past Exam Question) ID:
📗 [4 points] Suppose the states are integers between and . The initial state is , and the goal state is . The successors of a state \(i\) are \(2 i\) and \(2 i + 1\), if exist. How many states are expanded using a ? Include both the initial and goal states.
📗 Note: use the convention used in the lectures, push the states with larger index into the stack first (i.e. expand the states with the smaller index first).
📗 DFS (Depth First Search) uses Stack for the frontier: remove from the front, add to the front: Wikipedia.
📗 DFS is incomplete if \(D = \infty\).
📗 DFS is not optimal.
📗 Time complexity is \(T = 1 + b^{D-d+1} + ... + b^{D-1} + b^{D}\).
➩ This count includes the root and the goal.
📗 Space complexity is \(S = \left(b - 1\right) D + 1\).
TopHat Quiz
(Past Exam Question) ID:
📗 [4 points] Suppose the states are integers between and . The initial state is , and the goal state is . The successors of a state \(i\) are \(2 i\) and \(2 i + 1\), if exist. How many states are expanded using a ? Include both the initial and goal states.
📗 Note: use the convention used in the lectures, push the states with larger index into the stack first (i.e. expand the states with the smaller index first).
➩ DFS and stops if path length is larger than \(1\).
➩ DFS and stops if path length is larger than \(2\).
➩ ...
➩ DFS and stops if path length is larger than \(d\).
📗 IDS is complete.
📗 IDS is optimal when the cost of all actions are 1.
📗 Time complexity is \(T = \left(d + 1\right) + d b + \left(d - 1\right) b^{2} + ... + 3 b^{d-2} + 2 b^{d-1} + 1 b^{d}\).
➩ This count includes the root and the goal, and the first DFS step is only on the root.
📗 Space complexity is \(S = \left(b - 1\right) d + 1\).
TopHat Discussion
📗 What is a good search strategy for the water jugs game (note: there are cycles, so the search tree can be a search graph if the search strategy does not keep track of the visited states): Link.
📗 Additional information can be given in the form of a heuristic cost from any state to the goal state: this is an estimate or guess of the minimum cost from \(s\) to a goal state: Wikipedia.
➩ The cost from the initial state to a state \(s\) is denoted by \(g\left(s\right)\).
➩ The cost from the state \(s\) to the goal state is denoted by \(h^\star\left(s\right)\), which is unknown during the search.
➩ The estimated value of \(h^\star\left(s\right)\) is called the heuristic cost, denoted by \(h\left(s\right)\).
📗 Heuristic costs can be used to speed up the search.
TopHat Discussion
📗 Search for a page containing an image. Describe how you find the image.
📗 UCS (Uniform Cost Search, or Dijkstra's Algorithm) expands the state with the lowest current cost \(g\left(s\right)\): Wikipedia.
📗 It is BFS with a priority queue based on \(g\left(s\right)\), and it is equivalent to BFS if the cost of every action is 1.
📗 UCS is complete.
📗 UCS is optimal.
TopHat Quiz
📗 [4 points] Given initial state \(S\) and goal state \(G\), write down the expansion path (list of expanded nodes) for . In case the diagram is not clear, the edge costs are (heuristic on the diagonal).
📗 (Best First) Greedy Search expands the state with the lowest heuristic cost \(h\left(s\right)\): Wikipedia.
📗 BFGS is not an abbreviation of Greedy Search, since it usually stands for Broyden Fletcher Goldfarb Shanno algorithm (a version of a gradient descent algorithm).
📗 Greedy uses a priority queue based on \(h\left(s\right)\).
📗 Greedy is incomplete.
📗 Greedy is not optimal.
TopHat Quiz
📗 [4 points] Given initial state \(S\) and goal state \(G\), write down the expansion path (list of expanded nodes) for . In case the diagram is not clear, the edge costs are (heuristic on the diagonal).
📗 A Search expands the state with the lowest total cost \(g\left(s\right) + h\left(s\right)\): Wikipedia (for A*).
📗 A Search uses a priority queue based on \(g\left(s\right) + h\left(s\right)\).
📗 A is complete.
📗 A is not optimal.
TopHat Quiz
📗 [4 points] Given initial state \(S\) and goal state \(G\), write down the expansion path (list of expanded nodes) for . In case the diagram is not clear, the edge costs are (heuristic on the diagonal).
📗 Answer: .
TopHat Quiz
📗 [4 points] Given initial state \(S\) and goal state \(G\), write down the expansion path (list of expanded nodes) for . In case the diagram is not clear, the edge costs are (heuristic on the diagonal).
📗 \(h_{2}\) dominates \(h_{1}\) (or \(h_{1}\) is dominated by \(h_{2}\)) if \(h_{1}\left(s\right) \leq h_{2}\left(s\right) \leq h^\star\left(s\right)\) for every state \(s\).
📗 A* with a dominated heuristic is less informed and worse for A*.
📗 If optimality is not required, then a heuristic that is close but slightly over estimates the true cost can be preferred.
TopHat Quiz
(Past Exam Question) ID:
📗 [3 points] Let \(h_{1}\) be an admissible heuristic from a state to the optimal goal, A* search with which ones of the following \(h\) will be admissible?
📗 Iterative Deepening A* (IDA*) expands states with limit on \(g\left(s\right) + h\left(s\right)\) instead of depth in IDS: Link, Wikipedia.
📗 Beam Search keeps a priority queue with a limited size: Wikipedia.
Example
📗 A* search is often used to control robotic arms. States are usually represented by rotations angles instead of positions. The search space is called the configuration space: Link.
Review Note
Review of search algorithms:
📗 Ignore edge cost (equivalently, assume edge costs are all 1)
BFS:
Expand the shallowest node first
Complete, optimal (if cost is all 1)
Time complexity ~ \(b^{d}\)
Space complexity ~ \(b^{d}\)
DFS:
Expand deepest node first
Incomplete (when D is infinite), hence not optimal
Time complexity ~ \(b^{D}\)
Space complexity ~ \(b D\)
IDS:
"Search like BFS, fringe like DFS"
Complete, optimal (if cost is all 1)
Time complexity: ~ \(b^{d}\)
Space complexity: ~ \(b d\)
Preferred algorithms if heuristic is unknown
📗 Consider edge cost
UCS:
(Generalization of BFS)
Expand least-cost node first (first half cost \(g\left(s\right)\))
Complete, optimal
\(g\left(s\right)\): first half cost, known.
\(h^\star\left(s\right)\): second half cost, unknown.
\(h\left(s\right)\): an estimate/guess for second half cost; user-specified
BFG:
Expand node with lowest \(h\left(s\right)\)
Incomplete, not optimal.
Issue: ignores \(g\left(s\right)\)
A Search:
Expand node with lowest \(g\left(s\right) + h\left(s\right)\)
Complete, but not optimal.
Issue: \(h\left(s\right)\) may be an overestimate of \(h^\star\left(s\right)\), preventing us to expand a node with low true \(h^\star\left(s\right)\).
testbt,ltb,ltd,ucs,bfg,as,asf,adq
📗 Notes and code adapted from the course taught by Professors Jerry Zhu, Yingyu Liang, and Charles Dyer.
📗 Please use Ctrl+F5 or Shift+F5 or Shift+Command+R or Incognito mode or Private Browsing to refresh the cached JavaScript.