📗 Search problems solve for a sequence of optimal actions to minimize the total cost of these actions.
📗 Reinforcement learning is similar but the cost is unknown so the agent has the learn the costs (usually in the form of reward) while finding the optimal actions.
➩ Board games.
➩ Robotic control.
➩ Autonomous vehicle control.
➩ Economic models.
➩ Large language models.
📗 The total number of possible action sequences is large, so efficient algorithms are needed to search only the relevant ones.
📗 A search problem is defined by a state space graph (search graph): Wikipedia.
➩ \(V\) is the set of nodes (or vertices) representing the states (outcome from a sequence of actions).
➩ \(E\) is the set of edges (or arcs) representing the possible actions at each state.
➩ \(c\) is the cost (or weights) associated with each edge, assumed to be positive.
📗 The search graph is usually in the form of a tree.
➩ At the beginning of a search, only the initial state is expanded.
➩ The list of states that can be reached from some state \(s\) by performing some action is called successors of \(s\).
➩ Expanding a state means generating the list of successors and adding them to the search tree.
➩ The leafs of the search tree is called the frontier (or the fringe).
➩ The search is over when a goal state is expanded. The path from the initial state to the goal state is called a solution to the search problem.
➩ The solution is optimal if it has the smallest cost.
📗 A search strategy is the order in which the states are expanded.
TopHat Discussion
📗 [1 points] people with one flashlight (torch) want to go across a river. The bridge can hold two people at a time, and they must cross with the flashlight. The time it takes for each person to cross the river:
A
B
C
D
What is the minimum total time required for everyone to cross the river?
📗 BFS (Breadth First Search) uses Queue for the frontier: remove from the front, add to the back: Wikipedia.
📗 BFS is complete.
📗 BFS is optimal when the cost of every action is 1.
📗 Time complexity is \(T = 1 + b + b^{2} + ... + b^{d}\).
➩ This count includes the root and the goal.
📗 Space complexity is \(S = b^{d+1}\).
📗 Bidirectional search does BFS from the initial and goal states at the same time and stops when they meet.
TopHat Quiz
(Past Exam Question) ID:
📗 [4 points] Suppose the states are integers between and . The initial state is , and the goal state is . The successors of a state \(i\) are \(2 i\) and \(2 i + 1\), if exist. How many states are expanded using a ? Include both the initial and goal states.
📗 Note: use the convention used in the lectures, push the states with larger index into the stack first (i.e. expand the states with the smaller index first).
📗 DFS (Depth First Search) uses Stack for the frontier: remove from the front, add to the front: Wikipedia.
📗 DFS is incomplete if \(D = \infty\).
📗 DFS is not optimal.
📗 Time complexity is \(T = 1 + b^{D-d+1} + ... + b^{D-1} + b^{D}\).
➩ This count includes the root and the goal.
📗 Space complexity is \(S = \left(b - 1\right) D + 1\).
TopHat Quiz
(Past Exam Question) ID:
📗 [4 points] Suppose the states are integers between and . The initial state is , and the goal state is . The successors of a state \(i\) are \(2 i\) and \(2 i + 1\), if exist. How many states are expanded using a ? Include both the initial and goal states.
📗 Note: use the convention used in the lectures, push the states with larger index into the stack first (i.e. expand the states with the smaller index first).
➩ DFS and stops if path length is larger than \(1\).
➩ DFS and stops if path length is larger than \(2\).
➩ ...
➩ DFS and stops if path length is larger than \(d\).
📗 IDS is complete.
📗 IDS is optimal when the cost of all actions are 1.
📗 Time complexity is \(T = \left(d + 1\right) + d b + \left(d - 1\right) b^{2} + ... + 3 b^{d-2} + 2 b^{d-1} + 1 b^{d}\).
➩ This count includes the root and the goal, and the first DFS step is only on the root.
📗 Space complexity is \(S = \left(b - 1\right) d + 1\).
TopHat Discussion
📗 What is a good search strategy for the water jugs game (note: there are cycles, so the search tree can be a search graph if the search strategy does not keep track of the visited states): Link.
testbt,ltb,ltdq
📗 Notes and code adapted from the course taught by Professors Jerry Zhu, Yingyu Liang, and Charles Dyer.
📗 Content from note blocks marked "optional" and content from Wikipedia and other demo links are helpful for understanding the materials, but will not be explicitly tested on the exams.
📗 Please use Ctrl+F5 or Shift+F5 or Shift+Command+R or Incognito mode or Private Browsing to refresh the cached JavaScript.
📗 You can expand all TopHat Quizzes and Discussions: , and print the notes: , or download all text areas as text file: .
📗 If there is an issue with TopHat during the lectures, please submit your answers on paper (include your Wisc ID and answers) or this Google form Link at the end of the lecture.