📗 For some problems, every state is a solution, only some states are better than other states specified by a cost function (sometimes score or reward): Wikipedia.
📗 The search strategy will go from state to state, but the path between states is not important.
📗 Local search assumes similar (nearby) states have similar costs, and search through the state space by iteratively improving the costs to find an optimal state.
📗 The successor states are called neighbors (or move set).
📗 Simulated annealing uses a process similar to heating solids (heating and slow cooling to toughen and reduce brittleness): Wikipedia.
➩ Each time, a random neighbor is generated.
➩ If the neighbor has a lower cost, move to the neighbor.
➩ If the neighbor has a higher cost, move to the neighbor with a small probability: \(p = e^{- \dfrac{\left| f\left(s'\right) - f\left(s\right) \right|}{T\left(t\right)}}\), where \(f\) is the cost and \(T\left(t\right)\) is the temperature and decreasing in \(t\).
➩ Stop until bored.
📗 Simulated annealing is a version of Metropolis-Hastings algorithm: Wikipedia.
Example
📗 The traveling salesman problem is often solved by simulated annealing: Link.
📗 Genetic algorithm starts with a fixed population of initial states, and the successors are found through cross-over and mutation: Wikipedia.
📗 Each state in the population with \(N\) states has probability of reproduction proportional to the fitness (or negatively proportional to the costs): \(p_{i} = \dfrac{F\left(s_{i}\right)}{F\left(s_{1}\right) + F\left(s_{2}\right) + ... + F\left(s_{N}\right)}\).
📗 If the states are encoded by strings, cross-over means swapping substrings at a fixed point: for example, abcde and ABCDE cross-over at position 2 results in abCDE and ABcde.
📗 If the states are encoded by strings, mutation means randomly updating substrings with a small probability called the mutation rate: for example, abcde can be updated to abCde or aBcDe or ... with small probabilities.
TopHat Quiz
(Past Exam Question) ID:
📗 [4 points] When using the Genetic Algorithm, suppose the states are \(\begin{bmatrix} x_{1} & x_{2} & ... & x_{T} \end{bmatrix}\) = , , , . Let \(T\) = , the fitness function (not the cost) is \(\mathop{\mathrm{argmax}}_{t \in \left\{0, ..., T\right\}} x_{t} = 1\) with \(x_{0} = 1\) (i.e. the index of the last feature that is 1). What is the reproduction probability of the first state: ?
📗 The parents do not survive in the standard genetic algorithm, but if reproduction between two copies of the same states is allowed, the parents can survive.
📗 The fitness or cost functions can be replaced by the ranking.
📗 In theory, cross-over is much more efficient than mutation.
Example
📗 Many problems can be solved by genetic algorithm (but in practice, reinforcement learning techniques are more efficient and produce better policies).