📗 [3 points] Given the variance matrix of a data set \(V\) = , a principal component \(u\) = , what is the projected variance of the data set in the direction \(u\)?

📗 Answer: . Calculate

📗 [3 points] Suppose the UCB1 (Upper Confidence Bound) Algorithm is used to select arms in a multi-armed bandit problem, and in round \(t\) = , the arms pulls and empirical means \(\hat{\mu}\) for the arms are summarized in the following table, and in period \(t + 1\), an arm is pulled according to the UCB1 Algorithm and the reward is . Compute the updated empirical means of the arms after period \(t + 1\), i.e. updated \(\hat{\mu}_{1}, \hat{\mu}_{2}, ...\). Use \(c\) = .

📗 Answer (comma separated vector): . Calculate

📗 [3 points] In an infinite horizon MDP (Markov Decision Process), there are \(n\) = states: initial state \(s_{0}\), and absorbing states \(s_{1}, s_{2}, ..., s_{n-1}\). In state \(s_{0}\), the agent can stay or move to any other state, but in all other absorbing states the agent can only choose to stay. The reward from staying in those states are summarized in the following table. Compute the Q value (under the optimal policy, not from Q learning) \(Q\left(s_{0}, \text{stay}\right)\). Use the discount factor \(\gamma\) = .

📗 Answer: . Calculate

📗 [3 points] Given the pairwise distance matrix \(d\), what is the linkage distance between the cluster {} and {}? The columns and rows are indexed \(1, 2, 3, ...\), i.e. row \(i\) column \(j\) is the distance between point \(i\) and point \(j\).

📗 Answer: . Calculate

📗 [2 points] Given attention weight from \(q\) to \(k_{1}\), \(w_{1}\) = , from \(q\) to \(k_{2}\), \(w_{2}\) = , given values \(v_{1}\) = , \(v_{2}\) = , calculate the output vector.

📗 Answer (comma separated vector): . Calculate

📗 [3 points] Given the variance matrix \(\hat{\Sigma}\) is a diagonal matrix, what is the smallest value of \(K\) so that the Manhattan distance between the vector \(\begin{bmatrix} 1 \\ 1 \\ ... \\ 1 \end{bmatrix}\) with ones (\(1\)'s) and its reconstruction using the first \(K\) principal components is less than or equal to ?

📗 Answer: . Calculate

📗 [4 points] When using the Genetic Algorithm, suppose the states are \(\begin{bmatrix} x_{1} & x_{2} & ... & x_{T} \end{bmatrix}\) = , , , . Let \(T\) = , the fitness function (not the cost) is \(\mathop{\mathrm{argmin}}_{t \in \left\{1, ..., T + 1\right\}} x_{t} = 1\) with \(x_{T + 1} = 1\) (i.e. the index of the first feature that is 1). What is the reproduction probability of the first state: ?

📗 Answer: . Calculate

📗 [2 points] In simulated annealing one accepts a transition from \(s\) to an inferior neighbor \(t\) with probability \(\exp\left(\dfrac{- \left| f\left(s\right) - f\left(t\right) \right|}{T}\right)\), where \(T\) is the temperature parameter. Suppose \(f\left(s\right)\) = and \(f\left(t\right)\) = . At what temperature is the transition probability ?

📗 Answer: . Calculate

📗 [4 points] In a by grid, Tom is located at (, ) and Jerry is located at (, ). Tom uses to find Jerry and the successors of a state (one cell in the grid) are the four neighboring states on the grid (the cells above, below, to the left and to the right). What is the imum number of states that need to be expanded to find (and expand) the goal state? The order in which the successors are added can be arbitrary. Do not count repeated expansion of the same state. Include both the initial and the goal states.

📗 Answer: . Calculate

📗 [3 points] Suppose there are \(n\) = states, \(1\) (initial state), \(2\), ..., \(n\) (goal state) and two heuristics \(h_{1}\) and \(h_{2}\), described in the table below, are both admissible. State \(i\) has successors \(i + 1, i + 2, ..., n\), for each \(i = 1, 2, ..., n - 1\). What is the minimum possible total cost from state \(1\) to state \(n\).

📗 Note: do not assume the cost between a state and its successor is 1. 

📗 Answer: . Calculate

📗 [3 points] Let \(h_{1}\) be an admissible heuristic from a state to the optimal goal, A* search with which ones of the following \(h\) will be admissible? Enter the correct choices as a list, comma separated, without parentheses, for example, "1, 2, 4".

📗 Choices:

📗 Answer (comma separated vector): .

📗 [4 points] Consider the following zero-sum game tree. player moves first. Draw a new game tree by re-ordering the children of each internal node (including the root), such that the new game is equivalent to the tree above, but alpha-beta pruning will prune as many nodes as possible. (You do not have to submit the drawing.) Enter the number of nodes pruned.

📗 Note: in case the diagram is not clear, the values on the leafs (each sub-branch is a row): .

📗 Answer: .

📗 [3 points] There are \(n\) = cookies. The brother first proposes a division of these cookies into two piles (two integers adding up to \(n\)) and then the sister take one of the two piles. Both the brother and the sister want to maximize the number of cookies they take. What is the value of the game to the brother (measured by the number of cookies he gets)? Enter an integer.

📗 Answer: . Calculate

📗 [4 points] Given the following game payoff table, suppose the row player uses a mixed strategy playing U with probability \(p\), and column player uses a pure strategy. What is the smallest and largest value of \(p\) in a mixed strategy Nash equilibrium?

📗 Answer (comma separated vector): . Calculate

📗 [1 points] Please enter any comments including possible mistakes and bugs with the questions or your answers. If you have no comments, please enter "None": do not leave it blank.

📗 Answer: .