📗 [4 points] Given the training set below and find the label of the decision tree that achieves 100 percent accuracy. Enter ${\hat{y}}_1,{\hat{y}}_2,{\hat{y}}_3,{\hat{y}}_4$ as a vector.

📗 The training set:

📗 The decision tree:

📗 Answer (comma separated vector): . Calculate

📗 [4 points] Given a neural network with 1 hidden layer with $2$ hidden units, suppose the current hidden layer weights are $w^{\left(1\right)}$ = $\left[\begin{array}{cc}{w}_{11}& w_{12}\\ w_{21}& w_{22}\end{array}\right]$ = $\left[\begin{array}{cc}-0.17& -0.23\\ -0.36& 0.15\end{array}\right]$, and the output layer weights are $w^{\left(2\right)}$ = $\left[\begin{array}{c}{w}_1\\ w_2\end{array}\right]$ = $\left[\begin{array}{c}0.44\\ 0.32\end{array}\right]$. Given an instance (item) $x$ = $\left[\begin{array}{c}0.8\\ 0.97\end{array}\right]$ and $y$ = $0$, the activation values are $a^{\left(1\right)}$ = $\left[\begin{array}{c}{a}_1\\ a_2\end{array}\right]$ = $\left[\begin{array}{c}0.97\\ 0.41\end{array}\right]$ and $a^{\left(2\right)}$ = $0.84$. What is updated weight $w_{21}^{\left(1\right)}$ after one step of stochastic gradient descent based on $x$ with learning rate $\alpha$ = $0.86$? The activation functions are all logistic and the cost is square loss.

📗 Reminder: logistic activation has gradient ${\displaystyle \frac{\partial a_i}{\partial z_i}}=a_i(1-a_i)$, tanh activation has gradient ${\displaystyle \frac{\partial a_i}{\partial z_i}}=1-a_i^2$, ReLU activation has gradient ${\displaystyle \frac{\partial a_i}{\partial z_i}}=1_{\{a_i\ge 0\}}$, and square cost has gradient ${\displaystyle \frac{\partial C_i}{\partial a_i}}=a_i-y_i$.

📗 Answer: . Calculate

📗 [4 points] Suppose the only three support vectors in a data set is $\left[\begin{array}{c}-2\\ 2\end{array}\right]$ with label $0$ and $\left[\begin{array}{c}-4\\ 5\end{array}\right]$ with label $1$ and $x$ with label $1$, let the margin (the distance between the plus and minus planes) be $2$. What is $x$? If there are multiple possible values, enter one of them, if there are none, enter $-1,-1$.

📗 Answer (comma separated vector): . Calculate

📗 [4 points] Given the two training points $\left[\begin{array}{c}-2\\ -2\end{array}\right]$ and $\left[\begin{array}{c}-4\\ 2\end{array}\right]$ and their labels $0$ and $1$. What is the kernel (Gram) matrix if the RBF (radial basis function) Gaussian kernel with $\sigma$ = $8$ is used? Use the formula $K_{i{i}^{\prime }}=e^{-{\displaystyle \frac{1}{2{\sigma }^2}}{(x_i-x_{i^{\prime }})}^{\mathrm{\top }}(x_i-x_{i^{\prime }})}$.

📗 Answer (matrix with multiple lines, each line is a comma separated vector): . Calculate

📗 [4 points] Suppose the squared loss is used to do stochastic gradient descent for logistic regression, i.e. $C={\displaystyle \frac{1}{2}}{\displaystyle \sum _{i=1}^n{(a_i-y_i)}^2}$ where $a_i={\displaystyle \frac{1}{1+e^{-w{x}_i-b}}}$. Given the current weight $w$ = $0.41$ and bias $b$ = $0.38$, with $x_i$ = $0.32$, $y_i$ = $1$, $a_i$ = $0.63$ (no need to recompute this value), with learning rate $\alpha$ = $0.68$. What is the updated bias after the iteration? Enter a single number.

📗 Answer: . Calculate

📗 [4 points] A convolutional neural network has input image of size $30$ x $30$ that is connected to a convolutional layer that uses a $5$ x $5$ filter, zero padding of the image, and a stride of 1. There are $3$ activation maps. (Here, zero-padding implies that these activation maps have the same size as the input images.) The convolutional layer is then connected to a pooling layer that uses $3$ x $3$ max pooling, a stride of $3$ (non-overlapping, no padding) of the convolutional layer. The pooling layer is then fully connected to an output layer that contains $5$ output units. There are no hidden layers between the pooling layer and the output layer. How many different weights must be learned in this whole network, not including any bias.

📗 Answer: . Calculate

📗 [2 points] Let $w$ = $\left[\begin{array}{c}-1\\ -1\end{array}\right]$ and $b$ = $-5$. For the point $x$ = $\left[\begin{array}{c}-2\\ 1\end{array}\right]$, $y$ = $1$, what is the smallest slack value $\xi$ for it to satisfy the margin constraint?

📗 Answer: . Calculate

📗 [4 points] Given a linear SVM (Support Vector Machine) that perfectly classifies a set of training data containing $7$ positive examples and $7$ negative examples. What is the maximum possible number of training examples that could be removed and still produce the exact same SVM as derived for the original training set?

📗 Answer: .

📗 [3 points] Consider a $4$-dimensional feature space where each feature takes integer value from 0 to $3$ (including 0 and $3$). What is the smallest and largest Euclidean distance between the two distinct (non-overlapping) points in the feature space?

📗 Answer (comma separated vector): . Calculate

📗 [2 points] Consider the following directed graphical model over binary variables: $A\to B\leftarrow C$ with the following training set.

📗 Answer: . Calculate

📗 [3 points] Statistically, cats are often hungry around 6:00 am (I am making this up). At that time, a cat is hungry $\frac{3}{5}$ of the time (C = 1), and not hungry $\frac{2}{5}$ of the time (C = 0). What is the entropy of the binary random variable C? Reminder that log based 2 of x can be found by log(x) / log(2) or log2(x).

📗 Answer: . Calculate

📗 [4 points] What is the convolution between the image $\left[\begin{array}{cc}4& 4\\ 3& 6\end{array}\right]$ and the filter $\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 1& 0\end{array}\right]$ using zero padding? Remember to flip the filter first.

📗 Answer (matrix with multiple lines, each line is a comma separated vector): . Calculate

📗 [3 points] Suppose the vocabulary is the alphabet plus space (26 letters + 1 space character), what is the (maximum likelihood) estimated trigram probability $\hat{\mathbb{P}}\{a|x,y\}$ with Laplace smoothing (add-1 smoothing) if the sequence $x,y$ never appeared in the training set. The training set has $500$ tokens in total. Enter -1 if more information is required to estimate this probability.

📗 Answer: . Calculate

📗 [3 points] Given a Bayesian network $A\to B\to C\to D$ of 4 binary event variables with the following conditional probability table (CPT), what is the probability that none of the events happen, $\mathbb{P}\{\mathrm{\neg }A,\mathrm{\neg }B,\mathrm{\neg }C,\mathrm{\neg }D\}$?

📗 Answer: . Calculate

📗 [1 points] Please enter any comments including possible mistakes and bugs with the questions or your answers. If you have no comments, please enter "None": do not leave it blank.

📗 Answer: .