📗 [4 points] Consider a Linear Threshold Unit (LTU) perceptron with initial weights $w$ = $\left[\begin{array}{c}0.2\end{array}\right]$ and bias $b$ = $-0.3$ trained using the Perceptron Algorithm. Given a new input $x$ = $\left[\begin{array}{c}1\end{array}\right]$ and $y$ = $0$. Let the learning rate be $\alpha$ = $0.5$, compute the updated weights, $w^{\prime },b^{\prime }$ = $\left[\begin{array}{cc}{w}_1& \mathrm{b}\end{array}\right]$:

📗 Answer (comma separated vector): . Calculate

📗 [2 points] Consider a rectified linear unit (ReLU) with input $x$ and a bias term. The output can be written as $y$ = $max(0,-3x+2)$. Here, the weight is $-3$ and the bias is $2$. Write down the smallest input value $x$ that produces a specific output $y$ = $0$.

📗 The red curve is a plot of the activation function, given the y-value of the green point, the question is asking for its x-value.

📗 Answer: . Calculate

📗 [3 points] Statistically, cats are often hungry around 6:00 am (I am making this up). At that time, a cat is hungry $\frac{2}{3}$ of the time (C = 1), and not hungry $\frac{1}{3}$ of the time (C = 0). What is the entropy of the binary random variable C? Reminder that log based 2 of x can be found by log(x) / log(2) or log2(x).

📗 Answer: . Calculate

📗 [4 points] List English letters from A to Z: ABCDEFGHIJKLMNOPQRSTUVWXYZ. Define the distance between two letters in the natural way, that is $d(A,A)=0$, $d(A,B)=1$, $d(A,C)=2$ and so on. Each letter has a label, I, M, P, Q, T, U are labeled 0, and the others are labeled 1. This is your training data. Now classify each letter using kNN (k Nearest Neighbor) for odd $k=1,3,5,7,...$. What is the smallest $k$ where all letters are classified the same (same label, i.e. either all labels are 0s or all labels are 1s). Break ties by preferring the earlier letters in the alphabet. Hint: the nearest neighbor of a letter is the letter itself.

📗 Answer: . Calculate

📗 [2 points] Given the training data "am am Groot am am", with the unigram model, what is the probability of observing the new sentence "Groot Groot Groot" given the first word is Groot? Use MLE (Maximum Likelihood Estimate) without smoothing and do not include the probability of observing the first word.

📗 Answer: . Calculate

📗 [3 points] There are two biased coins in my pocket: coin A has $\mathbb{P}\left\{H|A\right\}$ = $\frac{2}{3}$, coin B has $\mathbb{P}\left\{H|B\right\}$ = $\frac{1}{2}$. I took out a coin from the pocket at random with probability of A is $\frac{3}{5}$. I flipped it three times (independently) and the outcome is HTT. What is the probability that the coin was B?

📗 Answer: . Calculate

📗 [2 points] In a corpus with $180$ word tokens, the phrase "Fort Night" appeared $27$ times (not Fortnite). In particular, "Fort" appeared $90$ times and "Night" appeared $67$. If we estimate probability by frequency (the maximum likelihood estimate) without smoothing, what is the estimated probability of P(Night | Fort)?

📗 Answer: . Calculate

📗 [4 points] Given the following training set, add one instance $\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]$ with $y$ = $1$ so that all instances are support vectors for the Hard Margin SVM (Support Vector Machine) trained on the new training set.

📗 Note: in the diagram, currently, the two support vectors are connected by the grey line and the black line represents the SVM classification boundary. After adding one point, you should be able to make all seven points support vectors with the classification boundary given by the green line.

📗 Answer (comma separated vector): .

📗 [2 points] Given the following image gradient, suppose gradient vectors are put into one of the four bins according to the gradient direction: bin 1: $(0,{\displaystyle \frac{\pi }{2}}]$, bin 2: $({\displaystyle \frac{\pi }{2}},\pi ]$, bin 3: $[-{\displaystyle \frac{\pi }{2}},-\pi )$, bin 4: $[0,-{\displaystyle \frac{\pi }{2}})$, which bin does the gradient of the center element (pixel) fall into?

📗 Calculator (you can use the function atan2(y, x)): . Calculate

📗 Answer: .

📗 [4 points] Given the following transition matrix for a bigram model with words "I" (label 0), "am" (label 1) and "Groot" (label 2): $\left[\begin{array}{ccc}0.35& 0.22& 0.43\\ 0.25& 0.34& 0.41\\ 0.38& 0.23& 0.39\end{array}\right]$. Row $i$ column $j$ is $\mathbb{P}\{w_t=j|w_{t-1}=i\}$. Two uniform random numbers between 0 and 1 are generated to simulate the words after "I", say $u_1$ = $0.87$ and $u_2$ = $0.04$. Using the CDF (Cumulativ Distribution Function) inversion method (inverse transform method), which two words are generated? Enter two integer labels (0, 1, or 2), not strings.

📗 Answer (comma separated vector): . Calculate

📗 [2 points] There is a total of $16$ red or green balls in a bag. How many red balls and how many green balls are there so that the entropy of the color of a randomly selected ball is maximized?

📗 Answer (comma separated vector): .

📗 [4 points] Consider an unbiased estimator $X$ for a parameters $\theta$. We have $\mathbb{E}\left[X\right]$ = $2$, $Var\left[X\right]$ = $9$, $\mathbb{E}\left[Y\right]$ = $-3$, $Var\left[Y\right]$ = $5$. We would like a modified estimator $Z=X-Y$ to have a reduced variance compared to $X$. For what covariance $Cov[X,Y]$ can we achieve $Var\left[Z\right]\le Var\left[X\right]$? Note: there are many possible answers, enter only one of them.

📗 Answer: . Calculate

📗 [3 points] Let a dataset consist of $n$ = $6$ points in $\mathbb{R}$, specifically, the first $n-1$ points are $\left[\begin{array}{ccccc}-3& 0& 2& 3& 6\end{array}\right]$ and the last point $x_n$ is unknown. What is the smallest value of $x_n$ above which $x_{n-1}$ is among $x_n$'s 3-nearest neighbors, but $x_n$ is NOT among $x_{n-1}$'s 3-nearest neighbor? Note that the 3-nearest neighbors of a point in the training set include the point itself.

📗 Answer: . Calculate

📗 [3 points] Assume the prior probability of having a female child (girl) is the same as having a male child (boy) and both are 0.5. The Smith family has $4$ kids. One day you saw one of the Smith children, and she is a girl. The Wood family has $4$ kids, too, and you heard that at least one of them is a girl. What is the chance that the Smith family has a boy? What is the chance that the Wood family has a boy?

📗 Answer (comma separated vector): . Calculate

📗 [1 points] Please enter any comments including possible mistakes and bugs with the questions or your answers. If you have no comments, please enter "None": do not leave it blank.

📗 Answer: .