# Videos on Past Exam Questions
📗 Perceptron:
Why does the (batch) perceptron algorithm work?
Link
Why cannot use linear regression for binary classification?
Link
How to use Perceptron update formula?
Link
How to find the size of the hypothesis space for linear classifiers?
Link (Part 1)
📗 Gradient Descent:
Why does gradient descent work?
Link
Computation of Hessian of quadratic form
Link
Computation of eigenvalues
Link
Gradient descent for linear regression
Link
What is the gradient descent step for cross-entropy loss with linear activation?
Link (Part 1)
What is the sign of the next gradient descent step?
Link (Part 2)
Which loss functions are equivalent to squared loss?
Link (Part 3)
How to compute the gradient of the cross-entropy loss with linear activation?
Link (Part 4)
How to find the location that minimizes the distance to multiple points?
Link (Part 3)
📗 Logistic Regression:
How to derive logistic regression gradient descent step formula?
Link
Gradient descent for logistic activation with squared error
Link
How to compute logistic gradient descent?
Link
📗 Neural Network:
How to construct XOR network?
Link
How derive 2-layer neural network gradient descent step?
Link
How derive multi-layer neural network gradient descent induction step?
Link
How to find the missing weights in neural network given data set?
Link
How many weights are used in one step of backpropogation?
Link (Part 2)
📗 Regularization:
Comparison between L1 and L2 regularization.
Link
How to compute cross validation accuracy?
Link
📗 Hard Margin Support Vector Machine:
How to find the margin expression for SVM?
Link
Compute SVM classifier
Link
How to find the distance from a plane to a point?
Link
How to find the formula for SVM given two training points?
Link
What is the largest number of points that can be removed to maintain the same SVM?
Link (Part 4)
What is minimum number of points that can be removed to improve the SVM margin?
Link (Part 5)
How many training items are needed for a one-vs-one SVM?
Link (Part 2)
Which items are used in a multi-lcass one-vs-one SVM?
Link (Part 7)
📗 Soft Margin Support Vector Machine:
What is the gradient descent step for SVM hinge loss with linear activation?
Link (Part 1)
How to compute the subgradient?
Link (Part 2)
What happens if the lambda in soft-margin SVM is 0?
Link (Part 3)
How to compute the hinge loss gradient?
Link (Part 1)
📗 Kernel Trick:
Why does the kernel trick work?
Link
How to find feature representation for sum of two kernel (Gram) matrices?
Link
What is the kernel SVM for XOR operator?
Link
How to convert the kernel matrix to feature vector?
Link
How to find the kernel (Gram) matrix given the feature representation?
Link (Part 1)
How to find the feature vector based on the kernel (Gram) matrix?
Link (Part 4)
How to find the kernal (Gram) matrix based on the feature vectors?
Link (Part 10)
📗 Entropy:
How to do entropy computation?
Link
How to find the information gain given two distributions (this is the Avatar question)?
Link
What distribution maximizes the entropy?
Link (Part 1)
How to create a dataset with information gain of 0?
Link (Part 2)
How to compute the conditional entropy based on a binary variable dataset?
Link (Part 3)
How to find conditional entropy given a dataset?
Link (Part 9)
When is the information gain based on a dataset equal to zero?
Link (Part 10)
How to compute entropy of a binary variable?
Link (Part 1)
How to compute information gain, the Avatar question?
Link (Part 2)
How to compute conditional entropy based on a training set?
Link (Part 3)
📗 Decision Trees:
What is the decision tree for implication operator?
Link
How many conditional entropy calculations are needed for a decision tree with real-valued features?
Link (Part 1)
What is the maximum and minimum training set accuracy for a decision tree?
Link (Part 2)
How to find the minimum number of conditional entropies that need to be computed for a binary decision tree?
Link (Part 9)
What is the maximum number of conditional entropies that need to be computed in a decision tree at a certain depth?
Link (Part 4)
📗 Nearest Neighbor:
How to do three nearest neighbor 3NN?
Link
How to find a KNN decision boundary?
Link
What is the accuracy for KNN when K = n or K = 1?
Link (Part 1)
Which K maximizes the accuracy of KNN?
Link (Part 3)
How to work with KNN with distance defined on the alphabet?
Link (Part 4)
How to find the 1NN accuracy on training set?
Link (Part 8)
How to draw the decision boundary of 1NN in 2D?
Link (Part 1)
How to find the smallest k such that all items are classified as the same label with kNN?
Link (Part 2)
Which value of k maximizes the accuracy of kNN?
Link (Part 3)
📗 K-Fold Validation:
How to compute the leave-one-out accuracy for kNN with large k?
Link
What is the leave-one-out accuracy for KNN with K = n?
Link (Part 2)
How to compute cross validation accuracy for KNN?
Link (Part 5)
What is the leave-one-out accuracy for n-1-NN?
Link (Part 5)
How to find the 3 fold cross validation accuracy of a 1NN classifier?
Link (Part 12)
📗 Convolution and Image Gradient:
How to compute the convolution between two matrices?
Link (Part 1)
How to compute the convolution between a matrix an a gradient (Sobel) filter?
Link (Part 2)
How to find the 2D convolution between two matrices?
Link
How to find a discrete approximate Gausian filter?
Link
How to find the HOG features?
Link
How to compute the gradient magnitude of pixel?
Link (Part 3)
How to compute the convolution of a 2D image with a Sobel filter?
Link (Part 2)
How to compute the convolution of a 2D image with a 1D gradient filter?
Link (Part 8)
How to compute the convolution of a 2D image with a sparse 2D filter?
Link (Part 13)
How to find the gradient magnitude using Sobel filter?
Link (Part 3)
How to find the gradient direction bin?
Link (Part 4)
📗 Convolutional Neural Network:
How to count the number of weights for training for a convolutional neural network (LeNet)?
Link
How to find the number of weights in a CNN?
Link
How to compute the activation map after a pooling layer?
Link (Part 1)
How to find the number of weights in a CNN?
Link (Part 2)
How to compute the activation map after a max-pooling layer?
Link (Part 11)
How many weights are there in a CNN?
Link (Part 11)
How to find the number of weights and biases in a CNN?
Link (Part 1)
How to find the activation map after a pooling layer?
Link (Part 2)
📗 Probability and Bayes Rule:
How to compute the probability of A given B knowing the probability of A given not B?
Link (Part 4)
How to compute the marginal probabilities given the ratio between the conditionals?
Link (Part 1)
How to compute the conditional probabilities given the same variable?
Link (Part 1)
What is the probability of switch between elements in a cycle?
Link (Part 2)
Which marginal probabilities are valid given the joint probabilities?
Link (Part 3)
How to use the Bayes rule to find which biased coin leads to a sequence of coin flips?
Link
Please do NOT forget to submit your homework on Canvas!
Link
How to use Bayes rule to find the probability of truth telling?
Link (Part 6)
How to estimate fraction given randomized survey data?
Link (Part 12)
How to write down the joint probability table given the variables are independent?
Link (Part 13)
Given the ratio between two conditional probabilities, how to compute the marginal probabilities?
Link (Part 1)
What is the Boy or Girl paradox?
Link (Part 3)
How to compute the maximum likelihood estimate of a conditional probability given a count table?
Link (Part 1)
How to compare the probabilities in the Boy or Girl Paradox?
Link (Part 3)
📗 N-Gram Model and Markov Chains:
How to compute the MLE probability of a sentence given a training document?
Link (Part 1)
How to find maximum likelihood estimates for Bernoulli distribution?
Link
How to generate realizations of discrete random variables using CDF inversion?
Link
How to find the sentence generated given the random number from CDF inversion?
Link (Part 3)
How to find the probability of observing a sentence given the first and last word using the transition matrix?
Link (Part 14)
How many conditional probabilities need to be stored for a n-gram model?
Link (Part 2)
📗 Bayesian Network:
How to compute the joint probability given the conditional probability table?
Link
How to compute conditional probability table given training data?
Link
How to do inference (find joint and conditional probability) given conditional probability table?
Link
How to find the conditional probabilities for a common cause configuration?
Link
What is the size of the conditional probability table?
Link
How to compute a condition probability given a Bayesian network with three variables?
Link
What is the size of a conditional probability table of two discrete variables?
Link (Part 2)
How many joint probabilities are needed to compute a marginal probability?
Link (Part 3)
How to compute the MLE conditional probability with Laplace smoothing given a data set?
Link (Part 2)
What is the number of conditional probabilities stored in a CPT given a Bayesian network?
Link (Part 3)
How to compute the number of probabilities in a CPT for variables with more than two possible values?
Link (Part 14)
How to find the MLE of the conditional probability given the sum of two variables?
Link (Part 5)
How many joint probabilities are used in the computation of a marginal probability?
Link (Part 4)
How to find the size of an arbitrary Bayesian network with binary variables?
Link (Part 3)
📗 Navie Bayes and Hidden Markov Model:
How to use naive Bayes classifier to do multi-class classification?
Link
How to find the size of the conditional probability table for a Naive Bayes model?
Link
How to compute the probability of observing a sequence under an HMM?
Link
What is the number of conditional probabilities stored in a CPT given a Naive Bayes model?
Link (Part 4)
How to find th obervation probabilities given an HMM?
Link (Part 2)
What is the size of the CPT for a Naive Bayes network?
Link (Part 1)
How to detect virus in email messages using Naive Bayes?
Link (Part 2)
What is the relationship between Naive Bayes and Logistic Regression?
Link
📗 Hierarchical Clustering
How to update distance table for hierarchical clustering?
Link
How to do hierarchical clustering for 1D points?
Link
How to do hierarchical clustering given pairwise distance table?
Link
📗 K-Means Clustering
What is the relationship between K Means and Gradient Descent?
Link
How to update cluster centers for K-means clustering?
Link
How to find the cluster center so that a fixed number of items are assigned to each K-means cluster?
Link
How to find the cluster center so that one of the clusters is empty?
Link (Part 9)
📗 PCA
Why is PCA solving eigenvalues and eigenvectors?
Part 1,
Part 2,
Part 3
How to compute projection?
Link
How to compute new features based on PCA?
Link
How to compute the projected variance?
Link (Part 8)
📗 Reinforcement Learning
How to compute value function given policy?
Link
How to compute optimal value function?
Link
📗 Uninformed Search
How to get expansion path for BFS?
Link
How to get expansion path for DFS?
Link
How to get expansion path for IDS?
Link
What is the shape of tree for IDS to search the quickest?
Link
How to do backtracking for search problems?
Link
How to compute time complexity for multi-branch trees?
Link
How to find the best case time complexity?
Link (Part 4)
What is the shape of the tree that minimizes the time complexity of IDS?
Link (Part 8)
What is the minimum number of nodes searched given the goal depth?
Link (Part 4)
How to find the number of states expanded during search for a large tree?
Link (Part 12)
How to find all possible configurations of the 3-puzzle?
Link (Part 1)
How to find the time complexity on binary search tree with large number of nodes?
Link (Part 2, Part 3)
How to find the shape of a search tree such that IDS is the quickest?
Link (Part 1)
📗 Informed Search
How to get expansion path for UCS?
Link
How to get expansion path for BFGS?
Link
How to get expansion path for A?
Link
How to get expansion path for A*?
Link
How to check if a heuristic is admissible?
Link
How to find the expansion sequence for uniform cost search?
Link
Which functions of two admissible heuristic are still admissible?
Link
How to do A search on a maze?
Link (Part 2)
📗 Hill Climbing
How to do hill climbing on 2D state spaces?
Link
How to do hill climbing for SAT problems?
Link
What is the number of flips needed to move from one binary sequence to another?
Link (Part 7)
What is the local minimum of a linear function with three variables?
Link (Part 14)
How to use hill climbing to solve the graph coloring problem?
Link (Part 7)
How to do hill climbing on 3D state spaces?
Link (Part 1)
How to find the shortest sequence of flipping consecutive entries to reach a specific configuration?
Link
📗 Simulated Annealing
How to find the probability of moving in simulated annealing?
Link
Which temperature would minimize the probability of moving in simulated annealing?
Link (Part 2)
📗 Genetic Algorithm
How to find reproduction probabilities?
Link
How to find the state with the highest reproduction probability given the argmax-argmin fitness functions?
Link (Part 1, Part 2)
How to compute reproduction probabilities?
Link
📗 Extensive Form Game
How to solve the lions game?
Link
How to solve the pirate game?
Link
How to solve the wage competition game (sequential version)?
Link
How to solve a simple game with Chance?
Link
How to figure out which branches can be pruned using Alpha Beta algorithm? Simple
Link, Complicated
Link
How to solve the Rubinstein Bargaining problem?
Link
How to figure out which nodes are alpha-beta pruned?
Link
How to find the solution of the II-nim game?
Link (Part 2)
How to find the solution of a game with Chance?
Link (Part 11)
How to compute the value of a game with Chance?
Link (Part 11)
How to reorder the branches so that alpha-beta pruning will prune the largest number of nodes?
Link (Part 13)
What is the order of the branches that maximizes the number of alpha-beta pruned nodes?
Link (Part 13)
How to reorder the subtrees so that alpha-beta would prune the largest number of nodes?
Link (Part 1)
How to find the value of the game for II-nim games?
Link (Part 2)
How to solve for the SPE for a game with Chance?
Link (Part 3)
📗 Normal Form Game
How to find the Nash equilibrium of a zero sum game?
Link
How to do iterated elimination of strictly dominated strategies (IESDS)?
Link
How to find the mixed strategy Nash equilibrium of a simple 2 by 2 game?
Link
What is the median voter theorem?
Link
How to guess and check a mixed strategy Nash equilibrium of a simple 3 by 3 game?
Link
How to solve the mixing probabilities of the volunteer's dilemma game?
Link
What is the Nash equilibrium of the vaccination game?
Link
How to find the mixed strategy best responses?
Link
How to compute the Nash equilibrium for zero-sum matrix games?
Link
How to draw the best responses functions with mixed strategies?
Link
How to compute the pure Nash equilibrium of the high way game?
Link (Part 5)
What is the value of a mixed strategy Nash equilibrium?
Link (Part 6)
How to compute the pure Nash equilibrium of the vaccination game?
Link (Part 5)
How to find the value of the battle of the sexes game?
Link (Part 6)
How to redesign the game to implement a Nash equilibrium?
Link (Part 10)
How to find all Nash equilibria using best response functions?
Link (Part 1)
How to compute the Nash equilibrium of the pollution game?
Link (Part 3)
How to compute a symmetric mixed strategy Nash equilibrium for the volunteer's dilemma game?
Link (Part 10)
How to perform iterated elimination of strictly dominated strategies?
Link (Part 14)
How to compute the Nash equilibrium where only one player mixes?
Link (Part 1)
How to compute the mixed Nash for the battle of sexes game?
Link (Part 1)
How to compute the game with indifferences where only one player mixes?
Link (Part 2)
How to modify the game so that a specific entry is the Nash?
Link (Part 1)
What is the Nash equilibrium of a the highway game?
Link (Part 2)
What is the Nash equilibrium of the pollution game?
Link (Part 3)
How to find the Nash equilibrium of the vaccination game?
Link (Part 4)