# Other Materials

Part 1 (Generative Models): Link
Part 2 (Natural Language): Link
Part 3 (Sampling): Link
Part 4 (Probability Distribution): Link
Part 5 (Bayesian Network): Link
Part 6 (Network Structure): Link
Part 7 (Naive Bayes): Link

Zipf's Law: Link
Markov Chain: Link
Google N-Gram: Link

Simple Bayes Net: Link, Link 2
ABNMS: Link, pathfinder: Link


How to find the HOG features? Link
How to count the number of weights for training for a convolutional neural network (LeNet)? Link
Example (Quiz): How to find the 2D convolution between two matrices? Link
Example (Homework): How to find a discrete approximate Gausian filter? Link

# Keywords and Notations

Distance: (Euclidean) $\rho (x,x^{\prime })={\Vert x-x^{\prime }\Vert }_2=\sqrt{{\displaystyle \sum _{j=1}^m{(x_j-x_j^{\prime })}^2}}$, (Manhattan) $\rho (x,x^{\prime })={\Vert x-x^{\prime }\Vert }_1={\displaystyle \sum _{j=1}^m|x_j-x_j^{\prime }|}$, where $x,x^{\prime }$ are two instances.
K-Nearest Neighbor classifier: ${\hat{y}}_i$ = mode $\{y_{\left(1\right)},y_{\left(2\right)},...,y_{\left(k\right)}\}$, where mode is the majority label and $y_{\left(t\right)}$ is the label of the $t$-th closest instance to instance $i$ from the training set.

Unigram model: $\mathbb{P}\{z_1,z_2,...,z_d\}={\displaystyle \prod _{t=1}^d\mathbb{P}\left\{z_t\right\}}$ where $z_t$ is the $t$-th token in a training item, and $d$ is the total number of tokens in the item.
Maximum likelihood estimator (unigram): $\hat{\mathbb{P}}\left\{z_t\right\}={\displaystyle \frac{c_{z_t}}{{\displaystyle \sum _{z=1}^m{c}_z}}}$, where $c_z$ is the number of time the token $z$ appears in the training set and $m$ is the vocabulary size (number of unique tokens).
Maximum likelihood estimator (unigram, with Laplace smoothing): $\hat{\mathbb{P}}\left\{z_t\right\}={\displaystyle \frac{c_{z_t}+1}{\left({\displaystyle \sum _{z=1}^m{c}_z}\right)+m}}$.
Bigram model: $\mathbb{P}\{z_1,z_2,...,z_d\}=\mathbb{P}\left\{z_1\right\}{\displaystyle \prod _{t=2}^d\mathbb{P}\left\{z_t|z_{t-1}\right\}}$.
Maximum likelihood estimator (bigram): $\hat{\mathbb{P}}\left\{z_t|z_{t-1}\right\}={\displaystyle \frac{c_{z_{t-1},z_t}}{c_{z_{t-1}}}}$.
Maximum likelihood estimator (bigram, with Laplace smoothing): $\hat{\mathbb{P}}\left\{z_t|z_{t-1}\right\}={\displaystyle \frac{c_{z_{t-1},z_t}+1}{c_{z_{t-1}}+m}}$.