Prev: L27, Next: L29

# Lecture

📗 The lecture is in person, but you can join Zoom: 8:50-9:40 or 11:00-11:50. Zoom recordings can be viewed on Canvas -> Zoom -> Cloud Recordings. They will be moved to Kaltura over the weekends.
📗 The in-class (participation) quizzes should be submitted on TopHat (Code:741565), but you can submit your answers through Form at the end of the lectures too.
📗 The Python notebooks used during the lectures can also be found on: GitHub. They will be updated weekly.


# Lecture Notes


TopHat Exercise
➩ Guess which face is real?
➩ Link

📗 Image Pixel Intensity Features
➩ Gray scale images have pixels valued from 0 to 255 (integers, 0 being black, 255 being white), or from 0 to 1 (real values, 0 being black, 1 being white).
➩ These pixel values are called pixel intensities.
➩ The pixel value matrices can be flattened into a vector (i.e. \(k\) pixel by \(k\) pixel image can be written as a list of \(k^{2}\) numbers and used as input features.

Handwriting Example

➩ Random MNIST Link features:
➩ Plot from features or draw on the canvas:


➩ Note: this plots 0 as white and 1 as black, unlike matplotlib.imshow or skimage.imshow.

 Image Preprocessing
➩ scikit-image can be used for basic image processing tasks: Link.
➩ skimage.io.imread (read image from file), skimage.io.imshow (show image): Doc.
➩ The images are stored as numpy.array, which can be flattened using numpy.ndarray.flatten: Doc.

📗 Color Images
➩ RGB color images have color pixels represented by three numbers Red, Green, Blue, each valued from 0 to 255 or 0 to 1.
➩ Intensity can be computed as \(I = \dfrac{1}{3} \left(R + G + B\right)\).
➩ Other perceptual luminance-preserving conversion can be done too, for example \(I = 0.2125 R + 0.7154 G + 0.0721 B\): Link.
➩ skimage.color.rgb2gray (convert into gray-scale): Link
➩ The pixel value 3D arrays can be converted into pixel intensity matrices and flattened into a feature vector.

Image Feature Example
➩ Compute and visualize the pixel intensity features of an image of Spot (the cow): Link
➩ Code to compute pixel intensities: Notebook.

 Feature Engineering
➩ Sometimes more complicated features are constructed for images to preserving information about neighboring pixels.
➩ Features can be learned in a supervised manner (convolutional neural networks) or a unsupervised manner (dimensionality reduction).

📗 Convolution
➩ Convolution between an image and a filter (sometimes called kernel) is the dot product of every region of an image and the filter (sometimes flipped).
➩ Convolution with special filters detects special features: Link
➩ skimage.filters has a list of special filters: Doc.

Name Example Use
Identity \(\begin{bmatrix} 0 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{bmatrix}\) nothing
Gaussian \(\begin{bmatrix} 0.0625 & 0.125 & 0.0625 \\ 0.125 & 0.25 & 0.125 \\ 0.0625 & 0.125 & 0.0625 \end{bmatrix}\) blur
X gradient \(\begin{bmatrix} 0 & 0 & 0 \\ 1 & 0 & -1 \\ 0 & 0 & 0 \end{bmatrix}\) vertical edges
Left (Right) Sobel \(\begin{bmatrix} 1 & 0 & -1 \\ 2 & 0 & -2 \\ 1 & 0 & -1 \end{bmatrix}\) vertical edges (blurred)
Y gradient \(\begin{bmatrix} 0 & 1 & 0 \\ 0 & 0 & 0 \\ 0 & -1 & 0 \end{bmatrix}\) vertical edges
Top (Bottom) Sobel \(\begin{bmatrix} 1 & 2 & 1 \\ 0 & 0 & 0 \\ -1 & -2 & -1 \end{bmatrix}\) vertical edges (blurred)


Convolution Examples
➩ Compute and visualize the edge features for Spot.
➩ Code to compute edge features: Notebook.

 Image Gradient Features
➩ One popular face recognition algorithms uses HOG (Histogram of Gradients) features.
(1) Compute the X and Y gradient for each pixel (either X or Y gradient filters or Sobel filters),
(2) Compute a histogram for pixel gradients in each of the 8 orientations for every 16 by 16 pixel regions (8 numbers for every 16 x 16 region).
(3) Concatenate the histograms to use as the feature vector.
➩ skimage.feature.hog computes the HOG features: Link.

HOG Feature Examples
➩ Compute and visualize the HOG feature for Spot.
➩ Code to compute HOG: Notebook.


 Notes and code adapted from the course taught by Yiyin Shen Link and Tyler Caraza-Harter Link






Last Updated: November 18, 2024 at 11:43 PM