📗 The lecture is in person, but you can join Zoom: 8:50-9:40 or 11:00-11:50. Zoom recordings can be viewed on Canvas -> Zoom -> Cloud Recordings. They will be moved to Kaltura over the weekends.
📗 The in-class (participation) quizzes should be submitted on TopHat (Code:741565), but you can submit your answers through Form at the end of the lectures too.
📗 The Python notebooks used during the lectures can also be found on: GitHub. They will be updated weekly.
TopHat Exercise
➭ Guess which face is real?
➭ Link
📗 Image Pixel Intensity Features
➭ Gray scale images have pixels valued from 0 to 255 (integers, 0 being black, 255 being white), or from 0 to 1 (real values, 0 being black, 1 being white).
➭ These pixel values are called pixel intensities.
➭ The pixel value matrices can be flattened into a vector (i.e. \(k\) pixel by \(k\) pixel image can be written as a list of \(k^{2}\) numbers and used as input features.
Handwriting Example
➭ Random MNIST Link features:
➭ Plot from features or draw on the canvas:
➭ Note: this plots 0 as white and 1 as black, unlike matplotlib.imshow or skimage.imshow.
📗 Image Preprocessing
➭ scikit-image can be used for basic image processing tasks: Link.
➭ skimage.io.imread (read image from file), skimage.io.imshow (show image): Doc.
➭ The images are stored as numpy.array, which can be flattened using numpy.ndarray.flatten: Doc.
📗 Color Images
➭ RGB color images have color pixels represented by three numbers Red, Green, Blue, each valued from 0 to 255 or 0 to 1.
➭ Intensity can be computed as \(I = \dfrac{1}{3} \left(R + G + B\right)\).
➭ Other perceptual luminance-preserving conversion can be done too, for example \(I = 0.2125 R + 0.7154 G + 0.0721 B\): Link.
➭ skimage.color.rgb2gray (convert into gray-scale): Link
➭ The pixel value 3D arrays can be converted into pixel intensity matrices and flattened into a feature vector.
Image Feature Example
➭ Compute and visualize the pixel intensity features of an image of Spot (the cow): Link
➭ Code to compute pixel intensities: Notebook.
📗 Feature Engineering
➭ Sometimes more complicated features are constructed for images to preserving information about neighboring pixels.
➭ Features can be learned in a supervised manner (convolutional neural networks) or a unsupervised manner (dimensionality reduction).
📗 Convolution
➭ Convolution between an image and a filter (sometimes called kernel) is the dot product of every region of an image and the filter (sometimes flipped).
➭ Convolution with special filters detects special features: Link
➭ skimage.filters has a list of special filters: Doc.
Convolution Examples
➭ Compute and visualize the edge features for Spot.
➭ Code to compute edge features: Notebook.
📗 Image Gradient Features
➭ One popular face recognition algorithms uses HOG (Histogram of Gradients) features.
(1) Compute the X and Y gradient for each pixel (either X or Y gradient filters or Sobel filters),
(2) Compute a histogram for pixel gradients in each of the 8 orientations for every 16 by 16 pixel regions (8 numbers for every 16 x 16 region).
(3) Concatenate the histograms to use as the feature vector.
➭ skimage.feature.hog computes the HOG features: Link.
HOG Feature Examples
➭ Compute and visualize the HOG feature for Spot.
➭ Code to compute HOG: Notebook.
📗 Notes and code adapted from the course taught by Yiyin Shen Link and Tyler Caraza-Harter Link