📗 A degree \(n\) polynomial, \(c_{n} x^{n} + c_{n-1} x^{n-1} + c_{n-2} x^{n-2} + ... + c_{2} x^{2} + c_{1} x + c_{0}\) is specified by a list of coefficients \(c_{n}, c_{n-1}, ..., c_{1}, c_{0}\) usually stored in a vector.
📗 The computation can be done efficiently using Horner's method: \(\left(\left(\left(\left(c_{n} x + c_{n-1}\right) x + c_{n-2}\right) x + ... + c_{2}\right) x + c_{1}\right) x + c_{0}\).
📗 Many elementary functions can be evaluated and approximated using polynomials.
📗 A set of \(n\) points (with distinct \(x\) values, and ordered according to \(x\)) can define \(n - 1\) cubic polynomial segments satisfying the following conditions.
➩ The \(i\)-th polynomial segment connects point \(i\) and point \(i - 1\).
➩ The segments form a smooth curve. Technically, \(C^{\left(2\right)}\) smoothness is desired: the first and second derivatives should be continuous.
📗 With \(n\) points, there are \(4 n - 4\) coefficients to compute. Suppose the coefficient for segment \(i\) connecting \(\left(x_{i}, y_{i}\right)\) and \(\left(x_{i+1}, y_{i+1}\right)\) is \(c_{i1}, c_{i2}, c_{i3}, c_{i4}\).
➩ \(c_{i1} x_{i}^{3} + c_{i2} x_{i}^{2} + c_{i3} x_{i} + c_{i4} = y_{i}\) for \(i = 1, 2, ..., n - 1\) so that the segment passes through the left end point.
➩ \(c_{i1} x_{i+1}^{3} + c_{i2} x_{i+1}^{2} + c_{i3} x_{i+1} + c_{i4} = y_{i+1}\) for \(i = 1, 2, ..., n - 1\) so that the segment passes through the right end point.
➩ \(3 c_{i1} x_{i+1}^{2} + 2 c_{i2} x_{i+1} + c_{i3} = 3 c_{\left(i+1\right)1} x^{2}_{i+1} + 2 c_{\left(i+1\right)2} x_{i+1} + c_{\left(i+1\right)3}\) for \(i = 1, 2, ..., n - 2\) so that the first derivative is continuous.
➩ \(6 c_{i1} x_{i+1} + 2 c_{i2} = 6 c_{\left(i+1\right)1} x_{i+1} + 2 c_{\left(i+1\right)2}\) for \(i = 1, 2, ..., n - 2\) so that the second derivative is continuous.
➩ There are \(4 n - 6\) equations. \(2\) more equations are needed.
📗 There are many ways to specify the \(2\) extra equations.
➩ If the third derivative for the first and last two segments are continuous, \(c_{11} = c_{21}\) and \(c_{\left(n-1\right)1} = c_{\left(n-2\right)1}\), the resulting spline is called a not-a-knot spline.
➩ If the second derivative at the end points are \(0\), or \(6 c_{11} x_{1} + 2 c_{12} = 0\) and \(6 c_{\left(n-1\right)1} x_{n} + 2 c_{\left(n-1\right)2} = 0\), the resulting spline is called a natural spline.
📗 Suppose \(x, y\) are two row vectors with length \(n\).
📗 spline(x, y).coefs finds the coefficients of the cubic spline that interpolates \(\left(x, y\right)\) while satisfying the not-a-knot condition.
📗 spline(x, y, x0) evaluates the cubic spline interpolating \(\left(x, y\right)\) at \(x = x_{0}\).
📗 csape(x, y, 'variational').coefs finds the coefficients of the natural cubic spline. It requires MATLAB's Curve Fitting Toolbox.
📗 spline(x, y).coefs are local coefficients meaning the equation for segment \(i\) and \(x \in \left[x_{i}, x_{i+1}\right]\) is \(c_{i1} \left(x - x_{i}\right)^{3} + c_{i2} \left(x - x_{i}\right)^{2} + c_{i3} \left(x - x_{i}\right) + c_{i4}\).
📗 The polynomial can be expanded to find the coefficients of the original spline problem.
📗 polyfit(x, y, d) with \(d < n\), finds the polynomial that is the closest to the points \(\left(x, y\right)\).
📗 The total distance from the points to the polynomial is defined as the sum of the squared differences between the \(y\) value at \(x\) and the value of the polynomial at \(x\), \(D = \displaystyle\sum_{i=1}^{n} \left(y_{i} - \displaystyle\sum_{j=0}^{d} c_{i} x_{i}^{j}\right)^{2}\) = \(\displaystyle\sum_{i=1}^{n} \left(y_{i} - c_{d} x_{i}^{d} - c_{d-1} x_{i}^{d-1} - ... - c_{1} x_{i} - c_{0}\right)^{2}\).
📗 polyfit finds the coefficients of the polynomial that minimizes \(D\). These coefficients are also called regression coefficients.
📗 regress(y', X) finds the regression coefficients for \(c_{n} x_{n} + c_{n-1} x_{n-1} + ... + c_{1} x_{1} + c_{0} x_{0}\), where \(x_{0}, x_{1}, ..., x_{n}\) are columns of \(X\). It requires MATLAB's Statistics and Machine Learning Toolbox.
📗 regress(y', X) solves \(\left(X^\top X\right) c = X^\top y\) instead of \(X c = y\). This is because \(X^\top X\) is always invertible (meaning \(\left(X^\top X\right)^{-1}\) always exists), so \(c = \left(X^\top X\right)^{-1} X^\top y\).
📗 When \(X\) is the degree \(d\) Vandermonde matrix for \(x\), meaning \(x_{i} = x^{i}\) for \(i = d, d - 1, ..., 0\), regress(y, X) finds the same coefficients as polyfit(x, y, d).
TopHat Quiz
📗 What are the values of \(c, d\) such that the curve \(y = c x^{2} + d\) best fits \(\left(x, y\right)\) where x = 1:10; y = 1 + x .^ 2;.
📗 polyfit(f(x), y, 1) find the regression coefficients for \(c_{1} f\left(x\right) + c_{0}\), for example, polyfit(sin(x), y, 1) finds \(c_{1}, c_{0}\) such that the total distance from the points to the function \(y = c_{1} \sin\left(x\right) + c_{0}\) is minimized.
📗 regress(y', [f1(x'), f2(x') ...]) finds the regression coefficients for \(c_{1} f_{1}\left(x\right) + c_{2} f_{2} \left(x\right) + ...\), for example, regress(y', [sin(x'), cos(x'), ones(length(x), 1)]) finds \(c_{2}, c_{1}, c_{0}\) such that the total distance from the points to the function \(y = c_{2} \sin\left(x\right) + c_{1} \cos\left(x\right) + c_{0}\) is minimized.
TopHat Quiz
📗 What are the values of \(c, d\) such that the curve \(y = c x^{d}\) best fits \(\left(x, y\right)\) where x = 1:10; y = x;.
➩ A: v = polyfit(log(x), log(y), 1); [exp(v(2)), v(1)]
➩ B: v = polyfit(x, log(y), 1); [exp(v(2)), v(1)]
➩ C: v = polyfit(log(x), log(y), 1); [v(2), v(1)]
➩ D: v = polyfit(x, log(y), 1); [v(2), v(1)]
TopHat Quiz
📗 What are the values of \(c, d\) such that the curve \(y = c e^{d x}\) best fits \(\left(x, y\right)\) where x = 1:10; y = exp(x);.
➩ A: v = polyfit(log(x), log(y), 1); [exp(v(2)), v(1)]
📗 Notes and code adapted from the course taught by Professors Beck Hasti and Michael O'Neill.
📗 You can expand all TopHat Quizzes and Discussions: .
📗 If there is an issue with TopHat during the lectures, please submit your answers on paper (include your Wisc ID and answers) or this Google form Form at the end of the lecture.