%%Successive Numeric Approximation

% Idea: Solve a mathematical problem using a numeric algorithm to
% estimate the solution
% General structure: Use the current estimate to generate a better
% estimate for the solution
% This technique is applied to difficult mathematical problems that
% can't be solved analytically (e.g. calculus rules)
% These methods are used in built-in MATLAB functions like fzero,
% integral, fminunc, etc.

%Issues to consider:

% Does our algorithm always produce better estimates than the current
% estimate?
% If it does, this means it will converge to a solution!

% How quickly does our algorithm generate a "good" estimate? Known
% as the "convergence rate" of the algorithm.

% Different ways to calculate error in the current estimate:
% Absolute error - |approx value - exact value|
% Relative error - absolute error/|exact value|
% Percent error - relative error * 100%

% Issue: If we knew the exact value, we wouldn't be running the
% algorithm because we don't need to solve it approximately
% Can estimate the error in problem dependent ways
% Ex: For finding a root of the function f, we want a point where
% f(x) = 0. If we have a current estimate y, we can estimate the
% error using f by looking at |f(y) - f(x)| = |f(y)|

% We consider an algorithm to have converged when we calculate an
% error that is below a tolerance epsilon, which is some small
% positive number (e.g. 10^-12)

% Convergence rates -  an indication of how quickly we converge to
% the actual solution
%   Linear: error is reduced by a factor of the previous error
%   for example, new error = (1/2)*old error.
%   Quadratic: error is reduced by a factor of the previous error
%   squared. For example, new error = (old error)^2.
%   Cubic: error is reduced by a factor of the previous error cubed.
%   For example, new error = (old error)^3.

%Note: for quadratic and cubic convergence rates, often need a good
%estimate to start that has a relatively small error.

%Succesive numeric approximation is an extremely common tool that
%applies to many problems (just checkout the MATLAB documentation to
%see a bunch of them). We will only focus on root finding.
