Schedule
In general, classes will be held on MWF every week, and most lectures will be 60 minutes, but may take 70 minutes on a few occasions. (I will be absent on a number of class days, and the longer lectures will make up for these absences.)

General Course Information
Prerequisite
 Linear Algebra, some Analysis. See guidebook for specifics.

The course will involve some programming to test algorithms. One useful option is to use Matlab with the free addon cvx. A second option (particularly appealing if you took 524 recently) is to use Julia with the JuMP optimization toolbox. A third option is to use Python, but I can provide less support for this.
Text
 J. Nocedal and S. J. Wright, Numerical Optimization, Second Edition, Springer, 2006. (It's essential to get the second edition!) Here is the current list of typos.
References
 D. P. Bertsekas, with A. Nedic and A. Ozdaglar, Convex Analysis and Optimization, Athena Scientific, Belmont, MA, 2003.
 Nesterov, Y., Introductory Lectures on Convex Optimization, Kluwer, 2004.
 S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press. Available here.
 D. P. Bertsekas, Nonlinear Programming, Second Edition, Athena Scientific, Belmont, MA, 1999.
 R. Fletcher, Practical Methods of Optimization, 2nd Edition, Wiley, Chichester & New York, 1987.
 R. T. Rockafellar and R. J.B. Wets, Variational Analysis, Springer, 1998. (This is a more advanced book and an invaluable reference.)
 A. Ruszczynski, Nonlinear Optimization, Princeton University Press, 2006.
 S. J. Wright, PrimalDual InteriorPoint Methods, SIAM, 1997.

Course Outline
This will likely be adadpted as the semester proceeds, but most of the following topics will be covered.
 Introduction
 Optimization paradigms and applications
 Mathematical background: convex sets and functions, linear algebra, topology, convergence rates
 Smooth unconstrained optimization: Background
 Taylor's theorem
 Optimality conditions
 FirstOrder Methods
 Steepest descent. Convergence for convex and nonconvex cases.
 Accelerated gradient. Convergence for convex case.
 Line search methods based on descent directions
 Conjugate gradient methods
 Conditional gradient for optimization over closed convex sets
 Higherorder methods
 Newton's method
 Linesearch Newton
 Trustregion Newton and cubic regularization
 Conjugate gradientNewton
 QuasiNewton methods
 Limitedmemory quasiNewton
 Stochastic optimization
 Basic methods and their convergence properties
 reducedvariance approaches
 Differentiation
 Adjoint calculations
 Automatic differentiation
 Leastsquares and nonlinear equations
 Linear least squares: direct and iterative methods
 Nonlinear least squares: GaussNewton, LevenbergMarquardt
 Newton’s method for nonlinear equations
 Merit functions for nonlinear equations, and line searches
 Optimization with linear constraints
 Normal cones to convex sets
 Farkas Lemma and firstorder optimality conditions (KKT)
 Gradient projection algorithms
