Computer Sciences Dept.

CS 726: Nonlinear Optimization I

Fall 2015 (also ISyE, Stat, Math)

  • Michael C. Ferris
  • Office: 4381 CS&S
  • Telephone: 262-4281
  • E-mail: I will not respond to questions about class material via email. We will use Piazza for this.
  • Office Hours: 12:00 - 1:00 Mondays, 11:00 - 12:00 Wednesdays
  • 9:55 - 10:45 MWF, 2540 Eng Hall
Mailing list
Piazza: questions and answers Getting Started

Piazza is an online tool for asking and answering questions. Piazza is available to you anywhere you have access to the internet. It is delivered and supported by a company called Piazza. Their home page is:


Logging In to Piazza

Once you're logged in, you can stay logged in on that computer. You will need to relogin if you check Piazza from another computer.

Course URL

General Course Information (

Course Outline

Theory and algorithms for nonlinear optimization, focusing on unconstrained optimization. Line-search and trust-region methods; quasi-Newton methods; conjugate-gradient and limited-memory methods for large-scale problems; derivative-free optimization; algorithms for least-squares problems and nonlinear equations; gradient projection algorithms for bound-constrained problems; and simple penalty methods for nonlinearly constrained optimization.

  • Continuous optimization paradigms
  • Representative applications
  • Mathematical background, including convex sets and functions
  • Unconstrained optimization: Theory and algorithms
    • Optimality conditions
    • Gradient methods and Newton's method
    • Line search methods
    • Quasi-Newton methods
  • Derivative-Free optimization
    • Model-based methods
    • Pattern-search methods
  • Large-scale unconstrained optimization:
    • Conjugate gradient methods (linear and nonlinear)
    • Limited-memory quasi-Newton methods
  • Duality
  • Nonlinear equations and least-squares problems
  • Optimization with bound constraints:
    • Projections and gradient projection algorithms
    • Enhancements of gradient projection using second-order information and quasi-Newton techniques.
  • Constrained nonlinear programming algorithms
    • Constraint elimination
    • Penalty methods

Required Text

  • I will use a set of notes specially prepared for this course. They will cover the first part of the Nocedal and Wright book.
  • Numerical Optimization, J. Nocedal and S.J. Wright, Springer Series in Operations Research, Springer-Verlag, New York, 2006 (2nd edition).

Other References:

  • Convex Optimization, S. Boyd and L. Vandenberghe, Cambridge University Press, UK 2004.
  • Nonlinear Optimization, Andrzej Ruszczynski, Princeton University Press, NJ 2006.
  • Nonlinear Programming, 2nd Edition, Dimitri Bertsekas, Athena Scientific, Belmont, MA 1999.
  • Practical Methods of Optimization, 2nd Edition, R. Fletcher, Wiley, 1987.
  • Practical Optimization, P. Gill, W. Murray and M. Wright, Academic Press, 1981.
  • Nonlinear Programming Theory and Algorithms , M. S. Bazaraa, H. D. Sherali and C. M. Shetty, Second Edition, Wiley, New York 2006.
  • The MATLAB PRIMER (Third Edition): An introduction to the basic commands and utilities that you may need in Matlab.


Programming Assignments and Homeworks

    All assignments need to be written up entirely separately. You may discuss the problems informally with others in the class, but the discussions must not include code or explicit solutions to any of the problems.
  • N Assignments total. Most homeworks will be handed in either in hard copy or using the drop box facility of Learn@UW. The drop box menu is found in the top menu bar once you have logged into the system. Most of the assignments will require the use of MATLAB, which will also be used extensively in the lectures. No homework or project accepted in mailbox of instructor. Further details will be provided when the assignments are passed out. Homework due at beginning of class one week after assigned unless otherwise noted.
  • Examinations are closed book, with the exception that 1 handwritten sheet (standard size paper) can be brought in to the examination.
  • Midterm Examination: Monday October 12 at 7:15 - 9:15pm in EH 2535.
  • Final Examination - Monday, December 21 at 10:05 am - 12:05 pm in 1221 CS. Previous exams (from 730) are given below, relevant questions are: 2003 (Q1), 2004 (Q2), 2005 (Q1)
  • Final Exam 2003
  • Final Exam 2004
  • Final Exam 2005
  • Final Exam 2009
  • Final Exam 2011
  • Prereq: Familiarity with basic analysis (e.g. Math 521) and either Math 443 or 320, or consent of instructor


  • Grades for the class will be available at Learn@UW. You will need to log-on, move to the course page, and use the "Grades" tab at the top of the page.
  • Approximately: 10% Scribing, 30% Homework, 20% Midterm, 40% Final
  • Each person should sign up for one class to scribe. Lectures should be written in Latex using the style that can be found on pizza. Homework solutions must be provided as a pdf file (or matlab m files) in a single zip (not rar or any other format) file on Learn@UW. Both scribing and homeworks will be marked as 0/5/10 by the instructor. 5 marks will be given for a decent attempt at the assignment. 10 marks will be given for a good to excellent solution, but no feedback will be given apart from this. An additional 0/5/10 marks will be given for activity (answering questions, etc) on piazza that will go towards the homework score.
  • You may discuss the assignments with your classmates. However, you may not share any code, copy solution from another person, or carry out an assignment together. Discussion should only involve verbal communication. All assignments need to be written up entirely separately. Learning is a collaborative process, so you are encouraged to verbally discuss the course work, including homework, with other students. However, work that you hand in must have been prepared by you alone. However, it is imperative that you understand the material on your own. You (obviously) can't ask your study-partners for help on the exam, so it is up to the student to ensure that they have individually mastered the material. An effective way of ensuring this is to work on the homework assignments alone before seeking help or collaborating with others. If it is perceived that homeworks are being completed by teams of individuals, a modification of the course grading allocation may be necessary.

    Submitting someone else's work as your own is academic misconduct. Such cheating and plagiarism will be dealt with in accordance with University procedures (see the Academic Misconduct Guide for Students) .

  • You may use supplementary material to enhance your understanding of the course material. If you use external references of any kind, even if they are not quoted verbatim, YOU MUST CITE THEM! This rule will be strictly enforced and violations will be dealt with harshly. Again, I encourage you to seek outside sources, but you must acknowledge the source of any ideas that are not your own.

CS Department Computing Information


This page was updated September 2, 2015.

Computer Sciences | UW Home