CS 730: Nonlinear Optimization II - Spring 2011

Schedule

Lecture: 11:00-12:15 MWF, Engineering Hall 2239
Class Mailing List: compsci730-1-s11@lists.wisc.edu (here's the archive)
Mail Instructor/TAs
Course URL: http://www.cs.wisc.edu/~swright/cs730.html

In general, lectures will be 60 minutes, but may take up the full 75-minutes slot on a few occasions. I will be absent on a number of class days, and the longer lectures will make up for these absences.

The lecture schedule is posted below - it is subject to change but each week's schedule will be finalized by the preceding week.

  • Week 1: Wed 1/19 (60 mins), Fri 1/21 (65 mins)
  • Week 2: Mon 1/24 (55 mins), Wed 1/26 (55 mins), Fri 1/28 (60 mins)
  • Week 3: Mon 1/31 (60 mins), Wed 2/2 (cancelled: blizzard), Fri 2/4 (60 mins)
  • Week 4: Mon 2/7 (70 mins), Wed 2/9 (60 mins), Fri 2/11 (60 mins)
  • Week 5: Mon 2/14 (60 mins), Wed 2/16 (60 mins), Fri 2/18 (60 mins)
  • Week 6: Mon 2/21 (60 mins), Wed 2/23 (60 mins), Fri 2/25 (65 mins)
  • Week 7: Mon 2/28 (60 mins), Wed 3/2 (60 mins), Fri 3/4 (50 mins)
  • Week 8: Mon 3/7 (60 mins), Wed 3/9 (60 mins) (MIDTERM 7:15-9:15), Fri 3/11 (no class)
  • (Spring Break: 3/14 - 3/18)
  • Week 9: Mon 3/21 (55 mins), Wed 3/23 (65 mins), Fri 3/25 (60 mins)
  • Week 10: Mon 3/28 (60 mins), Wed 3/30 (60 mins), Fri 4/1 (60 mins)
  • Week 11: Mon 4/4 (no lecture), Wed 4/6 (no lecture), Fri 4/8 (60 mins)
  • Week 12: Mon 4/11 (60 mins), Wed 4/13 (60 mins), Fri 4/15 (60 mins)
  • Week 13: Mon 4/18 (60 mins), Wed 4/20 (65 mins), Fri 4/22 (60 mins)
  • Week 14: Mon 4/25 (60 mins), Wed 4/27 (60 mins), Fri 4/29 (60 mins)
  • Week 15: Mon 5/2 (60 mins), Wed 5/4 (65 mins), Fri 5/6 (no class)

 

Instructor: Steve Wright

Office: 4379 CS
Phone: 262-4838
Email:
Office Hours: Tuesday 4-5, Friday 9:30-10:30

General Course Information

Prerequisite

  • CS / ISyE 726 or equivalent (see me if you think you have done an equivalent course).

Text

  • J. Nocedal and S. J. Wright, Numerical Optimization, Second Edition, Springer, 2006. (It's essential to get the second edition! The version published in China in 2006 is a reprint of the first edition.) Here is the current list of typos.

References

  • D. P. Bertsekas, with A. Nedic and A. Ozdaglar, Convex Analysis and Optimization, Athena Scientific, Belmont, MA, 2003.
  • S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press. Available here.
  • D. P. Bertsekas, Nonlinear Programming, Second Edition, Athena Scientific, Belmont, MA, 1999.
  • R. Fletcher, Practical Methods of Optimization, 2nd Edition, Wiley, Chichester & New York, 1987.
  • R. T. Rockafellar and R. J.-B. Wets, Variational Analysis, Springer, 1998. (This is a more advanced book and an invaluable reference.)
  • A. Ruszczynski, Nonlinear Optimization, Princeton University Press, 2006.

(All have been placed on reserve at the Wendt Library.)

Lecture Notes (updated sporadically).

These notes are for my own benefit and hence are terse, and refer heavily to the text. I supply them to help you fill in any missing details in the notes that you take in class.

 

Course Outline

  • Geometric viewpoint of constrained optimization
    • Convex sets, cones, projections
    • Tangent and normal to polyhedral sets
    • Theorems of the alternative, separation results
    • First-order conditions: polyhedral case
  • Optimality conditions for nonlinear programming
    • Constraint qualifications
    • First-order conditions and saddle points
    • Second-order conditions and critical cones
    • Degeneracy
  • Duality for nonlinear programming, including Wolfe and Fenchel duality
  • Nonlinear programming algorithms
    • Fundamentals: merit functions and filters, Maratos effect.
    • Interior-point and augmented Lagrangian methods
    • Sequential quadratic programming
  • Second-order cone programming and semidefinite programming: applications, barrier methods.

Assessment

(The scheme below is provisional and subject to change until the third week of semester.)

Keep track of your grades through the learn@uw system.

  • Approximately 8 homework assignments, 35% of grade. We may grade only 4-5 homeworks.
    • The electronic handin system may be used for some homeworks - see here for details
    • Homework is due at the beginning of class on the designated date.
    • No homeworks will be accepted by TAs, in mailbox or in person.
    • No homework or project is accepted in mailbox of instructor.
    • You may discuss homework with classmates. However, you may not share any code, carry out the assignment together, or copy solutions from another person. Discussion should be verbal only. The submitted version must be worked out, written, and submitted by you alone.
    • Submitting someone else's work as your own is academic misconduct. Such cheating and plagiarism will be dealt with in accordance with University procedures (see the Academic Misconduct page).
  • MIDTERM, 25% of grade. Wednesday 3/9/11, 7:15pm-9:15pm, Room 1257 CS. You may bring one sheet of paper, handwritten on both sides, into the exam.
  • FINAL, 40% of grade. Sunday 5/8/11, 2:45pm-4:45pm, Room EH 2239 (same as classroom). You may bring one sheet of paper, handwritten on both sides, into the exam.

Homeworks

Previous Exams

Handouts and Examples

Miscellaneous