CS 703
Section 1
Fall 2023
News
Watch this space for the latest updates.
The class mailing list is
compsci703-1-f23@g-groups.wisc.edu.
An archive of mail sent to the class can be found
here.
Covid accommodations
My colleagues are reporting that students in their classes are testing positive for Covid.
Please err on the side of caution.
If you test positive, please do the following:
Abstracts for the final-project presentations posted
The abstracts are available here.
Last updated:
5:56 PM, Monday, December 8, 2023
Contents
General Information
Course Subject, Number, and Title
Computer Sciences 703: Program Verification and Synthesis
Meeting Times and Location
Mondays, Wednesdays, and Fridays, 2:30 PM - 3:45 PM, on the days indicated in the
course schedule, in 1240 CS.
On September 15 and October 20, there are conflicting events in 1240 CS, so we wil meet on
Zoom.
Depending on the Covid situation, other classes may be broadcast on that link, too.
Credits
3
How the Course Meets the Credit-Hour-Policy Standards
Modified ``3-Credit Course, Option B'':
The classroom is reserved for three 75-minute class periods each week over the
semester;
however, on average we will meet for two 75-minute class periods per week.
Until late October, we will front-load the classes and meet for three
75-minute class periods most weeks, after which there will be no
classes, and you will have five uninterrupted weeks to work on a
project of your own choosing.
In the final week of the term, we will hold enough classes for each project-group to make
an 18-minute presentation about the work they carried out.
See the course schedule for the dates on which
classes will be held.
The class carries the expectation that, on average, students will work on course learning activities
(reading, writing, problem sets, projects, etc.) for about 3 hours out of the classroom
for every class period.
More information about the expectations for student work can be found elsewhere on this page.
Instructional Mode
Lecture by the instructor, and at the end of the term, a short project presentation by each project-group.
UW Policies
The course will be conducted in conformance with the UW policies found on the page at
this link.
In particular, policies about accommodating religious observances are detailed at
this link.
Please note that ``students are responsible for notifying instructors within the first two weeks of classes of the date(s) when they request relief due to a religious observance.''
Instructor
Professor Thomas Reps
Office: 6361 Computer Sciences
Office hours: By appointment, typically on Zoom
E-mail: reps at cs.wisc.edu
Home page: www.cs.wisc.edu/~reps/
Textbook and Notes
There is no required textbook for the course.
Much of the reading material for the course is available on-line;
see the course bibliography.
There are on-line notes for some of the topics we will cover in this
directory.
The relevant notes cover the following topics:
- Static program analysis
- Automatic differentiation and back-propagation (PDF)
For material about program synthesis, you may wish to consult [D'Antoni and Polikarpova 2022].
Over the course of the semester, we will accumulate a set of PowerPoint slides that will be posted in the Course Schedule.
and possibly other topics as time permits.
Assignments
There may be a few problem sets (2-3), possibly a paper critique or two, and a course project.
The bulk of the coursework will be the project.
Course Project
For the course project, you will work on a problem with a partner.
The first step is to find a partner and write a project proposal (1-2 pages) that describes
the following items:
- Background: A short summary of the background needed to appreciate the problem
- Problem statement: A concise statement of the problem to be investigated
- Rationale: A discussion of why the problem is interesting
- Objectives and scope: A description of what you propose to do
- Explain the elements that you will have to build
- Explain the elements that you can pick up from open-source sites
- Explain the experiment(s) and/or performance measurement(s) that you
plan to carry out. A good way to make it clear what you hope to do is as follows:
- State the hypotheses that you hope to refute.
- Complete the following sentence with 2-4 bullet points:
``The experiments are designed to shed light on the following questions: . . .''
Then explain what you plan to measure; how you will measure it (if it is not obvious); and
where you will obtain test cases.
- List the tasks, broken down into two or three milestones
A list of possible project topics can be found
here.
However, you are not required to choose one of the topics from the list;
you may propose a topic of your own devising.
If you are already doing research in some area of CS, you may wish to exploit
your existing domain expertise by exploring some issue in that area that is related to verification or synthesis.
So that you have an idea of the scope of an appropriate project, here are the abstracts of the
presentations from the Fall 2015 version of a related course:
Fall 2015 CS 701 abstracts.
To help keep you on track, I will impose five deadlines:
- Due date (for proposal): 5:15 PM, Friday, October 27, 2023 [7.5 weeks]. (Send as a PDF file, by e-mail to reps at cs.wisc.edu)
-
Milestone 1: I am interested in evidence that
you are making forward progress. I will leave it up
to you as to what makes the most sense to turn in.
For some projects, it could be some worked examples
and a summary of planned next steps;
for others, it could be an implementation plan with completed
steps checked off. Please turn in an updated proposal (with
changes marked with changebars [\usepackage{changebar}
if you are using LaTeX]), and your new material added as
``Appendix A: Milestone 1''.
Due date (for Milestone 1 document): 5:15 PM, Friday, November 10, 2023 [9.5 weeks]. (Send as a PDF file, by e-mail to reps at cs.wisc.edu).
-
Milestone 2: Description of progress, implementation
plan with completed steps checked off, and experimentation
plan. Please turn in an updated proposal (with changes marked
with changebars, and your new material added as
``Appendix B: Milestone 2''.
Due date (for Milestone 2 document): 5:15 PM, Wednesday, November 22, 2023 [11 weeks]. (Send as a PDF file, by e-mail to reps at cs.wisc.edu).
-
Presentation: oral presentations
will be given during class at the end of the semester.
Presentations will be 20 minutes for 3-person groups, 17 minutes for 2-person groups, and 14 minutes for 1-person groups.
In addition, we will have 3 minutes for questions/discussion.
You will need to e-mail me an abstract (in plaintext)
giving the title, project participants, and a two-paragraph to
three-paragraph summary of what will be presented.
Also, sign up for one of the slots on 12/6, 12/8, 12/11, and 12/13 on the Google doc
here.
Due date (for abstract): 5:15 PM, Sunday, December 3 [12.5 weeks]. (Send as plaintext, by e-mail to reps at cs.wisc.edu.)
Due date (for presentations): December 6, 8, 11, and 13 [13-14 weeks].
-
Final Writeup: The final writeup should be modeled
after a typical conference paper. There is no length
requirement or limit, but I would expect it to be somewhere
around 6-10 pages in one of the full-page, double-column
conference formats.
Due date: 5:15 PM, Friday, December 15, 2023 [14.5 weeks]. (Send as a PDF file, by e-mail to reps at cs.wisc.edu).
Grading
The course will be graded on the conventional (A-F) system.
- Course project: 80%
- Project presentations: 20%
The final grades will be curved.
I may be looking for a few volunteers to prepare LaTeX notes for one
or more of the term's classes -- more about this topic later in the
semester.
Students who do such a task may receive extra credit.
Policy on Collaborative Work
A certain amount of discussion of problem sets is permitted.
You should not ``go to someone,'' nor Google, nor ChatGPT simply to get an answer;
the kind of collaboration intended is a discussion with someone else
in which you discover the answer jointly.
You are not to make a habit of it over the course of the semester.
(Points will be deducted if you do an excessive amount
of the course work in collaboration with other people.)
When you do work in collaboration with someone else,
make a note of it on the solution set that you turn in.
In all cases, you are to do your own write-up.
For the end-of-course project, you are encouraged to work in groups of two.
Course Summary
CS 703 is a graduate course on program verification and program synthesis.
The official course listing does not accurately reflects the material
that I plan to teach, which is described below.
Topics to be covered include logic, verification, SAT solving,
abstract interpretation, static-analysis path problems, and program
synthesis, including algorithms relevant to those topics.
A smattering of other topics will be covered, such as semantics, some
type theory, and material both related to -- and relevant for --
machine learning.
It is useful, but not mandatory, for students to have completed an
undergraduate compiler course, such as Wisconsin's CS 536.
Criteria to Use for Reading/Critiquing Assignments
Please use the following criteria (taken from Ben Liblit's CS706 class)
for the critiques that you write for the reading/critiquing assignments:
-
Stated goals and solution. What problem are the authors trying to
solve? What are the bounds on this problem, i.e., what are they not
trying to solve? What techniques or tools do the authors offer to
solve the problem at hand? How do the authors know they have solved
the problem? Do the authors test or validate their approach
experimentally? Does the solution meet the stated goals, or does it
fall short in some way? Avoid simply quoting the authors' own
abstract. Restating in your own words demonstrates your understanding.
-
Cool or significant ideas. What is new here? What are the main
contributions of the paper? What did you find most interesting? Is
this whole paper just a one-off clever trick or are there fundamental
ideas here which could be reused in other contexts?
-
Fallacies and blind spots. Did the authors make any assumptions or
disregard any issues that make their approach less appealing? Are
there any theoretical problems, practical difficulties, implementation
complexities, overlooked influences of evolving technology, and so on?
Do you expect the technique to be more or less useful in the future?
What kind of code or situation would defeat this approach, and are
those programs or scenarios important in practice?
Note: we are not interested in flaws in presentation, such as trivial
examples, confusing notation, or spelling errors. However, if you have
a great idea on how some concept could be presented or formalized
better, mention it.
-
New ideas and connections to other work. How could the paper be
extended? How could some of the flaws of the paper be corrected or
avoided? Also, how does this paper relate to others we have read, or
even any other research you are familiar with? Are there similarities
between this approach and other work, or differences that highlight
important facets of both?
Course Schedule
What follows is a tentative schedule of topics:
DATE TOPIC READINGS PURPOSE
=====================================================================================================
9/6 Introduction; course mechanics PPT Slides Introduction
9/8 Overview of six formalisms for semantics [Nielson & Nielson 1999] Background
PPT Slides
-----------------------------------------------------------------------------------------------------
9/11 First-order logic, axiomatic semantics, [Hoare 1969] Background
verification conditions PPT Slides
9/13 Weakest preconditions, strongest Background
postconditions, symbolic execution PPT Slides
9/15 ZOOM CLASS (Event occurring in 1240 CS) Lecture Notes Background
Static program analysis PPT Slides
Recording, Part I
Recording, Part II
-----------------------------------------------------------------------------------------------------
9/18 Intro to abstract interpretation; [Cousot & Cousot 1977] Background
lattices and fix-points of equations PPT Slides
on lattice elements
9/20 Automating abstract interpretation via [R., Sagiv, & Yorsh 2004] Synthesis in service of analysis
the strongest-consequence problem PPT Slides
9/22 Introduction to program synthesis [Gulwani 2010] Synthesis
(Guest lecturer: Loris D'Antoni) [D'Antoni and Polikarpova 2022]
PPT Slides
-----------------------------------------------------------------------------------------------------
9/25 Automating abstract interpretation [Thakur & R. 2012] Synthesis in service of analysis
+ refutation as lattice search PPT Slides + abs. int. in service of verification
9/27 Propositions as types [Wadler 2015] Synthesis
PPT Slides
9/29 CDCL SAT solving [Marques-Silva et al. 2008], Verification
(also [Tichy & Glase 2006],
[Mitchell 2005],
and [Eén and Sörensson 2003])
PPT Slides
---------------------------------------------------------------------------------------------------------
10/2 What is a path problem? Lecture Notes Program analysis
PPT Slides
10/4 Automatic differentiation Lecture Notes Interlude: ML
and back-propagation [Olah 2015]
PPT Slides
10/6 Collecting semantics as a path problem Lecture Notes Program analysis
PPT Slides
----------------------------------------------------------------------------------------------------
10/9 Abstract semantics as a path problem Lecture Notes Program analysis
PPT Slides
10/11 Functional approach to interprocedural Lecture Notes, Part I Program analysis
dataflow analysis Lecture Notes, Part II
[Sharir & Pnueli 81]
PPT Slides
10/13 Program analysis via CFL-reachability [Reps 1998] Program analysis
[RHS 1995]
PPT Slides
-----------------------------------------------------------------------------------------------------
10/16 Weighted pushdown systems [RLK 2007] Program analysis
PPT Slides about points-to analysis via CFL-reachability
PPT Slides about WPDSs
10/18 Synthesis with hard + soft constraints [KMDRR 2022] Synthesis in service of analysis
(Kanghee Park, guest lecturer) [Park et al. 2023]
PPT Slides
10/20 ZOOM CLASS (Event occurring in 1240 CS)
A few topics about synthesis: Synthesis
(i) CEGIS [Solar-Lezama et al. 2008, Sect. 5.1]
(ii) E-graphs, tree automata, and blog, [Wang et al. 2021], slides, [Koppel 2021]
version-space algebra [Polozov and Gulwani 2015, Sect. 4]
PPT Slides
------------------------------------------------------------------------------------------------
10/23 Counter-example-guided abstraction [Ball & Rajamani 2000] Program analysis via synthesis for abstraction refinement
refinement (CEGAR) [Ball & Rajamani 2001a,2001b]
PPT Slides
10/25 Analysis of concurrent programs [Farzan & Kincaid 2012] Program analysis
[Farzan & Kincaid 2013]
PPT Slides
10/27 Newtonian program analysis Lecture Notes Program analysis
[RTP16], [RTP17]
PPT Slides
10/27 Project proposal due (5:15 PM)
------------------------------------------------------------------------------------------------
10/30 Tips on writing a research paper PPT Slides Writing
11/1 No class
11/3 Retrograde debugging Page with links to the video recording
PPT Slides
------------------------------------------------------------------------------------------------
11/6 No class
11/8 No class
11/10 No class
11/10 Milestone 1 due (5:15 PM)
------------------------------------------------------------------------------------------------
11/13 No class
11/15 No class
11/17 No class
------------------------------------------------------------------------------------------------
11/20 No class
11/22 No class
11/22 Milestone 2 due (5:15 PM)
11/23-11/26 Thanksgiving Break
------------------------------------------------------------------------------------------------
11/27 No class
11/29 No class
12/1 No class
12/3 Presentation abstract due (5:15 PM)
------------------------------------------------------------------------------------------------
12/4 No class
12/6 FINAL PROJECT PRESENTATIONS
12/8 FINAL PROJECT PRESENTATIONS
------------------------------------------------------------------------------------------------
12/11 FINAL PROJECT PRESENTATIONS
12/13 FINAL PROJECT PRESENTATIONS
12/15 FINAL PROJECT WRITEUPS DUE (5:15 PM, as a PDF file, by e-mail to reps at cs.wisc.edu)
------------------------------------------------------------------------------------------------
Bibliography
- [Ball & Rajamani 2000]
-
Ball, T. and Rajamani, S.K.,
Bebop: A Symbolic Model Checker for Boolean Programs. SPIN 2000: 113-130.
[PDF]
- [Ball & Rajamani 2001a]
-
Ball, T. and Rajamani, S.K.,
Bebop: A path-sensitive interprocedural dataflow engine. PASTE 2001: 97-103.
[PDF]
- [Ball & Rajamani 2001b]
-
Ball, T. and Rajamani, S.K., Automatically Validating Temporal Safety Properties of Interfaces,
SPIN 2001.
[PDF]
- [Cousot & Cousot 1977]
-
Cousot, P. and Cousot, R.,
Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints.
POPL 1977: 238-252.
[PDF]
- [D'Antoni and Polikarpova 2022]
-
D'Antoni, L. and Polikarpova, N.,
Program Synthesis: The Book.
Draft of Sept. 29, 2022.
[PDF]
- [Eén and Sörensson 2003]
-
Eén, N., and N. Sörensson. An extensible SAT-solver.
International conference on theory and applications of satisfiability testing. Springer, Berlin Heidelberg, 2003.
[PDF]
- [Farzan & Kincaid 2012]
-
Farzan, A. and Kincaid, Z.
Verification of parameterized concurrent programs by modular reasoning about data and control.
POPL 2012: 297-308.
[PDF]
- [Farzan & Kincaid 2013]
-
Farzan, A. and Kincaid, Z.
Duet: Static analysis for unbounded parallelism.
CAV 2013: 191-196.
[PDF]
- [Gulwani 2010]
-
Gulwani, S.,
Dimensions in program synthesis. PPDP 2010: 13-24.
[PDF]
- [Gulwani 2012]
-
Gulwani, S.,
Synthesis from examples: Interaction models and algorithms. SYNASC 2012: 8-14.
[PDF]
- [Hoare 1969]
-
Hoare, C.A.R.,
An axiomatic basis for computer programming. Commun. ACM 12(10): 576-580 (1969)
[PDF]
- [KMDRR 2022]
-
Kalita, P., Muduli, S., D'Antoni, L., Reps, T., and Roy, S.,
Synthesizing abstract transformers. Proc. ACM Program. Lang. 6(OOPSLA2): 1291-1319 (2022)
[PDF]
- [Koppel 2021]
-
James Koppel:
Version space algebras are acyclic tree automata.
CoRR abs/2107.12568 (2021)
[PDF]
- [Marques-Silva et al. 2008]
-
Marques-Silva, J., Lynce I., and Malik, S.,
Conflict-driven clause learning SAT solvers.
Handbook of Satisfiability, Armin Biere, Marijn Heule, Hans van Maaren and Toby Walsch (eds.),
IOS Press, 2008
[PDF]
- [Mitchell 2005]
-
Mitchell, D.G.,
EATCS Bulletin (The Logic in Computer Science Column), Volume 85,
February 2005, pages 112-133.
[PDF]
- [Nielson & Nielson 1999]
-
Nielson, F. and Nielson, H.R.,
Semantics with Applications: A Formal Introduction, 2nd Ed., John Wiley & Sons, 1999.
[PDF]
- [Nielson, Nielson, & Hankin 1999]
-
Nielson, F., Nielson, H.R., and Hankin, C.,
Principles of program analysis, Springer, 1999.
[PDF]
- [Olah 2015]
-
Olah, C.,
Calculus on computational graphs: Backpropagation.
Aug. 2015.
[Blog post]
- [Park et al. 2023]
-
Park, K., D'Antoni, L., and Reps, T.W.,
Synthesizing specifications. To appear in PACMPL (OOPSLA), 2023.
[PDF]
- [Polozov and Gulwani 2015]
-
Polozov, O. and Gulwani, S.,
FlashMeta: a framework for inductive program synthesis. OOPSLA 2015:
[pdf]
- [Reps 1998]
-
Reps, T.,
Program analysis via graph reachability.
Information and Software Technology 40, 11-12 (November/December 1998),
pp. 701-726.
[pdf]
- [RHS 1995]
-
Reps, T., Horwitz, S., and Sagiv, M.,
Precise interprocedural dataflow analysis via graph reachability.
POPL 1995.
[pdf]
- [RLK 2007]
-
Reps, T., Lal, A., and Kidd, N.,
Program analysis using weighted pushdown systems.
In Proc. 27th Conf. on Foundations of Software Technology and Theoretical Computer Science (FSTTCS), 2007.
[pdf]
- [RSY 2004]
-
Reps, T., Sagiv, M., and Yorsh, G.,
Symbolic implementation of the best transformer.
Verification, Model Checking, and Abstract Interpretation (VMCAI), 2004.
[pdf,
ppt]
- [RTP 2016]
-
Reps, T., Turetsky, E., and Prabhu, P.,
Newtonian program analysis via tensor product.
POPL 2016.
[pdf,
ppt]
- [RTP 2017]
-
Reps, T., Turetsky, E., and Prabhu, P.,
Newtonian program analysis via tensor product.
ACM Trans. on Program. Lang. and Syst. 39(2): 9:1-9:72 (2017).
[pdf,
ppt]
- [SP 1981]
-
Sharir, M. and Pnueli, A.,
Two approaches to interprocedural data flow analysis,
Program Flow Analysis: Theory and Applications,
Chapter 7, Prentice-Hall, 1981.
[On-line copy (from campus only):
PDF]
- [Solar-Lezama et al. 2008]
-
Solar-Lezama, A., Jones, C.G., and Bodík, R.,
Sketching concurrent data structures. PLDI 2008.
[pdf]
- [Tichy & Glase 2006]
-
Tichy, R., and Glase, T., Clause learning in SAT,
University of Potsdam, 2006.
[pdf]
- [Wang et al. 2021]
-
Wang, Y.R., Koppel, J., Haan, A., and Pollock, J.,
E-Graphs, VSAs, and Tree Automata: a Rosetta Stone.
Unpublished.
[pdf]
- [Wadler 2015]
-
Wadler, P., Propositions as types. Commun. ACM 58(12): 75-84 (2015).
[PDF]