Computer Sciences Department logo

CS 368-4 (2012 Fall) — Introduction to Scripting for CHTC

In this 1-credit course, you will learn to use the Python scripting language and the UW’s Center for High-Throughput Computing (CHTC) facility to solve a variety of research computing challenges: data manipulation, data analysis, statistical analysis, image processing, workflow automation, etc.

From July 2010 to June 2011, researchers on campus used 45 million hours of computing time in the CHTC — for FREE! If computation is or could be part of your research, you might benefit from access to CHTC and basic scripting skills to harness its full power.

The course is ideal for students, faculty, and staff whose research work entails significant computing.

No prior programming or Linux experience is required. In the past, students without prior experience have completed the course successfully. However, without prior programming or Linux experience, the course is challenging and the student should expect to devote extra effort accordingly. Resources will be available prior to the start of the course to help those who need extra background knowledge and skills.

Basics

Dates October 22 (Monday) through December 13 (Thursday) — 8 weeks
Times MoTh 1:20–2:10 p.m.
Location Engineering Hall 2349
Instructor Tim Cartwright : CS 4265 : cat [@] cs [.] wisc [.] edu
Mailing List compsci368-4-f12-hhh [@] lists [.] wisc [.] edu

Office Hours

Mondays, 3–4 p.m. and Thursdays, 3–4 p.m. in CompSci 4265 (my office).

You can also make an appointment (by email is probably best) to talk to me. I work full time here in the CS building, so will often be available. However, dropping by my office without an appointment is not guaranteed to work.

Course Books & Other Resources

There are no books to buy for this course. There is a Python book that I suggest using, and it is available for free online through the UW library system. Also, there are plenty of good resources available online. Here are a few (including the book) to get started:

If you really want to buy a physical book, there are many books on Python. The Learning Python book, cited above, is fine. I have personally looked at the following book and thought it looked OK, too; plus, it focuses on teaching the basics of computing and programming along the way, so it is aimed at new programmers. However, I make no recommendations for purchasing a book.

Introduction to Computing Using Python: An Application Development Focus by Ljubomir Perkovic. ISBN 978-0-470-61846-2. Wiley, December 2011.

If you are new to programming or using the command line, I wrote a few pointers to get started. For more advanced students, I wrote a basic page on writing unit tests in Python.

Also, check out the other CS 368 sections for the 2012 Fall semester:

Syllabus

Slides, code samples, and homework assignments are posted on the day of the corresponding class. All reading assignments are from Learning Python (4th Ed.), unless otherwise as indicated by a hyperlink.

Details

# Day Topic Reading Slides Homework
1 October 22 (Mon) Overview & Python Basics Ch.1: A Python Q&A Session [skim]
Ch.2: How Python Runs Programs [optional]
Ch.3: How You Run Programs
Ch.4: Introducing Python Object Types [overview]
Ch.5: Numeric Types
Ch.7: Strings
PDF homework
2 October 25 (Thu) More Basic Syntax Ch.6: The Dynamic Typing Interlude
Ch.10: Introducing Python Statements
Ch.11: Assignments, Expressions, and Prints
Ch.12: if Tests and Syntax Rules
Ch.13: while and for Loops [just while loops today]
PDF homework
3 October 29 (Mon) Collections Ch.8: Lists and Dictionaries
Ch.9: Tuples, Files, and Everything Else [just Tuples]
Ch.13: while and for Loops [for loops]
PDF homework
4 November 1 (Thu) I/O and Exceptions Ch.9: Tuples, Files, and Everything Else [just Files]
Ch.32: Exception Basics
PDF homework
5 November 5 (Mon) Data, Functions, and Classes Ch.16: Function Basics
Ch.17: Scopes
Ch.18: Arguments
Ch.25: OOP: The Big Picture [optional]
PDF homework
6 November 8 (Thu) Modules and the Standard Library Ch.21: Modules: The Big Picture
Ch.22: Module Coding Basics
Python (2.4.3) Global Module Index
PDF homework
7 November 12 (Mon) Regular Expressions Python module re (link is for 2.4.3)
[Very advanced:] Mastering Regular Expressions
PDF homework
8 November 15 (Thu) System Interaction Python modules (links are for 2.4.3):
os, sys, getopt, optparse, subprocess
PDF homework
9 November 19 (Mon) Introduction to CHTC and HTCondor Jobs HTCondor 7.8 Manual:
Chapter 1: Overview
Chapter 2: Users’ Manual (at most, 2.1–2.7)
PDF homework
November 22 (Thu) NO CLASS — THANKSGIVING
10 November 26 (Mon) More Complex Jobs HTCondor 7.8 Manual:
Chapter 2: Users’ Manual (at most, 2.1–2.7)
condor_q, condor_status,
condor_submit, condor_prio
PDF homework
11 November 29 (Thu) Workflows with DAGMan HTCondor 7.8 Manual:
Chapter 2: Users’ Manual (skim section 2.10)
condor_submit_dag
PDF homework
12 December 3 (Mon) Scripting Workflows I: Parameter Sweeps PDF homework
13 December 6 (Thu) Scripting Workflows II: DAGMan PDF homework
14 December 10 (Mon) Wrapper Scripts PDF homework
15 December 13 (Thu) Writing Scientific Code in Python PDF