Computer Sciences Department logo

CS 368-4 (2012 Fall) — Day 9 Homework

Due Monday, November 26, at the start of class.

Goal

Run a job (or many!) on CHTC using HTCondor. This is our gateway into tapping the full potential of CHTC via scripting.

Tasks

This assignment is quite straightforward, but I will walk you through some of the set-up steps, just to be sure.

  1. Using your CHTC account information, log on to submit-368.chtc.wisc.edu. You can ssh to the machine from any other machine, including your own desktop or laptop. If you are not sure how to ssh to a remote machine, check out the Remote Logins section of my Notes for Beginners page.
  2. On the submit machine, make sure you can run basic commands like condor_status and condor_q. If those commands fail, contact me right away.
  3. Copy the following Python script into a new file (perhaps named something like homework_09.py): 
    #!/usr/bin/env python

"""Homework for CS 368-4 (2012 Fall)
Assigned on Day 09, 2012-11-19
Written by Tim Cartwright
Submitted to CHTC by YOUR NAME
"""

import getpass
import os
import platform
import socket
import sys
import time

input_filename = None
if len(sys.argv) > 1:
    input_filename = sys.argv[1]

print >> sys.stderr, __doc__
print 'Time    :', time.strftime('%Y-%m-%d (%a) %H:%M:%S %Z')
print 'Host    :', getpass.getuser(), '@', socket.gethostname()
uname = platform.uname()
print "System  :", uname[0], uname[2], uname[4]
print "Version :", platform.python_version()
print "Program :", sys.executable
print 'Script  :', os.path.abspath(__file__)
print

if input_filename is None:
    print 'No input filename given, all done!'
    sys.exit(0)

print 'Reading input file at', os.path.abspath(input_filename)
input_file_object = open(input_filename, 'r')
input_lines = input_file_object.readlines()
print 'Found %d lines in the input file' % (len(input_lines))
input_file_object.close()

output_file_object = open('homework-09-output.txt', 'w')
print >> output_file_object, 'First line of input file:'
print >> output_file_object, input_lines[0].rstrip('\n')
output_file_object.close()
print 'Wrote output file'
  4. Edit the script above to contain your own name instead of the YOUR NAME placeholder!
  5. Use the shell to make the script executable by itself: 
    chmod 0755 homework_09.py

    If you named the script something other than homework_09.py, substitute your actual script file name in command above instead.

  6. Write a simple input text file of your own devising. Make sure it has a few lines of text.
  7. Make sure that you can run your script successfully from the command line. First, without a command-line argument: 
    ./homework_09.py

    And then, with a single command-line argument, which is the name of your input text file:

    ./homework_09.py input-09.txt

    If you cannot run your script like this, neither can HTCondor! Fix any problems with the script and/or your command until this step works perfectly.

    When you provide a command-line argument, be sure to check out the output file created by the script: homework-09-output.txt.

  8. OK, everything is ready for you now. Create your HTCondor submit file, and run your job! Try it without a command-line argument first, and once that works, try adding the command-line argument and input file. Get comfortable with all of the basic HTCondor commands and submit-file statements presented in the slides.
  9. Be sure to look at all of the different files created by your job. There may be interesting details in there!

HTCondor Tips

Here are some things to observe and/or watch out for.

Extra Challenges

Like writing code with Python itself, the more you play around with the basics of HTCondor and running jobs, the more you will learn and be ready for the next step(s). So, here are some things to try:

Reminders

Do the work yourself, consulting reasonable reference materials as needed; any reference material that gives you a complete or nearly complete solution to this problem or a similar one is not OK to use. Asking the instructors for help is OK, asking other students for help is not. All standard UW policies concerning student conduct (esp. UWS 14) and information technology apply to this course and assignment.

Hand In

All homework assignments must be turned in by email! See the email page for more details about formatting, etc.

For this assignment, you must submit several files in order for your assignment to be complete:

All of these files should be from a single run of the script (you may choose which version to send me, with or without the input file).