CS 368-4 (2012 Fall) — Day 13 Homework

Due Monday, December 10, at the start of class.

Goal

Create a pretty image by running a Mandelbrot generator on several “tiles” and then stitch the tiles together to create a single, large image.

This exercise is an extension of Homework #12 to create the entire workflow.

Background Information

See Homework #12 for background information, which is still relevant here.

Tasks

There are two main tasks to complete: design a workflow to solve this problem, and then implement it with a Python script and, depending on your design, perhaps some other supporting files.

Part I: Design

As in Homework #12, your final script must accept the following command-line arguments, which define what to draw. Keep them in mind as you design your solution.

• cx (Mandelbrot center x)
• cy (Mandelbrot center y)
• w (Mandelbrot width)
• p (pixel size)
• n (tiles per side)

Consider a fairly simple set of command-line arguments:

Sketch your workflow for this scenario, using circles for nodes and arrows to connect them. Be sure each node is needed — avoid extraneous nodes. Give each node a unique label.

Next, write down the details for each step: prerequisites or preconditions, inputs (including command-line arguments), process (i.e., executable), and output. Do you want or need any pre- or post-scripts on any nodes? If so, write down the same details for them. The clearer you can be about this part of the design, the easier the implementation will be. However, if you are not sure of details now, you can always come back and figure them out later.

Finally, consider the DAGMan design of your solution. We have talked about lots of different approaches to building a DAG. Which will you use? Some questions to consider:

• Will there be one submit file for all goatbrot nodes or one per tile?
• How and where will the command-line arguments to goatbrot be set?
• How and where will the command-line arguments to montage be set?
• Will each DAGMan node have its own directory or not?
• Which DAGMan features (JOB…DIR, PRE_SKIP, PRIORITY, SCRIPT, VARS) do you need or want to use?
• Will you write the HTCondor submit files manually, fully generate them, or generate them from templates?

There is not one correct answer to these questions! Each decision has pros and cons. Do your best to consider your options and make choices. Again, you can always defer some decisions until later, or change decisions already made!

Part II: Implementation

Given your experience with Homework #12 and your design decisions, implementation should be fairly straightforward. Be sure to reuse as much code from Homework #12 as possible! Do not reinvent the wheel. Of course, some things will certainly change for this assignment.

Some important notes (read carefully!):

• The montage program is not installed on the execute machines of CHTC. Therefore, we will use a different universe setting in the corresponding HTCondor submit file to tell HTCondor to run the montage program right on the submit machine. Do not use the vanilla universe for your montage node! Doing so will guarantee failure. Instead, use this line:

  universe = scheduler

  Also, because this job will run locally (i.e, on the submit machine), you should remove the HTCondor submit file statements for should_transfer_files, when_to_transfer_output, rank, and +WantRHEL6Job (think about why they are no longer needed). The remaining HTCondor submit file statements, such as executable and queue, are still needed.

• Remember to create goatbrot output files that sort into correct tile order. This is no different than in Homework #12, just a reminder.

Extra Challenges

Some ideas for extra learning:

• Repeated from Homework #12: Try making your script work for images that are not square. You will need to add command-line arguments to your script, and it makes some aspects a bit more complicated. However, the same overall approach should still work.
• Notice how many files are left over from a successful run of the workflow? Clean up unnecessary files automatically. But, do not clean up files if there are any failures, because the files might be helpful to diagnose what went wrong. There is more than one way to do this task.
• For the very adventuresome: Suppose I wrote a simple text file with different values for cx, cy, w, p, and n — one set of values per line. Now I want to run a single command that, ultimately, will result in one image per line in the file. Given everything you have done so far, how would you implement this new system? Either sketch out your approach, or actually implement it if you have time.

Reminders

Do the work yourself, consulting reasonable reference materials as needed. Any resource that provides a complete solution or offers significant material assistance toward a solution not OK to use. Asking the instructor for help is OK, asking other students for help is not. All standard UW policies concerning student conduct (esp. UWS 14) and information technology apply to this course and assignment.

Hand In

All homework assignments must be turned in by email! See the email page for more details about formatting, etc.

For this assignment, submit your Python script and any required support files (templates, HTCondor submit files, whatever).