CS 744 Big Data Systems, Fall 2021 – Assignment 0

You will do your assignments for CS 744 using CloudLab. CloudLab is a research facility which provides bare-metal access and control over a substantial set of computing, storage, and networking resources. If you haven’t worked in CloudLab before, you need to register a CloudLab account.

This small assignment walks you through the CloudLab registration process and shows you how to start an experiment in CloudLab.

Most importantly, it introduces our policies on using CloudLab that will be enforced throughout the semester.

Register a CloudLab account

To register an account, please visit http://cloudlab.us and create an account using your Wisconsin email address as login. Note that an SSH public key is required to access the nodes CloudLab assigns to you; if you are unfamiliar with creating and using ssh keypairs, we recommend taking a look at the first few steps in GitHub’s guide to generating SSH keys. (Obviously, the steps about how to upload the keypair into GitHub don’t apply to CloudLab.) Click on Join Existing Project and enter UWMadison744-F21 as the project name. Then click on Submit Request. The project leader will approve your request. If you already have a CloudLab account, simply request to join the UWMadison744-F21 project.

Start An Experiment

To start a new experiment, go to your CloudLab dashboard and click on the Experiments tab in the upper left corner, then select Start Experiment. This will lead to the profile selection panel. Click on Change Profile, and select a profile from the list. For example, if you choose the single-node-ubuntu18-fa21 profile in the UWMadison744-F21 project you will be able to launch 1 machine with the Ubuntu 18 image. Select the profile and click on Next to move to the next panel. Here you should name your experiment with WisconsinLogin-ExperimentName. The purpose of doing this is to prevent everyone from picking random names and ending up confusing each other since everyone in the UWMadison744-F21 project can see a full list of experiments created. You also need to specify from which cluster you want to start your experiment. Each cluster has different hardwares. For more information on the hardware CloudLab provides, please refer to this. Once you select the cluster you can instantiate the experiment. Once the experiment is ready you will see the ssh login command. Try to login to the machine and check for the number of CPU cores available and memory available on the node.

Policies on Using CloudLab Resources

• The nodes you receive from CloudLab are real hardware machines sitting in different clusters. Therefore, we ask you not to hold the nodes for too long. CloudLab gives users 16 hours to start with, and users can extend it for a longer time. Manage your time efficiently and only hold onto those nodes when you are working on the assignment. You should use a private git repository to manage your code, and you must terminate the nodes when you are not using them. If you do have a need to extend the nodes, do not extend them by more than 1 day. We will terminate any cluster running for more than 48 hours.
• As a member of the UWMadison744-F21 project, you have permissions to access another member’s private user space. Stick to your own space and do not access others’ to peek at/copy/use their code, or intentionally/unintentionally overwriting files in others’ workspaces. For more information related to this, please refer to to http://studentconduct.wiscweb.wisc.edu/academic-integrity/