Finish it soon so that you can start Assignment #1
This assignment will verify you can access Microsoft Azure and introduce you to it's management interfaces. You will setup a cluster of 4 VMs using Azure Command-Line Interface (Azure CLI). This cluster will be used to carry out the remaining assignments during the course of the semester.
After completing this assignment, students should:
Microsoft Azure offers a one-month free trial for any new customer and provides Azure credits of $200 at no charge, for a month. Students are encouraged to use these credits to carry out the course assignments.
Initially, only one student in a group should sign up for the free trial using your wisc id (Eg: firstname.lastname@example.org). The other student should sign up for the free trial a month later. You can sign up for the free trial here. You will need a phone number (to receive your confirmation code) and a credit/debit card (which will be charged for any usage beyond the free trial).
The Azure CLI is a set of open-source, cross-platform commands for working with the Azure platform. It provides an easy way to setup a cluster using custom scripts. We will be using the Azure CLI to set up the cluster. In order to install the Azure CLI on your PC, you need to perform either of the following steps:
In order to check if the Azure CLI installation was successful, you should type azure help in the command line. If the installation was successful, the expected output is the CLI usage information.
Microsoft Azure provides two models of deployement - Classic model and Resource Manager model. The latter deployment model introduced the concept of a resource group. A resource group enables the deployment, management, and monitoring of all resources that share a common life cycle, in a single coordinated operation. You will be using the Resource Manager deployment model to setup your cluster.
The cluster that you will deploy consists of 4 VMs connected by a virtual network. Each VM runs Ubuntu 14.04 and has 3 disks associated with it - OS disk, data disk and temporary scratch disk.
To deploy the cluster, you need to download the deployment scripts archive and perform the following steps:
The cluster deployment would take a few minutes. Once the deployment is successful, verify you can connect to the VMs using SSH by typing the following command in the command prompt: ssh ubuntu@cs838fall2016group<group_number><vm_number>.eastus.cloudapp.azure.com. The <vm_number> placeholder takes values 1, 2, 3 or 4. Replace <group_number> and <vm_number> with appropriate values.
The cluster deployment may fail incase the CPU core limit is reached(Expected Error: error: QuotaExceeded : Operation results in exceeding quota limits of Core. Maximum allowed: 4, Current in use: 0, Additional requested: 16). By default, for a free trial Azure may limit the number of cores in a region to 4. You can check the limit for your account by typing the command: azure vm list-usage eastus. The cluster deployment requires 16 cores and the limit on the number of cores can be increased via the Microsoft Azure Portal:
It normally takes a few hours for the request to be processed. Once the core limit is increased, you can go ahead and retry the cluster deployment.
The cluster deployment template associates a 50GB data disk with each VM. In order to use the data disk, you need to SSH into each VM to partition, format, and mount the data disk. You need to perform the following steps in every VM of the cluster:
To ensure that the data disk is remounted automatically after a reboot, it must be added to the /etc/fstab file. You need to perform the following steps:
UUID=<data_disk_uuid> /workspace ext4 defaults,discard 1 2
For more detailed instructions regarding data disk provisioning, please refer the Azure CLI documentation regarding the same.
To ensure that the $200 Azure credits are not wasted, students must always deallocate all the 4 VMs when the cluster is not in use. Merely stopping the VMs is of no good - you will still be charged for compute hours as well as storage. However, when the VMs are deallocated, you will be only charged for the storage of the OS disk (/dev/sda) and the data disk (/dev/sdc). Another thing to note is that when the VMs are deallocated, any data on the temporary disk (/dev/sdb mounted on /mnt) will be lost.
In order to deallocate the 4 VMs, type the following command: azure vm deallocate group<group_number> vm1;azure vm deallocate group<group_number> vm2;azure vm deallocate group<group_number> vm3;azure vm deallocate group<group_number> vm4. Replace <group_number> with your group number.
In order to start the 4 VMs after deallocation, type the following command: azure vm start group<group_number> vm1;azure vm start group<group_number> vm2;azure vm start group<group_number> vm3;azure vm start group<group_number> vm4. Replace <group_number> with your group number.
You can manage you cluster and keep a track of the Azure credits using the Microsoft Azure Portal. View your account usage in Azure. (It may take up to a day for the usage to be registered.) Confirm that you incurred no actual charges apart from the deduction in the Azure credits.
You do not need to submit anything for this assignment. It is assumed you will complete this assignment for your own benefit.