Getcube


What is getcube?

Getcube is a "partition manager" that allocates nodes into named units cubes which may be used to refer to those nodes as a whole. This is similar to a filesystem in concept. The filesystem allocates and names units of blocks into a file, and files into directories.

For example: create a cube from nodes 1..4, and call it hacked.

The name is a remnant from the single most useful parallel computer we ever had, the Intel iPSC Hypercube. It was the perfect parallel computing environment; we actually did work on it, instead of constantly maintaining it.


How does it work?

Getcube consists of two pieces;
  1. A daemon, cube_mgr, which is started from the COPS init.d script. Cube_mgr lives in /p/cops/sbin. It maintains the current configuration of cubes in a text database located in /usr/adm/cops/config.
  2. A utility, cube which communicates with the daemon using rpc. Cube is used to to query and modify the state of the getcube database. Cube lives in /p/cops/bin. For backward compatability with an older version of the cube utilities, alternative executables called getcube, lscube, showcube and rmcube are also present in that directory.

Properties of Cubes

To be expanded on later.

Cube commands

In the following commands, if no [partition] is specified, the root partition is used. The '.' character is used as the path seperator in partition names.

ls [partition]
lists any sub-partitions of the partition.
show [partition]
display information about the nodes which comprise a a particular partition.
get [-l lon | -n non ] [-s] partition-name
Create a new partition with the given name. The options which specify the construction of the new partition are
-l <list of nodes>
a comma-seperated list of node numbers from the enclosing partition. For example, if the enclosing partition has 8 nodes, the numbers 0-7 are valid node numbers
-n <number of nodes>
is followed by the number of nodes the new partition should contain.
-m mode
A Unix-like protection system is currently used to control access to partitions and nodes. This specified which mode the partition should be created with. There is currently no method of changing the mode of a partition after it is created.
-s
Specify the cube should be shared.
rm partition.name
removes the named partition, freeing the nodes to be used by someone else.

Example use of Getcube

It is of no small coincidence that the name of the "hypothetical user" in the following is markos. Markos helped me create the original version of this document.

Acquiring nodes

A hypothetical user named markos wants to do some work on a parallel machine or cluster of workstations managed by getcube. He wants four nodes to do some testing for the next several hours. He will be benchmarking and can't suffer the timing problems of a multi-user system; he doesn't want to share them with other users. Markos does the following ...

cops% getcube -n4 .markos
getcube 'markos' ok

If markos wanted to use a subset of his nodes, he could now use getcube to divide his nodes into whatever subsets he wanted to used. The configuration software configures entire cubes, so if markos wanted to have some of his nodes configured differently from the rest, he would need to place them in their own partition.

Markos would like to test a shore client and server; he only needs to use 2 of his 4 nodes to do this. He could configure all of them to use ip over myrinet, but I'll use this as a place to demonstrate getcube.

cops% getcube -n2 .markos.tcp
getcube 'markos.tcp' ok
Creates a 2 node partition which we will configure for IP over myrinet.

You can use 'showcube' to find what nodes you actually are using.

cow% showcube markos.tcp
name	.markos.tcp
numnodes	2
user	2411
group	2411
mode	0775
0	cops04	cops04	none	none	1
1	cops05	cops05	none	none	1
showcube 'markos.tcp' ok

How to undo all of this?

cow% rmcube markos.tcp
cow% rmcube markos
and we're done.
HTML Shipyard
Bolo's Home Page
Last Modified: Fri Aug 9 11:59:13 CDT 1996
bolo (Josef Burger) <bolo@cs.wisc.edu>