Big Cluster Project 11 April 2006

(Note: this page is under construction and hasn't been announced yet, so is in massive flux.)

Come help us make a big parallel cluster on 11 April 2006!

If you are in CS 521, you'll get five points credit for participating (details below).
It's after instruction ends but before exams start, so you have no excuse.
It'll be right before Tuesday Tea.
Brett has a budget from Google for pizza that he's eager to spend on this event.
It'll be fun!

We want to get a lot of computers, where "a lot" is on the order of 40.

We want to run a cool parallel program on the cluster. If you have a suggestion, add it to the list below. The suggestions we have so far

Finding large prime numbers
LINPACK

Something graphical -- something interesting to watch -- would be more fun than something purely textual like prime numbers.

CS 521 credit

To get CS 521 credit, you need to do the following:

Burn a bootable BCCD CD (see below).
Boot your laptop from that CD at home to make sure that the burn was correct, that your system will boot, etc.
- Note what kind of network card you have so that you can boot faster on the spot.
Bring your computer(s) and the CD to (tbd location). Laptops are preferable, since we might be limited in the number of power outlets we will have.
- If you don't have a laptop, you can bring in your desktop; we will have at least one monitor that we can swap around.
- If you don't have a laptop, you can borrow one for four hours (not renewable) from Koerner Library or Woodward Library, Circulation Desk, entrance level.
Be fully prepared to boot your own computer, and to help out anyone who showed up who is not in 521 (i.e. who has even less experience than you have).

Making a bootable CD

There are a number of different distributions of MPI/LAM, but we will use BCCD

. It is specifically designed for educational use, and has a PPC version so that Mac users can play too.

Linux

Making a bootable CD on Linux is nowhere near as easy as it ought to be. It's probably not something that you do all the time, so you might have challenges with media, drivers, kernels, applications, or all of the above.

On Linux, there are some unspecified issues with cdburner for kernels newer than 2.5; there is some documentation on the Web that says they have been fixed in 2.6.11, but DuckySherwood had problems even with 2.6.11.

If you have trouble burning from Linux, you can is burn a CD from Windows and then boot that from Linux. Note that if everything is working, you should not need to burn anything except an iso file (Windows) or an iso.img file (Linux) to your CD.

Note that if everything is working, you should be able to stick the CD in your drive and restart; you should not need to hold down magic key combinations.
If you have trouble, see also the note in the Mac section below about "raw" mode.
Ducky successfully burned BCCD from Windows and booted on Linux (Kannotix distro, 2.6.11 kernel) using just what Windows offered (::RecordNow, which appears to be distributed with IBM Thinkpads).
Ducky found that she had trouble booting from the CD when she had previously booted into Linux from her hard drive -- she had to boot into Windows, then reboot with the CD in the tray.

Mac

When burning a CD on the Mac, it is important to burn it in "raw" format. (A raw image contains special error detection and correction codes that the CDR software will generate as it is writing the ISO image to the CD. You cannot simply add the image file to the CD layout as you would when recording normal files off your hard drive.) You can't just use the standard method of burning a file onto disk like you would an MP3.

Here are instructions from mkLinux on how to burn a raw image:

Download a BCCD image
Open Disk Copy, which can be found in /Applications/Utilities
Choose "Burn Image..." from the "Image" menu.
Locate the image you wish to burn (i.e. maindisk.img).
Insert a blank CD-R or CD-RW.
Click the "Burn" button!

This is slightly different from the UI Ducky found with version 10.2 (Panther) of OS X:

Download a BCCD image
Open Disk Utility from /Applications/Utilities
Choose "Burn..." from the "Images" menu
Select the .iso BCCD image from the file picker
When it prompts you for a blank CD, insert one
Click the "Burn" button!

Note that when you boot from the CD on your Mac, you must hold down the "c" button to make it boot from the CD.

Ducky had trouble rebooting into OS X immediately after booting into BCCD. She found that Control-Command-Power after an non-starting boot worked.

On the Day Of

On the day of the event, several things need to happen.

Equipment

We need some extra equipment:

Switches -- Alan Wagner will bring those
Spare CAT5 cables (from where? @@@)
A drive somewhere with the code to run (source, since we'll have Macs on the network as well)

Boot

DHCP Server: One machine needs to boot as a DHCP server. DHCP server, follow this sequence:

Boot up to the BCCD splash screen
Hit F3, then type framebuffer_mode_number startdhcp (framebuffer_mode_number just refers to what screen resolution to use; 4 is 1024x768.)
Follow directions.
- Say NO when it asks if it should autoconfigure with DHCP and YES to everything else
- When it asks for IP addresses, just type in the ones they use in the examples.

All the other machines need to be clients. On the Mac, it will boot you straight through this sequence. On x86 machines:

Boot up to the BCCD splash screen.
Hit F3
Type framebuffer_mode_number automode (framebuffer_mode_number just refers to what screen resolution to use; 4 is 1024x768.)
Follow directions; say YES to everything.
We will give you a password for bccd at the event.

Everybody then:

You might want to sign in as root and unmount all of your own partitions:
- df to get a list of the mounted partitions
- umount partition for all of your partitions, (e.g. umount /mnt/rw/discs/disc0/part3/home/fred)
- Log out.
Sign in as bccd, with the password given earlier.
Answer yes when it asks if you want to run a heartbeat.

Switch to LAM

@@@ do we need to do this? or do we just stick with MPICH?

vi ~/.bashrc
- edit PATH so that the line reads export PATH=/lam-mpi/bin:$PATH
source ~/.bashrc in all appropriate shells (or log out and log back in)
For each node, rebuild the library cache:
- su - root
- ldconfig -v | less
- exit (back to bccd)
bccd-allowall (Answer yes.)
bccd-snarfhosts
recon -v ~/machines
lamboot -v ~/machines

Windows users have the option of

startx

but Ducky found that her mouse didn't work on the Mac.

Compile the target code

Everybody needs to compile the target code.

The DHCP server machine needs to run bccd-syncdir dirname ~/machines; it will show up for all machines in a directory named something like /tmp/6g2w98s.
cp -r dirname/cs521.arch ~bccd/cs521 (e.g. cp -r /tmp/6g2w98s/cs521.x86 ~bccd/cs521)
cd ~bccd/cs521
make
wait for instructions, eat pizza, etc.

As more machines come online, you might need to refresh your system state:

bccd-allowall
bccd-snarfhosts
recon -v ~/machines

Setup needs

There are a few one-of things to do ahead of time:

Instructions on what everyone needs to bring -- Ducky
Instructions for what to do on the day of -- Ducky
Locate switches -- Alan
Locate CAT5 cables
Reserve a room with an adequate amount of tables and space -- ?
- Estimate head/node count, adjust room if needed
Post information to class -- Ducky w/input from Alan
Post information more widely -- Ducky w/input from Alan
Figure out what code to run -- class
Make sure code compiles on Macs and Linux both with near-forehead install (meaning two directories) -- Ducky

On the day of:

Switches -- Alan
Extra CAT5 cables (?)
Pizza -- Google via Brett
Printouts of instructions -- Ducky (maybe a projector with instructions as well?)
DHCP server boot
Put code in ~bccd/cs521

Topic revision: r8 - 2006-03-20 - TWikiGuest

~~Edit~~
~~Attach~~