Difference: CLUMEQ (2 vs. 3)

Revision 32006-05-21 - bleuant

Line: 1 to 1

META TOPICPARENT	name="WESTGRID"

CLUMEQ is a super computer available to McGill

students (though I'm sure any student attending a Canadian university may have access). See details here: http://www.clumeq.mcgill.ca/

. Unfortunately, it is some what outdated both in terms of software and hardware. However, it is a large parallel system.

Line: 6 to 6

Here are some things found out about CLUMEQ that might help someone who is new to CLUMEQ (after reading the CLUMEQ FAQ

Changed:

<
<

0) Forget about CLUMEQ. WESTGRID's http://www.westgrid.ca/support/Facilities#head-7d986acf67ff63ef2a2e9d0553cd80bba84c2aa3|glacier

(http://www.westgrid.ca

) is much bigger and better in almost every single way. See the WESTGRID wiki.

>
>

0) Forget about CLUMEQ. WESTGRID's glacier

(http://www.westgrid.ca

) is much bigger and better in almost every single way. See the WESTGRID wiki.

Changed:

<
<

Note: 2006-05-05 CLUMEQ is about to get a $30,000,000 upgrade. It might be worth compairing the two systems in 2007.\

>
>

Note: 2006-05-05 CLUMEQ is about to get a $30,000,000 upgrade. It might be worth compairing the two systems in 2007.

1) Opening an account is more of a formality than anything. Everyone gets accepted.

2) Matlab is available but only in single processor format.

Changed:

<
<

3) I have made a Matlab script that will take an M-file (whose primary function is to run many identical test cases with different parameters after some setup code at the top of the M-file) and create a queue to submit to the CLUMEQ queue. See a graphics explanation here

. See the code here clumeq.m.

>
>

. See the code here qsuball.m

Deleted:

<
<

Note: Albert has made a better launcher file, and it can be found in the WESTGRID wiki. However, it seems to fail to input jobs into the proper queues of (single/multi/big/clumeq). It doesn't affect the processes.

Changed:

<
<

4) Only 16 single processor processes will be executed at any one time. 12 of which may be yours.

>
>

4) One may only have 12 single processor processes executing at any one time.

5) The admin, Patrice, if a very useful and responsive. When troubles arise, do email him.

Line: 26 to 25

7) CLUMEQ makes 2 files, stdout and errout, after running your job queue file (ie: PBS file). However, the error file sometimes cuts off if Matlab is writing to that error file (unknown why). This makes debugging Matlab sometimes difficult. Beware....

Changed:

<
<

8) I have made a matlab script that will kill all the user queues. Check it out here qdelall.m. One uses it like this under matlab: qdelall('your_user_name');

>
>

8) I have made a matlab script that will kill all the user queues. Check it out here qdelall.m

9) Sometimes there aren't enough licenses free to execute your program. This is a pain. However, someone has solved this Stupid Cluster Tricks

. Good luck with this.

Line: 39 to 38

Note: This can be done for every single toolbox required. However, to prevent possible license-required deadlocks

, I highly suggest one manually (within your matlab script, and most probably during your initilisation procedure) control the checking out of required matlab toolbox licenses as the matlab license()

function does not allow one to release a toolbox license. But, of course, this only works if one can initiate a maltab session.

Changed:

<
<

11) To solve the above 2 problems, one can take the ultimate step and compile the M-file into a system executable. This is done via matlab's compiler

. However, one must be aware of the licensing limitations

associated with a stand-alone system executable from the matlab compiler. Of course, if one uses a call in the "Can not be compiled" column, then one must revert back to the solution above. Gee, this is getting complicated.

>
>

11) To solve the above problem (not being able to start a matlab session due to lack of licenses), one may have a PBS call that doesn't execute your script until a license is available. Just add "#PBS -l software=MATLAB" to your PBS file (this is done automatically in qsuball.m

Note: Some PBS systems don't accept this option.

12) If the system does have matlab but lacks licenses for a particular toolbox (ie: it only has 1 license for a required toolbox-- sorta meaningless in a parallel system) or (yet worst) has the wrong version of matlab, then one can take the ultimate step and compile the M-file into a system executable. This is done via matlab's compiler. However, one must be aware of the licensing limitations associated with a stand-alone system executable from the matlab compiler. Of course, if one uses a call in the "Can not be compiled" column, then one must revert back to the solution above.

13) If all else fails, then the ultimate matlab licensing solution would involved a system executable which attempts to launch a matlab engine. Whilst monitoring the standard error output stream (ie: stderr), if it detects an error "-4" (the error number associated with insufficient matlab licenses), then it can relaunch the matlab engine until it doesn't fail. The problem with this solution is that it is racing against all the other processes in a polling manner-- quite inelegant. So really this isn't the ultimate solution.

Deleted:

<
<

12) The ultimate matlab licensing solution would involved a system executable which attempts to launch a matlab engine

. Whilst monitoring the standard error output stream (ie: stderr), if it detects an error "-4" (the error number associated with insufficient matlab licenses), then it can relaunch the matlab engine until it doesn't fail. The problem with this solution is that it is racing against all the other processes in a polling manner-- quite inelegant. So really this isn't the ultimate solution.

View topic | History: r4 < r3 < r2 < r1 | More topic actions...