Imager Technical Information

Title

modified: dd mmm yyyy

Purchasing Software and The Software License Czar

modified: 27 Apr 2009

• When purchasing software, faculty should email imager-tech so that the software license czar (currently Rock Leung) can follow up on the licensing info. (Note-- please see the Ordering Software page for further information)
• The software license czar's lab duties include: keep track of software licensing info, show nonsensitive info on wiki, keep sensitive info in area viewable by imager-tech only, send out renewal reminders to profs 1 month before expiry date.

Snapshot Failure Due To Large Data Changes

modified: 10 Nov 2008

• The following are answers from Mike Sanderson (tech staff) about the snapshot failure.
• When there is a snapshot failure due to large data change, what exactly is lost? If a snapshot fails, the entire snapshot will not be created. We take hourly, daily, and weekly snapshots. We retain 25 hourly, 8 daily, and 3 weekly snapshots. We currently have weekly snapshots from Nov 8, Nov 1, and Oct 25 run at 2:00 AM on netapp2. If the next weekly snapshot at 2:00 AM Nov 15 fails, we will retain the snapshots from Nov 8, Nov 1, and Oct 25. To clarify the problem we were having a little, it was not the snapshots in /ubc/cs/research that we were having problems with. The issue was on netapp3 - the filer dedicated to retaining weekly snapshot data. The intent was to retain 52 weeks of snapshots on netapp3, effectively replacing our tape backups with online disk-based backups. The combination of large changes in snapshot volume and disk space increases on netapp1 and netapp2 that were not mirrored on netapp3 resulted in 100% usage of the disk space on netapp3 - regular space and snapshot space. By deleting the oldest snapshots, we were able to do the transfers of the newest snapshots. Because the retention is so large, it takes much longer for the large snapshots to roll off the end and the space to be recovered.
• Note that (almost) all non -raid* filesystems in /ubc/cs/research/* share the same snapshot space as .../research/imager. The snapshot space is based on the total size of a "volume" - think of it as an entire disk - which we split up into research group space via quotas.
  • 1.3T 670G 565G 55% /ubc/cs/research/imager/.snapshot
  • 635G 213G 423G 34% /ubc/cs/home/.snapshot
• Would it be possible to get a directory that does not have snapshot utilities? The RAID space (disk space not on the filers) was intended for this purpose, though there are problems with this space. If NetApp filer space is needed because of performance needs or some other reason, this can be discussed. Tech staff will need some lead time with it however. We are approaching the available capacity of netapp2. Mike will be meeting with Bill this week to discuss a non-budgeted purchase of additional space.

Performance Issues with Raid Space

modified: 03 Sep 2008

• Imager has raid space (found in /ubc/cs/research/): imager-raid1, psm-raid1, spiff-raid, pai-raid.
• The raid space lives on a Xyretec disk and has performance problems. Sometimes, the raid is slow. Tibi performed some tests and Mike tried to figure out what caused the raid to be slow. However, the cause is still not known. Perhaps profiling during testing might help analyze performance. If the raid is problematically slow, please notify the help desk.
• Tibi's tests show that imager-raid1 and imager-raid2 have similar performance
• It is documented that sequential I/O over NFS when submitting several jobs in parallel to the SGE can bring the system to a halt. Therefore, optimizing the I/O activity when submitting jobs to the SGE on the clusters likely will improve the performance quite a bit. It is possible that some of the scripts submitted do a lot of sub-optimal I/O over the network thus slowing all of us down. Here is a document tht explains the basic SGE commands as well as some best practices to optimize the I/O over the network (page 31) including some sample code. http://www.biostat.jhsph.edu/bit/SGE_lecture.ppt.pdf

Computing Committee's Levels of Service Document

modified: 11 Jun 2008

• The Computing Committee's Levels of Service Document can be found on the computing web page:
  • https://www.cs.ubc.ca/local/administration/committees/computing/resources.shtml
• This document describes the type and amount of support provided by the technical staff to support research activities.

SuSe 10.3 Issues

modified: 06 Jun 2008

• http://www.cs.ubc.ca/~hharun/suse103/103wys.txt

Renamed User Paths for Autodesk 3DS Max on Hamish

modified: 07 May 2008

• (temporary posting) The User Paths for Autodesk 3DS Max on hamish were renamed so that drives will be writable. The changes are as follows:

Sun Grid Engine

modified: 28 Feb 2008

• The Sun Grid Engine is scheduling software used to run jobs on 3 clusters of machines. The 3 clusters are beta, icics, and arrow; the icics cluster is available to the dept. The documentation appears on the BETA wiki page under the link "SunGridEngine". To access the SGE users can either ssh into a beta machine or submit jobs through departmental servers okanagan or begbie.

Departmental Servers

modified: 28 Feb 2008

• A list of departmental servers can be found here: http://www.cs.ubc.ca/local/computing/system/serverinfo.shtml

Video Editing Resources

modified: 21 Feb 2008

• The Video Editing Studio (x718) has two machines: a Mac with Final Cut software, and a Windows machine with Adobe Premiere Elements 3.0 software. Most of the equipment in x718 is actually owned by ICICS, purchased through a CFI award. It is available to everyone in Imager (and to others). This is a shared resource, for all ICICS researchers and their students. Fob access to the Video Editing Studio can be obtained through Ron Fussell. The Video Editing Studio should be reserved online at http://www.icics.ubc.ca/reserve-a-lab/.
• hamish is a general purpose Imager machine in x660 which has video editing software (Adobe Premiere Pro 1.5). Users should use the D drive to store their projects instead of the C drive since tech staff needs the C drive for maintenance.

Remote Access Policy for haggis and hamish

modified: 22 Nov 2007

• haggis and hamish are general purpose Imager machines. Users can remotely log on to them or use them in x660.
• The first person who logs on (remotely or not) owns the machine. The next person gets a warning message and cannot log on until the first person logs off. Please be aware and do not to press the reset button on these computers in case someone is logged on remotely.

Imager Printer

modified: 05 Nov 2008

• The printer in x652 (iccs-x652) has been replaced on Nov 5, 2008 to an HP P4015x.
• Canceling Print Jobs: User X can cancel user Y's print job by running lprm on the cascade server. This command removes the first job in the print queue.

Account Expiration

modified: 04 Oct 2007

• There are 3 things associated with a person: the account, the home directory, and the imager filespace in /imager/people/. When a person's account expires, this procedure applies only to the person's imager filespace. Tech staff will have a semi-automatic way of deleting/archiving a user's imager filespace according to the user's type. When accounts are first created, they are assigned an expiration date. When the expiration date is reached, the supervisor receives a notification email requesting confirmation for account archival/deletion. After supervisor confirmation, the account will expire. The files will be chown'd to the supervisor in a "purgatory" archive for 12 months before they are permanently deleted. While students should make sure their important files are archived in the right place before they leave, this policy is meant to handle the reality that this does not always occur. The time period of 12 months was chosen since many paper deadlines are annual conferences, so it's likely that the need to hunt down files would happen within the year window. This policy covers both undergraduate interns and MSc students.
• Visitors/postdocs and PhD student accounts will never be automatically deleted. Yearly reminders of active visitor accounts will be sent to supervisors, since most collaborations continue for many years. When a supervisor decides that a visitor account is no longer required and should be deleted, they should to be archived with the same 12-month purgatory process as above. After PhD students graduate, accounts should stay active, although pressure to reduce disk usage can be applied (preferably by the supervisor). Rationale: PhD students spend a significant portion of their life here, and their email is listed as contact on (hopefully many) papers. In many cases they continue to collaborate with their former supervisor or fellow students.

Hardware Purchase Procedure

modified: 18 Oct 2007

• When purchasing new hardware, email Ming Lau, Glen Lee, and Ciaran Llachlan Leavitt with the specifications (mlau,glee,llachlan).
• When ordering a new machine, only provide Ming with the specs that you really care about, and tech staff will fill in the rest with compatible parts.

Machine Installation (or Reinstallation) Process

modified: 26 Jan 2009

• When a new machine name is chosen, Glen will add machine to appropriate groups (i.e. imager linux group).
• Hardware staff will leave a note on the machine to let user know that tech staff still needs to complete the setup (nVidia drivers, screen resolution).
• The process that tech staff should follow for new machine installation includes the following.
  • Check screen resolution to see if it works.
  • Check that nVidia drivers are installed for nVidia cards.
  • Test glxgears to make sure graphics card works.

Sudo Access Policy and Joint Admin Model

modified: 04 Oct 2007

• The Joint Admin Model exists for Imager machines so that representatives have admin privileges as well as the tech staff.
• Sudo access is given to the Imager representatives for Linux, Windows, and Macs. This is currently Tibi for Linux and Christopher for Windows and Macs.
• Sudo access should only be used in urgent situations after hours when help desk is not available. Help desk must be used whenever possible.
• When sudo access is used, email help desk notifying them of what was done on the machine.
• Tech staff will keep track of sudo accesses and will report un-emailed incidents.

Update Policy and Conference Freezes

modified: 15 Nov 2007

• Update policy applies to Imager computers on subnet 54.
• The calendar on the Conference Freeze Dates page will list the freeze periods and the machines to be frozen during each period. If you would like your machine to be frozen for an upcoming conference please add your machine name, the conference, and the conference deadline to the calendar. Imager tech members will look at this page monthly to decide upon freeze periods. The page will also list conferences for which machines do not need to be frozen, and these conferences will be indicated as "no freeze". Tech staff will check the calendar for freeze periods and will not update the listed machines until the freeze period is over. No updates (including security updates) will occur on listed machines during their freeze periods.
• In order to synchronize the machines and leave them in a known state, a last update and reboot is performed just before the freeze period starts. Glen will send a notification email with the machine names on the freeze list and the date and time that those machines will be updated then rebooted.
• Vulnerability patching: During the freeze period, if there is a vulnerability that requires an upgrade, there are 3 options:
  • patch the machine and see if it works
  • take the machine off the network (no internet or cs access)
  • move the machine to subnet 54 which is the subnet for user administered boxes
• There will be a 48 hour notification on updates during non-frozen times.
• Look at the file /var/log/yum.log to see a list of recent updates to Linux machines.

Imager Filespace

modified: 27 Oct 2008

• The imager filespace is located at /ubc/cs/research/imager . /imager (2.2TB on NetApps2) contains both project/ and people/.
  • 2.2TB as of Oct 2008 (500GB purchased by Kelly and Robert)
  • 1.7TB as of Feb 2008
• There is also 1TB of old raid space which was imager's old disk space before moving to NetApps. This raid space (not on NetApps) is divided between two directories: /ubc/cs/research/imager-raid1 (600GB) and /ubc/cs/research/psm-raid1 (400GB).
• NetApps info:
  • Space capacity for NetApps = 7TB for a tray.
  • NetApps1 = smaller capacity, faster, expensive, $10/GB; NetApps2 = larger capacity, slower, cheaper, $5/GB
  • Imager gets 1.7TB of NetApps space for free and needs to pay for additional space. There is a policy on how much space out of the 1.7TB should be allocated to home dirs.

What changed with NetApps

modified: 26 Sep 2007

• NetApps documention link: http://www.cs.ubc.ca/local/computing/accounts/backups.shtml
• The above link contains information on how to use the backup/restore features on the new file servers.
• Recommendations for setting permissions: For most files = -rw------- (default, allows only you to read and write them.) C/C++ programs and other programs = -rwx------ (also default, allows only you to read, execute and write them.) See the man page on umask for information on how default permissions are set. For more information go to: http://www.cs.ubc.ca/local/computing/unix/permissions.shtml