PsmPackageSystem < Imager

---++ Description

The PSM Lab has a number of addition programs and libraries that may be of value to people working in the lab.  While there is department-wide space provided to do so, this has proven troublesome in the past.  Conflicting versions, out-of-date software, and poorly configured permissions has often made getting software in the department system more of a hassle than it is worth.  The software we use most often is kept separate from the department-wide installs for tighter control.

This software is made available by extending the same UBC =use= script as the department-wide system.  The =use= script searches a set of directories for programs, libraries, includes and documentation and will modify the appropriate environmental variables to make these available to you.  The PSM lab employs the same system for making software available by adding more places for =use= to search.

Consult =man use= and http://www.cs.ubc.ca/local/computing/unix/software.shtml for full information on how to use =use=.

The software, system and documentation are maintained by [[mailto:mmt@cs.ubc.ca Matthew Trentacoste]].  Please contact him with any questions or problems.

---++ Installation

The only installation required is to set some environmental variables to instruct =use= where to search for the additional software.  Below is a section of =csh= script that will set up all of the necessary environmental variables for you.  If you are using the standard CS department configuration, can paste this into your =~/csh_init/environment= file.  If you are using something non-standard or are using =bash=, I trust that you can figure out the appropriate modifications to make it work with your setup.

Because the system has to work for multiple operating systems and hardware platforms, a bit of work is involved to determine what system the user is currently on.  Binaries must be installed for each OS and hardware, and because we don't have the luxury of picking different NFS shares to mount to the same place, we need to choose the right architecture.  After that is accomplished, the necessary paths are set up and things should work.

<verbatim>
#=== Determine our system ==========

# Determine what operating system we are using
set OS_DIST=`uname -s`

# Figure out what distro and version we are running
if ("$OS_DIST" == 'Linux') then
    if (-f /etc/SuSE-release) then
   # If Suse
   set OS_NAME='SuSE'
       if { fgrep -q openSUSE /etc/SuSE-release >& /dev/null } then
           set OS_VERS=`head -n 1 /etc/SuSE-release | sed 's/[\(\)]//g' | awk '{print "_" $3 "-" $2}'`
       else
           set OS_VERS=`head -n 1 /etc/SuSE-release | awk '{print $3; }'`
       endif
    else    if (-f /etc/fedora-release) then 
   # If Fedora
   set OS_NAME='Fedora'
       set OS_VERS=`head -n 1 /etc/fedora-release | awk '{print $5; }'`
    else    if (-f /etc/redhat-release) then
   # If Redhat
   set OS_NAME='Redhat'
       set OS_VERS=`head -n 1 /etc/redhat-release | awk '{print $5; }'`
    elseif (-f /etc/debian_version) then
   # If Debian
   set OS_NAME='Debian'
       set OS_VERS=`head -n 1 /etc/debian_version | awk '{print $1; }'`
    else
   set OS_NAME='None'
   set OS_VERS='0'
    endif
else
# If other (namely Sun and Apple), then use the uname results
    set OS_NAME=$OS_DIST
    set OS_VERS=`uname -r`
endif

# Set the system name to locate our binaries with
setenv USERPKGARCHSYS $OS_NAME$OS_VERS

#=== Set use system ================

# Set generic location and description
setenv USERPKGBASE   /ubc/cs/research/imager/project/psm/Packages
setenv USERPKGROOT   $USERPKGBASE/Generic
setenv USERPKGDESC   'PSM Lab software'

# Set architecture directory based on what system you are on
setenv USERPKGARCH   $USERPKGBASE/Arch/$USERPKGARCHSYS
</verbatim>

If this was accomplished correctly, you should be able to open a new shell or type =source ~/.cshrc=, then the command =env= and see something similar to at the bottom of the output:

<verbatim>
USERPKGBASE=/imager/project/psm/Packages
USERPKGROOT=/imager/project/psm/Packages/Generic
USERPKGDESC=PSM Lab software
USERPKGARCHSYS=Suse9.1
USERPKGARCH=/imager/project/psm/Packages/Arch/Suse9.1
</verbatim>

---++ Usage

Usage of the system follows exactly like the normal =use= script.  To get a full listing of all =use= software, you type =use -L=.  You should see the CS local and public sections as normal and the bottom of the list, there should be something similar to:

<verbatim>
------------------------------------------------------------------------
PSM Lab software      Description (* production version)
------------------------------------------------------------------------
OGLutil              OpenGL utility library
OpenEXR-1.2.2        OpenEXR high dynamic range image library
RenderOGL            RenderOGL Rendering package
pfstools-1.2.1       MPI high dynamic range toolkit
------------------------------------------------------------------------
</verbatim>

From there, =use PACKAGE_NAME= will give you access to that software is normal.

_One cautionary note is that the =use= scripts will pick packages from the CS local section before the PSM section.  If there is a collision, like with !OpenEXR-1.2.2, then the CS local one will be chosen.  This hasn't been a problem, but if it is, it is always possible to tag our packages as =PSM-PACKAGE_NAME= to differentiate.  Talk to [[mailto:mmt@cs.ubc.ca Matthew Trentacoste]] if there are any issues._


---++ Bash

The Bash equivalent of the above is:

<verbatim>
#=== Determine our system ==========
# Determine what operating system we are using
OS_DIST=`uname -s`;

# Figure out what distro and version we are running
if [ "$OS_DIST" = "Linux" ]; then
   if [ -f /etc/SuSE-release ]; then
      OS_NAME='SuSE'
      if [[ `cat /etc/SuSE-release | grep openSUSE` ]]; then
         OS_VERS=`head -n 1 /etc/SuSE-release | sed 's/[\(\)]//g' | awk '{print "_" $3 "-" $2}'`
      else
         OS_VERS=`head -n 1 /etc/SuSE-release | awk '{print $3; }'`
      fi
   elif [ -f /etc/fedora-release ]; then
      OS_NAME='Fedora'
      OS_VERS=`head -n 1 /etc/fedora-release | awk '{print $5; }'`
   elif [ -f /etc/redhat-release ]; then
      OS_NAME='Redhat'
      OS_VERS=`head -n 1 /etc/redhat-release | awk '{print $5; }'`
   elif [ -f /etc/debian_version ]; then
      OS_NAME='Debian'
      OS_VERS=`head -n 1 /etc/debian_version | awk '{print $1; }'`
   else
      OS_NAME='None'
      OS_VERS='0'
   fi
else
   # If other (namely Sun and Apple), then use the uname results
   OS_NAME=${OS_DIST}
   OS_VERS=`uname -r`
fi

# Set the system name to locate our binaries with
export USERPKGARCHSYS=${OS_NAME}${OS_VERS}

#=== Set use system ================

# Set generic location and description
export USERPKGBASE=/ubc/cs/research/imager/project/psm/Packages
export USERPKGROOT=${USERPKGBASE}/Generic
export USERPKGDESC='PSM Lab software'

# Set architecture directory based on what system you are on
export USERPKGARCH=${USERPKGBASE}/Arch/${USERPKGARCHSYS}
</verbatim>


---++ Libraries and Compiling

While the =use= script will add things to the $PATH variable, it does not automatically configure everything necessary to compile software with libraries and header in the PSM =use= software.  Additional environmental variables can be set to instruct GCC to search in the proper places.  Paste the following code below the other section in your =~/csh_init/environment= file.

<verbatim>
#=== Add gcc libs/includes =========

# Set the include directory list to search for PSM libraries
if ($?C_INCLUDE_PATH) then
    setenv C_INCLUDE_PATH ${C_INCLUDE_PATH}:$USERPKGROOT/include
else
    setenv C_INCLUDE_PATH $USERPKGROOT/include
endif

# Set the include directory list to search for PSM libraries
if ($?CPLUS_INCLUDE_PATH) then
    setenv CPLUS_INCLUDE_PATH ${CPLUS_INCLUDE_PATH}:$USERPKGROOT/include
else
    setenv CPLUS_INCLUDE_PATH $USERPKGROOT/include
endif

# Set the library directory list to search for PSM libraries
if ($?LIBRARY_PATH) then
    setenv LIBRARY_PATH ${LIBRARY_PATH}:$USERPKGARCH/lib
else
    setenv LIBRARY_PATH $USERPKGARCH/lib
endif
</verbatim>

#AddingSoftware
---++ Adding Software

*NOTE:* _These instructions are provided as an example, and aren't guaranteed to work if you have multiple additions to the =use= search system configured.  Make sure to check.  If at all in doubt, contact [[mailto:mmt@cs.ubc.ca][Matthew Trentacoste]]._

Installing additional software in the PSM space is similar to the instructions provided at http://www.cs.ubc.ca/support/unix-software-usage (old url: http://www.cs.ubc.ca/local/computing/unix/software.shtml) with the exception of where exactly the software is placed.  Following the example on the CS department page, <br/>
instead of =/cs/public/generic/src/whizzy-3.14= use =$USERPKGBASE/Source/whizzy-3.14= <br/>
instead of =/cs/public/generic/lib/pkg/whizzy-3.14= use =$USERPKGROOT/whizzy-3.14= <br/>
instead of =/cs/public/lib/pkg/whizzy-3.14= use =$USERPKGARCH/whizzy-3.14= <br/>
Following the directions with those revised paths will install it in the PSM space.

See the attached "[[%ATTACHURL%/CompileMPlayer][CompileMPlayer]]" script for an example. <br/>
Alternative: Use =~nasarouf/psm/installer &lt;name&gt;= where =$USERPKGBASE/Source/&lt;name&gt;.tar.gz= is the package you want to unzip and install. sets the PFSTools "pkg-config" path so that libraries depending on it can find it.


#UpgradingOperatingSystem
---++ Upgrading the Operating System

When the department moves from one version of the OS to another (eg SuSE 11.3 to SuSE 11.4) the older packages will no longer be available. To initialise the package system under the new OS, follow these steps:

   * create directory $USERPKGARCH if it does not already exist (ensure group RW permissions)
   * create directory $USERPKGARCH/lib
   * install each package as before

Note that as of May 2011, helpdesk reports that the "use" script is broken (and has been for the past few years). It does not properly execute the "enable" and "disable" scripts mentioned on the Imager local resources webpage. This makes it difficult to install libraries and then compile against them, because the use script does not properly set various environment variables (CMAKE_LIBRARY_PATH, LIBRARY_PATH, PKG_CONFIG_PATH, PYTHON_PATH etc). One way to work around this is to have a $USERPKGARCH/lib and a $USERPKGROOT/include directory, point your various environment variables to those paths in your .bash_profile file, and then symlink each package's libraries and headers into those directories upon package installation. This has worked so far under SuSE 11.1 ... 11.3 but it does make uninstalling/upgrading packages tricky, and there is a potential for name conflict. An alternative would be to write an enable script for each package then then do "use whizzy; whizzy_enable" each time you want to use the package.
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
ext	CompileMPlayer	r1	manage	0.8 K	2009-07-27 - 23:27	BradAtcheson	Script for downloading and building MPlayer
This topic: Imager > WebHome > PsmSpace > PsmPackageSystem
Topic revision: r10 - 2011-05-31 - BradAtcheson