Compilation in BlueGene/L

From Zori Wiki

Preamble

This is a work in progress. Most links on this page are dead ends.

Get Organized

The compilation process for BG/L is more elaborate than for other platforms. The compute nodes have a different architecture than the login nodes where the compilation is done. To facilitate the cross-compile, it is necessary to compile zori and the required libraries for bot the login nodes and the compute nodes. Obviously, it will be important to keep these sets separate and organized. There's some flexibility here, but this is how I do it.

 In your home directory, create the following subdirectories:

   local:     This is where the libraries for the login nodes will be installed. 

   local_bgl: This is where the libraries for the compute nodes will be installed. 

   zori_aux:  This is where the sources for the libraries will be saved, and where the compilation occurs. 

   zori:      Zori sources for the login nodes. 

   zori_bgl:  Zori sources for the compute nodes. 

 We found it convenient to write Xlc and xlC wrappers for BG/L cross-compilers that have the same names as the regular compilers. 

The non-mpi wrappers should be saved in local/bin_bgl and the mpi wrappers in local/bin_bgl_mpi

 Get the required libraries (http://www.zori-code.com/wiki/index.php/Compilation_Instructions#Requirements) and save their tarballs in zori_aux. 

 Unpack the tarballs and move a copy of the tar files in zori_aux/tar 

Compilation for the login nodes

Compilation for the login nodes follows the GNU make procedure.

 Note that this is not a parallel install.
 That's because mpicc always does a cross compile
 This may cause problems building zori later. We'll see.

Compile zavg for the login node.

This is enough to compile zavg for the login node now.

 cd ~/zori 

 export OLDPATH=$PATH 

 export PATH=$HOME/local/bin/:$PATH 

 ACLOCAL_FLAGS='-I $HOME/local/share/aclocal' \ 

 CC='gcc -I$HOME/local/include -L$HOME/local/lib' \ 

 ./autogen.sh --enable-debug 

 make zavg 

 export PATH=$OLDPATH 

Gear up for the cross compile.

 We've written some go_scripts to make this a little easier. 

 Copy go_gsl (http://www.zori-code.com/wiki/index.php?title=go_scripts), go_popt (http://www.zori-code.com/wiki/index.php?title=go_scripts), go_glib (http://www.zori-code.com/wiki/index.php?title=go_scripts) and go_xml into zori_aux (http://www.zori-code.com/wiki/index.php?title=go_scripts). 

 Unpack the gsl, glib and xml tarballs again, and move them into zori_aux adding _bgl to the extracted directories. 

 pkgconfig is used only for the compilation process, it's not needed for the compute nodes. 

 IBM has modified a version of HDF5-1.6.4 so that it runs on BG/L. 
 As far as I know, it's not yet being distributed publicly. 
 If you can get it somehow, unpack it into zori_aux/hdf5-1.6.4_bgl. 

 When you're done, things should look like this.
   > ls ~/zori_aux
   glib-2.4.6      go_gsl   gsl-1.6      hdf5-1.6.4_bgl      pkgconfig-0.17.2  tar
   glib-2.4.6_bgl  go_popt  gsl-1.6_bgl  libxml2-2.6.19      popt-1.7
   go_glib         go_xml   hdf5-1.6.4   libxml2-2.6.19_bgl  popt-1.7_bgl

Cross-compiling for the BG/L compute nodes.

 modify ./myconf.sh to specify prefix=$HOME/local_bgl

 modify go_popt to specify prefix=$HOME/local_bgl

 modify go_gsl to specify prefix=$HOME/local_bgl

 repair wavelet/dwt.c:59 so that second last arg is an int (see go_gsl)

 modify go_xml to specify prefix=$HOME/local_bgl

 at my_dlfcn.h:28 replace <bits/dlfcn.h> with "my_bits_dlfcn.h" 

 at gmodule/gmodule-dl.c:31 replace <dlfcn.h> with "my_dlfcn.h" 

 modify go_glib to specify prefix=$HOME/local_bgl 

 in gobject/Makefile and tests/gobgject/Makefile 

 change the definition of glib_genmarshal so that it reads 

 glib_genmarshal=$(HOME)/local/bin/glib-genmarshal 

 I'd really like to automate this, but I'm not so good with sed 

Cross-Compiling Zori for BG/L compute nodes.

 We have all the neccessary libraries now. Let's compile Zori. 

   cd zori_bgl
   ./go_zori
 It will try to compile zopt and fail. Don't worry. That's expected.
 (I haven't yet gotten zopt running on BG/L ).

I'm heading for bed now. More tomorrow. Ignore everything below this line.

• pkgconfig

wget http://pkgconfig.freedesktop.org/releases/pkgconfig-0.17.2.tar.gz

• glib

wget ftp://ftp.gtk.org/pub/gtk/v2.4/glib-2.4.8.tar.gz