Compiling FEniCS 1.4.0 on ARCHER

Compiling FEniCS 1.4.0 on ARCHER

So, today, I decided: why not? let’s compile FEniCS 1.4.0 on ARCHER.
What can possibly go wrong?
Before attempting anything, I switched to the gnu compilers. I suppose some day I should try the Cray Compiler – but why risk it, when I know gcc works.

module swap PrgEnv-cray PrgEnv-gnu

Then I thought: maybe we should try with Cray PETSC etc. - after all, it
is there, nicely set up - maybe even optimised for the machine? I need a load of
other things - some of which I have compiled myself, some of which are already
there…

module use /work/e319/shared/modules
module load eigen/3.2.0
module load python/2.7.6 
module load numpy/1.8.0
module load ply/3.4
module load boost/1.55
module load swig/2.0.10           # needed on the compute nodes
module load cmake/2.8.12.2        # yes, my own version - available on the compute nodes
module load scientific-python/2.8 # needed by FIAT
module load cray-hdf5-parallel/1.8.12
module load cray-petsc/3.4.3.1
export PETSC_DIR=$CRAY_PETSC_PREFIX_DIR
module load cray-trilinos/11.6.1.0
export TRILINOS_DIR=$CRAY_TRILINOS_PREFIX_DIR
module load cray-tpsl/1.4.0
export SCOTCH_DIR=$CRAY_TPSL_PREFIX_DIR
export PARMETIS_DIR=$CRAY_TPSL_PREFIX_DIR

Now, to get it all working, first of all, we need ffc working. That is mostly
python, so fairly easy to fix up:-

wget https://bitbucket.org/fenics-project/ffc/downloads/ffc-1.4.0.tar.gz
tar xf ffc-1.4.0.tar.gz
cd ffc-1.4.0
python setup.py install --prefix=/work/e319/shared/packages/fenics-1.4.0

which can also be repeated for instant, ufl and fiat.
Now, on to dolfin. In cmake, it is essential to disable the build tests: -DDOLFIN_SKIP_BUILD_TESTS=true, and the MPI autodetection: -DDOLFIN_AUTO_DETECT_MPI=false.
CMake doesn’t know that Cray PETSc has some crazy names for its libraries, so we have to tell it somewhere. I just hack a line into cmake/modules/FindPETSc.cmake :-

set(PETSC_LIB_BASIC "-lcraypetsc_gnu_real")

But now, there is a problem with Trilinos. This looks more serious.

CMake Error at /opt/cray/trilinos/11.6.1.0/GNU/48/sandybridge/lib/cmake/Trilinos/TrilinosTargets.cmake:412 (message):
  The imported target "teuchoscore" references the file

     "/opt/cray/trilinos/11.6.1.0/GNU/48/sandybridge/lib/libteuchoscore.so.11.6.1"

OK, maybe we can manage without Trilinos for the moment… module rm trilinos…
Now it compiles.. but does it work? More later…

Written with StackEdit.