Cray CC for FEniCS

Cray CC is c++11 compliant…

I never thought this was going to happen, but I just randomly checked the latest Cray C++ compiler, and bar a few minor details, it is now c++11 compliant! When you’re using a HPC machine, it is always nice to be able to use the manufacturer’s own compiler - I guess it gives you some kind of satisfaction, and a (probably misplaced) sense that any optimisations will be targeted to the hardware. Building FEniCS with Cray CC is at least now possible. How does it work?

Well, I just installed all the pure python modules of FEniCS as usual (see blog posts passim), and then load the PrgEnv-cray environment. Now, the fun begins. First of all, the Cray c++ compiler CC does print a lot of nonsense to the console, so we need to suppress that with -h msglevel_4, which is supposed to stop “warnings” and only print “errors”. Something is a bit wrong with this, as it still prints “Warning 11709” (quite a lot). OK, so -h nomessage=11709.
Everything is running inside CMake, so it’s just a question of adding a line like:

 -DCMAKE_CXX_FLAGS="-h std=c++11 -h msglevel_4 -h nomessage=11709"

Hmm… another problem… the FEniCS developers have used restrict as a function name, and it is a reserved word for Cray CC. So - I’ll have to rename it to something else. Anyway, if we do that, it all kind-of works. It is very slow though. It might take gcc 10 minutes to plough through the complete FEniCS dolfin build, but it takes Cray CC about an hour and a half… I’m sure it’s working hard on optimising… and at the end of it… you get: libdolfin.a and it is huge. Oh, wait, I wanted a shared library…

So now we can set export CRAYPE_LINK_TYPE=dynamic and try again, but it will fail linking unless you enable -h PIC (position independent code). And that now causes another problem with Eigen3 which has some inline asm which fails. OK - add options, add options…

-DCMAKE_CXX_FLAGS="-DEIGEN_NO_CPUID -h pic -h std=c++11 -h msglevel_4 -h nomessage=11709"

I tried this on two systems, one with Cray CC 8.4, another with CC 8.5 - I guess there must be some bug fixes, because the 8.4 build crashed a few times with Segmentation Faults (never great in a compiler).

CC-2116 crayc++: INTERNAL  
  "/opt/cray/cce/8.4.1/CC/x86-64/lib/ccfe" was terminated due to receipt of signal 013:  Segmentation fault.

Well, the CC 8.5 install completed. I have still to check if it actually works… next time.

Written with StackEdit.