Skip to content

add new easyconfigs for ictce/5.3.0#229

Merged
boegel merged 10 commits intoeasybuilders:developfrom
fgeorgatos:194_contrib_ictce_5.3.0_easyconfigs
Apr 27, 2013
Merged

add new easyconfigs for ictce/5.3.0#229
boegel merged 10 commits intoeasybuilders:developfrom
fgeorgatos:194_contrib_ictce_5.3.0_easyconfigs

Conversation

@fgeorgatos
Copy link
Copy Markdown
Contributor

Hi,

this is to populate the builds for the latest ictce toolchain; on my end successful modules
are listed below; not included ones may not have serious problems (eg. broken download) or, there may be a more serious issue behind it; nevertheless, here is the full list (ie. 177 are fine out of ~215):

ABySS/1.3.4-ictce-5.3.0-Python-2.7.3
AMOS/3.1.0-ictce-5.3.0
ASE/3.6.0.2515-ictce-5.3.0-Python-2.7.3
Armadillo/2.4.4-ictce-5.3.0-Python-2.7.3
Autoconf/2.69-ictce-5.3.0
BFAST/0.7.0a-ictce-5.3.0
BWA/0.6.2-ictce-5.3.0
Bison/2.5-ictce-5.3.0
Bonnie++/1.03e-ictce-5.3.0
Boost/1.49.0-ictce-5.3.0-Python-2.7.3
Boost/1.51.0-ictce-5.3.0
Bowtie2/2.0.2-ictce-5.3.0
CGAL/4.0-ictce-5.3.0-Python-2.7.3
CLHEP/2.1.1.0-ictce-5.3.0
CMake/2.8.4-ictce-5.3.0
CVXOPT/1.1.5-ictce-5.3.0-Python-2.7.3
ClustalW2/2.1-ictce-5.3.0
Corkscrew/2.0-ictce-5.3.0
Cython/0.16-ictce-5.3.0-Python-2.7.3
Docutils/0.9.1-ictce-5.3.0-Python-2.7.3
Doxygen/1.8.1.1-ictce-5.3.0
ELinks/0.12pre5-ictce-5.3.0
ESPResSo/3.1.1-ictce-5.3.0-parallel
ESPResSo/3.1.1-ictce-5.3.0-serial
Eigen/3.1.1-ictce-5.3.0
FASTX-Toolkit/0.0.13.2-ictce-5.3.0
FFC/1.0.0-ictce-5.3.0-Python-2.7.3
FFTW/3.3.1-ictce-5.3.0
FFTW/3.3.3-ictce-5.3.0
FFTW/3.3.3-ictce-5.3.0-single
FIAT/1.0.0-ictce-5.3.0-Python-2.7.3
GEOS/3.3.5-ictce-5.3.0
GMP/5.0.5-ictce-5.3.0
GPAW/0.9.0.8965-ictce-5.3.0-Python-2.7.3
GSL/1.15-ictce-5.3.0
Greenlet/0.4.0-ictce-5.3.0-Python-2.7.3
HDF/4.2.7-patch1-ictce-5.3.0
HDF5/1.8.7-ictce-5.3.0
HDF5/1.8.9-ictce-5.3.0
HMMER/3.0-ictce-5.3.0
HPL/2.0-ictce-5.3.0
Harminv/1.3.1-ictce-5.3.0
Hypre/2.8.0b-ictce-5.3.0
Infernal/1.1rc1-ictce-5.3.0
Instant/1.0.0-ictce-5.3.0-Python-2.7.3
Iperf/2.0.5-ictce-5.3.0
JasPer/1.900.1-ictce-5.3.0
Jinja2/2.6-ictce-5.3.0-Python-2.7.3
Libint/1.1.4-ictce-5.3.0
M4/1.4.16-ictce-5.3.0
MCL/12.135-ictce-5.3.0
MDP/3.3-ictce-5.3.0-Python-2.7.3
MEME/4.8.0-ictce-5.3.0
METIS/4.0.1-ictce-5.3.0
METIS/5.0.2-ictce-5.3.0
MPFR/3.1.0-ictce-5.3.0
MUMmer/3.23-ictce-5.3.0
Mercurial/2.3.2-ictce-5.3.0-Python-2.7.3
Mesa/7.11.2-ictce-5.3.0-Python-2.7.3
MetaVelvet/1.2.01-ictce-5.3.0
MrBayes/3.1.2-ictce-5.3.0
NASM/2.07-ictce-5.3.0
NEURON/7.2-ictce-5.3.0
Oger/1.1.3-ictce-5.3.0-Python-2.7.3
OpenFOAM/2.1.1-ictce-5.3.0
OpenPGM/5.2.122-ictce-5.3.0
PAPI/5.0.1-ictce-5.3.0
PCRE/8.12-ictce-5.3.0
PETSc/3.3-p2-ictce-5.3.0-Python-2.7.3
ParMETIS/3.1.1-ictce-5.3.0
ParMETIS/4.0.2-ictce-5.3.0
Pasha/1.0.3-ictce-5.3.0
Pasha/1.0.5-ictce-5.3.0
Primer3/2.3.0-ictce-5.3.0
PyZMQ/2.2.0.1-ictce-5.3.0-Python-2.7.3-zmq2
PyZMQ/2.2.0.1-ictce-5.3.0-Python-2.7.3-zmq3
Python/2.7.3-ictce-5.3.0
Python/3.2.3-ictce-5.3.0
R/2.15.2-ictce-5.3.0
R/2.15.2-ictce-5.3.0-bare
RNAz/2.1-ictce-5.3.0
SAMtools/0.1.18-ictce-5.3.0
SCOOP/0.5.3-ictce-5.3.0-Python-2.7.3
SCOTCH/5.1.12b_esmumps-ictce-5.3.0
SHRiMP/2.2.3-ictce-5.3.0
SLEPc/3.3-p1-ictce-5.3.0-Python-2.7.3
SOAPdenovo/1.05-ictce-5.3.0
SWIG/2.0.4-ictce-5.3.0-Python-2.7.3
ScientificPython/2.8-ictce-5.3.0-Python-2.7.3
Shapely/1.2.15-ictce-5.3.0-Python-2.7.3
Sphinx/1.1.3-ictce-5.3.0-Python-2.7.3
Stow/1.3.3-ictce-5.3.0
SuiteSparse/3.7.0-ictce-5.3.0-withparmetis
Szip/2.1-ictce-5.3.0
Tar/1.26-ictce-5.3.0
Tcl/8.5.12-ictce-5.3.0
Theano/0.5.0-ictce-5.3.0-Python-2.7.3
Tk/8.5.12-ictce-5.3.0
Trilinos/10.12.2-ictce-5.3.0-Python-2.7.3
Trinity/2012-10-05-ictce-5.3.0
UDUNITS/2.1.24-ictce-5.3.0
UFC/2.0.5-ictce-5.3.0-Python-2.7.3
UFL/1.0.0-ictce-5.3.0-Python-2.7.3
VTK/5.10.1-ictce-5.3.0
Velvet/1.2.07-ictce-5.3.0
ViennaRNA/2.0.7-ictce-5.3.0
Viper/1.0.0-ictce-5.3.0-Python-2.7.3
WPS/3.3.1-ictce-5.3.0-dmpar
WRF/3.3.1-ictce-5.3.0-dmpar
XCrySDen/1.5.53-ictce-5.3.0
Yasm/1.2.0-ictce-5.3.0
ZeroMQ/2.2.0-ictce-5.3.0
ZeroMQ/3.2.2-ictce-5.3.0
a2ps/4.14-ictce-5.3.0
aria2/1.15.1-ictce-5.3.0
byacc/20120526-ictce-5.3.0
bzip2/1.0.6-ictce-5.3.0
cURL/7.27.0-ictce-5.3.0
ccache/3.1.9-ictce-5.3.0
cflow/1.4-ictce-5.3.0
cgdb/0.6.5-ictce-5.3.0
expat/2.1.0-ictce-5.3.0
flex/2.5.35-ictce-5.3.0
flex/2.5.37-ictce-5.3.0
fmri/1.4-8-ictce-5.3.0-R-2.15.2
freetype/2.4.10-ictce-5.3.0
g2lib/1.2.4-ictce-5.3.0
git/1.7.12-ictce-5.3.0
glproto/1.4.16-ictce-5.3.0
gnuplot/4.6.0-ictce-5.3.0
google-sparsehash/2.0.2-ictce-5.3.0
gperf/3.0.4-ictce-5.3.0
grib_api/1.9.18-ictce-5.3.0
gzip/1.5-ictce-5.3.0
ictce/5.3.0
jellyfish/1.1.10-ictce-5.3.0
libdrm/2.4.27-ictce-5.3.0
libgtextutils/0.6.1-ictce-5.3.0
libpciaccess/0.13.1-ictce-5.3.0
libpng/1.5.10-ictce-5.3.0
libpng/1.5.11-ictce-5.3.0
libpng/1.5.13-ictce-5.3.0
libpthread-stubs/0.3-ictce-5.3.0
libreadline/6.2-ictce-5.3.0
libtool/2.4.2-ictce-5.3.0
libunistring/0.9.3-ictce-5.3.0
libxc/2.0.1-ictce-5.3.0
libxcb/1.8-ictce-5.3.0-Python-2.7.3
libxml2/2.8.0-ictce-5.3.0
libxml2/2.8.0-ictce-5.3.0-Python-2.7.3
libxml2/2.9.0-ictce-5.3.0
lzo/2.06-ictce-5.3.0
makedepend/1.0.4-ictce-5.3.0
matplotlib/1.1.1-ictce-5.3.0-Python-2.7.3
matplotlib/1.2.0-ictce-5.3.0-Python-2.7.3
mc/4.6.1-ictce-5.3.0
mpiBLAST/1.6.0-ictce-5.3.0
nano/2.2.6-ictce-5.3.0
ncurses/5.9-ictce-5.3.0
netCDF/4.1.3-ictce-5.3.0
netCDF/4.2-ictce-5.3.0
numactl/2.0.8-ictce-5.3.0
numpy/1.6.2-ictce-5.3.0-Python-2.7.3
parallel/20130122-ictce-5.3.0
paycheck/1.0.2-ictce-5.3.0-Python-2.7.3
petsc4py/3.3-ictce-5.3.0-Python-2.7.3
pkg-config/0.27.1-ictce-5.3.0
scipy/0.11.0-ictce-5.3.0-Python-2.7.3
setuptools/0.6c11-ictce-5.3.0-Python-2.7.3
util-linux/2.22.2-ictce-5.3.0
xcb-proto/1.7-ictce-5.3.0-Python-2.7.3
xorg-macros/1.17-ictce-5.3.0
xproto/7.0.23-ictce-5.3.0
zlib/1.2.5-ictce-5.3.0
zlib/1.2.7-ictce-5.3.0
zsync/0.6.2-ictce-5.3.0

Signed-off-by: Fotis Georgatos [email protected]

@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 25, 2013

The easyconfigs unit tests pass without an issue when this PR is combined with #194:

$ python -m test.easyconfigs.suite

.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 1185 tests in 164.792s

OK

This basically means that all easyconfigs parse, can be instantiated as easyblocks, and that all dependencies can be resolved. So, excellent PR. I won't go in and nitpick on code style here, since we'll take care of that in an automated way soon anyway.

I'm a bit reluctant to merge it in though, only because of the further explosion of the amount of easyconfig files.
We're planning to enhance the easyconfig format to support 'fat' easyconfigs, that allow to include specifications for different versions/toolchains/... in a single file. Once that's supported, we'll concatenate all easyconfigs for each software package together in a single file.

So, currently adding more easyconfigs that only differ by a toolchain version will only add to that workload... @stdweird: What's your view on this?

@stdweird
Copy link
Copy Markdown
Contributor

giving the unittests ran by @boegel and @fgeorgatos probably actually build these, i'd say why not merge them in. but it illustrates the need for automated new format and accompanying tools.

@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 26, 2013

Some builds fail. For these, the easyconfigs should be pulled out of this PR, and new ones should be created where the problems can be followed up. @fgeorgatos: Can you take care of that?
(I will update this list as tests progress)

../src/x86/ffi64.c(41): error: invalid redeclaration of type name "__int128_t"
  typedef struct { int64_t m[2]; } __int128_t;

The ictce-4.0.6 still works though, so it seems like the icc compiler used by this toolchain became stricter?
The weird thing is that Python uses a very similar patch, since it includes libffi.

ld: cannot find -lmkl_solver_lp64_sequential
checking for gcc... mpicc
checking for C compiler default output file name... configure: error: C compiler cannot create executables
See `config.log' for more details.

in config.log:

ld: cannot find -lmkl_solver_lp64_sequential

@fgeorgatos
Copy link
Copy Markdown
Contributor Author

done. making separate PRs now

On Fri, Apr 26, 2013 at 11:47 AM, Kenneth Hoste [email protected]:

Some builds fail. For these, the easyconfigs should be pulled out of this
PR, and new ones should be created where the problems can be followed up.
@fgeorgatos https://github.com/fgeorgatos: Can you take care of that?
(I will update this list as tests progress)


Reply to this email directly or view it on GitHubhttps://github.com//pull/229#issuecomment-17064417
.

@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 26, 2013

More failures, mostly due to dependencies:

  • these should be included in ictce-5.3.0: libffi, guile, libmatheval, h5utils, Meep, libctl builds #238 instead:
    • guile-1.8.8-ictce-5.3.0: depends on libffi
    • libctl-3.2.1-ictce-5.3.0: depends on guile
    • Meep-1.2-ictce-5.3.0: depends on libctl
    • libmatheval-1.1.8-ictce-5.3.0: depends on guile
    • h5utils-1.12.1-ictce-5.3.0: depends on libmatheval
  • python-meep-1.4.2-ictce-5.3.0-Python-2.7.3: depends on Meep

Can probably be fixed easily:

  • R-3.0.0-ictce-5.3.0-bare: download of sources fails, so source_urls needs a fix?

Separate PRs are required for:

  • NCBI-Toolkit-9.0.0-ictce-5.3.0: triggers internal compiler error (ICE):
/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL5/harpertown/software/icc/2013.3.163/bin/intel64/icpc  -c  -we70 -pthread -O1 -xHOST -ftz -fp-speculation=safe -fp-model source  -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64  -I/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL5/harpertown/software/imkl/11.0.3.163/mkl/include -I/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL5/harpertown/software/Boost/1.51.0-ictce-5.3.0/include -D_MT -D_REENTRANT -D_THREAD_SAFE -I/tmp/easybuild_build/NCBI-Toolkit/9.0.0/ictce-5.3.0/ncbi_cxx--9_0_0/ICC1311163-ReleaseMT64/inc -I/tmp/easybuild_build/NCBI-Toolkit/9.0.0/ictce-5.3.0/ncbi_cxx--9_0_0/include  /tmp/easybuild_build/NCBI-Toolkit/9.0.0/ictce-5.3.0/ncbi_cxx--9_0_0/src/connect/test/test_ncbi_pipe_connector.cpp -o test_ncbi_pipe_connector.o 
": internal error: 0_7849
ld: cannot find -lmkl_solver_lp64
  • pyTables-2.4.0-ictce-5.3.0-Python-2.7.3: unclear cause, maybe partially broken Python build?
== 2013-04-26 12:11:13,105 main.filetools ERROR EasyBuild crashed with an error (at easybuild/tools/build_log.py:69 in caller_info): cmd "python setup.py build" exited with exitcode 1 and output:
Traceback (most recent call last):
  File "setup.py", line 5, in <module>
    import ctypes
  File "/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL5/harpertown/software/Python/2.7.3-ictce-5.3.0/lib/python2.7/ctypes/__init__.py", line 10, in <module>
    from _ctypes import Union, Structure, Array
ImportError: No module named _ctypes
  • BLAST-2.2.27-ictce-5.3.0: triggers ICE:
/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL5/harpertown/software/icc/2013.3.163/bin/intel64/icpc  -c  -we70 -pthread -xHOST -ftz -fp-speculation=safe -fp-model source   -O3 -axW -ip  -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64  -I/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL5/harpertown/software/imkl/11.0.3.163/mkl/include -I/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL5/harpertown/software/Boost/1.51.0-ictce-5.3.0/include -D_MT -D_REENTRANT -D_THREAD_SAFE -I/tmp/easybuild_build/BLAST/2.2.27/ictce-5.3.0/ncbi-blast-2.2.27+-src/c++/ICC1311163-ReleaseMT64/inc -I/tmp/easybuild_build/BLAST/2.2.27/ictce-5.3.0/ncbi-blast-2.2.27+-src/c++/include  /tmp/easybuild_build/BLAST/2.2.27/ictce-5.3.0/ncbi-blast-2.2.27+-src/c++/src/algo/blast/api/rpstblastn_options.cpp -o rpstblastn_options.o 
icpc: command line remark #10279: option '-axW' is deprecated and will be removed in a future release. See '-help deprecated'
": internal error: backend signals
  • ROOT-v5.34.01-ictce-5.3.0 (and GATE-v6.2-ictce-5.3.0 that depends on it): syntax errors and related problems (??):
icc -O -fPIC -wd1476 -Iinclude -DR__HAVE_CONFIG  -pthread -UR__HAVE_CONFIG -DROOTBUILD -I/tmp/easybuild_build/ROOT/v5.34.01/ictce-5.3.0/root/core/utils/src -o core/utils/src/RStl_tx
cint/cint/lib/iccstrm/iostrm.h(85): error: class "std::basic_ostream<char, std::char_traits<char>>" has no member "_Mysb"
  static ostream::_Mysb *G__store_cout;
                  ^
cint/cint/lib/iccstrm/iostrm.h(86): error: class "std::basic_ostream<char, std::char_traits<char>>" has no member "_Mysb"
  static ostream::_Mysb *G__store_cerr;
                  ^
cint/cint/lib/iccstrm/iostrm.h(87): error: class "std::basic_istream<char, std::char_traits<char>>" has no member "_Mysb"
  static istream::_Mysb *G__store_cin;
...
  • NCL-6.0.0-ictce-5.3.0: undefined stuff, cause unclear:
mpicc -ansi -fPIC -O3 -xHOST -ftz -fp-speculation=safe -fp-model source -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -I/usr/local/include -fPIC -O3 -xHOST -ftz -fp-speculation=safe -c
NclNetCdf.c(363): warning #810: conversion from "void *" to "int" may lose significant bits
        if ((int)(rec->options[NC_SUPPRESS_CLOSE_OPT].values) == 0) {
bash-3.2$ sed -n "34473,34480p" NCL-6.0.0-ictce-5.3.0.o4010124
mpicc -ansi -fPIC -O3 -xHOST -ftz -fp-speculation=safe -fp-model source -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -I/usr/local/include -fPIC -O3 -xHOST -ftz -fp-speculation=safe -c
NclNetCdf.c(363): warning #810: conversion from "void *" to "int" may lose significant bits
        if ((int)(rec->options[NC_SUPPRESS_CLOSE_OPT].values) == 0) {
            ^

NclNetCdf.c(411): error: identifier "NC_CHUNKED" is undefined
        int storage = NC_CHUNKED;
ld: cannot find -lmkl_solver_ilp64
  • DOLFIN-1.0.0-ictce-5.3.0-Python-2.7.3: dependencies fail, and EB is told to act on those as genuine failures:
Performing Test PARMETIS_TEST_RUNS - Failed

A couple of tests are still running: ALADIN-36t1_op2bf1-ictce-5.3.0 and OpenFOAM-2.1.1-ictce-5.3.0

@fgeorgatos
Copy link
Copy Markdown
Contributor Author

OK, all done; comments on the 3 respective feedback sections:

#238 libffi related, being finalized by the time you read this
#240 addresses the need for latest R and fixes another issue of older R versions along
#241 includes the 8 easyconfigs that will take more effort; maybe easy, maybe not,not for now

@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 26, 2013

The two remaining builds that were running (ALADIN and OpenFOAM), worked fine.

@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 26, 2013

@fgeorgatos: GATE-v6.2-ictce-5.3.0 should be moved to #241 as well, because it depends on ROOT. And R-3.0.0-ictce-5.3.0-bare has to be removed, as it was moved to #240.

These are blockers to merge this in, so please fix.

@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 26, 2013

Also get rid of WPS-3.3.1-ictce-5.3.0-dmpar, and open a separate PR for it. It seems like it's getting miscompiled by the Intel compilers included in ictce/5.3.0, see below.

This is a bit strange, since it seems to have worked on your end...

== 2013-04-26 21:33:43,491 main.filetools ERROR EasyBuild crashed with an error (at easybuild/tools/build_log.py:69 in caller_info): cmd "mpirun --file=/tmp/mpdboot -machinefile /tmp/nodes -np 1 /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL5/harpertown/software/WPS/3.3.1-ictce-5.3.0-dmpar/WPS/metgrid.exe" exited with exitcode 173 and output:
Processing domain 1 of 2
 Processing 2000-01-24_12
    FILE
forrtl: severe (173): A pointer passed to DEALLOCATE points to an object that cannot be deallocated
Image              PC                Routine            Line        Source             
metgrid.exe        00000000005FAFEE  Unknown               Unknown  Unknown
metgrid.exe        00000000005F9A86  Unknown               Unknown  Unknown
metgrid.exe        00000000005A9B02  Unknown               Unknown  Unknown
metgrid.exe        000000000054AE8B  Unknown               Unknown  Unknown
metgrid.exe        0000000000586B80  Unknown               Unknown  Unknown
metgrid.exe        00000000004A5C8B  Unknown               Unknown  Unknown
metgrid.exe        000000000045F8BA  Unknown               Unknown  Unknown
metgrid.exe        000000000045C088  Unknown               Unknown  Unknown
metgrid.exe        00000000004114BA  Unknown               Unknown  Unknown
metgrid.exe        000000000040996C  Unknown               Unknown  Unknown
libc.so.6          000000300081D994  Unknown               Unknown  Unknown
metgrid.exe        0000000000409879  Unknown               Unknown  Unknown
rank 0 in job 1  localhost_55857   caused collective abort of all ranks
  exit status of rank 0: return code 173 

@fgeorgatos
Copy link
Copy Markdown
Contributor Author

This is a bit strange, since it seems to have worked on your end...

After this experience, I am really happy for the course we took:

  • I supplied on purpose all possible easyconfigs, so that we had a chance to cross-check builds;
    there are builds that work for you but not my side and vice versa; now we have good overview of status
    thanks to dual-site build reports and therefor, I suggest to adopt this approach also in future cases.

READY TO MERGE!

@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 27, 2013

OK, all easyconfigs contributed here worked on my end, so merging them in.
@fgeorgatos: Don't forget to include the WPS-3.3.1-ictce-5.3.0-dmpar easyconfig somewhere else, so it doesn't get lost.

boegel added a commit that referenced this pull request Apr 27, 2013
…onfigs

add new easyconfigs for ictce/5.3.0
@boegel boegel merged commit 0e19968 into easybuilders:develop Apr 27, 2013
@fgeorgatos fgeorgatos deleted the 194_contrib_ictce_5.3.0_easyconfigs branch April 27, 2013 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants