Psi 4.0b5 by wpoely86 · Pull Request #457 · easybuilders/easybuild-easyconfigs

wpoely86 · 2013-10-06T09:38:53Z

This is the one to rule them all.

This PR replaces #443, #434, #439, #433 and #439.

Test on raichu and everything works.

Depends on easybuilders/easybuild-easyblocks#270

I've added two env vars to the configure script: one for the install directory for the source and one for the install directory of the objects.

A variable is changed without proper mutex locking.

I've added two env vars to the configure script: one for the install directory for the source and one for the install directory of the objects.

Hang should be fixed now.

…add mpi PSI goalf easyconfig

…into psi * 'psi' of https://github.com/boegel/easybuild-easyconfigs: remove respecifying default max_fail_ratio of 0.5 in NWChem easyconfig really fix typo in patch file fix typo in patch file complete mpi patch file for PSI-4.0b5 align PSI easyconfigs, correct filename of -mt PSI goalf easyconfig, add mpi PSI goalf easyconfig

wpoely86 · 2013-10-11T07:58:08Z

All merged, ready to go! (At least!)

boegel · 2013-10-11T14:20:31Z

I retested all of this on gengar, with PSI being installed on gengar scratch, which worked fine.
I also tested this on delcatty, with PSI being installed on gengar scratch (NFS mounted). In that case, I'm still seeing PSI tests occasionally hanging...

I don't think the problem is in the build however, so I won't hold this PR any longer.

Just to clarify things: when you tested on raichu, on which filesystem was the target installation dir for PSI?

Psi 4.0b5

wpoely86 · 2013-10-13T14:12:13Z

@boegel, I've tested on raichu, both on /tmp and on the scratch. In both case everything worked fine. If you can reproduce those hangs, I'm still interested in the backtraces.

boegel · 2013-10-18T06:17:43Z

I'm seeing the hang again, this time with PSI 4.0b4 on delcatty using delcatty's scratch (so not related to using NFS or not, indeed). This might be because the thread-pool patch is not applied to PSI 4.0b4 though, can you confirm?

It's hanging on:

user  35362  0.0  0.0   4396  1080 ?        S    Oct17   0:00                  \_ make tests TESTFLAGS=-u -q
user  35363  0.0  0.0   9356  1372 ?        S    Oct17   0:00                      \_ /bin/sh -c (cd tests; echo Running test suite...; make) || exit 1;
user  35364  0.0  0.0   9360   936 ?        S    Oct17   0:00                          \_ /bin/sh -c (cd tests; echo Running test suite...; make) || exit 1;
user  35365  0.0  0.0   4664  1348 ?        S    Oct17   0:00                              \_ make
user  42653  0.0  0.0   9356  1396 ?        S    Oct17   0:00                                  \_ /bin/sh -c make -C cisd-h2o-clpse; true
user  42654  0.0  0.0   4396  1092 ?        S    Oct17   0:00                                      \_ make -C cisd-h2o-clpse
user  42663  0.0  0.0  25900  2384 ?        S    Oct17   0:00                                          \_ perl ../runtest.pl /tmp/me/easybuild_build/PSI/4.0b4/ictce-4.1.13/psi4.0b4/tests/cisd-h2o-clpse/input.dat cisd-h2o-clpse.test false
user  42664  0.0  0.1 2671244 95908 ?       Sl   Oct17   0:11                                              \_ ../../bin/psi4 /tmp/me/easybuild_build/PSI/4.0b4/ictce-4.1.13/psi4.0b4/tests/cisd-h2o-clpse/input.dat output.dat

Here's the backtrace:

(gdb) bt
#0  0x00002aaaab46243c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000e6833d in psi::detci::tpool_queue_close(psi::detci::tpool*, int) ()
#2  0x0000000000e6330e in psi::detci::s3_block_vdiag(psi::detci::stringwr*, psi::detci::stringwr*, double**, double**, double*, int, int, int, int, int, int, int, int, double**, double*, double*, double*, int*, int*) ()
#3  0x0000000000e4403d in psi::detci::sigma_block(psi::detci::stringwr**, psi::detci::stringwr**, double**, double**, double*, double*, int, int, int, int, int, int, int, int, int, int, int, int, int, int, int, int) ()
#4  0x0000000000e44400 in psi::detci::sigma_b(psi::detci::stringwr**, psi::detci::stringwr**, psi::detci::CIvect&, psi::detci::CIvect&, double*, double*, int, int) ()
#5  0x0000000000e42ce2 in psi::detci::sigma(psi::detci::stringwr**, psi::detci::stringwr**, psi::detci::CIvect&, psi::detci::CIvect&, double*, double*, int, int) ()
#6  0x0000000000e3ebb9 in psi::detci::sem_iter(psi::detci::CIvect&, psi::detci::stringwr**, psi::detci::stringwr**, double*, double, double, double, double, int, int, int, _IO_FILE*, int) ()
#7  0x0000000000e227dc in psi::detci::diag_h(psi::detci::stringwr**, psi::detci::stringwr**) ()
#8  0x0000000000e1e89e in psi::detci::detci(psi::Options&) ()
#9  0x0000000000748cbd in py_psi_detci() ()
#10 0x00000000007691fe in boost::python::objects::caller_py_function_impl<boost::python::detail::caller<double (*)(), boost::python::default_call_policies, boost::mpl::vector1<double> > >::operator()(_object*, _object*) ()
#11 0x00002aaaac8d5f71 in boost::python::objects::function::call(_object*, _object*) const ()
   from /user/scratchdelcatty/replica/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Boost/1.53.0-ictce-4.1.13-Python-2.7.3/lib/libboost_python.so.1.53.0
#12 0x00002aaaac8d5c85 in boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous namespace)::bind_return, void>::invoke(boost::detail::function::function_buffer&) () from /user/scratchdelcatty/replica/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Boost/1.53.0-ictce-4.1.13-Python-2.7.3/lib/libboost_python.so.1.53.0
#13 0x00002aaaac8e0cfb in boost::python::handle_exception_impl(boost::function0<void>) ()
   from /user/scratchdelcatty/replica/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Boost/1.53.0-ictce-4.1.13-Python-2.7.3/lib/libboost_python.so.1.53.0
#14 0x00002aaaac8d7837 in function_call ()
   from /user/scratchdelcatty/replica/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Boost/1.53.0-ictce-4.1.13-Python-2.7.3/lib/libboost_python.so.1.53.0
#15 0x00002aaaad9f64eb in PyObject_Call (func=0x1244480c, arg=0x80, kw=0x5b) at Objects/abstract.c:2529
#16 0x00002aaaadaccac6 in call_function (pp_stack=0x1244480c, oparg=128) at Python/ceval.c:4044
#17 0x00002aaaadac4e14 in PyEval_EvalFrameEx (f=0x1244480c, throwflag=128) at Python/ceval.c:2666
#18 0x00002aaaadacb36d in PyEval_EvalCodeEx (co=0x1244480c, globals=0x80, locals=0x5b, args=0xffffffffffffffff, argcount=0, kws=0x2d, kwcount=0, defs=0x0, defcount=0, closure=0x0)
    at Python/ceval.c:3253
#19 0x00002aaaada2fea0 in function_call (func=0x1244480c, arg=0x80, kw=0x5b) at Objects/funcobject.c:526
#20 0x00002aaaad9f64eb in PyObject_Call (func=0x1244480c, arg=0x80, kw=0x5b) at Objects/abstract.c:2529
#21 0x00002aaaadacc105 in ext_do_call (func=0x1244480c, pp_stack=0x80, flags=91, na=-1, nk=0) at Python/ceval.c:4334
#22 0x00002aaaadac49c1 in PyEval_EvalFrameEx (f=0x1244480c, throwflag=128) at Python/ceval.c:2705
#23 0x00002aaaadacb36d in PyEval_EvalCodeEx (co=0x1244480c, globals=0x80, locals=0x5b, args=0xffffffffffffffff, argcount=0, kws=0x2d, kwcount=0, defs=0x0, defcount=0, closure=0x0)
    at Python/ceval.c:3253
#24 0x00002aaaadaccde2 in call_function (pp_stack=0x1244480c, oparg=128) at Python/ceval.c:4042
#25 0x00002aaaadac4e14 in PyEval_EvalFrameEx (f=0x1244480c, throwflag=128) at Python/ceval.c:2666
#26 0x00002aaaadacb36d in PyEval_EvalCodeEx (co=0x1244480c, globals=0x80, locals=0x5b, args=0xffffffffffffffff, argcount=0, kws=0x2d, kwcount=0, defs=0x0, defcount=0, closure=0x0)
    at Python/ceval.c:3253
#27 0x00002aaaadacac29 in PyEval_EvalCode (co=0x1244480c, globals=0x80, locals=0x5b) at Python/ceval.c:667
#28 0x00002aaaadaff833 in PyRun_StringFlags (str=0x1244480c "[", start=128, globals=0x5b, locals=0xffffffffffffffff, flags=0x0) at Python/pythonrun.c:1316
#29 0x00002aaaac8e5192 in boost::python::exec(boost::python::str, boost::python::api::object, boost::python::api::object) ()
   from /user/scratchdelcatty/replica/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Boost/1.53.0-ictce-4.1.13-Python-2.7.3/lib/libboost_python.so.1.53.0
#30 0x00000000007534aa in psi::Python::run(_IO_FILE*) ()
#31 0x000000000071dbb4 in main ()

wpoely86 · 2013-10-18T07:22:20Z

Yeah, this is with 99% certainty the same bug. Apply PSI-4.0b5-thread-pool.patch and everything should be fine.

I'm puzzled that you can trigger this bug so easily. The threads have to follow a certain pattern for it to trigger: the only way I could reproduce it as on a NFS share but I don't understand the correlation between NFS and the scheduler. Once loaded into memory, NFS should not make any difference.
Anyway, as far as I can see, this bug is already in PSI for a long long time.

boegel · 2013-10-18T14:25:20Z

@wpoely86: OK, fixed in #471

wpoely86 added 12 commits October 5, 2013 14:48

Update Python-2.7.5 to the goalf-1.5.12-no-OFED toolchain

6079c25

Added PSI 4.0b5 with goalf-1.5.12-no-OFED

1a733e6

Added thread pool patch to PSI-4.0b5-goalf-1.5.12-no-OFED

b0dd763

Added patch to PSI for new-plugin

1824b9e

I've added two env vars to the configure script: one for the install directory for the source and one for the install directory of the objects.

PSI: reorder new-plugin patch and added comments

e679d69

Updated Boost to 1.53 with ictce-5.5.0 and python-2.7.5

7b62e19

Update PSI to 4.0b5 with ictce-5.5.0 and MPI

c3e094c

PSI-4.0b5-ictce-5.5.0: Added fix for potential thread bug

ed89334

A variable is changed without proper mutex locking.

Added patch to PSI for new-plugin

36050bb

I've added two env vars to the configure script: one for the install directory for the source and one for the install directory of the objects.

PSI: second thread pool patch

40dab5d

Hang should be fixed now.

PSI update to 4.0b5 with ictce-5.5.0 multithreaded

b280477

PSI-4.0b5-ictce-5.5.0: use correct version of patch

e032764

wpoely86 mentioned this pull request Oct 6, 2013

Psi 4.0b5 with ictce-5.5.0 and MPI #434

Closed

boegel and others added 9 commits October 10, 2013 17:58

drop incorrect patch file spec

4775e5e

PSI: add MPI patch to 4.0b5

3f0b55d

align PSI easyconfigs, correct filename of -mt PSI goalf easyconfig, …

d86b963

…add mpi PSI goalf easyconfig

complete mpi patch file for PSI-4.0b5

c91f480

fix typo in patch file

fc17d5f

really fix typo in patch file

ae62f61

remove respecifying default max_fail_ratio of 0.5 in NWChem easyconfig

e1b2780

PSI: avoid duplicate patch

a3f01e8

boegel added a commit that referenced this pull request Oct 11, 2013

Merge pull request #457 from wpoely86/psi

8a31b2d

Psi 4.0b5

boegel merged commit 8a31b2d into easybuilders:develop Oct 11, 2013

wpoely86 deleted the psi branch October 13, 2013 14:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Psi 4.0b5#457

Psi 4.0b5#457
boegel merged 21 commits intoeasybuilders:developfrom
wpoely86:psi

wpoely86 commented Oct 6, 2013

Uh oh!

wpoely86 commented Oct 11, 2013

Uh oh!

boegel commented Oct 11, 2013

Uh oh!

wpoely86 commented Oct 13, 2013

Uh oh!

boegel commented Oct 18, 2013

Uh oh!

wpoely86 commented Oct 18, 2013

Uh oh!

boegel commented Oct 18, 2013

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

wpoely86 commented Oct 6, 2013

Uh oh!

wpoely86 commented Oct 11, 2013

Uh oh!

boegel commented Oct 11, 2013

Uh oh!

wpoely86 commented Oct 13, 2013

Uh oh!

boegel commented Oct 18, 2013

Uh oh!

wpoely86 commented Oct 18, 2013

Uh oh!

boegel commented Oct 18, 2013

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants