Test report by @migueldiascosta
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in this PR)
sms - Linux centos linux 7.6.1810, AMD EPYC 7601 32-Core Processor, Python 2.7.5
See https://gist.github.com/4662d3b27915c5527127e6bb2d8b40f7 for a full test report.

Going in, thanks @jmbeuken!

@jmbeuken Thanks for being thorough on this!

Can you clarify what the problem is exactly when AtomPAW 4.1.0.6 is used as a dependency for ABINIT 8.10.3?
Are there specific error messages that pop up than we could mention here, so people can quickly find this solution to the problem?

With the AtomPAW 4.1.0.6 version, some tests of the testsuite fail because the tests were badly "built" (it was not visible with the version AtomPAW 4.1.0.5 which built pseudopotentials of "bad quality")

We can not consider this a "bug"; certainly the tests do not pass but ABINIT and AtomPAW are correct.

@jmbeuken Would it make sense to run the test suite as a part of the installation process performed by EasyBuild?

It could be....
But the problem we had with this ABINIT 8.10.3/AtomPAW 4.1.0.6 configuration, is purely my fault... :-(
We have a testfarm that tests ABINIT with about twenty builders....
Unfortunately, I didn't have time to deploy AtomPAW 4.1.0.6 on all builders before doing the PR on Easybuild :-(
As it worked with AtomPAW 4.1.0.5, I wrongly deduced that it would work with version 4.10.6 ( modification of the fourth digit! ) and so, I didn't start the tests.... :-(
It was the first time I contributed to Easybuild, and I was more attentive to configs than to numerical test results : -
So, from today, no more PR on Easybuild without testfarm control ;-)
For your information, the testsuite ( > 600 tests ) take 30mins (8 cores) on a machine equipped with 2x Xeon Silver 4110 [email protected]

@jmbeuken actually, it may be my fault :)

I think it was me that first added runtest = 'check', to the 8.4.4 easyconfig, and it has been carried forward since, but now that you mention it and I have a better look, it should be runtest = 'tests_in'? (runtest takes the make target for running the tests)

I don't think 30min is a problem, installations with EasyBuild are typically run in the background or as a batch job, but if it is people can always comment runtest (or change to tests_fast?)

@migueldiascosta
'make check' is a subset of tests (fast tests) and used internally to check the 'make distcheck'
'make tests_in' and 'make tests_fast' are no longer supported / maintained ( no rule )

the testsuite has been completely rewritten in python since 8.2.2 ( tests/runtest.py )

The main problem is the interpretation of the final results...
Depending on the OS, the version of the compilers, the algebraic libraries,..., some tests may fail ( and it is not serious... -> compiler bug, instability of the test,...)
When it runs on the testfarm, these tests are automatically excluded so as not to warn the developers

@migueldiascosta
Currently, 'make check' is executed? I don't see this command and or the results in the travis log...

@jmbeuken it's not part of the uploaded test report, but you should be able to find it in the easybuild log in the installation directory (example at https://gist.github.com/migueldiascosta/ba3267a62b85cb8d5fddf89a5e9bbc81)