@boegel can you please enable coveralls from marketplace https://github.com/marketplace/coveralls
This PR will try to send report to coveralls from travis automatically. I dont know if coverage is setup properly. Right now I only ran it for python module test.framework.suite

@shahzebsiddiqui I think that coveralls is now available for the easybuild-framework repo.

We'll need to fix the issue where coverage report -m is tripping over the non-existing generic/foobar.py easyblock that is injected at runtime by one of the tests though...

I guess coverage is picking up the use of that files, and expects it to still be there when collecting the report. Can we make it ignore missing files?

@boegel I cant seem to find where generic/foobar.py easyblock is stored relative to Travis build. Yes there is a way to ignore files during coverage report that can be configured in .coveragerc

See https://coverage.readthedocs.io/en/coverage-5.0.3/source.html

I think something like

[run]
omit = 
  - <path>/generic/foobar.py

coverage can even exclude by lines. Most useful when ignoring methods that you know are not to be tested or part of development that you can blacklist for time being. See https://coverage.readthedocs.io/en/coverage-5.0.3/excluding.html

I can fix it but dont know the full path to easyblock relative to Travis build directory.

@shahzebsiddiqui The issue is that this file is only created when a specific test is running, and then cleaned up afterwards (it's only stored for a short while in a temporary location).

So, the path changes on every run (to ensure a unique temporary location).

It may be possible to tweak the test a bit to avoid that coveralls thinks the file should still be there though...

@boegel well i am going to try one more thing clearly from the log it looks like a coverall issue.

Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.7/bin/coveralls", line 8, in <module>
    sys.exit(wear())
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/coveralls/__init__.py", line 78, in wear
    from coveralls.control import coveralls
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/coveralls/control.py", line 2, in <module>
    from coveralls.report import CoverallsReporter
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/coveralls/report.py", line 6, in <module>
    from coverage.report import Reporter
ImportError: cannot import name 'Reporter'

I found a similar issue reported so i will try to remove python-coveralls and just use coveralls to test if it works in travis.
z4r/python-coveralls#74

@boegel all Travis builds were complete, only one build failed and it is okay since it was Python 2.6 unable to install coveralls. Anyhow i think it is a good step to get coveralls. Once you merge this, all incoming PRs that pass Travis build will get uploaded to coveralls and you can view the coverage report. As of now the develop branch is at 72.642%

See coverage report @ https://coveralls.io/github/easybuilders/easybuild-framework

Thanks a lot for looking into this @shahzebsiddiqui!