Skip to content

[ictce/5.3.0] gputools/0.28 for R/3.0.2 (REVIEW)#546

Merged
boegel merged 11 commits intoeasybuilders:developfrom
fgeorgatos:contrib_Rgputools
Jun 15, 2016
Merged

[ictce/5.3.0] gputools/0.28 for R/3.0.2 (REVIEW)#546
boegel merged 11 commits intoeasybuilders:developfrom
fgeorgatos:contrib_Rgputools

Conversation

@fgeorgatos
Copy link
Copy Markdown
Contributor

@fgeorgatos fgeorgatos commented Nov 19, 2013

Hi,
this one looks surprisingly simple but it proved to be very tedious business (spin-off of #282); reason is you MUST:

first, be careful with the exact git commit ID that corresponds to v0.28, this is not exactly a 1-1 relationship :-P

  1. untar/retar manually the github tarball ; reason: incompatibility github vs R-untar, see easyblocks thread:
    Provide a workaround for tar/pax dogfight in R packages easybuild-easyblocks#313
  2. you still need to supply full path from within R, eg. dyn.load("/hello/big/world"), while R_LIB does get populated
  3. you need do this on GPU-equiped node, otherwise you receive an obscure message about missing libcuda.so.1
  4. e5cb024 works OK, while next commits (all called v0.28!) break down somewhere along the commit sequence.
    Let's focus on the e5cb024 case for now, that should be the one to get out working first.
    I need your help w. arguing the possible workarounds, I have a specified manual reproducible case at this point.

Replacing #282. Using a test case here may be recommended.

Signed-off-by: Fotis Georgatos <[email protected]>
@hpcugentbot
Copy link
Copy Markdown

Automatic reply from Jenkins: Can I test this?

@fgeorgatos
Copy link
Copy Markdown
Contributor Author

forgot to mention that, this issue relies on R/3.0.2-ictce-5.3.0-bare which is by coincidence provided via #549 !

@fgeorgatos
Copy link
Copy Markdown
Contributor Author

nope: mea culpa, #549 has goolf only, R/3.0.2-ictce-5.3.0-bare is part of v1.9 anyhow - sorry for the noise

@boegel
Copy link
Copy Markdown
Member

boegel commented Dec 12, 2013

Jenkins: ok to test

@boegel
Copy link
Copy Markdown
Member

boegel commented Dec 29, 2013

@fgeorgatos: This needs a couple of kinks worked out.

0 & 4) The versioning makes little sense, you're supposed to be able to order versions... How about this?

version = '0.28'
versionsuffix = '-<commit>'
  1. we need a proper fix for this, e.g. add an option to the generic easyblock to specify that the tarball should be unpacked to install it
  2. looks very fishy
  3. this seems to indicate that the libcuda.so supplied by goolfc isn't found? that's bad, needs to be fixed

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this definitely has to go

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aha!! #1150 has the fix for this ;-) thanks, @pescobar !

addendum: the potential fix is now proposed instead inside the R easyblock:
easybuilders/easybuild-easyblocks#495

@boegel
Copy link
Copy Markdown
Member

boegel commented Feb 14, 2014

@fgeorgatos: this needs more work before it can go in, exactly because it's so tedious as you describe...

@boegel boegel modified the milestones: v1.12, v1.11 Mar 14, 2014
@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 17, 2014

@pforai, @wpoely86: this needs testing on a GPU system, up for it?

@fgeorgatos fgeorgatos changed the title add Rgputools easyconfigs Contrib Rgputools easyconfigs Apr 29, 2014
@fgeorgatos
Copy link
Copy Markdown
Contributor Author

Test report by @fgeorgatos
FAILED
Build succeeded for 0 out of 1
Linux debian 6.0.10, Intel(R) Xeon(R) CPU L5640 @ 2.27GHz, Python 2.6.6
See https://gist.github.com/455d55b8587f02d3a313 for a full test report.

@fgeorgatos fgeorgatos changed the title Contrib Rgputools easyconfigs [ictce/5.3.0] Rgputools/0.28 for R/3.0.2 (WIP) Jan 20, 2015
@boegel boegel modified the milestone: v1.15 Jun 24, 2015
@boegel
Copy link
Copy Markdown
Member

boegel commented Jun 10, 2016

@fgeorgatos: ping on this?

I'm cleaning up the oldest PRs, in an effort to reduce the pile of open PRs.

Is this still relevant? Any idea of the amount of extra effort required to get it ready for merge?

@fgeorgatos
Copy link
Copy Markdown
Contributor Author

I'd expect this to be mergeable, esp. if for ictce/5.5.0; where could we see the current build status?!

@boegel
Copy link
Copy Markdown
Member

boegel commented Jun 14, 2016

@fgeorgatos please take a look at fgeorgatos#37

@fgeorgatos I'm still experiencing problems when testing this:

** R
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  unable to load shared object '/user/scratchdelcatty/gent/gvo000/gvo00002/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Rgputools/0.28-ictce-5.3.0-e5cb024-R-3.0.2/gputools/libs/gputools.so':
  libcuda.so.1: cannot open shared object file: No such file or directory
Error: loading failed
Execution halted
ERROR: loading failed

Where is libcuda.so.1 supposed to come from?

@boegel boegel added this to the v2.9.0 milestone Jun 14, 2016
@gppezzi
Copy link
Copy Markdown
Contributor

gppezzi commented Jun 15, 2016

libcuda.so should be part of the nvidia driver

style fixes in Rgputools easyconfig, rename to 'gputools'
@fgeorgatos
Copy link
Copy Markdown
Contributor Author

fgeorgatos commented Jun 15, 2016

@boegel:

You really exercise my memory when you ask me something about a build, 3 years later :-p
Luckily, although I did not remember, I had already described well the build process in the issue, see:

3) you need do this on GPU-equipped node, otherwise you receive an obscure message about missing libcuda.so.1

@boegel
Copy link
Copy Markdown
Member

boegel commented Jun 15, 2016

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in this PR)
centos6 - Linux centos 6.5, AMD Opteron(tm) Processor 6136, Python 2.6.6
See https://gist.github.com/0406db964d7253de0932c8d17d191528 for a full test report.

@boegel
Copy link
Copy Markdown
Member

boegel commented Jun 15, 2016

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in this PR)
centos6 - Linux centos 6.5, AMD Opteron(tm) Processor 6136, Python 2.6.6
See https://gist.github.com/53044489eaf8d2803f7c7e8ef34c9f41 for a full test report.

@boegel boegel changed the title [ictce/5.3.0] Rgputools/0.28 for R/3.0.2 (WIP) [ictce/5.3.0] gputools/0.28 for R/3.0.2 (REVIEW) Jun 15, 2016
@boegel
Copy link
Copy Markdown
Member

boegel commented Jun 15, 2016

Green lights everywhere, going in, thanks @fgeorgatos!

@boegel boegel merged commit 1990f3b into easybuilders:develop Jun 15, 2016
@fgeorgatos fgeorgatos deleted the contrib_Rgputools branch June 16, 2016 02:40
@fgeorgatos
Copy link
Copy Markdown
Contributor Author

fyi. bug #1 made it's appearance again on my side ; here's how it manifests itself:

make: *** [rinterface.o] Error 1
ERROR: compilation failed for package ‘gputools’
 (at easybuild/tools/run.py:397 in parse_cmd_output)
== 2016-06-16 12:59:43,774 easyblock.py:2362 WARNING build failed (first 300 chars): cmd "R CMD INSTALL e5cb024.tar.gz   --library=/home/fgeorgatos
Warning in untar2(tarfile, files, list, exdir, restore_times) :
  skipping pax global extended headers
* installi
== 2016-06-16 12:59:43,775 easyblock.py:270 INFO Closing log for application name gputools version 0.28

I have my workaround, as described above, however I feel obliged to report...

@fgeorgatos
Copy link
Copy Markdown
Contributor Author

btw.
if people face trouble in the future, I recommend a simple test-case in the lines of:

library(gputools) 

set.seed(5446) 
p <- 20 
X <- matrix(rnorm(2^p),ncol = 2^(p/2)) 

system.time(d <- dist(X)) 
system.time(d <- gpuDist(X)) 

ref. http://dept.stat.lsa.umich.edu/~bobyuen/gpuwiki/

@boegel
Copy link
Copy Markdown
Member

boegel commented Jun 17, 2016

@fgeorgatos can you provide a more complete error/log for the unpack issue? what's the exact problem with the unpack? It seems we have something related in ConfigureMake that may provide a workaround.

@fgeorgatos
Copy link
Copy Markdown
Contributor Author

@boegel: well, the case had been described here:
easybuilders/easybuild-easyblocks#313
IMHO, it is either incomplete implementation on the python untar2 side or, overshooting from github

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants