@damianam If my memory serves me right, you brought this up at some point... Please review?

I remember hitting an issue with -r, but I don't remember the specifics of that use case. However, if I understand your PR, I am not sure it is fixing the root cause.

eb something.eb -Dr will work as expected (ie: enable the robot, do a dry-run)

eb something.eb -rD will not work as expected (ie: it will exit with an error, if, and just if, D doesn't exist)

In my opinion the order of the options shouldn't affect the behaviour of EB. It looks to me like we should make the parsing of options a bit smarter when grouping them together in a single switch.

@damianam the problem is how to distinguish between the valid use case of giving --robot an argument D (indicating it should go hunting in the D directory as well for easyconfigs), and the unintended use of -rD to go a --robot --dry-run-short.

The only way to distinguish is to actually check whether D is an existing directory imho...

When D is not there, I prefer issuing a clear error for the use of -rD, rather than silently doing a --robot --dry-run-single if D does not exist, to train the user to be aware of the fact that --robot takes an optional argument.

Note that the problem is also wider than e.g. -rD; for example "eb --robot foo.eb".
Although the intention is clear there, as it is now, --robot will consume the foo.eb as an argument, and you'll get a (confusing) error that no easyconfigs were specified...

To me it looks more natural to have a space between the option and the argument (ie: I'd type -r some_path, not -rsomepath), so things like -rD should never be a problem with that approach. That slightly breaks compatibility though, even though it might be for a good reason.

Then, in the more general case of --robot foo, we could have a smarter parser that checks what foo is. Does it look like a path, an eb file, or as another option? If it ends in .eb doesn't seem like a path, just the same as when it starts with -

@damianam the space is irrelevant, you still have the same problem where eb -r foo.eb may not do what you expect it to (i.e. error out because no easyconfig file is specified because the foo.eb gets eaten by -r)

We currently don't have the requirement that easyconfig files need to add in .eb; using eb test where test is an easyconfig file will work just fine, no reason to break that imho

Also, not that -r -D will already work as expected, since -D is a known option. And something silly like -r -X will not work, since -X is not a known option (the -X does not get eaten by -r because it starts with a -)

We currently don't have the requirement that easyconfig files need to add in .eb; using eb test where test is an easyconfig file will work just fine, no reason to break that imho

Well, this looks like a good enough reason ;-P. Looking from the other side, is there a reason to don't require easyconfigs to end in .eb?

There's no reason to require the .eb extension, we have other ways of checking whether a specified file is actually an easyconfig file (by looking at the contents). We're not MS Windows. ;-)

This was sort of a (potentially unintentional at first) design choice, and I see no reason to diverge from that now (although in practice, most people probably use the .eb extension all the time).

Sure, but with that "restriction" in place you don't even have to read the contents for this kind of thing, which will make it a bit more efficient.

Besides that, there are tons of examples in UNIX of similar things. The linux linkers just check for .so files for instance. A library called libsomething.wop won't work. And you can have a shell script that ends in .py, but that doesn't make it a good idea. And fortran compilers behave differently if the file ends in .f90 or in .F90.

My point is that even if it is technically possible to accept any kind of extension (or lack of), that doesn't necessarily make it a good idea. #extensionsmatter ;-P. Putting that restriction in place will allow us to fix this issue quickly and efficiently, and will even enforce good practices. I've had people pointing me to a directory with a bunch of files without extensions, and I had to manually check which ones were easyconfigs and which ones weren't. That is bad practice I think, enforcing that everyone uses something standard that makes sense is not a bad thing IMO. Moreover, easybuild has in place way more stricter conventions for easyconfig files already (<SW>-<version>-<toolchain>-<toolchain_version>-<suffix>?). Without them --robot can't find anything. Is it enforcing .eb extensions really that big of a deal?

In my opinion, activating the robot and settings the paths for the robot should be two different options. I've personally never used -r /some/dir.

@damianam I agree that using a .eb easyconfig files is a good idea (I also do it consistently), but up until now EasyBuild hasn't enforced this. Changing that would break backward compatibility, so we could only strictly enforce it in EasyBuild v4.0...
If people feel strongly about it though, we can already issue a big fat warning or something for easyconfig files that not have a .eb extension, or even require to use a configure flag like --skip-eb-extension-check (which could then be removed in EB v4.0); the latter does not break backward compatibility in my book as long as the error that would arise by default clearly mentions --skip-eb-extension-check (it should pop up rarely anyway, I hope).

However, that's an orthogonal discussion to what is being done in this PR, imho, since that would not fix the problem where people use eb -drf foo.eb where they should be using eb -dfr foo.eb; otherwise the f intended to enable --force gets still eaten by --robot without checking whether it's an existing directory...

@wpoely86 w.r.t. --robot accepting an optional argument: I use that quite often; there are good reasons to keep doing this together with also having --robot-paths, see http://easybuild.readthedocs.io/en/latest/Using_the_EasyBuild_command_line.html#controlling-robot-search-path

I was hoping that this would be a trivially accepted change, not to trigger a giant discussion...

Can you please let me know whether or not the actual change being made here is a good idea or not?
Related stuff (like the .eb requirement) can/should be discussed in a separate issue, ideally...

@boegel I am ok with a big fat warning, I think it makes sense.

I don't think it is an orthogonal discussion. Partly yes, but it is relevant to this one because it might help to solve the original issue in an easy way. What I would propose is:

• -frD kind of options, with the robot in the middle: The robot does not check for any path in the command line
• -fDr kind of options, with the robot at the end: The robot check for what comes later:
  • -fDr without anything after it: No path
  • -fDr /path: Add the path to the robot
  • -fDr file.eb: File detected: no robot path added
  • -fr -D: Option detected: no robot path added
• --robot: same as -fDr

With that we avoid this bug and break backwards compatibility in a very mild (and desirable?) way. Otherwise, if I understand the current fix right, you can no longer activate the robot without passing it an existing directory. That's not an issue for us, but some people might want to enable it without passing any extra directory.

@damianam I see your point about allowing -frD, since it should be safe to assume that no typo was made in that case, the intention is clear. That needs to be dealt with in the GeneralOption parser in vsc-base though, not here (and I suspect this is not going to be trivial)...

The proposed change is not supposed to break the use of just --robot or -r without providing a path, that would be too big of a change breaking backwards compatibility, and I don't think it does, but I'll make very sure it indeed doesn't.

moving this to v3.2.0 milestone, not worth blocking the v3.1.0 release over this

@damianam let's discuss this over a beer or two during the EUM ;-)

Picking up on this again to try and get this merged for EasyBuild v4.0...

@damianam I've made additional changes that should cover your suggestions:

• a single argument to --robot or -r that ends with .eb (or .yeb) is now no longer consumed by --robot (unless it's an existing directory)
• using -rD is now interpreted as -r -D by unrolling combos of single-letter arguments before passing them to the option parser