Why is a dedicated (vendor-specific) ioctl required? Isn't the generic gcm(aes) ioctl working for you?

This is because the generic gcm(aes) ioctl is defined with higher priority(300) than the HW accelerated one(200). And the HW accelerated one has the same generic name. So there is no way to select the vendor specific one unless I'm mistaken ?

First of all, I would check why this happens - in theory, HW-accelerated version should have a higher priority. Is there a specific reason why the Linux kernel developers defined the priorities this way?

Besides this, Linux kernel provides an interface that allows user space to change crypto algorithms' priority dynamically (at runtime). Linux kernel needs to be compiled with CONFIG_CRYPTO_USER=y/m. You could try using "crconf" - the Linux crypto layer configuration tool (https://sourceforge.net/projects/crconf) to change priorities, though I am not sure it's still in sync with the kernel (IIRC it uses some of kernel's internal headers).

I don't know why they set the priority lower than the software version. Maybe because the HW-accelerated one is slower ? It is still useful as it frees up cycles for the CPU.

IMO cryptodev is not the right place to implement policies in general, in this case setting / altering crypto algorithms' priorities.

You could ask on the linux-crypto mailing list or send a patch / RFC. This should clarify why priorities are set up as they are today and whether a change is acceptable.
If this is not viable, then use the 2nd option - a user-space tool to configure priorities at runtime.

@horiag @victor-dumas We'll have to see what's the right approach here but it seems Linux is already establishing policies - for good or worse- with its algorithm priorities

@horia, can you investigate the usability status of this third-party tool so we can direct people to it?
@victor-dumas please ask linux-crypto mailinglist to comment on this priority issue so we know their position in the future.

I understand it's not exactly zero work on top of a three line patch but with the right answers we'll be in better position to justify a direction. Thank you all.