@gribozavr on a general note, the idea is to replace the run_cmd and run_cmd_qa with RunAsyncLoopLog.run (or subclass thereof for qa) from vsc-base. the slowness that you encounter is fixed in https://github.com/hpcugent/vsc-base/blob/master/lib/vsc/utils/run.py#L444. the qa code in vsc-base/run is not finished however.

I discussed this a bit with Stijn, and it seems the sleeps are indeed a bit of overkill.

However, running unit tests on this branch revelas an issue. A simple test like the ones below fails with the sleeps removed...

(out, ec) = run_cmd_qa("echo question", {"question":"answer"})
self.assertEqual(out, "question\n")

So, simply removing the sleeps seems to break run_cmd_qa?

can you run this with debug and check what goes wrong? and try with recv_some(...,t=1)

Rerunning the test with the initial sleep that was removed in run_cmd_qa added again and removing the sleep in the loop fixes the problem, i.e. the following patch on top of this PR:

diff --git a/easybuild/tools/filetools.py b/easybuild/tools/filetools.py
index 910e10e..0663034 100644
--- a/easybuild/tools/filetools.py
+++ b/easybuild/tools/filetools.py
@@ -495,6 +495,7 @@ def run_cmd_qa(cmd, qa, no_qa=None, log_ok=True, log_all=False, simple=False, re
     except OSError, err:
         _log.error("run_cmd_qa init cmd %s failed:%s" % (cmd, err))

+    time.sleep(0.1)
     ec = p.poll()
     stdoutErr = ''
     oldLenOut = -1
@@ -561,7 +562,7 @@ def run_cmd_qa(cmd, qa, no_qa=None, log_ok=True, log_all=False, simple=False, re
                                                                                     ))

         # This sleep should not be needed.
-        time.sleep(1)
+        #time.sleep(1)
         ec = p.poll()

     # Process stopped. Read all remaining data

The run_cmd_qa testcase was rubbish, since it never wait for input as an actual interactive command would, so I corrected the test to:

(out, ec) = ft.run_cmd_qa("echo question; read x; echo $x", {"question": "answer"})
self.assertEqual(out, "question\nanswer\n")

The patch mentioned above breaks this test, the sleep in the loop is quite critical to run_cmd_qa to be able to ignore non-questions (see also https://github.com/hpcugent/easybuild-framework/pull/599/files#r3874464).

PR gribozavr#5 enhances the unit tests to make sure the proposed changes are OK.

Whether they'll cause any issues should become clear when a full regtest is run.

@boegel I think I now understand why that sleep was needed.

OK, this seems ready to merge in.