Skip to content

Reduce test boilerplate#13746

Merged
keszybz merged 3 commits intosystemd:masterfrom
keszybz:tests-reduce-boilerplate
Oct 8, 2019
Merged

Reduce test boilerplate#13746
keszybz merged 3 commits intosystemd:masterfrom
keszybz:tests-reduce-boilerplate

Conversation

@keszybz
Copy link
Member

@keszybz keszybz commented Oct 8, 2019

No description provided.

I *think* this was originally added to make it easier to see what was happening
in tests. Later we added the functionality to print the journal on failure, so
this redirection has stopped being useful.

In systemd#13719 (comment)
@filbranden shows that grep tries to write to stdout and fails. In general,
we should not assume that writing to the console it always possible. We have
special code to handle this in pid1 after all:

99    19:22:10.731965 fstat(1,  <unfinished ...>
99    19:22:10.731993 <... fstat resumed>{st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0), ...}) = 0
99    19:22:10.732070 write(1, "ExecStartPost={ path=/bin/echo ; argv[]=/bin/echo ${4_four_ex} ; ignore_errors=no ; start_time=[Mon 2019-10-07 19:22:10 PDT] ; stop_time=[Mon 209-10-07 19:22:10 PDT] ; pid=97 ; code=exited ; status=0 }\n", 203) = -1 EIO (Input/output error)
99    19:22:10.732174 write(2, "grep: ", 6) = -1 EIO (Input/output error)
99    19:22:10.732226 write(2, "write error", 11) = -1 EIO (Input/output error)
99    19:22:10.732263 write(2, ": Input/output error", 20) = -1 EIO (Input/output error)
99    19:22:10.732298 write(2, "\n", 1 <unfinished ...>
99    19:22:10.732325 <... write resumed>) = -1 EIO (Input/output error)
99    19:22:10.732349 exit_group(2)     = ?
99    19:22:10.732424 +++ exited with 2 +++

Removing the redirection should make the tests less flakey.

Replaces systemd#13719.

While at it, also drop NotifyAccess=all. I think it was added purposefully in
TEST-20-MAINPIDGAMES, and then cargo culted to newer tests.
Many tests were also masking systemd-machined.service. But machined
should only start when activated, so having it not masked shouldn't be
noticable. TEST-25-IMPORT needs it.
@mrc0mmand
Copy link
Member

TEST-31-DEVICE-ENUMERATION started consistently failing in CentOS CI:

Oct 08 09:30:49 systemd-testsuite systemd[462]: systemd-logind.service: Executing: /sbin/modprobe -abq drm
Oct 08 09:30:49 systemd-testsuite systemd[1]: dev-cdrom.device: Changed dead -> plugged
Oct 08 09:30:49 systemd-testsuite bash[466]: Oct 08 08:30:49 systemd-testsuite bash[466]: + grep -e '\.device: Changed plugged -> dead'
Oct 08 09:30:49 systemd-testsuite bash[466]: + exit 1
Oct 08 09:30:49 systemd-testsuite systemd[1]: Received SIGCHLD from PID 466 (bash).
Oct 08 09:30:49 systemd-testsuite systemd[1]: Child 466 (bash) died (code=exited, status=1/FAILURE)
Oct 08 09:30:49 systemd-testsuite systemd[1]: testsuite.service: Child 466 belongs to testsuite.service.
Oct 08 09:30:49 systemd-testsuite systemd[1]: testsuite.service: Main process exited, code=exited, status=1/FAILURE
Oct 08 09:30:49 systemd-testsuite systemd[1]: testsuite.service: Failed with result 'exit-code'.
Oct 08 09:30:49 systemd-testsuite systemd[1]: testsuite.service: Changed start -> failed
Oct 08 09:30:49 systemd-testsuite systemd[1]: testsuite.service: Job 195 testsuite.service/start finished, result=failed
Oct 08 09:30:49 systemd-testsuite systemd[1]: Failed to start Testsuite service.
Oct 08 09:30:49 systemd-testsuite systemd[1]: testsuite.service: Unit entered failed state.
Oct 08 09:30:49 systemd-testsuite systemd[1]: Child 462 (modprobe) died (code=exited, status=1/FAILURE)

Full journal from the CentOS 7 job

Journal from the KVM job should be available soon

@mrc0mmand
Copy link
Member

Ah, I see, it now matches the grep command itself, as it's logged into journal as well, thanks to set -x, ugh.

systemd#13746 (comment):
> [grep] now matches the grep command itself, as it's logged into journal as well, thanks to set -x.

Also, use journalctl --grep and -t to make things a bit quicker.
@mrc0mmand
Copy link
Member

I guess we can't use --grep without tweaking the CI configuration first:

Oct 08 11:33:59 systemd-testsuite bash[462]: Compiled without pattern matching support
Oct 08 11:33:58 systemd-testsuite systemd[459]: Not remounting /run/systemd/unit-root/var/tmp blacklisted by /run/systemd/unit-root/var/tmp, called for /run/systemd/unit-root
Oct 08 11:33:59 systemd-testsuite bash[462]: + exit 1

@keszybz
Copy link
Member Author

keszybz commented Oct 8, 2019

I guess we can't use --grep without tweaking the CI configuration first:

Can we do that? I think it'd be useful to dogfood.

mrc0mmand added a commit to systemd/systemd-centos-ci that referenced this pull request Oct 8, 2019
We do this already in the Arch Linux job, so let's keep the environment
consistent across all upstream jobs

See: systemd/systemd#13746 (comment)
@mrc0mmand
Copy link
Member

I guess we can't use --grep without tweaking the CI configuration first:

Can we do that? I think it'd be useful to dogfood.

I noticed that we already do that in the Arch jobs, so enabling it for the CentOS 7 job makes sense. I pushed the fix and re-triggered the failing job.

@keszybz
Copy link
Member Author

keszybz commented Oct 8, 2019

I'll merge this. After all, it's just tests and the change is rather trivial. This should fix TEST-40 flakiness.

@keszybz keszybz merged commit 69d0eb4 into systemd:master Oct 8, 2019
@filbranden
Copy link
Contributor

Thanks for taking care of this @keszybz !

Let's now see if this will take care of most of the flakiness...

@keszybz keszybz deleted the tests-reduce-boilerplate branch October 9, 2019 08:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

3 participants