Tiffany Rea (22a1b7c3) at 17 Mar 21:19
Add githooks for pre-push test run
Tiffany Rea (00ca595e) at 17 Mar 21:16
Related to https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/work_items/4271
Example push after installation
$ git push origin head --force
Running full unit test suite before push...
Running full Go test suite...
go: downloading github.com/stretchr/testify v1.11.0
=== RUN TestGetUser
=== PAUSE TestGetUser
=== RUN TestGetUser_ConnectionFailure
=== PAUSE TestGetUser_ConnectionFailure
=== RUN TestGetUsers
=== PAUSE TestGetUsers
=== RUN TestNewUserService
=== PAUSE TestNewUserService
=== RUN TestNewUserService_TrimsTrailingSlash
=== PAUSE TestNewUserService_TrimsTrailingSlash
=== CONT TestGetUser
=== RUN TestGetUser/returns_user_when_found
=== PAUSE TestGetUser/returns_user_when_found
=== RUN TestGetUser/returns_error_when_user_not_found
=== PAUSE TestGetUser/returns_error_when_user_not_found
=== RUN TestGetUser/returns_error_on_server_error
=== PAUSE TestGetUser/returns_error_on_server_error
=== RUN TestGetUser/returns_error_on_invalid_JSON_response
=== PAUSE TestGetUser/returns_error_on_invalid_JSON_response
=== CONT TestGetUser/returns_user_when_found
=== CONT TestNewUserService
=== RUN TestNewUserService/uses_default_HTTP_client_when_nil_provided
=== CONT TestGetUser/returns_error_on_invalid_JSON_response
=== CONT TestGetUsers
=== RUN TestGetUsers/returns_all_users
=== PAUSE TestGetUsers/returns_all_users
=== RUN TestGetUsers/returns_error_on_server_error
=== PAUSE TestGetUsers/returns_error_on_server_error
=== PAUSE TestNewUserService/uses_default_HTTP_client_when_nil_provided
=== CONT TestGetUser/returns_error_on_server_error
=== RUN TestNewUserService/uses_provided_HTTP_client
=== PAUSE TestNewUserService/uses_provided_HTTP_client
=== CONT TestNewUserService/uses_default_HTTP_client_when_nil_provided
=== CONT TestNewUserService/uses_provided_HTTP_client
--- PASS: TestNewUserService (0.00s)
--- PASS: TestNewUserService/uses_default_HTTP_client_when_nil_provided (0.00s)
--- PASS: TestNewUserService/uses_provided_HTTP_client (0.00s)
=== CONT TestGetUser/returns_error_when_user_not_found
=== RUN TestGetUsers/returns_error_on_invalid_JSON_response
=== CONT TestGetUser_ConnectionFailure
=== CONT TestNewUserService_TrimsTrailingSlash
=== RUN TestNewUserService_TrimsTrailingSlash/trims_single_trailing_slash
=== PAUSE TestGetUsers/returns_error_on_invalid_JSON_response
=== CONT TestGetUsers/returns_all_users
=== PAUSE TestNewUserService_TrimsTrailingSlash/trims_single_trailing_slash
=== CONT TestGetUsers/returns_error_on_invalid_JSON_response
=== RUN TestNewUserService_TrimsTrailingSlash/trims_multiple_trailing_slashes
=== PAUSE TestNewUserService_TrimsTrailingSlash/trims_multiple_trailing_slashes
=== CONT TestGetUsers/returns_error_on_server_error
=== RUN TestNewUserService_TrimsTrailingSlash/leaves_URL_without_trailing_slash_unchanged
=== PAUSE TestNewUserService_TrimsTrailingSlash/leaves_URL_without_trailing_slash_unchanged
=== CONT TestNewUserService_TrimsTrailingSlash/trims_single_trailing_slash
=== CONT TestNewUserService_TrimsTrailingSlash/leaves_URL_without_trailing_slash_unchanged
=== CONT TestNewUserService_TrimsTrailingSlash/trims_multiple_trailing_slashes
--- PASS: TestNewUserService_TrimsTrailingSlash (0.00s)
--- PASS: TestNewUserService_TrimsTrailingSlash/trims_single_trailing_slash (0.00s)
--- PASS: TestNewUserService_TrimsTrailingSlash/trims_multiple_trailing_slashes (0.00s)
--- PASS: TestNewUserService_TrimsTrailingSlash/leaves_URL_without_trailing_slash_unchanged (0.00s)
--- PASS: TestGetUser (0.00s)
--- PASS: TestGetUser/returns_error_when_user_not_found (0.00s)
--- PASS: TestGetUser/returns_error_on_invalid_JSON_response (0.00s)
--- PASS: TestGetUser/returns_user_when_found (0.00s)
--- PASS: TestGetUser/returns_error_on_server_error (0.00s)
--- PASS: TestGetUsers (0.00s)
--- PASS: TestGetUsers/returns_error_on_invalid_JSON_response (0.00s)
--- PASS: TestGetUsers/returns_all_users (0.00s)
--- PASS: TestGetUsers/returns_error_on_server_error (0.00s)
--- PASS: TestGetUser_ConnectionFailure (0.00s)
PASS
ok gitlab.com/gitlab-org/quality/quality-engineering/modular-feature-testing/golang/example 0.341s
β All pre-push tests passed!
Enumerating objects: 8, done.
Counting objects: 100% (8/8), done.
Delta compression using up to 10 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 1.40 KiB | 1.40 MiB/s, done.
Total 6 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)
remote:
remote: View merge request for trea-run-unit-test-git-hook:
remote: https://gitlab.com/gitlab-org/quality/quality-engineering/modular-feature-testing/-/merge_requests/5
remote:
To gitlab.com:gitlab-org/quality/quality-engineering/modular-feature-testing.git
+ d743a95...22a1b7c head -> trea-run-unit-test-git-hook (forced update)Tiffany Rea (d743a952) at 17 Mar 21:15
Add githooks for pre-push test run
Tiffany Rea (c813804d) at 17 Mar 17:41
Add .mise.toml file and mise installation
Tiffany Rea (00ca595e) at 17 Mar 17:29
Add githooks for pre-commit and pre-push
Thanks @willmeek Great suggestions. I think I will put this MR on hold and wait for !2 (diffs) to merge first. There will be conflicts to resolve after anyways. So I will leave this in draft until then.
I'm newbie here too
Good idea. I can add that
I did try to resolve the merge conflict after !1 (merged) but I missed this one
Maybe, we should not quarantine these kinds of tests. WDYT?
This spec was blocking deployment so we had to move it out of smoke suite. It is entirely up to product team to decide the fate of the test (quarantine, fix, unquarantine, delete it, or even replace it with test cases of lower levels testing type etc...).
we should always wait for the pipeline/job to finish before checking the output; that's the typical pattern
Agreed. This pattern is closely imitating end user's flows. I think the test is not fundamentally wrong. It surfaced a couple of the product's functionalities that can be improved, IMHO: The job has reached successful state but its runner became offline before final status is communicated/updated, and so the stop stuck in running indefinitely. Is there anything can be done to improve this? This can very be a real life scenario that can happen to customers.
Since this test is only caring if it is outputting secret data or not, why wait job completion? Just look at the output right?
Then we risk not knowing if reading from file variables could potentially fail the job somehow. End goal is still to have a green pipeline using these rules and config. And job completion is a good indication that job log is now complete and can be parsed. It's anti-pattern to have the test sits in a loop and keeps parsing the log until we find the text we want
Quarantine while waiting for grouppipeline authoring to investigate #593602
https://gitlab.com/gitlab-org/quality/test-failure-issues/-/work_items/28090
quarantine: tag with the specified quarantine type.#dx_maintainers Slack channel for someone to review and merge the merge request, rather than assigning it directly.Tiffany Rea (6b829c9b) at 13 Mar 22:36
Quarantine file variable spec for investigation
Thanks @mayra-cabrera I think grouppipeline authoring is a good start. I'm getting a quarantine MR ready in the mean time - Quarantine file variable spec for investigation (!227359 - merged)
Tiffany Rea (fe13b2a7) at 13 Mar 03:00
Merge branch 'jmc-dap-sidebar-fix' into 'master'
... and 1 more commit