feat(ci): Add junit reporters to jest/vitest, codecov test results by billyvg · Pull Request #15341 · getsentry/sentry-javascript

billyvg · 2025-02-07T17:35:24Z

I noticed that not all of our tests were being tracked in codecov. This adds jest-junit reporter for our jest tests and configures vitest to use junit reporter (when CI env var is set).

This brings the number of tests tracked from ~700 to 4.5k

codecov

The code changes predominantly center around integrating codecov reporting and 'jest-junit' into the project. While this is generally well done, there are some concerns to be addressed mainly regarding the error handling and reuse of code.

.github/workflows/build.yml

codecov · 2025-02-07T19:40:17Z

.github/workflows/build.yml

+        continue-on-error: true
+        uses: codecov/test-results-action@v1
+        with:
+          files: packages/**/*.xml


You have hard-coded the path to the test results. This could potentially cause problems when the project structure changes. Consider moving this into a configuration file or an environment variable so that any potential changes in the project structure are easier to manage.

.github/workflows/build.yml

package.json

codecov · 2025-02-07T19:40:17Z

vite/vite.config.ts

      enabled: true,
      reportsDirectory: './coverage',
    },
+    reporters: ['default', 'junit'],


You have chosen to use both the 'default' and 'junit' reporters. Ensure that using both these reporters together does not produce conflicting or duplicate reports.

codecov · 2025-02-07T19:40:17Z

yarn.lock

+  version "16.0.0"
+  resolved "https://registry.yarnpkg.com/jest-junit/-/jest-junit-16.0.0.tgz#d838e8c561cf9fdd7eb54f63020777eee4136785"
+  integrity sha512-A94mmw6NfJab4Fg/BlvVOUXzXgF0XIH6EmTgJ5NDPp4xoKq0Kr7sErb+4Xs9nZvu58pJojz5RFGpqnZYJTrRfQ==
+  dependencies:


The addition of the jest-junit package has introduced several new dependencies. Make sure these new dependancies do not cause conflicts with existing code or other packages.

codecov

The current code diff reflects an effort to integrate jest-junit to the project for test reporting and uploading those test results to Codecov in different jobs. However, there are some improvements necessary to ensure a more ideal behavior.

.github/workflows/build.yml

jest/jest.config.js

package.json

codecov · 2025-02-07T19:40:18Z

vite/vite.config.ts

      enabled: true,
      reportsDirectory: './coverage',
    },
+    reporters: ['default', 'junit'],


Ensure that the junit reporter does not conflict with other reporters. Also, make sure it's supported by the installed version of vitest.

codecov · 2025-02-07T19:40:18Z

vite/vite.config.ts

      reportsDirectory: './coverage',
    },
+    reporters: ['default', 'junit'],
+    outputFile: {


Double check the jest configurations and make sure that outputFile is a valid configuration option, as it's not recognized in the base jest configuration.

codecov

CodecovAI submitted a new review for 455666a

jest/jest.config.js

codecov · 2025-02-07T19:51:06Z

packages/angular/vitest.config.ts

    coverage: {},
    globals: true,
    setupFiles: ['./setup-test.ts'],
-    reporters: ['default'],


The 'default' reporter was removed from some vitest configurations, but it might be beneficial to leave this for local development. If you're aiming to use Junit reporter on CI and default reporter locally, you might want to consider a conditionally loaded configuration based on the environment (development vs CI).

codecov · 2025-02-07T19:51:06Z

vite/vite.config.ts

      enabled: true,
      reportsDirectory: './coverage',
    },
+    reporters: ['default', ...(process.env.CI ? ['junit'] : [])],


The test results filename should probably include the names of the individual test suites to prevent them from overwriting each other. Consider dynamically generating the output filename based on the test suite name.

billyvg · 2025-02-07T18:04:48Z

.github/workflows/build.yml

+        continue-on-error: true
+        uses: codecov/test-results-action@v1
+        with:
+          directory: dev-packages/node-integration-tests


looks like its working

billyvg · 2025-02-07T18:05:25Z

.github/workflows/build.yml

+        continue-on-error: true
+        uses: codecov/test-results-action@v1
+        with:
+          directory: packages/remix


looks like it's working

billyvg · 2025-02-07T20:32:30Z

.github/workflows/build.yml

+        continue-on-error: true
+        uses: codecov/test-results-action@v1
+        with:
+          files: packages/**/*.junit.xml


https://github.com/getsentry/sentry-javascript/actions/runs/13207000168/job/36874091470?pr=15341#step:9:70

billyvg · 2025-02-07T20:33:24Z

.github/workflows/build.yml

+        continue-on-error: true
+        uses: codecov/test-results-action@v1
+        with:
+          files: packages/**/*.junit.xml


https://github.com/getsentry/sentry-javascript/actions/runs/13207000168/job/36874097050?pr=15341#step:9:70

billyvg · 2025-02-07T20:34:19Z

This brings the number of tests tracked from ~700 to 4.5k

joseph-sentry

lgtm from my end, just one optional enhancement you can make to the jest config to get some nice formatting for test names in the codecov UI

i see now that specifying the files using globs was necessary for the browser unit tests, although i'd expect the CLI to find those files, so i'll make an issue to look into that here

jest/jest.config.js

Co-authored-by: joseph-sentry <[email protected]>

codecov · 2025-02-07T21:04:09Z

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
306	1	305	17

View the top 1 failed test(s) by shortest run time

transactions.test.ts Should send a transaction for instrumented server actions

Stack Traces | 4.01s run time

transactions.test.ts:53:5 Should send a transaction for instrumented server actions

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

chargome

lgtm!

billyvg added 2 commits February 7, 2025 12:35

feat(ci): Add junit reporters to jest/vitest, codecov test results

a375268

try glob

3c54638

billyvg added the ci-codecov-ai-review AI Review by Codecov label Feb 7, 2025

codecov bot reviewed Feb 7, 2025

View reviewed changes

tweaks

455666a

codecov bot reviewed Feb 7, 2025

View reviewed changes

billyvg removed the ci-codecov-ai-review AI Review by Codecov label Feb 7, 2025

billyvg commented Feb 7, 2025

View reviewed changes

billyvg marked this pull request as ready for review February 7, 2025 20:34

billyvg requested a review from a team as a code owner February 7, 2025 20:34

billyvg requested review from a team, chargome, joseph-sentry and s1gr1d and removed request for a team February 7, 2025 20:35

joseph-sentry mentioned this pull request Feb 7, 2025

[BUG] Not detecting reports even though they contain junit in the name codecov/test-results-action#110

Open

joseph-sentry approved these changes Feb 7, 2025

View reviewed changes

jest/jest.config.js Show resolved Hide resolved

billyvg and others added 3 commits February 7, 2025 15:57

Update jest/jest.config.js

5cc7979

Co-authored-by: joseph-sentry <[email protected]>

classnameTemplate for vitest

8095394

formatting

0765013

chargome approved these changes Feb 10, 2025

View reviewed changes

billyvg merged commit d8369f7 into develop Feb 10, 2025
145 checks passed

billyvg deleted the feat-ci-jest-junit branch February 10, 2025 16:31

Uh oh!

Conversation

billyvg commented Feb 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

codecov bot Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

codecov bot Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

codecov bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

codecov bot Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

codecov bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

codecov bot Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

billyvg Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

billyvg Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

billyvg Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

billyvg Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

billyvg commented Feb 7, 2025

Uh oh!

joseph-sentry left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Feb 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 1 Tests Failed:

Uh oh!

chargome left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

billyvg commented Feb 7, 2025 •

edited

Loading

codecov bot commented Feb 7, 2025 •

edited

Loading