Skip to content

CI Test for #35049 #35102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gregmagolan wants to merge 4 commits into from
Closed

CI Test for #35049 #35102

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b951461
test: use puppeteer in integration tests and to download correct chro…
gregmagolan Jan 31, 2020
d1144a4
build: update lock files in other integration tests
gregmagolan Feb 1, 2020
9ebfbe5
test: use puppeteer in aio build instead to remove CI_CHROMEDRIVER_VE…
gregmagolan Feb 7, 2020
6456b54
build: CI TEST [DO NOT MERGE]
gregmagolan Feb 7, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 36 additions & 12 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,22 @@ commands:
- attach_workspace:
at: *workspace_location

# Install shared libs used by Chrome that is either provisioned by
# rules_webtesting or by puppeteer.
install_chrome_libs:
description: Install shared Chrome libs
steps:
- run:
name: Install shared Chrome libs
command: |
sudo apt-get update
# Install GTK+ graphical user interface (libgtk-3-0), advanced linux sound architecture (libasound2)
# and network security service libraries (libnss3) & X11 Screen Saver extension library (libssx1)
# which are dependendies of chrome & needed for karma & protractor headless chrome tests.
# This is a very small install which takes around 7s in comparing to using the full
# circleci/node:x.x.x-browsers image.
sudo apt-get -y install libgtk-3-0 libasound2 libnss3 libxss1
Copy link
Copy Markdown
Contributor

@IgorMinar IgorMinar Feb 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we then also remove the browsers-executor executor from the circleci config and replace it with the default executor?

Copy link
Copy Markdown
Contributor Author

@gregmagolan gregmagolan Feb 7, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last holdout is still integration/bazel-schematics which I didn't want to touch yet as it is more complicated but all other tests including aio should be able to go to default-executor now. Good call.


# Initializes the CI environment by setting up common environment variables.
init_environment:
description: Initializing environment (setting up variables)
Expand Down Expand Up @@ -349,11 +365,11 @@ jobs:
no_output_timeout: 20m

test_aio:
# Needed because the AIO tests and the PWA score test depend on Chrome being available.
executor: browsers-executor
executor: default-executor
steps:
- custom_attach_workspace
- init_environment
- install_chrome_libs
# Compile dependencies to ivy
# Running `ngcc` here (instead of implicitly via `ng build`) allows us to take advantage of
# the parallel, async mode speed-up (~20-25s on CI).
Expand All @@ -376,11 +392,11 @@ jobs:
- run: yarn --cwd aio redirects-test

deploy_aio:
# Needed because before deploying the deploy-production script runs the PWA score tests.
executor: browsers-executor
executor: default-executor
steps:
- custom_attach_workspace
- init_environment
- install_chrome_libs
# Deploy angular.io to production (if necessary)
- run: setPublicVar_CI_STABLE_BRANCH
- run: yarn --cwd aio deploy-production
Expand All @@ -390,11 +406,11 @@ jobs:
viewengine:
type: boolean
default: false
# Needed because the AIO tests and the PWA score test depend on Chrome being available.
executor: browsers-executor
executor: default-executor
steps:
- custom_attach_workspace
- init_environment
- install_chrome_libs
# Build aio (with local Angular packages)
- run: yarn --cwd aio build-local<<# parameters.viewengine >>-with-viewengine<</ parameters.viewengine >>-ci
# Run unit tests
Expand Down Expand Up @@ -465,11 +481,11 @@ jobs:

# This job should only be run on PR builds, where `CI_PULL_REQUEST` is not `false`.
test_aio_preview:
# Needed because the test-preview script runs e2e tests and the PWA score test with Chrome.
executor: browsers-executor
executor: default-executor
steps:
- custom_attach_workspace
- init_environment
- install_chrome_libs
- run: yarn --cwd aio install --frozen-lockfile --non-interactive
- run:
name: Wait for preview and run tests
Expand Down Expand Up @@ -532,7 +548,8 @@ jobs:
# See comments inside the integration/run_tests.sh script.
integration_test:
executor:
# Needed because the integration tests expect Chrome to be installed (e.g cli-hello-world)
# Needed because the integration/bazel-schematics test expects Chrome to be installed
# TODO(gregmagolan): remove the dependency on local chrome from that test
name: browsers-executor
# Note: we run Bazel in one of the integration tests, and it can consume >2G
# of memory. Together with the system under test, this can exhaust the RAM
Expand Down Expand Up @@ -632,12 +649,11 @@ jobs:
webhook_url_env_var: SLACK_DEV_INFRA_CI_FAILURES_WEBHOOK_URL

aio_monitoring_next:
# This job needs Chrome to be globally installed because the tests run with Protractor
# which does not load the browser through the Bazel webtesting rules.
executor: browsers-executor
executor: default-executor
steps:
- custom_attach_workspace
- init_environment
- install_chrome_libs
- run:
name: Run tests against https://next.angular.io/
command: ./aio/scripts/test-production.sh https://next.angular.io/ $CI_AIO_MIN_PWA_SCORE
Expand Down Expand Up @@ -842,6 +858,14 @@ workflows:
- test_aio_preview:
requires:
- aio_preview
## ==> TEMP TEMP DO NO MERGE
- aio_monitoring_stable:
requires:
- setup
- aio_monitoring_next:
requires:
- setup
## TEMP TEMP DO NO MERGE <==
- integration_test:
requires:
- build-npm-packages
Expand Down
10 changes: 9 additions & 1 deletion aio/karma.conf.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
// Karma configuration file, see link for more information
// https://karma-runner.github.io/1.0/config/configuration-file.html
process.env.CHROME_BIN = require('puppeteer').executablePath();

module.exports = function (config) {
config.set({
Expand Down Expand Up @@ -30,7 +31,14 @@ module.exports = function (config) {
colors: true,
logLevel: config.LOG_INFO,
autoWatch: true,
browsers: ['Chrome'],
customLaunchers: {
ChromeHeadlessNoSandbox: {
base: 'ChromeHeadless',
// See /integration/README.md#browser-tests for more info on these args
flags: ['--no-sandbox', '--headless', '--disable-gpu', '--disable-dev-shm-usage', '--hide-scrollbars', '--mute-audio']
}
},
browsers: [process.env['CI'] ? 'ChromeHeadlessNoSandbox' : 'Chrome'],
browserNoActivityTimeout: 60000,
singleRun: false,
restartOnFileChange: true,
Expand Down
4 changes: 2 additions & 2 deletions aio/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@
"generate-stackblitz": "node ./tools/stackblitz-builder/generateStackblitz",
"generate-zips": "node ./tools/example-zipper/generateZips",
"build-404-page": "node scripts/build-404-page",
"update-webdriver": "webdriver-manager update --standalone false --gecko false $CI_CHROMEDRIVER_VERSION_ARG",
"update-webdriver": "node ../scripts/webdriver-manager-update.js",
"~~audit-web-app": "node scripts/audit-web-app",
"~~check-env": "node scripts/check-environment",
"~~clean-generated": "node --eval \"require('shelljs').rm('-rf', 'src/generated')\"",
Expand Down Expand Up @@ -118,7 +118,6 @@
"archiver": "^1.3.0",
"canonical-path": "1.0.0",
"chalk": "^2.1.0",
"chrome-launcher": "^0.10.7",
"cjson": "^0.5.0",
"codelyzer": "^5.1.2",
"cross-spawn": "^5.1.0",
Expand Down Expand Up @@ -155,6 +154,7 @@
"lunr": "^2.1.0",
"npm-run-all": "^4.1.5",
"protractor": "^5.4.2",
"puppeteer": "2.1.1",
"rehype": "^6.0.0",
"rehype-slug": "^2.0.0",
"remark": "^9.0.0",
Expand Down
11 changes: 6 additions & 5 deletions aio/scripts/audit-web-app.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,13 @@
*/

// Imports
const chromeLauncher = require('chrome-launcher');
const puppeteer = require('puppeteer');
const lighthouse = require('lighthouse');
const printer = require('lighthouse/lighthouse-cli/printer');
const logger = require('lighthouse-logger');

// Constants
const AUDIT_CATEGORIES = ['accessibility', 'best-practices', 'performance', 'pwa', 'seo'];
const CHROME_LAUNCH_OPTS = {chromeFlags: ['--headless']};
const LIGHTHOUSE_FLAGS = {logLevel: process.env.CI ? 'error' : 'info'}; // Be less verbose on CI.
const SKIPPED_HTTPS_AUDITS = ['redirects-http', 'uses-http2'];
const VIEWER_URL = 'https://googlechrome.github.io/lighthouse/viewer';
Expand Down Expand Up @@ -84,13 +83,15 @@ function formatScore(score) {
}

async function launchChromeAndRunLighthouse(url, flags, config) {
const chrome = await chromeLauncher.launch(CHROME_LAUNCH_OPTS);
flags.port = chrome.port;
const browser = await puppeteer.launch({
headless: true,
args: ['--remote-debugging-port=9222']});
flags.port = browser.port;

try {
return await lighthouse(url, flags, config);
} finally {
await chrome.kill();
await browser.close();
}
}

Expand Down
5 changes: 5 additions & 0 deletions aio/tests/deployment/e2e/protractor.conf.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@ exports.config = {
suite: 'full',
capabilities: {
browserName: 'chrome',
chromeOptions: process.env['CI'] ? {
binary: require('puppeteer').executablePath(),
// See /integration/README.md#browser-tests for more info on these args
args: ['--no-sandbox', '--headless', '--disable-gpu', '--disable-dev-shm-usage', '--hide-scrollbars', '--mute-audio']
} : {},
},
directConnect: true,
framework: 'jasmine',
Expand Down
5 changes: 5 additions & 0 deletions aio/tests/e2e/protractor.conf.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@ exports.config = {
],
capabilities: {
browserName: 'chrome',
chromeOptions: process.env['CI'] ? {
binary: require('puppeteer').executablePath(),
// See /integration/README.md#browser-tests for more info on these args
args: ['--no-sandbox', '--headless', '--disable-gpu', '--disable-dev-shm-usage', '--hide-scrollbars', '--mute-audio']
} : {},
},
directConnect: true,
baseUrl: 'http://localhost:4200/',
Expand Down
79 changes: 67 additions & 12 deletions aio/yarn.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1230,6 +1230,11 @@
resolved "https://registry.yarnpkg.com/@types/lunr/-/lunr-2.3.2.tgz#d4a51703315ed0e53c43257216f1014ce6491562"
integrity sha512-zcUZYquYDUEegRRPQtkZ068U9CoIjW6pJMYCVDRK25r76FEWvMm1oHqZQUfQh4ayIZ42lipXOpXEiAtGXc1XUg==

"@types/mime-types@^2.1.0":
version "2.1.0"
resolved "https://registry.yarnpkg.com/@types/mime-types/-/mime-types-2.1.0.tgz#9ca52cda363f699c69466c2a6ccdaad913ea7a73"
integrity sha1-nKUs2jY/aZxpRmwqbM2q2RPqenM=

"@types/minimatch@*":
version "3.0.3"
resolved "https://registry.yarnpkg.com/@types/minimatch/-/minimatch-3.0.3.tgz#3dca0e3f33b200fc7d1139c0cd96c1268cadfd9d"
Expand Down Expand Up @@ -3280,20 +3285,20 @@ [email protected]:
resolved "https://registry.yarnpkg.com/concat-map/-/concat-map-0.0.1.tgz#d8a96bd77fd68df7793a73036a3ba0d5405d477b"
integrity sha1-2Klr13/Wjfd5OnMDajug1UBdR3s=

concat-stream@^1.4.7, concat-stream@^1.5.2:
version "1.6.0"
resolved "https://registry.yarnpkg.com/concat-stream/-/concat-stream-1.6.0.tgz#0aac662fd52be78964d5532f694784e70110acf7"
[email protected], concat-stream@^1.5.0:
version "1.6.2"
resolved "https://registry.yarnpkg.com/concat-stream/-/concat-stream-1.6.2.tgz#904bdf194cd3122fc675c77fc4ac3d4ff0fd1a34"
integrity sha512-27HBghJxjiZtIk3Ycvn/4kbJk/1uZuJFfuPEns6LaEvpvG1f0hTea8lilrouyo9mVc2GWdcEZ8OLoGmSADlrCw==
dependencies:
buffer-from "^1.0.0"
inherits "^2.0.3"
readable-stream "^2.2.2"
typedarray "^0.0.6"

concat-stream@^1.5.0:
version "1.6.2"
resolved "https://registry.yarnpkg.com/concat-stream/-/concat-stream-1.6.2.tgz#904bdf194cd3122fc675c77fc4ac3d4ff0fd1a34"
integrity sha512-27HBghJxjiZtIk3Ycvn/4kbJk/1uZuJFfuPEns6LaEvpvG1f0hTea8lilrouyo9mVc2GWdcEZ8OLoGmSADlrCw==
concat-stream@^1.4.7, concat-stream@^1.5.2:
version "1.6.0"
resolved "https://registry.yarnpkg.com/concat-stream/-/concat-stream-1.6.0.tgz#0aac662fd52be78964d5532f694784e70110acf7"
dependencies:
buffer-from "^1.0.0"
inherits "^2.0.3"
readable-stream "^2.2.2"
typedarray "^0.0.6"
Expand Down Expand Up @@ -4960,6 +4965,16 @@ extglob@^2.0.4:
snapdragon "^0.8.1"
to-regex "^3.0.1"

extract-zip@^1.6.6:
version "1.6.7"
resolved "https://registry.yarnpkg.com/extract-zip/-/extract-zip-1.6.7.tgz#a840b4b8af6403264c8db57f4f1a74333ef81fe9"
integrity sha1-qEC0uK9kAyZMjbV/Txp0Mz74H+k=
dependencies:
concat-stream "1.6.2"
debug "2.6.9"
mkdirp "0.5.1"
yauzl "2.4.1"

[email protected]:
version "1.3.0"
resolved "https://registry.yarnpkg.com/extsprintf/-/extsprintf-1.3.0.tgz#96918440e3041a7a414f8c52e3c574eb3c3e1e05"
Expand Down Expand Up @@ -5025,6 +5040,13 @@ faye-websocket@~0.11.1:
dependencies:
websocket-driver ">=0.5.1"

fd-slicer@~1.0.1:
version "1.0.1"
resolved "https://registry.yarnpkg.com/fd-slicer/-/fd-slicer-1.0.1.tgz#8b5bcbd9ec327c5041bf9ab023fd6750f1177e65"
integrity sha1-i1vL2ewyfFBBv5qwI/1nUPEXfmU=
dependencies:
pend "~1.2.0"

figgy-pudding@^3.4.1, figgy-pudding@^3.5.1:
version "3.5.1"
resolved "https://registry.yarnpkg.com/figgy-pudding/-/figgy-pudding-3.5.1.tgz#862470112901c727a0e495a80744bd5baa1d6790"
Expand Down Expand Up @@ -8311,7 +8333,7 @@ [email protected]:
resolved "https://registry.yarnpkg.com/mime-db/-/mime-db-1.38.0.tgz#1a2aab16da9eb167b49c6e4df2d9c68d63d8e2ad"
integrity sha512-bqVioMFFzc2awcdJZIzR3HjZFX20QhilVS7hytkKrv7xFAn8bM1gzc/FOX2awLISvWe0PV8ptFKcon+wZ5qYkg==

mime-types@^2.1.12, mime-types@^2.1.16, mime-types@~2.1.17, mime-types@~2.1.18, mime-types@~2.1.19:
mime-types@^2.1.12, mime-types@^2.1.16, mime-types@^2.1.25, mime-types@~2.1.17, mime-types@~2.1.18, mime-types@~2.1.19:
version "2.1.26"
resolved "https://registry.yarnpkg.com/mime-types/-/mime-types-2.1.26.tgz#9c921fc09b7e149a65dfdc0da4d20997200b0a06"
integrity sha512-01paPWYgLrkqAyrlDorC1uDwl2p3qZT7yl806vW7DvDoxwXi46jsjFbg+WdwotBIk6/MbEhO/dh5aZ5sNj/dWQ==
Expand All @@ -8335,7 +8357,7 @@ [email protected], mime@^1.4.1:
resolved "https://registry.yarnpkg.com/mime/-/mime-1.6.0.tgz#32cd9e5c64553bd58d19a568af452acff04981b1"
integrity sha512-x0Vn8spI+wuJ1O6S7gnbaQg8Pxh4NNHb7KSINmEWKiPE4RKOplvijn+NkmYmmRgP68mc70j2EbeTFRsrswaQeg==

mime@^2.2.0, mime@^2.3.1, mime@^2.4.4:
mime@^2.0.3, mime@^2.2.0, mime@^2.3.1, mime@^2.4.4:
version "2.4.4"
resolved "https://registry.yarnpkg.com/mime/-/mime-2.4.4.tgz#bd7b91135fc6b01cde3e9bae33d659b63d8857e5"
integrity sha512-LRxmNwziLPT828z+4YkNzloCFC2YM4wrB99k+AV5ZbEyfGNWfG8SO1FUXLmLDBSo89NrJZ4DIWeLjy1CHGhMGA==
Expand Down Expand Up @@ -9535,6 +9557,11 @@ pbkdf2@^3.0.3:
safe-buffer "^5.0.1"
sha.js "^2.4.8"

pend@~1.2.0:
version "1.2.0"
resolved "https://registry.yarnpkg.com/pend/-/pend-1.2.0.tgz#7a57eb550a6783f9115331fcf4663d5c8e007a50"
integrity sha1-elfrVQpng/kRUzH89GY9XI4AelA=

performance-now@^2.1.0:
version "2.1.0"
resolved "https://registry.yarnpkg.com/performance-now/-/performance-now-2.1.0.tgz#6309f4e0e5fa913ec1c69307ae364b4b377c9e7b"
Expand Down Expand Up @@ -10002,7 +10029,7 @@ progress@^1.1.8:
version "1.1.8"
resolved "https://registry.yarnpkg.com/progress/-/progress-1.1.8.tgz#e260c78f6161cdd9b0e56cc3e0a85de17c7a57be"

progress@^2.0.3:
progress@^2.0.1, progress@^2.0.3:
version "2.0.3"
resolved "https://registry.yarnpkg.com/progress/-/progress-2.0.3.tgz#7e8cf8d8f5b8f239c1bc68beb4eb78567d572ef8"
integrity sha512-7PiHtLll5LdnKIMw100I+8xJXR5gW2QwWYkT6iJva0bXitZKa/XMrSbdmg3r2Xnaidz9Qumd0VPaMrZlF9V9sA==
Expand Down Expand Up @@ -10095,6 +10122,11 @@ proxy-addr@~2.0.5:
forwarded "~0.1.2"
ipaddr.js "1.9.0"

proxy-from-env@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/proxy-from-env/-/proxy-from-env-1.0.0.tgz#33c50398f70ea7eb96d21f7b817630a55791c7ee"
integrity sha1-M8UDmPcOp+uW0h97gXYwpVeRx+4=

prr@~1.0.1:
version "1.0.1"
resolved "https://registry.yarnpkg.com/prr/-/prr-1.0.1.tgz#d3fc114ba06995a45ec6893f484ceb1d78f5f476"
Expand Down Expand Up @@ -10161,6 +10193,22 @@ punycode@^2.1.0:
resolved "https://registry.yarnpkg.com/punycode/-/punycode-2.1.1.tgz#b58b010ac40c22c5657616c8d2c2c02c7bf479ec"
integrity sha512-XRsRjdf+j5ml+y/6GKHPZbrF/8p2Yga0JPtdqTIY2Xe5ohJPD9saDJJLPvp9+NSBprVvevdXZybnj2cv8OEd0A==

[email protected]:
version "2.1.1"
resolved "https://registry.yarnpkg.com/puppeteer/-/puppeteer-2.1.1.tgz#ccde47c2a688f131883b50f2d697bd25189da27e"
integrity sha512-LWzaDVQkk1EPiuYeTOj+CZRIjda4k2s5w4MK4xoH2+kgWV/SDlkYHmxatDdtYrciHUKSXTsGgPgPP8ILVdBsxg==
dependencies:
"@types/mime-types" "^2.1.0"
debug "^4.1.0"
extract-zip "^1.6.6"
https-proxy-agent "^4.0.0"
mime "^2.0.3"
mime-types "^2.1.25"
progress "^2.0.1"
proxy-from-env "^1.0.0"
rimraf "^2.6.1"
ws "^6.1.0"

[email protected]:
version "1.4.1"
resolved "https://registry.yarnpkg.com/q/-/q-1.4.1.tgz#55705bcd93c5f3673530c2c2cbc0c2b3addc286e"
Expand Down Expand Up @@ -13526,7 +13574,7 @@ ws@^1.0.1:
options ">=0.0.5"
ultron "1.0.x"

ws@^6.2.1:
ws@^6.1.0, ws@^6.2.1:
version "6.2.1"
resolved "https://registry.yarnpkg.com/ws/-/ws-6.2.1.tgz#442fdf0a47ed64f59b6a5d8ff130f4748ed524fb"
integrity sha512-GIyAXC2cB7LjvpgMt9EKS2ldqr0MTrORaleiOno6TweZ6r3TKtoFQWay/2PceJ3RuBasOHzXNn5Lrw1X0bEjqA==
Expand Down Expand Up @@ -13742,6 +13790,13 @@ yargs@^7.0.2:
y18n "^3.2.1"
yargs-parser "^5.0.0"

[email protected]:
version "2.4.1"
resolved "https://registry.yarnpkg.com/yauzl/-/yauzl-2.4.1.tgz#9528f442dab1b2284e58b4379bb194e22e0c4005"
integrity sha1-lSj0QtqxsihOWLQ3m7GU4i4MQAU=
dependencies:
fd-slicer "~1.0.1"

[email protected]:
version "0.1.2"
resolved "https://registry.yarnpkg.com/yeast/-/yeast-0.1.2.tgz#008e06d8094320c372dbc2f8ed76a0ca6c8ac419"
Expand Down
Loading