Skip to content

Commit c59be27

Browse files
llimllibllimllib
authored
Use local httpbin for integration testing (readmeio#185)
- add [go-httpbin](https://github.com/mccutchen/go-httpbin) as a local httpbin - I thought it would be easier to deploy and use a go binary than deal with a python httpbin, plus httpbin isn't under active development any longer - I had to make a few PRs to it but mostly it worked great - add a reverse proxy container - The only purpose of this container is to forward both 443 and 80 to the httpbin; at first I had been terminating TLS directly at the httpbin but that's hard with the test that requires port 80 - add a custom CA certificate, and TLS certificates generated by it - I used `[mkcert](https://github.com/FiloSottile/mkcert)` to make the certificates, and left a script showing how I did it - **note**: it is totally OK to check this private key in, it is useless for anything other than these tests - update all the dockerfiles - Since I had to run them a bunch of times in testing, moving the layers around so that the `ADD . /src` command came last made them much faster - Settle all but the node on on a single version of alpine - (can we upgrade the node version and library versions? Not for this PR, just curious) - install the custom CA certificate on each of them - update docker-compose to tell each environment to use the correct CA certificates - update the fixtures to expect `go-httpbin` style responses - mostly everything was the same, but there were minor differences - `go-httpbin` puts all headers into an array, even if there was only one - `go-httpbin` puts more stuff in the `form` field than does `httpbin` - refactor src/integration.test.ts a bit - I tried to have a light touch on this, but also I had to edit the code a bunch so I moved it around to make it a bit easier - update python target to use `r.text` instead of `r.json()` because the former prints out the json it received, while the latter prints out the _python representation_ of the json it received, which breaks our tests - I think this was working in the past because the alpine version was older and it was getting an older python and an older requests library closes readmeio#185
1 parent ae9f529 commit c59be27

48 files changed

Lines changed: 295 additions & 107 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docker-compose.yml‎

Lines changed: 43 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,40 +1,79 @@
11
version: '3'
22
services:
3+
httpbin:
4+
image: mccutchen/go-httpbin
5+
command: ['/bin/go-httpbin', '-port', '8080']
6+
7+
# the job of this container is to:
8+
# - host our fake "httpbin.org"
9+
# - terminate SSL
10+
# - forward both 443 and 80 to the httpbin
11+
#
12+
# see integrations/Caddyfile for its config
13+
reverse_proxy:
14+
image: caddy:2.6.4
15+
depends_on:
16+
- httpbin
17+
volumes:
18+
- ./integrations/https-cert:/https-cert:ro
19+
- ./integrations/Caddyfile:/etc/caddy/Caddyfile
20+
environment:
21+
- HTTPS_CERT_FILE=/https-cert/httpbin.org.pem
22+
- HTTPS_KEY_FILE=/https-cert/httpbin.org-key.pem
23+
networks:
24+
default:
25+
# on the docker-compose network, this proxy will be aliased as
26+
# httpbin.org. To make this work with HTTPS, each integration test
27+
# container needs to install the root CA contained in
28+
# ./integrations/https-cert/rootCA.pem
29+
aliases:
30+
- httpbin.org
31+
332
integration_node:
33+
depends_on:
34+
- reverse_proxy
435
build:
536
context: .
637
dockerfile: integrations/node.Dockerfile
7-
command: "npx jest src/integration.test.ts"
38+
command: 'npx jest src/integration.test.ts'
839
environment:
940
- HTTPBIN=true
1041
- INTEGRATION_CLIENT=node
1142
- NODE_ENV=integration
43+
- NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt
1244

1345
integration_php:
46+
depends_on:
47+
- reverse_proxy
1448
build:
1549
context: .
1650
dockerfile: integrations/php.Dockerfile
17-
command: "npx jest src/integration.test.ts"
51+
command: 'npx jest src/integration.test.ts'
1852
environment:
1953
- HTTPBIN=true
2054
- INTEGRATION_CLIENT=php
2155
- NODE_ENV=integration
2256

2357
integration_python:
58+
depends_on:
59+
- reverse_proxy
2460
build:
2561
context: .
2662
dockerfile: integrations/python.Dockerfile
27-
command: "npx jest src/integration.test.ts"
63+
command: 'npx jest src/integration.test.ts'
2864
environment:
2965
- HTTPBIN=true
3066
- INTEGRATION_CLIENT=python
3167
- NODE_ENV=integration
68+
- REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
3269

3370
integration_shell:
71+
depends_on:
72+
- reverse_proxy
3473
build:
3574
context: .
3675
dockerfile: integrations/shell.Dockerfile
37-
command: "npx jest src/integration.test.ts"
76+
command: 'npx jest src/integration.test.ts'
3877
environment:
3978
- HTTPBIN=true
4079
- INTEGRATION_CLIENT=shell

‎integrations/Caddyfile‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
:443 {
2+
reverse_proxy httpbin:8080
3+
tls /https-cert/httpbin.org.pem /https-cert/httpbin.org-key.pem
4+
}
5+
6+
:80 {
7+
reverse_proxy httpbin:8080
8+
}
9+

‎integrations/https-cert/httpbin.org-key.pem‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
-----BEGIN PRIVATE KEY-----
2+
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQCfqXaAuBLPmLnD
3+
uLFqOK4oJBLPds0+m+lkCVEwvcy8ph1/aVH4QvXf/i7kh5aBiO4+S+jx45CK5d/D
4+
cUHU4ivnTu5zA4eIaqvY3PqrSjIYOLMDScHksOfTAVC1Up8CU6ymJWSP+Iz9lDuM
5+
IG7g6N6v/vUhUTYWePbWfkl9GUFGakulwARxUSzjBpy/GaK9fI1SXb62DE4pdLoS
6+
H5wWdpdGXalQlxXZxEEnkBHwTWfdKghJRzA9dHtZU7OtdUkMgE6D5AkxhkRlPPue
7+
57VvVOD9lWbWI5rbOcQeorW5jVkH/R6xDEIlGyUOq6xwgnop+8tXQlgOFbgwZKBn
8+
OQrnqN77AgMBAAECggEAUyDebEJquN+hyL++z7lXI9s0WARY9IIk0ErxlNkdYhNz
9+
REVHwmTKs6caLy5RNHxg3tqTHG4JceghyxaK2hYwGazFBekOhf5UHwNfGBP3ZRkQ
10+
S2P5qeJZsUj8BoxP8dwzBgZuB2+3qMenAVxZnoxgdW3fn0szSBwPGLqD9LhTfh9D
11+
VPaTtk3xucLT7Q2p59Mr9i2LsLSpmlEqpou2kaqqlWqYbR7sogtb+NjkixojYOYh
12+
j7GV+0ryL6ngWGYbGhAo64PQg/szsYmacR8vvUSiiUuxL6AYOxS03q/aGJtQootF
13+
j2cHgk96u/6pwWAtnsu+QGqbN3bOHT0loKjkz5yvEQKBgQDOgwDwlFIerkBxlJi4
14+
NVrKMwPq22YFHQlFMHhLA8Y6cXe6NGBMSGcgfUIowyS+J+YHxlafKyHKoi8EVY/2
15+
aeWkAaTitC7pMKUm7m9IVZQuWxDBUdy3JaYBPqcXHSVtLz08NoW8gWvViGpGSo3B
16+
5Q3J9LMTjwUnKwcLwMor3b5t9wKBgQDF7FbJs7hd5lqZqeJKkUbfAbCShUDozJkI
17+
0Vcm1rCSIPMYNgMv5QBpZJHXzPjHRm1xOu/shTWN9ZQcozeorCcipB9W7V7cd8K6
18+
73L5EEv5up1LzpobsvWUPna+nPct9a4LAIP6tpVrUU6x/4dXTLyw8SzxUr1lpCdg
19+
TJgnZ2pmHQKBgQChlXGTzIogxXlZJcsFP4IlehtTlY3S7HBHefB1yaM+MXBb+wVq
20+
SrCehEPS/zXtr+xWIwO+ERKkqZgeTRCS3zM2y66HUDLwdQaUWrYqJAQI7WpDyVjo
21+
2QV2Ld6xwDV7pB3G0mZ8I8wLTWzSSR14HFBYuCWyZRLEHe+qa5QFFMEe6wKBgQCw
22+
cRiNh6H8U7bA9im9v/UmKSN3+0L6RirHKZhAD3QpUSZllwEQWV4cloNNlnTRcX9v
23+
SdNJTxeHDj6TqQ2dWJsqzpUBsWq3sCvw6jXcwyJ35l0Dj5Lizo8PMQA6vUO9vR8C
24+
v9roToy1ty4okFve/5HXS6l9GP2u9sADoUSjHBZGIQKBgD5TCut1OwE7Ph5eAQaH
25+
drsEkEEuO8h/GcReNCwddU8ISBoWB6y7VIc9cxeO7LeIpL9ued7AEp394mM05HGW
26+
szSs1fz/s7F9Rf+33CaEoAWBnSO5LhmcA5Ebof/8hl7qLLH13x9UDgPtudbRwwnA
27+
zHeF1qwdjvqOQVcebkyo0+Hl
28+
-----END PRIVATE KEY-----

‎integrations/https-cert/httpbin.org.pem‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
-----BEGIN CERTIFICATE-----
2+
MIIEMzCCApugAwIBAgIRAMzWAqhcZaKPLXJ5TE4l1TMwDQYJKoZIhvcNAQELBQAw
3+
dzEeMBwGA1UEChMVbWtjZXJ0IGRldmVsb3BtZW50IENBMSYwJAYDVQQLDB1sbGlt
4+
bGxpYkBhZGFtYSAoV2lsbGlhbSBNaWxsKTEtMCsGA1UEAwwkbWtjZXJ0IGxsaW1s
5+
bGliQGFkYW1hIChXaWxsaWFtIE1pbGwpMB4XDTIzMDYxODAwNDA1MFoXDTI1MDkx
6+
ODAwNDA1MFowUTEnMCUGA1UEChMebWtjZXJ0IGRldmVsb3BtZW50IGNlcnRpZmlj
7+
YXRlMSYwJAYDVQQLDB1sbGltbGxpYkBhZGFtYSAoV2lsbGlhbSBNaWxsKTCCASIw
8+
DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJ+pdoC4Es+YucO4sWo4rigkEs92
9+
zT6b6WQJUTC9zLymHX9pUfhC9d/+LuSHloGI7j5L6PHjkIrl38NxQdTiK+dO7nMD
10+
h4hqq9jc+qtKMhg4swNJweSw59MBULVSnwJTrKYlZI/4jP2UO4wgbuDo3q/+9SFR
11+
NhZ49tZ+SX0ZQUZqS6XABHFRLOMGnL8Zor18jVJdvrYMTil0uhIfnBZ2l0ZdqVCX
12+
FdnEQSeQEfBNZ90qCElHMD10e1lTs611SQyAToPkCTGGRGU8+57ntW9U4P2VZtYj
13+
mts5xB6itbmNWQf9HrEMQiUbJQ6rrHCCein7y1dCWA4VuDBkoGc5Cueo3vsCAwEA
14+
AaNgMF4wDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQMMAoGCCsGAQUFBwMBMB8GA1Ud
15+
IwQYMBaAFPmwtEydRa+QvWvDsgTF0I67b664MBYGA1UdEQQPMA2CC2h0dHBiaW4u
16+
b3JnMA0GCSqGSIb3DQEBCwUAA4IBgQB0KyoE2O4ZdLHlAa7rzk77kVX47xCfAso7
17+
zl0JA2Xrq74Ivx6mOpDIjVrYuWiDjb7KQtD//Zx0sSdj4VknXJ2eadSWvVAXhK9b
18+
PZ1aEAFPADtnqwJ2yWeT57TcFOGwVa14tU5FnETevQ55sprTzLJf9I0NQqoEBvf3
19+
gM36+Ov9a0DH8q2u24V1q1/jzA8wiOmuQLdwKeaY2tRF0VfH1VPPwuok6LfZ+w+c
20+
A/0Bkk1YZHioAKkejMJCqnrI8mU3KpAAm3ciaSbkYzo36XKpHEcTw5KuOZXmCnvf
21+
J/ebX7Upz32QMPNZGF/tRol/AEPEe9m0ze7TKo6ntKm2ITC3eU8qr64qmjI1YjyA
22+
LY8KC74xhM4m6+Wq3x8q+7WTrZG3LCneeHAimeaMqh6s1K8KWU1LCNo18UghfE7R
23+
yHSl/Lsii3UsPowyeg5s9MxEbxuZAv0rKR8TAGArFvKX27eS6JIVHjU2inkwJLim
24+
gTJEW1e0bjBTdS2mDNVRY4/oU5bMVbs=
25+
-----END CERTIFICATE-----

‎integrations/https-cert/make.sh‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
#!/usr/bin/env bash
2+
3+
# install https://github.com/FiloSottile/mkcert
4+
mkcert httpbin.org
5+
cp "$(mkcert -CAROOT)"/rootCA.pem .

‎integrations/https-cert/rootCA.pem‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
-----BEGIN CERTIFICATE-----
2+
MIIEvjCCAyagAwIBAgIRAIGeVWB23wmaHLVpiVNwUVgwDQYJKoZIhvcNAQELBQAw
3+
dzEeMBwGA1UEChMVbWtjZXJ0IGRldmVsb3BtZW50IENBMSYwJAYDVQQLDB1sbGlt
4+
bGxpYkBhZGFtYSAoV2lsbGlhbSBNaWxsKTEtMCsGA1UEAwwkbWtjZXJ0IGxsaW1s
5+
bGliQGFkYW1hIChXaWxsaWFtIE1pbGwpMB4XDTIzMDYxNzE2Mjc1NFoXDTMzMDYx
6+
NzE2Mjc1NFowdzEeMBwGA1UEChMVbWtjZXJ0IGRldmVsb3BtZW50IENBMSYwJAYD
7+
VQQLDB1sbGltbGxpYkBhZGFtYSAoV2lsbGlhbSBNaWxsKTEtMCsGA1UEAwwkbWtj
8+
ZXJ0IGxsaW1sbGliQGFkYW1hIChXaWxsaWFtIE1pbGwpMIIBojANBgkqhkiG9w0B
9+
AQEFAAOCAY8AMIIBigKCAYEAuHKpfZuUexcqU0F1UzFOyhyF0FQWFxIJu6fhq6rH
10+
/0jYwZdaIwiMiHwFMim42duYJlNZoFmD5msNZi4pP8T8reNUUhgt9ki57rsfGQ99
11+
arMXZ2o24TP1WMDz3+38jF2UVrT0ZFJ7A9vQYNUdKFxoNvZMC/VS0/Vwg31Jw1wA
12+
Je+AJauoMDjtOQNU6vMLloVrXu//S1bYI3k7ph/7HSAQsNhYA18tW9bClEkC0gtu
13+
mbtzIwvmiuu7bTUSlYQyHMtAE/1RZ9HfTamDtvfFu3k2/mJNuYecGwL0enNhdBng
14+
1ic4iXlDKu6UKBiZK35AgzMntrgusX2Bj7qOmOPoDN88D6qvvwXwqshg+ZYlRDY9
15+
nCUGAqGo8qs5gx4fEp8GUkCtqFPF/hmh4q8qrjV+XnLI0mtaI8Kql+uxgoa3n55q
16+
dvegCi8dRWXd13ZsrF7o58i5LfsENOYH+SyPtvUc9k7koLUUPK6GuDkNb5kHfX6L
17+
bbSJysK3vzL4A50KfvyYRjMHAgMBAAGjRTBDMA4GA1UdDwEB/wQEAwICBDASBgNV
18+
HRMBAf8ECDAGAQH/AgEAMB0GA1UdDgQWBBT5sLRMnUWvkL1rw7IExdCOu2+uuDAN
19+
BgkqhkiG9w0BAQsFAAOCAYEAdixhKiObZ0eyE1bCRjkVwZKnwAM6/mZpdP3JFpXD
20+
WaLfMZup2nCFJJP40bxq32DU99hqdlRVi8+tQfbsayOwPxGNxbB00R7bNRHNwx1h
21+
FyTCW3kEnbirFyOnN275PBA7n6ZZR3NO9MbxKLSMpDbf2t76Z1mUwCrophekcUlt
22+
XeTzlCItcRzt0g2JZReFds3zVjGxhhdR/n5jV30meY/rTyCjK5TDMCsBlpzsVD5s
23+
CfiSjUIqWVCREBri7VPZweygbYMqI9amrKM8dt1HNjbYppLGsn4ljVW/4hMEqJrw
24+
YJZ/yT9llZDrta1HpzTw50frOt4AppOuW6pvaXeurWvV9hMYLUgg3fo/CuXxCg3A
25+
8EEv3BtDHsb4QvpX7VLmFih/PfZ0E0i4RAH72IRFWpJo/wfUJkdowBk/AfoyG/cj
26+
0gBjNBj69Vd2AQrhX5VG4Qon7iB9YqmLbZItXQrKaonQ3obV19tHJdoW4grxVcq5
27+
dGqaxsY5xfGMztMqXH3ardCx
28+
-----END CERTIFICATE-----

‎integrations/node.Dockerfile‎

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,16 @@
1-
FROM node:14
1+
FROM node:14-alpine
2+
3+
COPY integrations/https-cert/rootCA.pem /root/integration-test.pem
4+
5+
# install the integration test certs
6+
RUN apk --no-cache add ca-certificates && \
7+
rm -rf /var/cache/apk/* && \
8+
cp /root/integration-test.pem /usr/local/share/ca-certificates/ && \
9+
update-ca-certificates
210

3-
ADD . /src
411
WORKDIR /src
512

6-
RUN apt-get update -qq
13+
ADD package.json /src/
714

815
# https://www.npmjs.com/package/axios
916
# https://www.npmjs.com/package/request
@@ -12,3 +19,7 @@ RUN npm install axios request
1219
# Installing node-fetch@2 because as of 3.0 is't now an ESM-only package.
1320
# https://www.npmjs.com/package/node-fetch
1421
RUN npm install node-fetch@2
22+
23+
RUN npm install
24+
25+
ADD . /src

‎integrations/php.Dockerfile‎

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,25 @@ WORKDIR /composer/
44
# https://packagist.org/packages/guzzlehttp/guzzle
55
RUN composer require guzzlehttp/guzzle
66

7-
FROM alpine:3.16
7+
FROM alpine:3.18
8+
9+
COPY integrations/https-cert/rootCA.pem /root/integration-test.pem
10+
COPY --from=builder /composer/vendor /src/vendor
11+
12+
# install the integration test certs
13+
RUN apk --no-cache add ca-certificates && \
14+
rm -rf /var/cache/apk/* && \
15+
cp /root/integration-test.pem /usr/local/share/ca-certificates/ && \
16+
update-ca-certificates
17+
18+
RUN apk update && \
19+
apk add nodejs npm php81 php81-fpm php81-opcache php81-curl
820

9-
ADD . /src
1021
WORKDIR /src
1122

12-
RUN apk update
13-
RUN apk add php81 php81-fpm php81-opcache php81-curl
14-
RUN apk add --update nodejs npm
15-
RUN ln /usr/bin/php81 /usr/bin/php
23+
# add package.json and run npm install so that we only re-do npm install if
24+
# package.json has changed
25+
ADD package.json /src/
1626
RUN npm install
1727

18-
COPY --from=builder /composer/vendor /src/vendor
28+
ADD . /src

‎integrations/python.Dockerfile‎

Lines changed: 18 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,23 @@
1-
FROM alpine
1+
FROM alpine:3.18
2+
3+
COPY integrations/https-cert/rootCA.pem /root/integration-test.pem
4+
5+
# install the integration test certs
6+
RUN apk --no-cache add ca-certificates && \
7+
rm -rf /var/cache/apk/* && \
8+
cp /root/integration-test.pem /usr/local/share/ca-certificates/ && \
9+
update-ca-certificates
10+
11+
RUN apk update && \
12+
apk add nodejs npm python3 py3-pip && \
13+
pip install requests
214

3-
ADD . /src
415
WORKDIR /src
516

6-
RUN apk update
7-
RUN apk add python3 py3-pip
8-
RUN apk add --update nodejs npm
17+
# add package.json and run npm install so that we only re-do npm install if
18+
# package.json has changed
19+
ADD package.json /src/
920
RUN npm install
1021

11-
# https://docs.python-requests.org/en/latest/
12-
RUN pip install requests
22+
ADD . /src
23+

‎integrations/shell.Dockerfile‎

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,21 @@
1-
FROM alpine
1+
FROM alpine:3.18
22

3-
ADD . /src
4-
WORKDIR /src
3+
COPY integrations/https-cert/rootCA.pem /root/integration-test.pem
4+
5+
# install the integration test cert, curl, and node
6+
RUN apk --no-cache add ca-certificates && \
7+
rm -rf /var/cache/apk/* && \
8+
cp /root/integration-test.pem /usr/local/share/ca-certificates/ && \
9+
update-ca-certificates && \
10+
apk update && \
11+
apk add curl && \
12+
apk add --update nodejs npm
513

6-
RUN apk update
14+
WORKDIR /src
715

8-
RUN apk add curl
9-
RUN apk add --update nodejs npm
16+
# add package.json and run npm install so that we only re-do npm install if
17+
# package.json has changed
18+
ADD package.json /src/
1019
RUN npm install
20+
21+
ADD . /src

0 commit comments

Comments
 (0)