Skip to content

Docker Dev Environment fixes#137

Open
AustinSchoen wants to merge 1 commit intohotsapi:masterfrom
AustinSchoen:dev_setup
Open

Docker Dev Environment fixes#137
AustinSchoen wants to merge 1 commit intohotsapi:masterfrom
AustinSchoen:dev_setup

Conversation

@AustinSchoen
Copy link
Copy Markdown

Apologize for the large PR, I didn't want to touch files that were already there, as I'm not sure what role they play in the CI workflow yet, so some things have been duplicated and edited, instead of changed in place.

This PR does a few things:

  • DB liveness check in the Parse code (This was to keep the container from dying if if the DB or migrations weren't quite ready)
  • provides a setup script for MacOS via brew/docker
  • Provides a development docker-compose file setup to handle waiting on dependencies amongst the other containers (such as the DB and migrations)
  • Provides a multi-stage Dockerfile to allow local dev image builds that will be used automatically with docker-compose up
  • additional database config for local dev

I tested these changes locally, and was able to get everything to spin up and access the web frontend, however I tried uploading a replay file and seeing an error https://share.getcloudapp.com/mXum1j60. I don't think that error is related to my changes and this changeset is large enough, so I planned to tackle this in another PR.

Copy link
Copy Markdown
Contributor

@rjnienaber rjnienaber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bear in mind that I was testing this on Linux so some of these comments might be due to it being different from Mac.

Comment thread .env.example.docker Outdated
APP_DEBUG=true
APP_LOG_LEVEL=debug

DB_HOST=mysql # this is the name of the compose database service
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the comment interferes with the artisan migration and I was getting an error until I removed it. It was creating a DSN that looked like the following:

mysql:host=mysql # this is the name of the compose database service;port=3306;dbname=hotsapi

instead of

mysql:host=mysql;port=3306;dbname=hotsapi

Comment thread docker-compose-dev.yml Outdated
MYSQL_USER: 'docker'
MYSQL_PASSWORD: 'docker'
healthcheck:
test: ['CMD', '/opt/goss', '-g', '/opt/goss/goss-mysql.yaml', 'validate']
Copy link
Copy Markdown
Contributor

@rjnienaber rjnienaber Dec 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the command be /opt/goss/goss ? Otherwise it just seems to be pointing at a directory.

Comment thread goss/goss-mysql.yaml Outdated
service:
mysql:
enabled: true
running: true
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if the service field has been auto-generated but mysql runs in the docker container as pid 1 and doesn't use a service manager.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, I generated this with the Goss autoadd, I'll find an alternative

Comment thread goss/goss-mysql.yaml
gid: 999
process:
mysql:
running: true
Copy link
Copy Markdown
Contributor

@rjnienaber rjnienaber Dec 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the process name in this case is mysqld. Also, would it make sense to add a section that waits for a listener on port 3306?

e.g.

tcp:3306:
  listening: true
  ip:
    - 0.0.0.0

Comment thread goss/README.md
@@ -0,0 +1,5 @@
# Goss
Goss is a standalone Go binary similar to servspec, which is used to validate server state: <https://github.com/aelsabbahy/goss>

Copy link
Copy Markdown
Contributor

@rjnienaber rjnienaber Dec 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps it would be better if the goss binary is downloaded from the release page when script/setup is called (if it's not available)?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea this is a better method that including it in the repo, I'll add download command to the setup script.

Comment thread script/setup
cd "${ROOT}"
cp .env.example.docker .env

docker-compose up
Copy link
Copy Markdown
Contributor

@rjnienaber rjnienaber Dec 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a missing step here? I had to run the following docker build command to ensure the hotsapidev/hotsapi:dev image was available:

$ docker build -t hotsapidev/hotsapi:dev -f docker/Dockerfile.hotsapi .

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm... docker-compose up should build that missing image if it doesn't already exist as part of the up command. Let me delete all my images and see what happens.

@poma
Copy link
Copy Markdown
Member

poma commented Jan 2, 2020

this one will take some time to review and test

@poma
Copy link
Copy Markdown
Member

poma commented Jan 3, 2020

Do we really need this level of complexity/automation? Why not just let developer make a few steps manually according to docs? Like

  1. customize .env (or rather set defaults that will just work)
  2. bring up db
  3. run artisan migrate
  4. bring up webserver

It's not that hard doing manually but will avoid some complexity in code. Also for dev environment it doesn't make much sense to keep worker, scheduler, parser, prune services. They can be tested by running corresponding artisan commands from console. Memcached could be turned off because we have file cache driver, and cloud can be emulated with filesystem too.

In my opinion writing a better docs and keeping code simple is a better way. I'll try to write a manual how to set up dev environment.

We might need this if we decide to implement some kind of staging environment that runs integration tests or something though.

@AustinSchoen
Copy link
Copy Markdown
Author

@poma definitely get your point in regards to added complexity, but it simplifies environment setup following a common pattern here: https://github.com/github/scripts-to-rule-them-all. I'm fine leaving it for later but I'd argue that lowering the barrier to entry would make it easier to get contributors, and we can always document and automate how the system works. To address a couple other points discussed out of band:

  • setup only installs the bare minimum packages to get the environment working, these would need to be installed regardless of the method used to run docker.
  • docker-compose can be used to launch the system in the latest version (3.7) as it was in 2.0.

@rjnienaber
Copy link
Copy Markdown
Contributor

That scripts-to-rule-them-all repo looks interesting. Going over the docs, it does seem like setup, in this case, is fulfilling more of a bootstrap role in the sense that it's installing dependencies homebrew, docker, docker-compose and goss.

I've been working a bit on migrations and developer setup over the last couple of days which means I've been setting up and blowing away the dev environment quite a bit. Including a wait in poma's setup steps has been working well for me.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants