Skip to content

Commit 598f433

Browse files
tiangolotiangolo
committed
🎉 First public commit
1 parent cfdda75 commit 598f433

105 files changed

Lines changed: 12894 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
.vscode

‎README.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Base Project
2+
3+
Generate a basic back end and front end stack.
4+
5+
## Features
6+
7+
* Full Docker integration (Docker based)
8+
* Docker Swarm Mode deployment
9+
* Docker Compose integration and optimization for local development
10+
* Production ready Python web server using Nginx and uWSGI
11+
* Python Flask back end with:
12+
* Flask-apispec: Swagger live documentation generation
13+
* Marshmallow: model and data serialization (convert model objects to JSON)
14+
* Webargs: parse, validate and document inputs to the endpoint / route
15+
* Secure password hashing by default
16+
* JWT token authentication
17+
* SQLAlchemy models (independent of Flask extensions, so they can be used with Celery workers directly)
18+
* Basic starting models for users and groups (modify and remove as you need)
19+
* Alembic migrations
20+
* CORS (Cross Origin Resource Sharing)
21+
* Celery worker that can import and use models and code from the rest of the back end selectively (you don't have to install the complete app in each worker)
22+
* REST back end tests based on Pytest, integrated with Docker, so you can test the full API interaction, independent on the database. As it runs in Docker, it can build a new data store from scratch each time (so you can use ElasticSearch, MongoDB, CouchDB, or whatever you want, and just test that the API works)
23+
* Easy Python integration with Jupyter Kernels for remote or in-Docker development with extensions like Atom Hydrogen or Visual Studio Code Jupyter
24+
* Angular front end with:
25+
* Docker server based on Nginx
26+
* Docker multi-stage building, so you don't need to save or commit compiled code
27+
* Docker building integrated tests with Chrome Headless
28+
* PGAdmin for PostgreSQL database, you can modify it to use PHPMyAdmin and MySQL easily
29+
* Swagger-UI for live interactive documentation
30+
* Flower for Celery jobs monitoring
31+
* Load balancing between front end and back end with Traefik, so you can have both under the same domain, separated by path, but served by different containers
32+
* Traefik integration, including Let's Encrypt HTTPS certificates automatic generation
33+
* GitLab CI (continuous integration), including front end and back end testing
34+
35+
## How to use it
36+
37+
Go to the directoy where you want to create your project and run:
38+
39+
```bash
40+
pip install cookiecutter
41+
cookiecutter https://github.com/senseta-os/base-project
42+
```
43+
44+
### Generate passwords
45+
46+
You will be asked to provide passwords and secret keys for several components. Open another terminal and run:
47+
48+
```bash
49+
< /dev/urandom tr -dc A-Za-z0-9 | head -c${1:-32};echo;
50+
```
51+
52+
Copy the contents and use that as password / secret key. And run that again to generate another secure key.
53+
54+
55+
## License
56+
57+
This project is licensed under the terms of the MIT license.

‎cookiecutter.json‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
{
2+
"project_name": "Base Project",
3+
"project_slug": "{{ cookiecutter.project_name|lower|replace(' ', '-') }}",
4+
"domain_main": "{{cookiecutter.project_slug}}.com",
5+
"domain_staging": "stag.{{cookiecutter.domain_main}}",
6+
"domain_branch": "branch.{{cookiecutter.domain_main}}",
7+
"domain_dev": "dev.{{cookiecutter.domain_main}}",
8+
9+
"docker_swarm_stack_name_main": "{{domain_main|replace('.', '-')}}",
10+
"docker_swarm_stack_name_staging": "{{domain_staging|replace('.', '-')}}",
11+
"docker_swarm_stack_name_branch": "{{domain_branch|replace('.', '-')}}",
12+
13+
"secret_key": "changethis",
14+
"first_superuser": "admin@{{cookiecutter.domain_main}}",
15+
"first_superuser_password": "changethis",
16+
17+
18+
"postgres_password": "changethis",
19+
"pgadmin_default_user": "admin@{{cookiecutter.domain_main}}",
20+
"pgadmin_default_user_password": "changethis",
21+
22+
"traefik_constraint_tag": "{{cookiecutter.domain_main}}",
23+
"traefik_constraint_tag_staging": "{{cookiecutter.domain_staging}}",
24+
"traefik_constraint_tag_branch": "{{cookiecutter.domain_branch}}",
25+
"traefik_public_network": "traefik-public",
26+
"traefik_public_constraint_tag": "traefik-public",
27+
28+
"flower_auth": "root:changethis",
29+
30+
"sentry_dsn": "",
31+
32+
"docker_image_backend": "backend",
33+
"docker_image_celeryworker": "celeryworker",
34+
"docker_image_frontend": "frontend",
35+
36+
"_copy_without_render": [
37+
"frontend/src/**/*.html",
38+
"frontend/node_modules/*"
39+
]
40+
}

‎{{cookiecutter.project_slug}}/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
.vscode

‎{{cookiecutter.project_slug}}/.gitlab-ci.yml‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
image: docker:latest
2+
3+
before_script:
4+
- apk add --no-cache py-pip
5+
- pip install docker-compose
6+
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
7+
8+
stages:
9+
- test
10+
- build
11+
- deploy
12+
13+
rest-tests:
14+
stage: test
15+
script:
16+
- docker-compose -f docker-compose.test.yml build
17+
- docker-compose -f docker-compose.test.yml up -d
18+
- docker-compose -f docker-compose.test.yml exec -T backend-rest-tests pytest
19+
- docker-compose -f docker-compose.test.yml down -v
20+
tags:
21+
- build
22+
- test
23+
24+
build-branch:
25+
stage: build
26+
script:
27+
- docker-compose -f docker-compose.branch.build.yml build
28+
- docker-compose -f docker-compose.branch.build.yml push
29+
except:
30+
- master
31+
- production
32+
- tags
33+
tags:
34+
- build
35+
- test
36+
37+
build-stag:
38+
stage: build
39+
script:
40+
- docker-compose -f docker-compose.stag.build.yml build
41+
- docker-compose -f docker-compose.stag.build.yml push
42+
only:
43+
- master
44+
tags:
45+
- build
46+
- test
47+
48+
build-prod:
49+
stage: build
50+
script:
51+
- docker-compose -f docker-compose.prod.build.yml build
52+
- docker-compose -f docker-compose.prod.build.yml push
53+
only:
54+
- production
55+
tags:
56+
- build
57+
- test
58+
59+
deploy-branch:
60+
stage: deploy
61+
script: docker stack deploy -c docker-compose.branch.yml --with-registry-auth {{cookiecutter.docker_swarm_stack_name_branch}}
62+
environment:
63+
name: staging
64+
url: https://{{cookiecutter.domain_branch}}
65+
except:
66+
- master
67+
- production
68+
- tags
69+
tags:
70+
- swarm
71+
- branch
72+
73+
deploy-stag:
74+
stage: deploy
75+
script: docker stack deploy -c docker-compose.stag.yml --with-registry-auth {{cookiecutter.docker_swarm_stack_name_staging}}
76+
environment:
77+
name: staging
78+
url: https://{{cookiecutter.domain_staging}}
79+
only:
80+
- master
81+
tags:
82+
- swarm
83+
- stag
84+
85+
deploy-prod:
86+
stage: deploy
87+
script: docker stack deploy -c docker-compose.prod.yml --with-registry-auth {{cookiecutter.docker_swarm_stack_name_main}}
88+
environment:
89+
name: production
90+
url: https://{{cookiecutter.domain_main}}
91+
only:
92+
- production
93+
tags:
94+
- swarm
95+
- prod

‎{{cookiecutter.project_slug}}/README.md‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# {{project_name}}
2+
3+
## Back end local development
4+
5+
* Update your local `hosts` file, set the IP `127.0.0.1` (your `localhost`) to `{{cookiecutter.domain_dev}}`. The `docker-compose.override.yml` file will set the environment variable `SERVER_NAME` to that host. Otherwise you would receive 404 errors.
6+
7+
* Modify your hosts file, probably in `/etc/hosts` to include:
8+
9+
```
10+
0.0.0.0 {{cookiecutter.domain_dev}}
11+
```
12+
13+
...that will make your browser talk to your locally running server.
14+
15+
* Start the stack with Docker Compose:
16+
17+
```bash
18+
docker-compose up -d
19+
```
20+
21+
* Start an interactive session in the server container that is running an infinite loop doing nothing:
22+
23+
```bash
24+
docker-compose exec server bash
25+
```
26+
27+
* Run the local debugging Flask server, all the command is in the `RUN` environment variable:
28+
29+
```bash
30+
$RUN
31+
```
32+
33+
* Your OS will handle redirecting `{{cookiecutter.domain_dev}}` to your local stack. So, in your browser, go to: http://{{cookiecutter.domain_dev}}.
34+
35+
Add and modify SQLAlchemy models to `./backend/app/app/models/`, Marshmallow schemas to `./backend/app/app/schemas` and API endpoints to `./backend/app/app/api/`.
36+
37+
Add and modify tasks to the Celery worker in `./backend/app/app/worker.py`.
38+
39+
If you need to install any additional package to the worker, add it to the file `./backend/app/Dockerfile-celery-worker`.
40+
41+
42+
### Back end tests
43+
44+
To test the back end run:
45+
46+
```bash
47+
# Build the testing stack
48+
docker-compose -f docker-compose.test.yml build
49+
# Start the testing stack
50+
docker-compose -f docker-compose.test.yml up -d
51+
# Run the REST tests
52+
docker-compose -f docker-compose.test.yml exec -T backend-rest-tests pytest
53+
# Stop and eliminate the testing stack
54+
docker-compose -f docker-compose.test.yml down -v
55+
```
56+
57+
The tests run with Pytest, modify and add tests to `./backend/app/app/rest_tests/`.
58+
59+
If you need to install any additional package for the REST tests, add it to the file `./backend/app/Dockerfile-rest-tests`.
60+
61+
If you use GitLab CI the tests will run automatically.
62+
63+
64+
### Migrations
65+
66+
* Start an interactive session in the server container that is running an infinite loop doing nothing:
67+
68+
```bash
69+
docker-compose exec server bash
70+
```
71+
72+
* After changing a model (for example, adding a column), inside the container, create a revision, e.g.:
73+
74+
```bash
75+
alembic revision -m "Add column last_name to User model"
76+
```
77+
78+
* Commit to the git repository the files generated in the alembic directory.
79+
80+
* After creating the revision, run the migration in the database (this is what will actually change the database):
81+
82+
```bash
83+
alembic upgrade head
84+
```
85+
86+
## Front end development
87+
88+
* Enter the `frontend` directory, install the NPM packages and start it the `npm` scrits:
89+
90+
```bash
91+
cd frontend
92+
npm install
93+
npm run start
94+
```
95+
96+
Check the file `package.json` to see other available options.
97+
98+
## Deployment
99+
100+
To deploy the stack to a Docker Swarm run, e.g.:
101+
102+
```bash
103+
docker stack deploy -c docker-compose.prod.yml --with-registry-auth {{cookiecutter.docker_swarm_stack_name_main}}
104+
```
105+
106+
Using the corresponding Docker Compose file.
107+
108+
If you use GitLab CI, it will automatically deploy it.
109+
110+
GitLab CI is configured assuming 3 environments following GitLab flow:
111+
112+
* `prod` (production) from the `production` branch.
113+
* `stag` (staging) from the `master` branch.
114+
* `branch`, from any other branch (a feature in development).

‎{{cookiecutter.project_slug}}/backend/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
__pycache__

‎{{cookiecutter.project_slug}}/backend/Dockerfile‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
FROM tiangolo/uwsgi-nginx-flask:python3.6
2+
3+
RUN pip install --upgrade pip
4+
RUN pip install flask flask-cors psycopg2 raven[flask] celery==4.1.0 passlib[bcrypt] SQLAlchemy==1.1.13 flask-apispec flask-jwt-extended alembic
5+
6+
# For development, Jupyter remote kernel, Hydrogen
7+
# Using inside the container:
8+
# jupyter notebook --ip=0.0.0.0 --allow-root
9+
ARG env=prod
10+
RUN bash -c "if [ $env == 'dev' ] ; then pip install jupyter ; fi"
11+
EXPOSE 8888
12+
13+
COPY ./app /app
14+
WORKDIR /app/
15+
16+
ENV STATIC_PATH /app/app/static
17+
ENV STATIC_INDEX 1
18+
19+
ENV PYTHONPATH=/app
20+
21+
EXPOSE 80

‎{{cookiecutter.project_slug}}/backend/Dockerfile-celery-worker‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
FROM python:3.6
2+
3+
RUN pip install psycopg2 raven pyyaml celery==4.1.0 SQLAlchemy==1.1.13 passlib[bcrypt]
4+
5+
# For development, Jupyter remote kernel, Hydrogen
6+
# Using inside the container:
7+
# jupyter notebook --ip=0.0.0.0 --allow-root
8+
ARG env=prod
9+
RUN bash -c "if [ $env == 'dev' ] ; then pip install jupyter ; fi"
10+
EXPOSE 8888
11+
12+
ENV C_FORCE_ROOT=1
13+
14+
COPY ./app /app
15+
WORKDIR /app
16+
17+
ENV PYTHONPATH=/app
18+
19+
CMD celery worker -A app.worker -l info -Q main-queue -c 1

‎{{cookiecutter.project_slug}}/backend/Dockerfile-rest-tests‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
FROM python:3.6
2+
3+
RUN pip install requests
4+
5+
# For development, Jupyter remote kernel, Hydrogen
6+
# Using inside the container:
7+
# jupyter notebook --ip=0.0.0.0 --allow-root
8+
RUN pip install jupyter
9+
EXPOSE 8888
10+
11+
RUN pip install faker==0.8.4 pytest
12+
13+
COPY ./app /app
14+
15+
ENV PYTHONPATH=/app
16+
17+
WORKDIR /app/app/rest_tests

0 commit comments

Comments
 (0)