Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 47 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,51 @@
## Project board
https://app.zenhub.com/workspaces/commit-zero-5da8decc7046a60001c6db44/board?filterLogic=any&repos=203630543,247773730,257676371,258369081
# Overview
A set of templates meant to work with [Zero], the templated result is a ready to scale infrastructure boilerplate built on top of AWS EKS baked with all best practices we have accumulated.

# AWS EKS Stack
## Repository structure
The root folder is used for declaring parameters required by the templates, and [Zero][zero] will gather the required parameters and parse the templates as individual repositories for user to maintain.
```sh
/ # file in the root directory is for initializing the user's repo and declaring metadata
|-- Makefile #make command triggers the initialization of repository
|-- zero-module.yml #module declares required parameters and credentials
|
| # files in templates become the repo for users
| templates/
| | # this makefile is used both during init and
| | # on-going needs/utilities for user to maintain their infrastructure
| |-- Makefile
| |-- terraform/
| | |-- bootstrap/ #initial setup
| | |-- environments/ #infrastructure setup
| | | |-- production/
| | | |-- staging/
| |-- kubernetes
| | |-- terraform
| | | |-- environments #k8s-ultities
| | | | |-- production/
| | | | |-- staging/
```

This is a [zero](https://github.com/commitdev/zero) module which sets up a
## AWS EKS Stack
The Zero-awk-eks stack is designed with scalability and maintainability in mind, this repo is a series of templates indented to be filled in with modules parameters, and executed by zero
This is a [Zero][zero] module which sets up a
hosting environment on AWS running Kubernetes. It will generate terraform output
which describes the environment mapped in this [architecture
diagram](./templates/docs/architecture-overview.svg).
which describes the environment mapped in this [architecture diagram][arch-diagram].

**Resource List**: [Link][resource-list]

**Prerequisites**
- Terraform installed
- Kubectl installed
- Wget installed
- A valid AWS account
- [Set up the AWS CLI](https://docs.aws.amazon.com/polly/latest/dg/setup-aws-cli.html)
- [A domain registered with Route53](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-register.html)
- Note: if you want to use different domain per environment (staging/poduction), you need to have multiple hosted zones. The available zones in Route53 can be found by running this command. `aws route53 list-hosted-zones`
- [Set up the AWS CLI][aws-cli]
- [A domain registered with Route53][aws-route53]
- Note: if you want to use different domain per environment (staging/production), you need to have multiple hosted zones. The available zones in Route53 can be found by running this command. `aws route53 list-hosted-zones`

## Getting Started

This is meant to be used with the `zero` tool and not directly. See
the [zero](https://github.com/commitdev/zero) repository for more
the [Zero][zero] repository for more
information. The tool will parse through configuration and fill in any
template variables which need to be encoded into the terraform configuration.

Expand All @@ -36,3 +61,15 @@ infrastructure:
- EC2 (t2.small): $0.023 USD / hr

EC2 instance sizing can be configured in [terraform/environments/staging/main.tf](terraform/environments/staging/main.tf)

## Other links
Project board: [zenhub][zenhub-board]

<!-- Links -->
[zero]: https://github.com/commitdev/zero
[arch-diagram]: ./docs/architecture-overview.svg
[resource-list]: ./docs/resources.md
<!-- External Links -->
[aws-cli]: https://docs.aws.amazon.com/polly/latest/dg/setup-aws-cli.html
[aws-route53]: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-register.html
[zenhub-board]: https://app.zenhub.com/workspaces/commit-zero-5da8decc7046a60001c6db44/board?filterLogic=any&repos=203630543,247773730,257676371,258369081
File renamed without changes.
4 changes: 2 additions & 2 deletions templates/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,11 @@ teardown-remote-state:
export AWS_PAGER='' && export AWS_DEFAULT_REGION=<% index .Params `region` %> && \
aws s3 rm s3://<% .Name %>-$(ENVIRONMENT)-terraform-state --recursive && \
aws s3 rb s3://<% .Name %>-$(ENVIRONMENT)-terraform-state --force && \
aws dynamodb delete-table --table-name <% .Name %>-$(ENVIRONMENT)-terraform-state-locks
aws dynamodb delete-table --region <% index .Params `region` %> --table-name <% .Name %>-$(ENVIRONMENT)-terraform-state-locks

teardown-secrets:
export AWS_PAGER='' && export AWS_DEFAULT_REGION=<% index .Params `region` %> && \
aws secretsmanager list-secrets --query "SecretList[?Tags[?Key=='project' && Value=='<% .Name %>']].[Name] | [0][0]" | xargs aws secretsmanager delete-secret --secret-id && \
aws secretsmanager list-secrets --region <% index .Params `region` %> --query "SecretList[?Tags[?Key=='project' && Value=='<% .Name %>']].[Name] | [0][0]" | xargs aws secretsmanager delete-secret --region <% index .Params `region` %> --secret-id && \
aws iam delete-access-key --user-name <% .Name %>-ci-user --access-key-id $(shell aws iam list-access-keys --user-name <% .Name %>-ci-user --query "AccessKeyMetadata[0].AccessKeyId" | sed 's/"//g') && \
aws iam delete-user --user-name <% .Name %>-ci-user

Expand Down
85 changes: 73 additions & 12 deletions templates/README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,32 @@
# Infrastructure

**Prerequisites**
- Terraform installed
- Kubectl installed
- Wget installed
- A valid AWS account
- [Set up the AWS CLI](https://docs.aws.amazon.com/polly/latest/dg/setup-aws-cli.html)
- [A domain registered with Route53](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-register.html)
- Note: if you want to use different domain per environment (staging/poduction), you need to have multiple hosted zones. The available zones in Route53 can be found by running this command. `aws route53 list-hosted-zones`

### Using the Kubernetes Cluster
# Overview
Your infrastructure should be up and running, your terraform repository is the source of truth for your infrastructure, here is [a list of components and resources][zero-resource-list] that comes with the EKS-stack

# How to
## Managing your Terraform
#### Why terraform
The repository follows infrastructure as code as a core principle, it allows repeatable and reproducible infrastructures and makes DevOps much more manageable; to learn more about it we suggest reading the [Terraform's workflow guide][tf-workflow].

#### Intended workflows
To make changes to the infrastructure you would modify the terraform code changing the components you wish to modify, then plan the changes with `terraform plan` to make sure you are making the desired changes; then apply the changes with `terraform apply` on your staging environment. Once you reach the satisfactory state, you should do the same on production environment and check-in the changes of your infrastructure code, as this repo should be the source of truth of your deployed infrastructure.
Our infrastructure is divided into a few areas.
1. Initial setup
- [remote state][tf-remote-state]
- [secrets][tf-secrets]
2. Infrastructure
- [production][tf-production-env]
- [staging][tf-staging-env]
3. Kubernetes utilities
- [production][tf-production-utilities]
- [staging][tf-staging-utilities]


#### Style guide, resources, and Configuring your infrastructure as code
For more information about the terraform in this repo, please see [Link][zero-tf-guide]


## Using the Kubernetes Cluster

Before using the cluster the first time you'll need to set up your local `kubectl` context:
```shell
Expand All @@ -26,13 +43,57 @@ kubectl get pods -A
To init and apply the terraform configs, simply run the `make` and specify the
environment. The default environment is `staging`
```shell
$ make ENV=<environment>
$ make ENVIRONMENT=<environment>
```


### Set up an application
Configure your k8s context

```shell
$ aws eks update-kubeconfig --name <cluster-name> --region us-east-1
```

#### Extra features built into my kubernetes cluster
Outlines and best practices utilities that comes with your EKS cluster.
Please see [Link][zero-k8s-guide]
- Dashboards
- Logging
- Monitoring
- Ingress / TLS certificates (auto provisioning)
- AWS IAM integration with Kubernetes RBAC
...

# Resources
### Infrastructure
This [architecture-diagram][architecture-diagram] displays the original setup you get from the terraform templates

Commonly used links in AWS console
|Resources |Links|
|--- |---|
|Route 53 |https://console.aws.amazon.com/route53/home |
|IAM |https://console.aws.amazon.com/iam/home#/users|
|ECR |https://console.aws.amazon.com/ecr/repositories|
|RDS |https://console.aws.amazon.com/rds|
### Suggested readings
- [Terraform workflow][tf-workflow]
- [Why do I want code as infrastructure][why-infra-as-code]
-


<!-- Links -->
[tf-workflow]: https://www.terraform.io/guides/core-workflow.html
[why-infra-as-code]: https://www.oreilly.com/library/view/terraform-up-and/9781491977071/ch01.html
<!-- code -->
[tf-remote-state]: ./terraform/bootstrap/remote-state
[tf-secrets]: ./terraform/bootstrap/secrets
[tf-production-env]: ./terraform/environments/production
[tf-staging-env]: ./terraform/environments/staging

[tf-production-utilities]: ./kubernetes/terraform/environments/production
[tf-staging-utilities]: ./kubernetes/terraform/environments/staging

[zero-tf-guide]: ./terraform/README.md
[zero-k8s-guide]: ./kubernetes/terraform/modules/kubernetes/README.md
[zero-architecture-diagram]: https://github.com/commitdev/zero-aws-eks-stack/blob/master/docs/architecture-overview.svg
[zero-resource-list]: https://github.com/commitdev/zero-aws-eks-stack/blob/master/docs/resources.md

4 changes: 0 additions & 4 deletions templates/kubernetes/terraform/modules/kubernetes/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
# kubernetes tf module

## Introduction

This Terraform module contains configuration to provision kubernetes resources.

## Core Components

[Nginx Ingress Controller](https://github.com/kubernetes/ingress-nginx/)
Expand Down
8 changes: 4 additions & 4 deletions templates/terraform/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@

- All environments should contain the following:

`main.tf`: Toplevel terraform configuration file that instantiates the `environment` module.
`main.tf`: Top level terraform configuration file that instantiates the `environment` module.

- Configuration should be pushed "top->down" from the `environment` module to it's submodules.

Expand All @@ -40,7 +40,7 @@
`main.tf`: Module entrypoint where instantiation of resources happens.
`backend.tf`: Terraform remote state configuration.
`provider.tf`: Provider configuration.
`variables.tf`: Environment-specific variables are desclared here.
`variables.tf`: Environment-specific variables are declared here.
`versions.tf`: Terraform version information.
`files/`: (DEPRECATED)

Expand Down Expand Up @@ -78,9 +78,9 @@

- When to use the Terraform Kuberenetes Provider and when to use manifests?

- Use the Terraform Kubernetes Provider (`provider "kubernetes"`) whenever you are provisioning a resource that could be considered relatively static (think Ingress, RoleBinding, CluterRoleBinding, etc).
- Use the Terraform Kubernetes Provider (`provider "kubernetes"`) whenever you are provisioning a resource that could be considered relatively static (think Ingress, RoleBinding, ClusterRoleBinding, etc).

- Use conventional Kubernetes manifests / `kubectl` when provisioning resouirces that could be considered dynamic (think Deployments).
- Use conventional Kubernetes manifests / `kubectl` when provisioning resources that could be considered dynamic (think Deployments).

## Application

Expand Down