This workshop is a continuation off: Deploy Trino on Kubernetes: Helm Setup with Hive Metastore & MinIO Integration
Medium Article Covering this repo.
- docker-desktop with kubernetes enabled
Set for your environment ~/.baschrc or ~/.zshrc or in the session where you run this.
export HELM_EXPERIMENTAL_OCI=1helm repo add argo https://argoproj.github.io/argo-helmList version
helm search repo argo-cdTime of writing this using 7.3.4.
make deploy-local-argocd Get password for UI.
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -dRun
kubectl port-forward service/datapains-argocd-server -n argocd 8080:443You can now login into the UI with:
ArgoCD UI
- user: admin
- password:
make create-secret [email protected]:Thelin90/datapains-airbyte.git SECRET_NAME=datapains-airbyte-creds SSH_KEY_PATH=<path-to-id_rsa>make apply-argocd-app REPO_NAME=datapains-airbyte APP_NAME=airbyteNote deployment can take a few minutes, check in argocd UI!.
UI is now accessible via: wwww.localhost:32767
make create-secret [email protected]:Thelin90/datapains-bi-tools.git SECRET_NAME=datapains-bi-tools-creds SSH_KEY_PATH=<path-to-id_rsa>make apply-argocd-app REPO_NAME=datapains-bi-tools APP_NAME=supersetPort Forward
kubectl port-forward svc/superset 8088:8088 -n supersetmake create-secret [email protected]:Thelin90/datapains-argo-workflow.git SECRET_NAME=datapains-argo-workflow-creds SSH_KEY_PATH=<path-to-id_rsa>make apply-argocd-app REPO_NAME=datapains-argo-workflow APP_NAME=argo-workflowNote deployment can take a few minutes, check in argocd UI!.
UI is now accessible via: wwww.localhost:32767
helm repo add prometheus-community https://prometheus-community.github.io/helm-chartshelm repo update- prometheus operator
- grafana
make apply-argocd-app REPO_NAME=datapains-monitoring APP_NAME=kube-prometheus-stackNote deployment can take a few minutes, check in argocd UI!.
make create-secret [email protected]:Thelin90/datapains-prometheus-k8s.git SECRET_NAME=datapains-prometheus-k8s-creds SSH_KEY_PATH=<path-to-id_rsa>make apply-argocd-app REPO_NAME=datapains-monitoring APP_NAME=prometheusNote deployment can take a few minutes, check in argocd UI!.
make apply-argocd-app REPO_NAME=datapains-monitoring APP_NAME=example-applicationFirst add spark namespace.
kubectl create namespace sparkmake apply-argocd-app REPO_NAME=datapains-spark-operator APP_NAME=spark-operatorWe will deploy trino through argocd, the repo we have available on datapains-trino-k8s
You will have to setup a secret key.
Create your secret based on your github SSH key.
make create-secret [email protected]:Thelin90/datapains-trino-k8s.git SECRET_NAME=datapains-trino-k8s-creds SSH_KEY_PATH=<path-to-id_rsa>NOTE!
Remember to build the hive metastore image, instructions here.
Now deploy the metastore:
make apply-argocd-app REPO_NAME=datapains-trino-k8s APP_NAME=metastoreNow deploy trino, once you see it being healthy in ArgoCD UI.
make apply-argocd-app REPO_NAME=datapains-trino-k8s APP_NAME=trino