[{"content":"Now that the cluster is up and has Argo CD controlling application deployment, I\u0026rsquo;m starting to move my homelab services out of docker-compose stacks and into my k8s cluster.\nI run MQTT (Eclipse Mosquitto in my case) as a message bus between zigbee2mqtt and zwave-js-ui and Home Assistant, so it\u0026rsquo;s the first service I\u0026rsquo;m moving into the k8s cluster.\nThis is part 6 of my Kubernetes homelab cluster setup series - Install MQTT into a k8s cluster.\nPart 1 - Setting up Talos with a Cilium CNI on proxmox Part 2 Add SSL to Kubernetes using Cilium, cert-manager and LetsEncrypt with domains hosted on Amazon Route 53 Part 3 - Set up Secret Management with SOPS Part 4 - Back up your Talos etcd cluster to a SMB share Part 5 - Install ArgoCD Part 6 - Install MQTT into a k8s cluster Goals Set up an ArgoCD Application that installs Eclipse Mosquitto into your cluster Software Versions Here are the versions of the software I used while writing this post.\nSoftware Version argocd 3.2.6 cilium 1.18.6 kubectl 1.34 kubernetes 1.34.1 mosquitto 2.1.2 talos 1.12.4 Pre-requisites To follow along with this post, you will need:\nA kubernetes cluster. I\u0026rsquo;m using the talos cluster from Part 1 - Setting up Talos with a Cilium CNI on proxmox, but this will work with any kubernetes cluster. A GitHub account. You can also use GitLab or your own git server, but all the examples in this post assume you\u0026rsquo;re storing your Argo CD repository on GitHub. ArgoCD installed in your cluster. See Part 5 - Install ArgoCD for installation instructions If you\u0026rsquo;ve been following along with this series, you already have helm and kubectl. Otherwise you can brew install them or follow the install instructions at helm.sh and kubectl. Installing Mosquitto I created a helm chart for Mosquitto here.\nBefore we install it, I\u0026rsquo;m going to show you the manifests I used to create the chart so you can see how everything works. I recommend you read through that before going to the easier gitops way to have Argo CD install the helm chart with your own values.yaml.\nPick a static IP Mosquitto needs a static IP to run on so that your IOT devices can find it. I recommend you make your life easier and add a mqtt-server.example.com entry in your homelab pointing at the IP though, instead of configuring your devices with the bare IP. I have too many devices to want to hassle with reconfiguring things later just because I decide to move the MQTT server - that\u0026rsquo;s what DNS is for.\nCreate the password secret Whether you install from the manifests directly or via Argo and Helm, you\u0026rsquo;re going to need to create a secret containing the password file. Here\u0026rsquo;s one I created with sops.\nFirst, you need to create a password file - we\u0026rsquo;ll use docker so we don\u0026rsquo;t have to install mosquitto outside of k8s just to make a password file.\ndocker run --rm -v $(pwd):/mnt eclipse-mosquitto mosquitto_passwd -b -c /mnt/mqtt_password iot your iot_password Replace the iot entry in this manifest with the one in the mqtt_password file you just created.\n# mqtt-sops-secret.yaml apiVersion: isindir.github.com/v1alpha3 kind: SopsSecret metadata: name: mqtt-secret namespace: mqtt spec: secretTemplates: - name: mqtt-secret stringData: password.txt: | iot:$7$1000$APd9OAc+QTR9rB8Zq8KOseIIWNl8AbyfDmqn4aHgwaSlFTKi7lrn1P/6+cHQDiJmzYtDaBCy7vNemrvxGr0PZg==$ZOqrxA2Y4vnHLtSC8HpQhGTV8b49IGiAAU2fhcFMxz1L9M42IBBk8yun03t1WTw6CUcfr3sBRuZ8Y6Gdt3FMag== Then use sops to encrypt the secret.\nsops encrypt -i mqtt-secret.yaml If you haven\u0026rsquo;t already set SOPS up in your cluster, I documented that in Part 3 - Set up Secret Management with SOPS.\nFinally, create the secret containing the password file.\nkubectl create namespace mqtt \u0026amp;\u0026amp; \\ kubectl apply -f mqtt-sops-secret.yaml Install Mosquitto using plain manifest files Make a mosquitto directory in the repository your Argo CD installation is using for configuration.\nCreate a Namespace Keep Mosquitto in its own namespace for tidiness.\n# mqtt-namespace.yaml apiVersion: v1 kind: Namespace metadata: labels: kubernetes.io/metadata.name: mqtt pod-security.kubernetes.io/enforce: privileged name: mqtt Create a PVC Create a PVC for mosquitto to store its data in\n--- # mqtt-data-pvc.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: mosquitto-data-pvc namespace: mqtt spec: accessModes: [\u0026#34;ReadWriteMany\u0026#34;] resources: requests: storage: 1Gi It needs a LoadBalancer so it can be reached from outside the cluster I run zigbee2mqtt and zwave-js-ui as part of my Home Assistant system. I haven\u0026rsquo;t moved them into the k8s cluster yet though, so I need mosquitto accessible from outside the cluster.\nThis service manifest works with the Cilium CNI we set up in Part 1 - Setting up Talos with a Cilium CNI on proxmox.\nChange the cilium.io/lb-ipam-pool-name and lbipam.cilium.io/ips keys in the manifest file to match your local setup.\n--- # mqtt-lb.yaml apiVersion: v1 kind: Service metadata: name: mqtt-lb namespace: mqtt annotations: cilium.io/lb-ipam-pool-name: \u0026#34;default-pool\u0026#34; cilium.io/assign-internal-ip: \u0026#34;true\u0026#34; lbipam.cilium.io/ips: \u0026#34;10.9.8.7\u0026#34; lbipam.cilium.io/sharing-key: \u0026#34;mqtt\u0026#34; labels: homelab.service: mqtt spec: type: LoadBalancer ports: - name: mqtt-tcp port: 1883 protocol: TCP targetPort: 1883 - name: mqtt-websocket port: 9001 protocol: TCP targetPort: 9001 selector: app: mosquitto Create a manifest for the ConfigMap and Deployment --- # mqtt-stack.yaml apiVersion: v1 kind: ConfigMap metadata: name: mosquitto-config namespace: mqtt data: mosquitto.conf: | persistence true persistence_location /mosquitto/data log_dest stdout # Explicitly bind to 0.0.0.0 to allow K8s Service traffic listener 1883 0.0.0.0 allow_anonymous false password_file /mosquitto/config/password.txt listener 9001 0.0.0.0 protocol websockets # # MQTT over TLS # listener 8883 # cafile /mosquitto/certs/ca.crt # certfile /mosquitto/certs/tls.crt # keyfile /mosquitto/certs/tls.key # # MQTT over WebSockets (Secure) # listener 9001 # protocol websockets # cafile /mosquitto/certs/ca.crt # certfile /mosquitto/certs/tls.crt # keyfile /mosquitto/certs/tls.key --- apiVersion: apps/v1 kind: Deployment metadata: name: mosquitto-deployment namespace: mqtt spec: replicas: 1 selector: matchLabels: app: mosquitto template: metadata: labels: app: mosquitto spec: securityContext: runAsNonRoot: true runAsUser: 1883 runAsGroup: 1883 fsGroup: 1883 fsGroupChangePolicy: \u0026#34;Always\u0026#34; seccompProfile: type: RuntimeDefault initContainers: - name: init-permissions image: busybox:latest command: - sh - -c - | # Copy from read-only secret to writable emptyDir # It will be owned by UID 1883 automatically cp /mnt/secret/password_file /mosquitto/writable-config/password.txt chmod 0600 /mosquitto/writable-config/password.txt securityContext: allowPrivilegeEscalation: false capabilities: drop: [\u0026#34;ALL\u0026#34;] runAsNonRoot: true runAsUser: 1883 seccompProfile: type: RuntimeDefault volumeMounts: - name: password-file-raw mountPath: /mnt/secret readOnly: true - name: writable-config mountPath: /mosquitto/writable-config containers: - name: mosquitto image: eclipse-mosquitto:latest securityContext: allowPrivilegeEscalation: false capabilities: drop: [\u0026#34;ALL\u0026#34;] readOnlyRootFilesystem: true runAsNonRoot: true runAsUser: 1883 seccompProfile: type: RuntimeDefault resources: limits: memory: \u0026#34;128Mi\u0026#34; requests: cpu: \u0026#34;100m\u0026#34; memory: \u0026#34;128Mi\u0026#34; ports: - containerPort: 1883 - containerPort: 9001 volumeMounts: - name: config mountPath: /mosquitto/config/mosquitto.conf subPath: mosquitto.conf - name: data mountPath: /mosquitto/data - name: writable-config mountPath: /mosquitto/config/password.txt subPath: password.txt - name: tmp mountPath: /tmp volumes: - name: config configMap: name: mosquitto-config - name: password-file-raw secret: secretName: mqtt-secret - name: writable-config emptyDir: {} - name: data persistentVolumeClaim: claimName: mosquitto-data-pvc - name: tmp emptyDir: {} Move all the manifest files into your mqtt directory, then install them with\nkubectl apply -f mqtt Install Mosquitto with ArgoCD The helm chart has only been tested on clusters using Cilium as their CNI. Clusters without Cilium should ignore the cilium annotations and work, I haven\u0026rsquo;t tested that.\nhelm repo add laboratorium-domesticum https://unixorn-argocd.github.io/laboratorium-domesticum \u0026amp;\u0026amp; \\ helm search repo laboratorium-domesticum --versions First, install the password secret from the manifest we created earlier.\nCreate a values.yaml file to configure the chart Add a directory in your configuration git repository. I put mine in configs/mosquitto/values.yaml.\nHere\u0026rsquo;s an example values.yaml you can base yours on.\nMake sure you change the ips and poolName keys in the cilium section. The mosquittoConfig section contains a reasonable configuration for using mqtt with zigbee2mqtt, zwave-js-ui and all the WIFI-based IOT devices I\u0026rsquo;ve used. Don\u0026rsquo;t increase the replicas in the mosquittoDeployment section or some of your devices and services will update one replica and some others and you will have weird errors. If you didn\u0026rsquo;t name your password secret mqtt-secret, update the existingSecret key in the mosquittoSecret section. The full documentation for the helm chart is here\n# values.yaml # Networking cilium: assign-internal-ip: \u0026#34;true\u0026#34; ips: \u0026#34;10.9.8.7\u0026#34; # Set the IP address of your MQTT server here pool-name: \u0026#34;default-pool\u0026#34; # The cilium IP pool that contains the IP sharing-key: \u0026#34;mqtt\u0026#34; # Changed from sharingKey # Mosquitto Deployment Settings mosquittoDeployment: replicas: 1 # Don\u0026#39;t image: repository: eclipse-mosquitto tag: \u0026#34;2.0.18\u0026#34; podSecurityContext: fsGroup: 1883 resources: limits: cpu: 100m memory: 128Mi requests: cpu: 100m memory: 128Mi # Configuration (Supports templates for dynamic values) mosquittoConfig: mosquittoConf: |- persistence true persistence_location /mosquitto/data log_dest stdout listener 1883 0.0.0.0 allow_anonymous false # This must match the mount path in our deployment password_file /mosquitto/config/password.txt # Password Secret Configuration mosquittoSecrets: # Set to false because you are providing the secret manually create: false # The name of your existing Kubernetes Secret existingSecret: \u0026#34;mqtt-secret\u0026#34; # This can be left empty since create is false passwordData: \u0026#34;\u0026#34; # Persistent Storage pvc: storageRequest: 1Gi # Probably overkill mqttLb: type: LoadBalancer # Add these lines here: poolName: \u0026#34;default-pool\u0026#34; sharingKey: \u0026#34;mqtt\u0026#34; ips: \u0026#34;10.9.8.7\u0026#34; revisionHistoryLimit: 3 Commit the file and push your changes to the default branch of your configuration repository.\nCreate an ArgoCD Application Manifest Here\u0026rsquo;s an ArgoCD multi-source application manifest you can use. It will configure Argo to use my chart to install Mosquitto using the values.yaml you just created in your git repository. I keep mine in an argocd-applications directory in my configuration repository.\n# mosquitto-argocd.yaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: mosquitto-mqtt namespace: argocd spec: project: default destination: server: https://kubernetes.default.svc # CHANGED: Now points to the \u0026#39;mqtt\u0026#39; namespace namespace: mqtt sources: - repoURL: \u0026#39;https://github.com/unixorn-argocd/laboratorium-domesticum.git\u0026#39; targetRevision: main path: charts/mqtt-cilium helm: valueFiles: - $config/configs/mosquitto/values.yaml - repoURL: \u0026#39;https://github.com/your_username/your_config_repo\u0026#39; targetRevision: HEAD ref: config syncPolicy: automated: prune: true selfHeal: true syncOptions: # Ensures the \u0026#39;mqtt\u0026#39; namespace is created if missing - CreateNamespace=true Apply it with kubectl\nkubectl apply -f argocd-applications/mosquitto-argocd.yaml And within a minute or so, you should see ArgoCD finish creating the deployment and it\u0026rsquo;ll be active on your LAN.\nIt\u0026rsquo;ll look something like this:\nand the detailed view should look similar to this:\nYou can see that there are two ReplicaSets - ArgoCD keeps old ReplicaSets to make rollback easier - they\u0026rsquo;re set to zero replicas since they\u0026rsquo;ve been superseded by a newer version.\n","permalink":"https://unixorn.github.io/post/homelab/k8s/06-install-mqtt/","summary":"\u003cp\u003eNow that the cluster is up and has Argo CD controlling application deployment, I\u0026rsquo;m starting to move my homelab services out of \u003ccode\u003edocker-compose\u003c/code\u003e stacks and into my k8s cluster.\u003c/p\u003e\n\u003cp\u003eI run MQTT (Eclipse \u003ca href=\"https://mosquitto.org/\"\u003eMosquitto\u003c/a\u003e in my case) as a message bus between \u003ca href=\"https://www.zigbee2mqtt.io/\"\u003ezigbee2mqtt\u003c/a\u003e and \u003ca href=\"https://github.com/zwave-js/zwave-js-ui\"\u003ezwave-js-ui\u003c/a\u003e and \u003ca href=\"https://www.home-assistant.io/\"\u003eHome Assistant\u003c/a\u003e, so it\u0026rsquo;s the first service I\u0026rsquo;m moving into the k8s cluster.\u003c/p\u003e","title":"Install MQTT into k8s"},{"content":"In part five of my kubernetes homelab series, we will install Argo CD into a homelab cluster so we can use gitops practices instead of ad-hoc kubectl commands.\nThis will make it much easier to recreate the cluster exactly if we decide to move it to new hardware or just want to reset things to a known-good state after experimentation.\nPart 1 - Setting up Talos with a Cilium CNI on proxmox Part 2 Add SSL to Kubernetes using Cilium, cert-manager and LetsEncrypt with domains hosted on Amazon Route 53 Part 3 - Secret Management with SOPS Part 4 - Back up your Talos etcd cluster to a SMB share Part 5 - Install Argo CD Part 6 - Install MQTT into a k8s cluster Why bother with gitops? It\u0026rsquo;s a homelab, and I\u0026rsquo;m the only administrator Using gitops to configure your cluster instead of running kubectl apply commands directly makes it possible to reproduce the exact state of the cluster later if you need to revert to a known-good state.\nIn a non-homelab environment, using gitops allows you to have a history of who made what changes when, easily control who can change which environments and to reproduce the same state in multiple clusters running in different data centers.\nIn our homelab environment, running kubectl apply commands directly makes it nearly impossible to reproduce the exact state of the cluster later if you need to revert to a known-good state. By using gitops powered by Argo CD, you can revert your configuration git repository to a known-good tag or commit, and Argo will automatically bring the cluster back to that state.\nEven if you\u0026rsquo;re the only one administering the cluster, it\u0026rsquo;s a good habit to get into - unless you\u0026rsquo;re very unlucky, work environments will require changes to production clusters to be done via some form of gitops for change management.\nGoals In this post we will install Argo CD into our homelab cluster and configure it to manage itself.\nSoftware Versions Here are the versions of the software I used while writing this post. Later versions should work, but this is what these instructions were tested with.\nSoftware Version argocd 3.2.6 helm 4.0.1 kubectl 1.34 kubernetes 1.34.1 talos 1.11.5 Pre-requisites To follow along with this post, you will need:\nA homelab kubernetes cluster. I\u0026rsquo;m using the talos cluster from Part 1 - Setting up Talos with a Cilium CNI on proxmox, but this will work with any kubernetes cluster. A GitHub account. You can also use GitLab or your own git server, but all the examples in this post assume you\u0026rsquo;re storing your Argo CD repository on GitHub. The argocd command line tool. Either install it with brew install argocd or follow the argocd install instructions page. If you\u0026rsquo;ve been following along with this series, you already have helm and kubectl. Otherwise you can brew install them or follow the install instructions at helm.sh and kubectl. Goals We\u0026rsquo;re going to bootstrap ArgoCD by installing it with helm, then we\u0026rsquo;ll aad it to its own configuration repository so we can do future upgrades with git.\nInstallation Add the helm repository We\u0026rsquo;re going to do our initial bootstrap with helm, so first add the argo helm repository.\nhelm repo add argo https://argoproj.github.io/argo-helm Find the latest chart version We want to extract the default values from the helm chart, so first run\nhelm search repo argo/argo-cd As I\u0026rsquo;m writing this, the current chart version is 9.3.7\n$ helm search repo argo/argo-cd NAME CHART VERSION\tAPP VERSION\tDESCRIPTION argo/argo-cd\t9.3.7 v3.2.6 A Helm chart for Argo CD, a declarative, GitOps... Now we can extract the defaults and see what we want to change\nhelm show values argo/argo-cd --version 9.3.7 \u0026gt; values.yaml Create argo-values.yaml In this post, we\u0026rsquo;re going to use all defaults except for server.insecure, so put this into argo-values.yaml\nconfigs: params: server.insecure: true Bootstrap Argo CD with helm Now we can bootstrap Argo CD. Why helm upgrade --install instead of helm install? If you have an old version of a chart installed and run helm install, you get an error. helm upgrade --install works whether or not the chart is already installed.\nhelm upgrade --install argocd argo/argo-cd \\ --version 9.3.7 \\ --values argo-values.yaml \\ --namespace argocd --create-namespace Confirm the install Depending on the horsepower of your k8s nodes, it can take a few minutes for argocd\u0026rsquo;s install to complete. Run kubectl get pod -n argocd until all the pods show that they\u0026rsquo;re ready - it should look similar to this:\n$ kubectl get pod -n argocd NAME READY STATUS RESTARTS AGE argocd-application-controller-0 1/1 Running 0 4m25s argocd-applicationset-controller-5f6775c8f7-pc6mt 1/1 Running 0 4m23s argocd-dex-server-7c454cdd97-x7b2g 1/1 Running 2 (4m5s ago) 4m25s argocd-notifications-controller-6485c849d7-9s6zq 1/1 Running 0 4m25s argocd-redis-6cf948f6d6-vqjx8 1/1 Running 0 4m25s argocd-repo-server-5b584fdbb8-pr6lg 1/1 Running 0 4m25s argocd-server-849bd4596-b52z4 1/1 Running 0 4m25s Create a GitHub token Argo CD reads its configuration from git repositories, so it needs a token to read your private repositories. These instructions were valid as on 2026-02-01. You don\u0026rsquo;t need to create a token if you\u0026rsquo;re only going to use public git repositories.\nGo to GitHub and select Settings, then Developer Settings at the bottom of the sidebar on the left Select Fine-grained-tokens, then Generate new token at the right side of the page. Give it a token name and description, then set an expiration time Time to decide on Repository Access level. I picked All repositories, but you can restrict it to just public repositories or even specific repositories Click the Add permissions button and grant Contents permission, then specify access - I didn\u0026rsquo;t want there to be any chance of it changing things behind my back so I gave it Read-only access. Connect to your Argo CD instance Before you connect to the web ui, you need to find the random password generated for the admin user when ArgoCD was installed.\nkubectl get secret argocd-initial-admin-secret -n argocd \\ -o jsonpath=\u0026#34;{.data.password}\u0026#34; | base64 -d Set up an HTTPRoute for argocd For quick debugging, you can use port forwarding to connect to Argo\u0026rsquo;s web UI.\nkubectl -n argocd port-forward svc/argocd-server 8080:80 8443:443 Now you can log into https://localhost:8080\nLong term though, it\u0026rsquo;s cleaner to add an HTTPRoute.\nIf you followed part 2, you have cert-manager set up in your cluster and a default Gateway. Here\u0026rsquo;s a HTTPRoute that will create an SSL certificate for argo.example.com and route any traffic to argo.example.com to your Argo CD.\napiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: argo-ssl-route namespace: argocd spec: parentRefs: - name: default-gateway namespace: default sectionName: https hostnames: - \u0026#34;argo.example.com\u0026#34; rules: - backendRefs: - name: argocd-server port: 80 Now you can open https://argo.example.com in your browser and have a secure SSL connection.\nFirst, reset the password Click User Info in the left sidebar\nOnce you\u0026rsquo;ve changed your password and added it to your password manager, you should delete the argocd-initial-admin-secret secret. It will be regenerated if you force a password reset in the future.\nkubectl -n argocd delete secret/argocd-initial-admin-secret Create an Argo CD chart We want Argo to be able to update or roll itself back, so let\u0026rsquo;s create an app for it and add that.\nCreate the argocd application in your Argo CD configuration repository Create git a repository for your Argo CD configuration. Add an argocd top-level directory to it. We\u0026rsquo;re going to create a helm chart for it, so you\u0026rsquo;ll need to create three files in the argocd directory - Chart.yaml, Chart.lock, and values.yaml.\nYou can copy them from my bootstrap-argocd repository, or fork that repository.\nHere are the manifest source files\nChart.yaml # Chart.yaml apiVersion: v2 name: argocd description: Argo CD Helm Chart type: application version: 9.3.7 dependencies: - name: argo-cd version: 9.3.7 repository: https://argoproj.github.io/argo-helm Chart.lock # Chart.lock dependencies: - name: argo-cd repository: https://argoproj.github.io/argo-helm version: 9.3.7 digest: sha256:688d3b74c5750ef87ff3cafd13b5879c8e7cda7be17ab252a289f3c24591ff0b generated: \u0026#34;2026-01-28T00:02:16.172701-07:00\u0026#34; values.yaml You could specify all the values you got when you ran helm show values argo/argo-cd --version 9.3.7 \u0026gt; values.yaml, but I prefer to only override the values I care about. If other values change in future versions, I want them automatically updated when I update argo.\n--- # values.yaml argo-cd: nameOverride: argocd configs: params: server.insecure: true Commit all the files, then push to GitHub.\nMake ArgoCD manage itself For these instructions, I\u0026rsquo;m going to use an example repository I set up on GitHub. Replace this url https://github.com/unixorn-exemplars/bootstrap-argocd.git with your repository url.\nAdd a new argocd app Log into your Argo CD instance and select New App from the upper left corner of the UI.\nSet the Application name to argocd Select default from the Project Name dropdown menu Leave the Sync policy as Manual for now Set the repository url to https://github.com/unixorn-exemplars/bootstrap-argocd.git Leave the revision as HEAD Set the path to argocd Since we\u0026rsquo;re running Argo on the same cluster we want to manage, set the cluster URL to https://kubernetes.default.svc Set the namespace to argocd. We want to use the same namespace that we had helm install argo into so that when we sync, it reuses the same resources that were installed by helm instead of making new ones. Click create. It should look similar to this:\nIf your configuration repository is a private repository, you will have to add your GitHub username and set the password to the PAT you created earlier.\nClick Create.\nSyncing the App Right now your app has been added, but until you sync it, it won\u0026rsquo;t be active. You should see\nIf you click on the app tile, it\u0026rsquo;ll show you a detailed map of what resources the app creates\nGo ahead and click Sync.\nBecause you\u0026rsquo;ve already installed argo with helm, the resources already exist and it should only take a few seconds to show that it is synced.\nEnabling automatic syncing One last thing to configure - right now, Argo will only sync when you explicitly tell it to. That\u0026rsquo;s not much good for gitops though, we want argo to detect when changes are merged to main and apply them to the cluster automatically.\nClick details, then scroll down to the bottom of the panel until you see sync policy\nEnable auto-sync, then select Prune Resources and Self Heal, and save. By default, Argo CD checks every 120 seconds, with a random offset of up to 60 seconds to prevent simultaneously polling multiple apps at once.\nPrune Resources tells argo that if a new version of the app no longer includes a resource, that resource should eb deleted during upgrade.\nSelf Heal tells argo that whenever it detects a deviation from the app\u0026rsquo;s configuration, it should correct it - not only during upgrades, all of the time.\nSummary You have:\nInstalled Argo CD in your cluster so that you can use it for gitops practices Set up an HTTPRoute so you can securely access your Argo CD instance with SSL Created a configuration GitHub repository you can use with Argo CD to install and configure apps in your cluster Created an argocd directory in that GitHub directory Configured Argo CD to use the argocd app in your repository to manage itself Configured Argo CD to automagically correct any deviation from the spec defined in the argocd directory ","permalink":"https://unixorn.github.io/post/homelab/k8s/05-install-argocd/","summary":"\u003cp\u003eIn part five of my kubernetes homelab series, we will install Argo CD into a homelab cluster so we can use gitops practices instead of ad-hoc \u003ccode\u003ekubectl\u003c/code\u003e commands.\u003c/p\u003e\n\u003cp\u003eThis will make it much easier to recreate the cluster exactly if we decide to move it to new hardware or just want to reset things to a known-good state after experimentation.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"../01-talos-with-cilium-cni-on-proxmox/\"\u003ePart 1 - Setting up Talos with a Cilium CNI on proxmox\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"../02-k8s-cilium-r53-and-cert-manager/\"\u003ePart 2 Add SSL to Kubernetes using Cilium, cert-manager and LetsEncrypt with domains hosted on Amazon Route 53\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"../03-secret-management-with-sops/\"\u003ePart 3 - Secret Management with SOPS\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"../04-backup-talos-etcd-to-smb/\"\u003ePart 4 - Back up your Talos etcd cluster to a SMB share\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ePart 5 - Install Argo CD\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"../06-install-mqtt/\"\u003ePart 6 - Install MQTT into a k8s cluster\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"why-bother-with-gitops-its-a-homelab-and-im-the-only-administrator\"\u003eWhy bother with gitops? It\u0026rsquo;s a homelab, and I\u0026rsquo;m the only administrator\u003c/h2\u003e\n\u003cp\u003eUsing gitops to configure your cluster instead of running \u003ccode\u003ekubectl apply\u003c/code\u003e commands directly makes it possible to reproduce the exact state of the cluster later if you need to revert to a known-good state.\u003c/p\u003e","title":"Install Argo CD in a homelab cluster"},{"content":"In this post, I will show how to access smb shares outside the cluster from a Kubernetes Pod. The example is backing up the etcd cluster in my Talos k8s cluster to a share, but you can use this for any service (like Plex or Jellyfin) that need access to files on a NAS.\nThis is part four of my Homelab Kubernetes series.\nPart 1 - Setting up Talos with a Cilium CNI on proxmox Part 2 Add SSL to Kubernetes using Cilium, cert-manager and LetsEncrypt with domains hosted on Amazon Route 53 Part 3 - Secret Management with SOPS Part 4 - Back up your Talos etcd cluster to a SMB share Part 5 - Install ArgoCD Part 6 - Install MQTT into a k8s cluster The Kubernetes project has created a standardized API specification to make it possible for third parties to write plugins that allow clusters to interact with file or block based storage systems without having to have code merged into the core project.\nIn this article, we\u0026rsquo;re going to:\nInstall the SMB CSI Driver Create a StorageClass, PersistentVolume and PersistentVolumeClaim Used a demo Deployment to confirm things are working Set up a backups PersistentVolume and PersistentVolumeClaim Create a CronJob that backs up the talos etcd cluster every night at 1:11 am Prerequisites A working kubernetes cluster. I\u0026rsquo;m using Talos for mine, but regular kubernetes or k3s clusters will work too. If you need to set up a new cluster, or configure an existing one to use Cilum, read part one of this series. helm \u0026amp; kubectl - if you don\u0026rsquo;t want to brew install them, install instructions are at helm.sh and kubectl. A SMB share on a NAS A separate user on the NAS that the cluster will use to access the share. Don\u0026rsquo;t just use your main NAS account, you want to be able to restrict what the cluster can do and which shares it can access. Software Versions Here are the versions of the software I used while writing this post. Later versions should work, but this is what these instructions were tested with.\nSoftware Version helm 4.0.1 kubectl 1.34 kubernetes 1.34.1 talos 1.11.5 SMB CSI Driver for Kubernetes 1.19.1 Installation Install with helm We\u0026rsquo;re going to use helm to install the SMB CSI Driver for Kubernetes from kubernetes-csi/csi-driver-smb.\n# Install the helm repository and CSI driver helm repo add csi-driver-smb https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/charts helm install csi-driver-smb csi-driver-smb/csi-driver-smb --namespace kube-system --version 1.19.1 Confirm pods are running kubectl --namespace=kube-system get pods --selector=\u0026#34;app.kubernetes.io/name=csi-driver-smb\u0026#34; You should see one copy of the csi-smb-controller pod, and one csi-smb-node-SOMETHING for each node in your cluster. On my homelab main talos cluster it looks like this:\nNAME READY STATUS RESTARTS AGE csi-smb-controller-66588dccff-4lfkt 4/4 Running 3 (7d6h ago) 7d16h csi-smb-node-5q22s 3/3 Running 3 (7d16h ago) 7d16h csi-smb-node-hzrss 3/3 Running 0 6d5h csi-smb-node-ms4gh 3/3 Running 0 6d6h csi-smb-node-nhk84 3/3 Running 0 7d16h Configuration Create a namespace We\u0026rsquo;re going to put all the resources used in this post into a separate backups namespace, so create it now.\nkubectl create namespace backups Create a secret We need to store login credentials for the server we\u0026rsquo;re going to connect to. We\u0026rsquo;re going to use sops (see part three - Secret Management with SOPS for install instructions) to create a secret containing the username and password.\nUsing SOPS # backups-smb-sopssecret.yaml apiVersion: isindir.github.com/v1alpha3 kind: SopsSecret metadata: name: backups-smb-sopssecret namespace: backups spec: secretTemplates: - name: backups-smb-secret stringData: username: k8s password: connect-to-nas Encrypt it, then apply it\nsops encrypt -i backups-smb-sopssecret.yaml \u0026amp;\u0026amp; \\ kubectl apply -f backups-smb-sopssecret.yaml` Insecure Alternative Alternatively we can define login credentials using a Secret manifest file. To do that, we\u0026rsquo;re going to specify the username and password as stringData fields so we don\u0026rsquo;t have to encode them ourselves with base64.\n# insecure-secret.yaml apiVersion: v1 kind: Secret metadata: name: backups-smb-secret namespace: backups type: Opaque stringData: username: k8s password: your-password-here and add it by running kubectl apply -f insecure-secret.yaml\nUsing the SMB Storage Class Now that we have our server credentials in a secret, we can set up a storageClass that connects to our server.\n# smb-storage-class.yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: smb-csi provisioner: smb.csi.k8s.io parameters: source: \u0026#34;//your.servers.fqdn.or.ip.address/blog-demo\u0026#34; # Define Samba share csi.storage.k8s.io/provisioner-secret-name: \u0026#34;backups-smb-secret\u0026#34; csi.storage.k8s.io/provisioner-secret-namespace: \u0026#34;backups\u0026#34; createSubDir: \u0026#34;true\u0026#34; # Creates subdirectories for each PersistentVolumeClaim csi.storage.k8s.io/node-stage-secret-name: \u0026#34;backups-smb-secret\u0026#34; # Define Samba credential secret csi.storage.k8s.io/node-stage-secret-namespace: \u0026#34;backups\u0026#34; # Define Samba credential secret namespace # Uncomment these two lines to make this the default storage class # for your cluster # annotations: # storageclass.kubernetes.io/is-default-class: \u0026#34;true\u0026#34; mountOptions: - dir_mode=0777 - file_mode=0777 - mfsymlinks - vers=3.0 # Define Samba version reclaimPolicy: Delete volumeBindingMode: Immediate allowVolumeExpansion: true Create a PersistentVolumeClaim This manifest will create a PVC with a 1Gi quota.\napiVersion: v1 kind: PersistentVolumeClaim metadata: name: smb-pvc spec: accessModes: - ReadWriteMany storageClassName: smb-csi resources: requests: storage: 1Gi Create a pod that uses your pvc apiVersion: v1 kind: Pod metadata: name: smb-test-pod spec: containers: - name: app image: busybox # talos requires resources specified instead of letting k8s YOLO them resources: requests: memory: \u0026#34;64Mi\u0026#34; cpu: \u0026#34;250m\u0026#34; limits: memory: \u0026#34;64Mi\u0026#34; command: - /bin/sh - -c - | echo \u0026#39;Demo smb-csi\u0026#39; \u0026gt; /mnt/smb/csi-demo.txt date \u0026gt;\u0026gt; /mnt/smb/csi-demo.txt echo \u0026#39;Sleeping 300 seconds so it stays visible in kubectl get pods\u0026#39; sleep 300 volumeMounts: - mountPath: \u0026#34;/mnt/smb\u0026#34; name: smb-volume volumes: - name: smb-volume persistentVolumeClaim: claimName: smb-pvc You can now see the pvc and pod in the cluster.\nkubectl get pod,pvc NAME READY STATUS RESTARTS AGE pod/smb-test-pod 1/1 Running 0 34s NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE persistentvolumeclaim/smb-pvc Bound pvc-5e3acec4-af9e-4cbd-8613-10c64ad7ebaa 1Gi RWX smb-csi \u0026lt;unset\u0026gt; 36s And if I log into the NAS, I can see the pvc subdirectory in my blog-demo share.\n$ cd /volume2/blog-demo $ ls -lah Permissions Size User Date Modified Name drwxrwx--- - admin 20 Jan 00:30 #recycle drwxrwxrwx@ - root 6 Jan 23:22 @eaDir drwxrwxrwx - talos 25 Jan 13:51 pvc-5e3acec4-af9e-4cbd-8613-10c64ad7ebaa $ cat pvc-5e3acec4-af9e-4cbd-8613-10c64ad7ebaa/csi-demo.txt Demo smb-csi Sun Jan 25 20:56:51 UTC 2026 Practical Usage: Back up Talos\u0026rsquo; etcd cluster Now that we\u0026rsquo;ve confirmed our configuration can connect to our NAS, let\u0026rsquo;s set up backups for the etcd cluster. These examples are Talos-specific, but still a good example of using SMB storage in cluster cron jobs.\nSet up the PersistentVolue and PersistentVolumeClaim to use for backups First, create a PersistentVolume that connects to our backups share.\nIMPORTANT: You must set persistentVolumeReclaimPolicy to Retain so the CSI doesn\u0026rsquo;t delete the pvc\u0026rsquo;s remote directory if we delete the pvc or we could accidentally wipe our backups volume!\nSince we\u0026rsquo;re doing static provisioning, we also need to set storageClassName to \u0026quot;\u0026quot; in both the PersistentVolume and the PersistentVolumeClaim that uses it.\n# backups.smb.setup.yaml apiVersion: v1 kind: PersistentVolume metadata: name: pv-backups-share namespace: backups spec: capacity: storage: 10Gi # Set an appropriate size accessModes: - ReadWriteMany # Allows the volume to be mounted by many nodes # IMPORTANT: Set persistentVolumeReclaimPolicy to Retain so the CSI doesn\u0026#39;t # delete the pvc\u0026#39;s remote directory if we delete the pvc or we could # accidentally wipe our backups volume! persistentVolumeReclaimPolicy: Retain storageClassName: \u0026#34;\u0026#34; # Must be an empty string for static provisioning csi: driver: smb.csi.k8s.io readOnly: false volumeHandle: k8s-backups-smb-unseen # A unique name for the volume handle volumeAttributes: source: \u0026#34;\\\\\\\\unseen.miniclusters.rocks\\\\k8s-backups\u0026#34; # Use escaped backslashes or forward slashes nodeStageSecretRef: name: backups-smb-secret namespace: backups # The namespace where you created the secret --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: pvc-backups-share namespace: backups spec: accessModes: - ReadWriteMany resources: requests: storage: 10Gi storageClassName: \u0026#34;\u0026#34; # Must match the empty string in the PV volumeName: pv-backups-share --- Create a talosconfig secret The backup script needs to run talosctl to extract a backup from the cluster\u0026rsquo;s etcd, so let\u0026rsquo;s create a secret containing it.\nkubectl create secret generic backups-talosconfig \\ --from-file talosconfig \\ --namespace backups Our CronJob will mount that secret into the backup pod as a file at /etc/talos/talosconfig.\nCreate a CronJob We\u0026rsquo;re going to use my unixorn/talosctl-backup image to run my tal-backup-etcd script to backup etcd. Source for both the docker image and tal-backup-etcd can be found on Github at unixorn/talosctl-etcd-backups.\nBy default, after a successful backup, tal-backup-etcd deletes any backups in the backup directory older than $KEEP_DAYS old.\n# etcd-backup.cronjob.yaml apiVersion: batch/v1 kind: CronJob metadata: name: cron-backup-void-etcd namespace: backups spec: # Standard cron syntax: minute hour day-of-month month day-of-week schedule: \u0026#34;11 1 * * *\u0026#34; timeZone: \u0026#34;America/Denver\u0026#34; successfulJobsHistoryLimit: 5 failedJobsHistoryLimit: 10 concurrencyPolicy: Forbid # Prevents a new backup if the old one is still running jobTemplate: spec: backoffLimit: 4 ttlSecondsAfterFinished: 604800 # Keep jobs for a week so I can examine their logs template: spec: restartPolicy: OnFailure containers: - name: app image: unixorn/talosctl-backup:latest env: - name: TZ value: \u0026#34;America/Denver\u0026#34; - name: BACKUP_D value: /mnt/backups/etcd-backups - name: KEEP_DAYS value: \u0026#34;30\u0026#34; - name: PRUNE_OK value: \u0026#34;true\u0026#34; - name: TALOSCONFIG value: \u0026#34;/etc/talos/talosconfig\u0026#34; - name: CONTROLPLANE_IP value: 10.9.8.7 resources: requests: memory: \u0026#34;64Mi\u0026#34; cpu: \u0026#34;250m\u0026#34; limits: memory: \u0026#34;64Mi\u0026#34; command: [ \u0026#34;sh\u0026#34;, \u0026#34;-c\u0026#34;, \u0026#34;/usr/local/bin/tal-backup-etcd ; ls -lah /mnt/backups/void-etcd\u0026#34; ] volumeMounts: - name: backups-volume mountPath: \u0026#34;/mnt/backups\u0026#34; - name: talosconfig mountPath: \u0026#34;/etc/talos\u0026#34; readOnly: true volumes: - name: backups-volume persistentVolumeClaim: claimName: pvc-backups-share - name: talosconfig secret: secretName: backups-talosconfig You can now install the CronJob with kubectl apply -f etc-backup.cronjob.yaml\nBonus Job manifest for immediate backups Having a cluster CronJob is great, but you probably don\u0026rsquo;t want to wait an hour to see if it works. It\u0026rsquo;s also handy to be able to run a backup immediately if you\u0026rsquo;re planning to do an experiment that might break your etcd.\nHere\u0026rsquo;s a Job manifest you can use to test or run backups at arbitrary times.\n# immmediate-backup.yaml apiVersion: batch/v1 kind: Job metadata: name: backups-void-etcd namespace: backups spec: ttlSecondsAfterFinished: 604800 # Keep results around for a week backoffLimit: 4 template: spec: restartPolicy: OnFailure containers: - name: app image: unixorn/talosctl-backup env: - name: TZ value: \u0026#34;America/Denver\u0026#34; - name: BACKUP_D value: /mnt/backups/etcd-backups - name: KEEP_DAYS value: \u0026#34;30\u0026#34; - name: PRUNE_OK value: \u0026#34;true\u0026#34; - name: TALOSCONFIG value: \u0026#34;/etc/talosconfig/talosconfig\u0026#34; - name: CONTROLPLANE_IP value: 10.9.8.7 resources: requests: memory: \u0026#34;64Mi\u0026#34; cpu: \u0026#34;250m\u0026#34; limits: memory: \u0026#34;64Mi\u0026#34; command: [ \u0026#34;sh\u0026#34;, \u0026#34;-c\u0026#34;, \u0026#34;/usr/local/bin/tal-backup-etcd ; ls -lah /mnt/backups/etcd-backups\u0026#34; ] volumeMounts: - name: backups-volume mountPath: \u0026#34;/mnt/backups\u0026#34; - name: talosconfig mountPath: \u0026#34;/etc/talosconfig\u0026#34; readOnly: true volumes: # Moved inside pod spec - name: backups-volume persistentVolumeClaim: claimName: pvc-backups-share - name: talosconfig secret: secretName: backups-talosconfig Use kubectl apply -f immediate-backup.yaml to run it.\nSummary You should now have:\nInstalled the SMB CSI driver into your cluster Created a StorageClass that stores files on a NAS Created a Deployment that shows how to keep a pod\u0026rsquo;s work files on an SMB share Created a PersistentVolume that connects to a backups share and does not delete files after pvcs using it have been deleted from your cluster Created a PersistentVolumeClaim that backup jobs can use to write backups to your SMB server Created a CronJob and a Job that you can use to back up your etcd cluster both via cron and at arbitrary times. Backed up your etcd ","permalink":"https://unixorn.github.io/post/homelab/k8s/04-backup-talos-etcd-to-smb/","summary":"\u003cp\u003eIn this post, I will show how to access smb shares outside the cluster from a Kubernetes Pod. The example is backing up the etcd cluster in my Talos k8s cluster to a share, but you can use this for any service (like Plex or Jellyfin) that need access to files on a NAS.\u003c/p\u003e\n\u003cp\u003eThis is part four of my Homelab Kubernetes series.\u003c/p\u003e","title":"Back up your Talos etcd cluster to a smb share"},{"content":"This is part 3 of my Kubernetes homelab cluster setup series - Secrets Management with SOPS.\nPart 1 - Setting up Talos with a Cilium CNI on proxmox Part 2 Add SSL to Kubernetes using Cilium, cert-manager and LetsEncrypt with domains hosted on Amazon Route 53 Part 3 - Secret Management with SOPS Part 4 - Back up your Talos etcd cluster to a SMB share Part 5 - Install ArgoCD Part 6 - Install MQTT into a k8s cluster The cluster is up, but it isn\u0026rsquo;t very usable yet. Before we add any services, we need to set up secrets management.\nIn this post, we\u0026rsquo;re going to add secret management to the cluster with sops and age so we can safely check our configuration into git.\nPrerequisites A working kubernetes cluster. I\u0026rsquo;m using Talos for mine, but regular kubernetes or k3s clusters will work too. If you need to set up a new cluster, or configure an existing one to use Cilum, read part one of this series. cilium, kubectl \u0026amp; helm - if you don\u0026rsquo;t want to brew install them, install instructions are at cilium.io, helm.sh and kubectl. sops and age. On a Mac, you can run brew install sops age. If you\u0026rsquo;re using Linux or Windows, use the age installation instructions and sops installation instructions. Goal I have my cluster configuration in git so that it\u0026rsquo;s easy to recreate if I break something while experimenting. I don\u0026rsquo;t want to commit secrets into git in cleartext though. Instead, I want to encrypt secrets in a way that the cluster can decrypt them, but they\u0026rsquo;re safe to check into source control.\nTo do this, we\u0026rsquo;re going to use sops to encrypt the sensitive parts of our manifest files.\nWhy sops and not something else? Sops supports encrypting individual keys in yaml, json, env and ini files. It will let you encrypt with many of the cloud services like AWS KMS or GCP KMS (and others, check the site), but since we\u0026rsquo;re doing this in a home lab, we\u0026rsquo;re going to use age. It also supports using GPG, but age is what the sops maintainers recommend, and it\u0026rsquo;s a lot more user friendly than GPG.\nSoftware Versions Here are the versions of the software I used while writing this post. Later versions should work, but this is what these instructions were tested with.\nSoftware Version age 1.3.1. helm 4.0.1 kubectl 1.34 kubernetes 1.34.1 sops 3.11.0 talos 1.11.5 Set up age First, generate a key pair Create a key pair with age-keygen\n$ age-keygen # created: 2026-01-18T18:12:39-07:00 # public key: age1234567890ABCDEF1234567890ABCDEF1234567890ABCDEF1234567890A AGE-SECRET-KEY-1234567890ABCDEF1234567890ABCDEF1234567890ABCDEF1234567890A $ It does not create any files. It\u0026rsquo;s up to you to store both the secret and private key. Don\u0026rsquo;t commit the secret key or there\u0026rsquo;s no point in encrypting.\nmacOS On my Mac, I store the secret key in the system keychain, using keychainctl from my tumult macOS cli helper script collection.\nUse keychainctl to add the private key with\nkeychainctl set cluster-age-secret-key password data for new item: Paste in your secret key.\nsops allows you to specify a command to run to retrieve the secret key - it will run it and read the key from the command\u0026rsquo;s stdout.\nWe can use keychainctl to retrieve the private key when decrypting a file with sops so we don\u0026rsquo;t have to store the file on disk unencrypted by adding SOPS_AGE_KEY_CMD to our environment\nexport SOPS_AGE_KEY_CMD=keychainctl get cluster-age-secret-key Linux sops will automatically look in $HOME/.config/sops/age/keys.txt (or $XDG_CONFIG_HOME/sops/age/keys.txt if set) for the age secret key. If you want to use an alternate location, set SOPS_AGE_KEY_FILE in your environment. This unfortunately leaves your key decrypted on the filesystem.\nIf you\u0026rsquo;re using a password manager, I recommend storing the secret key in that and setting SOPS_AGE_KEY_CMD to something that will print the key as shown in the macOS section above.\nYou can also store the age private key in the SOPS_AGE_KEY environment variable.\nSet up sops Now that you have an age key pair, let\u0026rsquo;s configure sops to use it.\nCreate .sops.yaml sops stores its configuration in .sops.yaml. At the top level of your git checkout, .sops.yaml with these contents:\n# .sops.yaml creation_rules: - path_regex: /*?.yaml encrypted_regex: \u0026#34;^(id|password|password_file|app-password|web-password|secret|secretboxencryptionsecret|bootstraptoken|secretboxencryption|token|ca|crt|key|access-key-id|secret-access-key|hostedZoneID|email|data|stringdata|api-token|encryption-token|encryption-key)$\u0026#34; age: age1234567890ABCDEF1234567890ABCDEF1234567890ABCDEF1234567890A This file contains keys to control sops\u0026rsquo; behavior.\nKey Description path_regex Let sops know what files it should examine for encryptable keys encrypted_regex A golang regex that tells sops which keys in our files need encryption age The public key to use for encrypting the secrets Add the sops operator to your cluster We\u0026rsquo;re going to add the sops operator to the cluster. This will handle automatically decrypting keys in manifests so you don\u0026rsquo;t have to decrypt them before running kubectl apply -f\nAdd the age private key to the cluster Put the private key in cluster-age-private.key, then run\nkubectl create secret generic sops-age-key-file \\ --from-file=key=cluster-age-private.key \\ --namespace sops Add an operator configuration Create a values file we can use with helm.\n--- # sops-values.yaml secretsAsFiles: - mountPath: /etc/sops-age-key-file name: sops-age-key-file secretName: sops-age-key-file extraEnv: - name: SOPS_AGE_KEY_FILE value: /etc/sops-age-key-file/key Install the sops operator Now that we have created a secret containing the age private key and a values file to configure the operator, we can install it with helm.\nhelm repo add sops https://isindir.github.io/sops-secrets-operator \u0026amp;\u0026amp; \\ helm update \u0026amp;\u0026amp; \\ helm upgrade --install sops sops/sops-secrets-operator \\ --namespace sops --create-namespace \\ -f sops-values.yaml Usage examples Now that the operator is in place, here are some usage examples. Note - never edit the files after sops has encrypted them. Not even the unencrypted keys - sops calculates a checksum of the entire file and your edits will make the validation fail.\nInstead, run sops decrypt -i something.yaml, edit it, then run sops encrypt -i something.yaml to re-encrypt it.\nEncrypting keys in place Here\u0026rsquo;s an example ClusterIssuer manifest I use to configure cert-manager to use LetsEncrypt and Route 53 DNS challenges.\n# example.yaml apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: letsencrypt-dns-r53 spec: acme: # Use the production server for real certificates; switch to the staging URL while testing. server: https://acme-v02.api.letsencrypt.org/directory # staging: https://acme-staging-v02.api.letsencrypt.org/directory email: user@example.com privateKeySecretRef: name: letsencrypt-dns-r53-account-key solvers: - dns01: route53: # Region where your hosted zone lives region: us-east-1 hostedZoneID: Z1234567890AB accessKeyIDSecretRef: name: route53-aws-secret key: access-key-id secretAccessKeySecretRef: name: route53-aws-secret key: secret-access-key I have my cluster configuration on GitHub, but I don\u0026rsquo;t want my hostedZoneID visible to people trying to use my manifests as an example. I also want to make sure that if they copy the file to use to start their configuration for their own cluster they don\u0026rsquo;t accidentally use my email.\nThe .sops.yaml example I gave is configured to encrypt email and hostedZoneID keys, so let\u0026rsquo;s encrypt this with sops encrypt -i example.yaml. We\u0026rsquo;re using -i so that sops encrypts the file in place, otherwise you\u0026rsquo;d have to run something like sops encrypt example.yaml \u0026gt; example.encrypted.yaml.\nThe resulting file will look something like this:\n# example.yaml apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: letsencrypt-dns-r53 spec: acme: # Use the production server for real certificates; switch to the staging URL while testing. server: https://acme-v02.api.letsencrypt.org/directory # staging: https://acme-staging-v02.api.letsencrypt.org/directory email: ENC[AES256_GCM,data:Ds7Zyk8e7DNCFcqrIHUvgA==,iv:h3MatsiOfK7IEG/kPY1QexGfOGtq/npxPovrFz4arDs=,tag:VOSyK4loBzk7+XfhpIUczg==,type:str] privateKeySecretRef: name: letsencrypt-dns-r53-account-key solvers: - dns01: route53: # Region where your hosted zone lives region: us-east-1 hostedZoneID: ENC[AES256_GCM,data:H52qvjdhqf2zmgyK2A==,iv:zI+N6nH17lch9H756+hABpRc28A/E82wxvaoaLB4RRw=,tag:AATtcJLJlggJjWheIpoiSQ==,type:str] accessKeyIDSecretRef: name: route53-aws-secret key: ENC[AES256_GCM,data:RzULmAT9/ZvgO2fLqw==,iv:Afyhbn8z9n+1Nf1LxsttGbmbncUDoNLJPHZY21LN1zk=,tag:0sH9ZX41xgW7nvE4kJM1VA==,type:str] secretAccessKeySecretRef: name: route53-aws-secret key: ENC[AES256_GCM,data:dyXBYf0R1XUUWejHH2uQyso=,iv:hKxW7otsTkF5f5/kXbAa0Yw0WPxa13NrN8N1Lo/Nbfo=,tag:21FyiXBxd1Bor9SARXaW7g==,type:str] sops: age: - recipient: age1t2fr4u6yvfja69s59rwt70rwa7vhu06sen0m6lxygsyn2dpgzc4s0ma7wy enc: | -----BEGIN AGE ENCRYPTED FILE----- YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBUUWNoZzFMUmdQQTI0RE8w VC8vQzQyV3VTdEhtTkdNS3dwcENKM3hkcjBFCnhLN2xDUVUvQWlLRTV5ZkRyd0lX YUZsWEZseGlrY0hOZUI1Zm1xYnZMWE0KLS0tIGVBdE0vVkdFUEJYcUZwRm1Icy9h OWd4TUZKL2NIOXI5SjRITmhHSkZaZTQKn6+S5b8rfGADzTGNazfiH+Li/se/2as3 g6LFWUkDMLq6CCRJoybbGFl/K/HdT+Eni3WMAL0ZYWIn5gh3qD2LRQ== -----END AGE ENCRYPTED FILE----- lastmodified: \u0026#34;2026-01-19T19:58:51Z\u0026#34; mac: ENC[AES256_GCM,data:dSgG36sbNaAqo1TyCEqWBVveniUNKHpsBT5A0A5wto6A+ohA6OmX3w/soGn0T4NU+5dIs8s0gJWZpR67rtwKu9+Wtf1dELw27NGpZu8Jak6zPZwqt0iktkOE7JYdtmyRR3VP3ykkkKOxEd4KoRhLdLkPPC3VrScwzbSX4K0AJVA=,iv:xzNZNTM0rYEf8Btexggmtum6H/iI6iexcIQYSG36zwk=,tag:G1pzGu+6RoC2WyhRfovUMA==,type:str] encrypted_regex: ^(id|password|password_file|app-password|web-password|secret|secretboxencryptionsecret|bootstraptoken|secretboxencryption|token|ca|crt|key|access-key-id|secret-access-key|hostedZoneID|email|data|stringdata|api-token|encryption-token|encryption-key)$ version: 3.11.0 Note that it\u0026rsquo;s added a sops section to the file, the confidential keys are encrypted, but the non-sensitive keys were left alone so the file is still readable by humans, it\u0026rsquo;s just had the secrets redacted.\nCreating k8s secrets with sops What if we want to create actual k8s secrets? Ones with key names that aren\u0026rsquo;t found by the encrypted_regex in .sops.yaml?\nThe sops operator creates a SopsSecret resource. If you create secretTemplates resources in a SopsSecret, the operator will create those keys when you apply the manifest, update them when you apply changes, and delete all secrets created by the SopsSecret when it is deleted.\nHere\u0026rsquo;s an example\n# sample-secret.raw.yaml apiVersion: isindir.github.com/v1alpha3 kind: SopsSecret metadata: name: foo-sopssecret namespace: blog spec: secretTemplates: - name: foo-secret stringData: username: myUsername password: \u0026#39;Pa$$word\u0026#39; - name: some-token stringData: token: 8675309thx1138 After running sops encrypt -i sample-secret.raw.yaml\n# sample-secret.raw.yaml apiVersion: isindir.github.com/v1alpha3 kind: SopsSecret metadata: name: foo-sopssecret namespace: blog spec: secretTemplates: - name: foo-secret stringData: username: myUsername password: ENC[AES256_GCM,data:gz3TVX4rEB8=,iv:ifDsVjQmYbwAHy073rP8uI2NNu073WqOnw/JZ+e6TqI=,tag:tgb0RkiZe6Xlhx4UOItIgg==,type:str] - name: some-token stringData: token: ENC[AES256_GCM,data:paithquCOGuWL0zG/Q4=,iv:7P+yp62d9e4wLb8JUCFwYcO14swntguAr8wwO6z22fc=,tag:VOkL/leu5zT20Ufj5GGskQ==,type:str] sops: age: - recipient: age1t2fr4u6yvfja69s59rwt70rwa7vhu06sen0m6lxygsyn2dpgzc4s0ma7wy enc: | -----BEGIN AGE ENCRYPTED FILE----- YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBJdWNPYmQyTEtvTkZ0OHk3 N2tHNS9sRU5PTml3OFUxUGR2aW9CbXdZMTB3ClhXWi9LbHlDYi82bEliVkNDY3R0 eCs1VUFSZm45Y0ZwaVZ3SzhYWjZ4NmMKLS0tIHF1RGZSY21SZWtkSXZsUG5MQjV4 TlNsZGR3WnVCZ05WWEQzN2tIMjhEQzQKXznPaFWnV8/qx1XGbSio/0XAa5/1SrrI EXm2by6UikdkSyngKk1sgvycM4z+JuvkKoxahH89RYe8ZX+9it4u1w== -----END AGE ENCRYPTED FILE----- lastmodified: \u0026#34;2026-01-19T20:11:54Z\u0026#34; mac: ENC[AES256_GCM,data:yo4wpmN9JXvE0DLjhG87me2470ezpQ1rFt9++yD06HnItyWQQhPRCkF0o8Z5J6JyJ++lM8pZLuzYjqJMy//Gt0neMFI1yPf7NzH3JzlO9fy8qpdFpQ0iyYdzTURNZ1qgZ9+dfeji5bqwHiLYY/GffcJvLPQoJUzD1+FHIWz4hXo=,iv:dmbQ/QKpWg6CXm/3cXNwlg71536SAKCl9CCE0R/I4xM=,tag:p3twWZya0nU/jQ5XSfsolg==,type:str] encrypted_regex: ^(id|password|password_file|app-password|web-password|secret|secretboxencryptionsecret|bootstraptoken|secretboxencryption|token|ca|crt|key|access-key-id|secret-access-key|hostedZoneID|email|data|stringdata|api-token|encryption-token|encryption-key)$ version: 3.11.0 Now I can run\n$ kubectl apply -f sample-secret.raw.yaml sopssecret.isindir.github.com/foo-sopssecret created $ and the operator will decrypt the keys and create the secrets, so when I run\n$ kubectl get secret -n blog foo-secret Opaque 2 11s some-token Opaque 1 11s $ I see the two secrets defined in the SopsSecret resource. And when I delete the SopsSecret, we can see that it deletes the regular kubernetes secrets defined in it.\n$ kubectl delete -n blog sopssecrets.isindir.github.com foo-sopssecret sopssecret.isindir.github.com \u0026#34;foo-sopssecret\u0026#34; deleted $ kubectl get secret -n blog No resources found in blog namespace. $ You should now be able to create secrets and encrypt individual yaml keys for your cluster using SOPS.\nGotchas I can\u0026rsquo;t decrypt a file I encrypted with sops There are two common reasons this can happen\nYou edited the file after you encrypted it. If you opened the encrypted file and saved it, your editor may have trimmed trailing whitespace from lines, which broke the checksumming. You don\u0026rsquo;t have SOPS_AGE_KEY, SOPS_AGE_KEY_FILE or SOPS_AGE_KEY_CMD defined in your environment Kubectl apply failed Confirm that you created the sops-age-key-file secret, and that it\u0026rsquo;s in the same namespace you installed sops.\nkubectl get secret -A | grep sops sops sh.helm.release.v1.sops.v1 helm.sh/release.v1 1 1d sops sops-age-key-file Opaque 1 1d ","permalink":"https://unixorn.github.io/post/homelab/k8s/03-secret-management-with-sops/","summary":"\u003cp\u003eThis is part 3 of my Kubernetes homelab cluster setup series - Secrets Management with \u003ca href=\"https://github.com/getsops/sops\"\u003eSOPS\u003c/a\u003e.\u003c/p\u003e","title":"Secret Management with SOPS"},{"content":"In part two of this homelab kubernetes setup series, we\u0026rsquo;re going to install \u0026amp; configure cert-manager to use LetsEncrypt with Route 53 so we can use SSL to connect to our services.\nPart 1 - Setting up Talos with a Cilium CNI on proxmox Part 2 - Add SSL to Kubernetes using Cilium, cert-manager and LetsEncrypt with domains hosted on Amazon Route 53 Part 3 - Set up Secret Management with SOPS Part 4 - Back up your Talos etcd cluster to a SMB share Part 5 - Install ArgoCD Part 6 - Install MQTT into a k8s cluster The tutorials I\u0026rsquo;ve seen for using cert-manager with a DNS challenge all use CloudFlare. I have my lab domain on Route 53 so this post will cover that instead.\nPre-requisites A domain hosted on Amazon Route 53 that you have administrative rights on. A working kubernetes cluster with Cilium installed and configured to be a Gateway. I\u0026rsquo;m using Talos for mine, but regular kubernetes or k3s clusters will work too. If you need to set up a new cluster, or configure an existing one to use Cilum, read part one of this series. cilium, kubectl \u0026amp; helm - if you don\u0026rsquo;t want to brew install them, install instructions are at cilium.io, helm.sh and kubectl. Software Versions Here are the versions of the software I used while writing this post. Later versions should work, but this is what these instructions were tested with.\nSoftware Version cert-manager 1.19.2 helm 4.0.1 kubectl 1.34 kubernetes 1.34.1 talos 1.11.5 Instructions I don\u0026rsquo;t have any services exposed to the internet, and I\u0026rsquo;m not interested in exposing a web server just to answer HTTP queries by LetsEncrypt, especially when there\u0026rsquo;s an objectively better option like using DNS challenges available.\nHere\u0026rsquo;s how to configure cert-manager to use a DNS01 challenge with LetsSencrypt instead of an HTTP one. This will let LetsEncrypt validate ownership of your domain by checking for DNS records created/updated by cert-manager during certificate creation and renewal. As a bonus, using DNS challenges also lets you create wildcard certificates for your domain.\nCreate an AWS user for cert-manager To validate certificate requests via DNS, cert-manager is going to need to use AWS credentials that have privilege to update DNS domains. Do not use your root IAM account! You should only ever use your root IAM account to create users that have the minimum privilege they need for explicit tasks. So instead, we\u0026rsquo;re going to create a user with access limited to updating domains hosted on Route 53.\nCreate an IAM policy First create a policy that grants control over Route53. Log in to the AWS console and select Identity and Access Management (IAM), then select Policies from the sidebar, and finally, hit the Create Policy button.\nCreating the policy first makes it easier to attach to an IAM group when we create one in the next step.\nI\u0026rsquo;ve already created a working policy, so instead of manually adding permissions, click JSON to the right of where it says Policy Editor and paste in this JSON snippet\n{ \u0026#34;Version\u0026#34;: \u0026#34;2012-10-17\u0026#34;, \u0026#34;Statement\u0026#34;: [ { \u0026#34;Effect\u0026#34;: \u0026#34;Allow\u0026#34;, \u0026#34;Action\u0026#34;: \u0026#34;route53:ListHostedZonesByName\u0026#34;, \u0026#34;Resource\u0026#34;: \u0026#34;*\u0026#34; }, { \u0026#34;Effect\u0026#34;: \u0026#34;Allow\u0026#34;, \u0026#34;Action\u0026#34;: [ \u0026#34;route53:ChangeResourceRecordSets\u0026#34;, \u0026#34;route53:ListResourceRecordSets\u0026#34;, \u0026#34;route53:GetHostedZone\u0026#34; ], \u0026#34;Resource\u0026#34;: [ \u0026#34;arn:aws:route53:::hostedzone/YOUR_DOMAINS_ZONE_ID\u0026#34; ] } ] } As of 2026-01-03, it should look like this:\nClick Next, and give your new policy a name and a description, for example blog-letsencrypt.\nCreate an IAM group You could attach a policy to a user, but using a group makes it easier to create other users with this set of permissions. I have a different IAM user that I use for nginx-proxy-manager on my standalone hosts, and I also use different users for my dev and prod clusters.\nNow create a group, click User Groups in the side bar, then hit the Create Group button. Give it a name like blog-le-users.\nUnder Attach permissions policy* select the blog-letsencrypt policy you just created.\nHit the Create Group button on the bottom right of the page.\nCreate an IAM user Now that the group and policy are created, you can finally create your R53 IAM user. Click Users in the side bar, then click the Add Users button on the right side of the screen.\nGive it a name like r53-acme-user and click Next.\nDon\u0026rsquo;t check the Provide user access to the AWS Management Console checkbox, this user will only be used by cert-manager. Select the blog-le-users group\nClick Next again.\nYou\u0026rsquo;ll see a review and create page:\nClick Create user on the bottom right of the page.\nOne last thing - you\u0026rsquo;ll need to create access credentials for the user. Click on Users in the side bar again, then your brand new r53-acme-user user.\nYou\u0026rsquo;ll see an info page about the user, click the Security Credentials tab\nScroll down to Access Keys, and click Create access key.\nClick Other on the next page\nClick Next. Put in letsencrypt for the description,and click Create access key again.\nYou\u0026rsquo;ll see something like\nYou only get one chance to see the secret key, so store it in your password manager. It isn\u0026rsquo;t a huge deal if you lose it, you can always create another access key for your user. Copy the access key \u0026amp; secret keys into your password manager for later use.\nInstall cert-manager We\u0026rsquo;re going to use helm to install cert-manager, so first we need to add its helm repository.\nAdd the Jetstack repo helm repo add jetstack https://charts.jetstack.io \u0026amp;\u0026amp; helm repo update Get the helm repository gpg keyring The cert-manager team provides a gpg keyring to validate the helm chart when we install it.\ncurl -LO https://cert-manager.io/public-keys/cert-manager-keyring-2021-09-20-1020CF3C033D4F35BAE1C19E1226061C665DF13E.gpg Install cert‑manager and the CRDs it needs Now that we have AWS credentials set up, let\u0026rsquo;s use helm to create a cert-manager namespace, verify the chart using the gpg keyring and install cert-manager along with the CRDs it needs.\nhelm install \\ cert-manager oci://quay.io/jetstack/charts/cert-manager \\ --version v1.19.2 \\ --namespace cert-manager \\ --create-namespace \\ --keyring ./cert-manager-keyring-2021-09-20-1020CF3C033D4F35BAE1C19E1226061C665DF13E.gpg \\ --verify \\ --set crds.enabled=true Confirm installation with helm list helm list --all-namespaces You should see something like\nNAME NAMESPACE REVISION\tUPDATED STATUS CHART APP VERSION cert-manager\tcert-manager\t1 2026-01-03 22:42:45.78814 -0700 MST deployed\tcert-manager-v1.19.2\tv1.19.2 Configure cert-manager Create an AWS credential secret Setting up an external secret provider is out of scope for this post. We\u0026rsquo;re going to store the AWS credentials in a secret. If you want to keep the secret file encrypted, have a look at Part 3 of this series, Set up Secret Management with SOPS.\nFor tidiness, we\u0026rsquo;re going to put all the cert-manager confguration yaml files in a cert-manager directory, so mkdir cert-manager.\nCreate a cert-manager/aws-secret.yaml file.\napiVersion: v1 kind: Secret metadata: name: route53-aws-secret namespace: cert-manager type: Opaque stringData: # Replace these with your actual keys (or use a Kubernetes external secret provider) access-key-id: YOUR_R53_IAM_USER_AWS_ACCESS_KEY_ID secret-access-key: YOUR_R53_IAM_USER_AWS_SECRET_ACCESS_KEY Create a ClusterIssuer Create cert-manager/cluster-issuer.yaml. Make sure you set the email field to your real email so LetsEncrypt can notify you about policy changes or issues.\napiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: letsencrypt-dns-r53 spec: acme: # Use the production server for real certificates; switch to the staging URL while testing. server: https://acme-v02.api.letsencrypt.org/directory # staging: https://acme-staging-v02.api.letsencrypt.org/directory email: your@email.example # \u0026lt;-- change to your real contact address privateKeySecretRef: name: letsencrypt-dns-r53-account-key solvers: - dns01: route53: region: us-east-1 # Region where your hosted zone lives hostedZoneID: \u0026#34;YOUR_ZONE_ID\u0026#34; # Articles I read said this was optional, but when I left it empty cert‑manager didn\u0026#39;t discover it and my certs never got to the ready state accessKeyIDSecretRef: name: route53-aws-secret key: access-key-id secretAccessKeySecretRef: name: route53-aws-secret key: secret-access-key Apply the configuration kubectl apply -f cert-manager # Apply all the yaml files in the directory Confirm that the ClusterIssuer is working Create a standlone certificate with the ClusterIssuer so that we can confirm that LetsEncrypt is working before we try to use it with an Ingress.\n# standalone-certificate.yaml apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: wildcard-yourdomain-com namespace: default spec: secretName: yourdomain-com-tls dnsNames: - \u0026#39;yourdomain.com\u0026#39; - \u0026#39;*.yourdomain.com\u0026#39; # This is a homelab, so I\u0026#39;m ok with using a wildcard certificate for all my services issuerRef: name: letsencrypt-dns-r53 kind: ClusterIssuer Make sure the secretName is unique to this certificate or you\u0026rsquo;ll get mysterious breakages, then apply it.\nkubectl apply -f standalone-certificate.yaml Wait a few seconds, then run\nkubectl get certificate --all-namespaces You should see something like\nNAMESPACE NAME READY SECRET AGE default wildcard-yourdomain-com-cert True yourdomain-com--tls. 1m The first time I made a certificate it took almost a minute, so don\u0026rsquo;t assume things are broken if the certificate doesn\u0026rsquo;t show as ready right away.\nBehind the scenes, cert-manager is creating a certificate, creating the required DNS entries, posting an ACME request to LetsEncrypt, waiting for LetsEncrypt to verify the DNS records and sign your certificate, and then finally loading the signed certfificate. It can take a literal minute if LetsEncrypt is busy.\nIf you run kubectl describe certificate wildcard-your-domain-com you should see something like this:\nName: wildcard-your-domain-cert Namespace: default Labels: \u0026lt;none\u0026gt; Annotations: \u0026lt;none\u0026gt; API Version: cert-manager.io/v1 Kind: Certificate Metadata: Creation Timestamp: 2026-01-03T07:18:54Z Generation: 1 Resource Version: 225621 UID: 12345678-90AB-CDEFGHIJK-LMNOPQRSTUVW Spec: Dns Names: www.yourdomain.com Issuer Ref: Kind: ClusterIssuer Name: letsencrypt-dns-r53 Secret Name: your-domain-com-tls Status: Conditions: Last Transition Time: 2026-01-03T07:19:02Z Message: Certificate is up to date and has not expired Observed Generation: 1 Reason: Ready Status: True Type: Ready Not After: 2026-04-03T06:20:29Z Not Before: 2026-01-03T06:20:30Z Renewal Time: 2026-03-04T06:20:29Z Revision: 1 Events: \u0026lt;none\u0026gt; Great, the cert exists and has status of ready, time to test it.\nCreate a playground workload Create a playground directory, and create playground/01-playground-nginx.yaml\n# 01-playground-nginx.yaml # Create the playground namespace apiVersion: v1 kind: Namespace metadata: labels: kubernetes.io/metadata.name: playground name: playground --- # Create an nginx deployment. apiVersion: apps/v1 kind: Deployment metadata: name: playground-nginx-app namespace: playground spec: selector: matchLabels: app: playground-nginx-pod # \u0026lt;-- must match pod labels exactly replicas: 1 template: metadata: labels: app: playground-nginx-pod spec: # Talos is very security oriented, so we have to set up the # security context explicitly # ---------- Pod‑level security settings ---------- securityContext: runAsNonRoot: true runAsUser: 101 # non‑root UID that the image can run as seccompProfile: type: RuntimeDefault # Uncomment if you need a shared FS group for volume writes # fsGroup: 101 # ---------- Containers ---------- containers: - name: nginx-playground-container image: nginxinc/nginx-unprivileged:latest # Alpine, but we force a non‑root UID imagePullPolicy: IfNotPresent ports: - containerPort: 8080 # talos requires us to specify our resources instead of # letting k8s YOLO them resources: requests: memory: \u0026#34;64Mi\u0026#34; cpu: \u0026#34;250m\u0026#34; limits: memory: \u0026#34;128Mi\u0026#34; cpu: \u0026#34;500m\u0026#34; securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL --- # And finally, create a service apiVersion: v1 kind: Service metadata: name: playground-nginx-service namespace: playground spec: selector: app: playground-nginx-pod ports: - name: http port: 80 # This is what the service is listening on, and what will be routed to targetPort: 8080 # Port the pods are listening on, don\u0026#39;t route directly here! Configure Cilium Create a Gateway We\u0026rsquo;re going to create a gateway that listens to both HTTP and HTTPS traffic. Create playground/02-playground-gateway\nWe\u0026rsquo;re specifying a specific IP address from our pool so that it stays static and we can create a DNS entry for it\n# 02-playground-gateway apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: playground-gateway # Note - we are _not_ putting this in a namespace so our workloads can # share it. spec: gatewayClassName: cilium addresses: - type: IPAddress value: 10.0.1.160 # Force the gateway to grab a specific IP from the pool listeners: - name: http protocol: HTTP port: 80 allowedRoutes: namespaces: from: All - name: https protocol: HTTPS port: 443 allowedRoutes: namespaces: from: All tls: certificateRefs: - kind: Secret name: yourdomain-com-tls Create the gateway now so you can find out what IP address it gets.\nkubectl apply -f playground/02-playground-gateway.yaml \u0026amp;\u0026amp; \\ kubectl get gateway --all-namespaces You\u0026rsquo;ll see something like\nNAMESPACE NAME CLASS ADDRESS PROGRAMMED AGE default playground-gateway cilium 10.0.1.160 True 1h For the rest of these instructions, we\u0026rsquo;re going to assume your gateway is 10.0.1.160.\nGo to Amazon\u0026rsquo;s R53 control panel and create an A record in your domain, ip-160.yourdomain.com with the address 10.0.1.160.\nCreate HTTPRoutes We\u0026rsquo;re going to create two HTTPRoutes, one for HTTP and one for HTTPS. Create playground/03-httproutes.yaml\n# 03-httproutes.yaml apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: playground-http-route namespace: playground spec: parentRefs: - name: playground-gateway namespace: default sectionName: http hostnames: - \u0026#34;ip-160.yourdomain.com\u0026#34; rules: - backendRefs: - name: playground-nginx-service port: 80 --- apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: playground-ssl-route namespace: playground spec: parentRefs: - name: playground-gateway namespace: default sectionName: https hostnames: - \u0026#34;ip-160.yourdomain.com\u0026#34; rules: - backendRefs: - name: playground-nginx-service port: 80 Apply and test the routes Now we\u0026rsquo;re finally ready to create our routes\nkubectl apply -f playground \u0026amp;\u0026amp; \\ kubectl get gateway \u0026amp;\u0026amp; echo \u0026amp;\u0026amp; \\ kubectl get httproute --all-namespaces You should see something like this:\nNAME CLASS ADDRESS PROGRAMMED AGE playground-gateway cilium 10.0.1.160 True 23h NAMESPACE NAME HOSTNAMES AGE playground playground-http-route [\u0026#34;ip-160.yourdomain.com\u0026#34;] 3m playground playground-ssl-route [\u0026#34;ip-160.yourdomain.com\u0026#34;] 3m You should now see the Welcome to nginx! page when you go to either http://ip-160.yourdomain.com or https://ip-160.yourdomain.com.\nBonus - create an http -\u0026gt; https redirect If I go to the http url instead of https, I want to be automagically redirected to https so I don\u0026rsquo;t accidentally send any login credentials on an unencrypted connection.\nUpdate our HTTProutes Our gateway is already configured to listen on both http and https, so we\u0026rsquo;re going to create an HTTPRoute that listens on port 80 and redirects any hits to port 443.\nCreate playground/04-redirect-http-to-https.yaml\n# 04-redirect-http-to-https.yaml kind: HTTPRoute apiVersion: gateway.networking.k8s.io/v1beta1 metadata: name: http-to-https-redirect namespace: default spec: parentRefs: - namespace: default name: playground-gateway sectionName: http rules: - filters: - type: RequestRedirect requestRedirect: scheme: https Now let\u0026rsquo;s replace the old http route\nkubectl delete httproute playground-http-route -n playground \u0026amp;\u0026amp; \\ kubectl apply -f playground/04-redirect-http-to-https.yaml \u0026amp;\u0026amp; \\ curl -iv ip160.yourdomain.com You should see something like this - on line 14 you can see it get a 302 and on line 29 it opens a new connection to port 443.\n* Host ip-160.yourdomain.com:80 was resolved. * IPv6: (none) * IPv4: 10.0.1.160 * Trying 10.0.1.160:80... * Established connection to ip-160.yourdomain.com (10.0.1.160 port 80) from 10.0.1.121 port 64799 * using HTTP/1.x \u0026gt; GET / HTTP/1.1 \u0026gt; Host: ip-160.yourdomain.com \u0026gt; User-Agent: curl/8.17.0 \u0026gt; Accept: */* \u0026gt; * Request completely sent off \u0026lt; HTTP/1.1 302 Found HTTP/1.1 302 Found \u0026lt; location: https://ip-160.yourdomain.com:443/ location: https://ip-160.yourdomain.com:443/ \u0026lt; date: Mon, 05 Jan 2026 00:19:16 GMT date: Mon, 05 Jan 2026 00:19:16 GMT \u0026lt; server: envoy server: envoy \u0026lt; content-length: 0 content-length: 0 * Ignoring the response-body * setting size while ignoring \u0026lt; * Connection #0 to host ip-160.yourdomain.com:80 left intact * Clear auth, redirects to port from 80 to 443 * Issue another request to this URL: \u0026#39;https://ip-160.yourdomain.com:443/\u0026#39; * Host ip-160.yourdomain.com:443 was resolved. * IPv6: (none) * IPv4: 10.0.1.160 * Trying 10.0.1.160:443... * ALPN: curl offers h2,http/1.1 * TLSv1.3 (OUT), TLS handshake, Client hello (1): * SSL Trust Anchors: * Native: Apple SecTrust * OpenSSL default paths (fallback) * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.3 (IN), TLS change cipher, Change cipher spec (1): * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): * TLSv1.3 (IN), TLS handshake, Certificate (11): * TLSv1.3 (IN), TLS handshake, CERT verify (15): * TLSv1.3 (IN), TLS handshake, Finished (20): * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.3 (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / TLS_CHACHA20_POLY1305_SHA256 / x25519 / RSASSA-PSS * ALPN: server did not agree on a protocol. Uses default. * Server certificate: * subject: CN=yourdomain.com * start date: Jan 3 06:20:30 2026 GMT * expire date: Apr 3 06:20:29 2026 GMT * issuer: C=US; O=Let\u0026#39;s Encrypt; CN=R13 * Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption * Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption * Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption * subjectAltName: \u0026#34;ip-160.yourdomain.com\u0026#34; matches cert\u0026#39;s \u0026#34;*.yourdomain.com\u0026#34; * SSL certificate verified via OpenSSL. * Established connection to ip-160.yourdomain.com (10.0.1.160 port 443) from 10.0.1.121 port 64800 * using HTTP/1.x \u0026gt; GET / HTTP/1.1 \u0026gt; Host: ip-160.yourdomain.com \u0026gt; User-Agent: curl/8.17.0 \u0026gt; Accept: */* \u0026gt; * Request completely sent off * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): \u0026lt; HTTP/1.1 200 OK HTTP/1.1 200 OK \u0026lt; server: envoy server: envoy \u0026lt; date: Mon, 05 Jan 2026 00:19:17 GMT date: Mon, 05 Jan 2026 00:19:17 GMT \u0026lt; content-type: text/html content-type: text/html \u0026lt; content-length: 615 content-length: 615 \u0026lt; last-modified: Tue, 28 Oct 2025 12:05:10 GMT last-modified: Tue, 28 Oct 2025 12:05:10 GMT \u0026lt; etag: \u0026#34;6900b176-267\u0026#34; etag: \u0026#34;6900b176-267\u0026#34; \u0026lt; accept-ranges: bytes accept-ranges: bytes \u0026lt; x-envoy-upstream-service-time: 1 x-envoy-upstream-service-time: 1 \u0026lt; \u0026lt;!DOCTYPE html\u0026gt; \u0026lt;html\u0026gt; \u0026lt;head\u0026gt; \u0026lt;title\u0026gt;Welcome to nginx!\u0026lt;/title\u0026gt; \u0026lt;style\u0026gt; html { color-scheme: light dark; } body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } \u0026lt;/style\u0026gt; \u0026lt;/head\u0026gt; \u0026lt;body\u0026gt; \u0026lt;h1\u0026gt;Welcome to nginx!\u0026lt;/h1\u0026gt; \u0026lt;p\u0026gt;If you see this page, the nginx web server is successfully installed and working. Further configuration is required.\u0026lt;/p\u0026gt; \u0026lt;p\u0026gt;For online documentation and support please refer to \u0026lt;a href=\u0026#34;http://nginx.org/\u0026#34;\u0026gt;nginx.org\u0026lt;/a\u0026gt;.\u0026lt;br/\u0026gt; Commercial support is available at \u0026lt;a href=\u0026#34;http://nginx.com/\u0026#34;\u0026gt;nginx.com\u0026lt;/a\u0026gt;.\u0026lt;/p\u0026gt; \u0026lt;p\u0026gt;\u0026lt;em\u0026gt;Thank you for using nginx.\u0026lt;/em\u0026gt;\u0026lt;/p\u0026gt; \u0026lt;/body\u0026gt; \u0026lt;/html\u0026gt; * Connection #1 to host ip-160.yourdomain.com:443 left intact Potential Problems curl to your http port doesn\u0026rsquo;t get redirected You curl to the http port of your host and instead of getting redirected, you see the nginx welcome page.\nMake sure you deleted the old route before you added the redirect. Check with kubectl get httproute --all-namespaces if you see both of them as here\n$ kubectl get httproute -A NAMESPACE NAME HOSTNAMES AGE default http-to-https-redirect 117s playground playground-http-route [\u0026#34;ip-160.yourdomain.com\u0026#34;] 200s playground playground-ssl-route [\u0026#34;ip-160.yourdomain.com\u0026#34;] 200s Run kubectl delete httproute playground-http-route -n playground and try again.\n","permalink":"https://unixorn.github.io/post/homelab/k8s/02-k8s-cilium-r53-and-cert-manager/","summary":"\u003cp\u003eIn part two of this homelab kubernetes setup series, we\u0026rsquo;re going to install \u0026amp; configure \u003ca href=\"https://cert-manager.io/\"\u003ecert-manager\u003c/a\u003e to use \u003ca href=\"https://letsencrypt.org\"\u003eLetsEncrypt\u003c/a\u003e with \u003ca href=\"https://aws.amazon.com/route53/\"\u003eRoute 53\u003c/a\u003e so we can use SSL to connect to our services.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"../01-talos-with-cilium-cni-on-proxmox/\"\u003ePart 1 - Setting up Talos with a Cilium CNI on proxmox\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ePart 2 - Add SSL to Kubernetes using Cilium, cert-manager and LetsEncrypt with domains hosted on Amazon Route 53\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"../03-secret-management-with-sops/\"\u003ePart 3 - Set up Secret Management with SOPS\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"../04-backup-talos-etcd-to-smb/\"\u003ePart 4 - Back up your Talos etcd cluster to a SMB share\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"../05-install-argocd/\"\u003ePart 5 - Install ArgoCD\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"../06-install-mqtt/\"\u003ePart 6 - Install MQTT into a k8s cluster\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe tutorials I\u0026rsquo;ve seen for using cert-manager with a DNS challenge all use CloudFlare. I have my lab domain on \u003ca href=\"https://aws.amazon.com/route53/\"\u003eRoute 53\u003c/a\u003e so this post will cover that instead.\u003c/p\u003e","title":"Add SSL to Kubernetes using Cilium, cert-manager and LetsEncrypt with domains hosted on Amazon Route 53"},{"content":"I\u0026rsquo;ve been meaning to set up a Talos linux kubernetes cluster in my homelab for a while and set one up over the holiday break. Here\u0026rsquo;s how I did it.\nAll the blogs and videos I looked at used the nginx ingress, which would be fine, except that the nginx ingress is a dead man walking and will be unsupported starting in March of 2026. No patches, no security updates, completely unsupported.\nBased on some advice on the hangops slack (Thanks, Brandon!) I wanted to use Cilium since it supports the Gateway API and can also do ARP announcements like MetallB.\nThis is part one of a series I\u0026rsquo;m writing as I get my homelab cluster up and running.\nPart 1 - Creating a Talos cluster with a Cilium CNI on Proxmox Part 2 - Add SSL to Kubernetes using Cilium, cert-manager and LetsEncrypt with domains hosted on Amazon Route 53 Part 3 - Set up Secret Management with SOPS Part 4 - Back up your Talos etcd cluster to a SMB share Part 5 - Install ArgoCD Part 6 - Install MQTT into a k8s cluster Pre-requisites proxmox will make it easier to rebuild your cluster if you make a mistake, but these instructions will work with bare metal as well. cilium, kubectl \u0026amp; helm cli tools. If you don\u0026rsquo;t want to brew install them or are not using a Mac, installation instructions are at cilium.io, helm.sh and kubectl. Software Versions Here are the versions of the software I used while writing this post. Later versions should work, but this is what these instructions were tested with.\nSoftware Version cilium 1.18.6 helm 4.0.1 kubectl 1.34 kubernetes 1.34.1 talos 1.11.5 Let\u0026rsquo;s get started.\nCreate a Proxmox VM control plane node There are a ton of videos and blogs describing getting started with proxmox, so I\u0026rsquo;m not going to go into a lot of detail here. The TL;DR is:\nGo to the Talos image factory. Create an image. Start with the Cloud Server option Select the latest Talos version Pick nocloud from the cloud type screen since it explicitly mentions proxmox in the description Select your architecture You should now be on the System Extensions page. Pick qemu-guest-agent and util-linux-tools Pick auto for the bootloader. You should now see Schematic Ready. Copy the ISO link Download the ISO into your Proxmox host\u0026rsquo;s ISO storage Start a new VM with at least 4 CPUs, 4 GB of RAM and at least a 16GB drive. Make sure you enable the qemu agent. If you\u0026rsquo;re planning to use this for real workloads later, you\u0026rsquo;ll want to go bigger - have a look at Sidero\u0026rsquo;s System Requirements page. I went with 100G per their recommendations. Wait until the Talos Dashboard appears, then copy the new node\u0026rsquo;s IP address On your DHCP server, find the IP from step 6, and set that as a static assignment. The server will reboot multiple times during installation, and if it changes IP you will have to update your kubeconfig and talosconfig files. If it changes IP addresses after you add a worker node, things will break so spare yourself future aggravation and give it a static assignment from the beginning. Once the Talos dashboard on the VM console shows that is in maintenance mode you can start to configure it. Again, make sure your DHCP is assigning it a static IP, that will save you aggravation later.\nWe\u0026rsquo;re going to make a single node cluster to simplify things since it\u0026rsquo;s just for learning. You can add worker nodes later very easily once you want to put real workloads in the cluster.\nConfigure the cluster control plane Setup some environment variables Set CLUSTER_NAME and CONTROL_PLANE_IP environment variables to make copying commands from this post easier.\nexport CLUSTER_NAME=sisyphus export CONTROL_PLANE_IP=10.0.1.51 Find out what disks are on the server The Talos installer needs to know what device is the node\u0026rsquo;s hard drive, so use talosctl to get the available disks.\ntalosctl get disks --insecure --nodes \u0026#34;$CONTROL_PLANE_IP\u0026#34; You\u0026rsquo;ll see something like\nNODE NAMESPACE TYPE ID VERSION SIZE READ ONLY TRANSPORT ROTATIONAL WWID MODEL SERIAL 10.0.1.51 runtime Disk loop0 2 73 MB true 10.0.1.51 runtime Disk sda 2 17 GB false virtio true QEMU HARDDISK 10.0.1.51 runtime Disk sr0 2 317 MB false ata true QEMU DVD-ROM QEMU_DVD-ROM_QM00003 Our node\u0026rsquo;s hard drive is sda, so\nexport DISK_NAME=sda Create a cluster patch file We\u0026rsquo;re going to use Cilium as the CNI and also have it replace kube-proxy, so let\u0026rsquo;s create the cluster with no CNI and disable kube-proxy. To do that, we\u0026rsquo;re going to create a patch file we can use when we generate the cluster\u0026rsquo;s configuration with talosctl.\n# cluster-patch.yaml cluster: network: cni: name: none proxy: # Disable kube-proxy, Cilium will replace it too disabled: true Generate the talos configuration talosctl gen config $CLUSTER_NAME \u0026#34;https://$CONTROL_PLANE_IP:6443\u0026#34; --install-disk \u0026#34;/dev/$DISK_NAME\u0026#34; --config-patch @cluster-patch.yaml export TALOSCONFIG=\u0026#34;$(pwd)/talosconfig\u0026#34; Initialize the cluster I like to test things in short-lived clusters so I don\u0026rsquo;t have to worry about breaking things that my internal services depend on. I like naming nodes clustername-role-number so that when I look at their proxmox console, it\u0026rsquo;s nice and clear what cluster the node is in and what its role is.\nHere\u0026rsquo;s how to create a patch file that sets the node name when we apply our configuration. We also want to enable scheduling on the control plane node since we\u0026rsquo;re setting up a single-node cluster.\nCreate a controlplane-1-patch.yaml that includes the hostname you want and sets allowSchedulingOnControlPlanes to true.\n# controlplane-1-hostname-patch.yaml machine: network: hostname: sisyphus-cn-1 cluster: allowSchedulingOnControlPlanes: true Apply the configuration to your control plane node to initialize the cluster with the merged controlpane.yaml and controlplane-1-patch.yaml files.\ntalosctl apply-config --insecure \\ --nodes $CONTROL_PLANE_IP \\ --file controlplane.yaml \\ --insecure \\ --config-patch @controlplane-1-patch.yaml Bootstrap etcd in the cluster ONLY DO THIS ONCE! Wait until you see etcd is waiting to join the cluster in the bottom portion of the dashboard. Depending on how fast your proxmox host is, this can take 5-10 minutes.\nThere will be some error messages and it will look like nothing is happening, be patient it will get back to ready. I think this is because we\u0026rsquo;re configuring the cluster to not include a CNI so we can use Cilium, and/or because we disable kube-proxy because Cilium replaces that functionality too.\nThe first time I stood a cluster up without CNI it took long enough that I thought I\u0026rsquo;d broken the configuration - it wasn\u0026rsquo;t till I kicked it off and then went to cook dinner that I gave it enough time to settle down.\nSo be patient, at least you only have to do this once per cluster.\ntalosctl bootstrap --nodes \u0026#34;$CONTROL_PLANE_IP\u0026#34; --talosconfig=./talosconfig Create a kubeconfig file talosctl kubeconfig sisyphus-kubeconfig --nodes $CONTROL_PLANE_IP --talosconfig=./talosconfig export KUBECONFIG=\u0026#34;$(pwd)/sisyphus-kubeconfig\u0026#34; Confirm that the cluster came up kubectl get nodes You\u0026rsquo;ll see something similar to\nNAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME sisyphus-cn-1 Ready control-plane 5m v1.34.1 10.0.1.51 \u0026lt;none\u0026gt; Talos (v1.11.5) 6.12.57-talos containerd://2.1.5 Install cilium First install the CRDs kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.2.0/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.2.0/config/crd/standard/gateway.networking.k8s.io_gateways.yaml kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.2.0/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.2.0/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.2.0/config/crd/standard/gateway.networking.k8s.io_grpcroutes.yaml kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.2.0/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml Confirm the gateway classes are present kubectl get crd gatewayclasses.gateway.networking.k8s.io gateways.gateway.networking.k8s.io httproutes.gateway.networking.k8s.io You should see something like this:\nNAME CREATED AT gatewayclasses.gateway.networking.k8s.io 2026-01-02T04:20:13Z gateways.gateway.networking.k8s.io 2026-01-02T04:20:14Z httproutes.gateway.networking.k8s.io 2026-01-02T04:20:15Z Set up the cilium helm repo helm repo add cilium https://helm.cilium.io/ helm repo update Install cilium cilium install \\ --version 1.18.1 \\ --helm-set=ipam.mode=kubernetes \\ --helm-set=kubeProxyReplacement=true \\ --helm-set=securityContext.capabilities.ciliumAgent=\u0026#34;{CHOWN,KILL,NET_ADMIN,NET_RAW,IPC_LOCK,SYS_ADMIN,SYS_RESOURCE,DAC_OVERRIDE,FOWNER,SETGID,SETUID}\u0026#34; \\ --helm-set=securityContext.capabilities.cleanCiliumState=\u0026#34;{NET_ADMIN,SYS_ADMIN,SYS_RESOURCE}\u0026#34; \\ --helm-set=cgroup.autoMount.enabled=false \\ --helm-set=cgroup.hostRoot=/sys/fs/cgroup \\ --helm-set=l2announcements.enabled=true \\ --helm-set=externalIPs.enabled=true \\ --set gatewayAPI.enabled=true \\ --helm-set=devices=e+ \\ --helm-set=operator.replicas=1 You\u0026rsquo;ll see something like\nℹ️ Using Cilium version 1.18.1 🔮 Auto-detected cluster name: sisyphus 🔮 Auto-detected kube-proxy has not been installed ℹ️ Cilium will fully replace all functionalities of kube-proxy I0101 21:25:52.110400 48637 warnings.go:110] \u0026#34;Warning: spec.SessionAffinity is ignored for headless services\u0026#34; This took several minutes to come up on my sisyphus cluster controller with 2 cores and 4GB RAM\nConfirm cilium status Confirm that Cilium is fully up and has no errors.\n❯ cilium status /¯¯\\ /¯¯\\__/¯¯\\ Cilium: OK \\__/¯¯\\__/ Operator: OK /¯¯\\__/¯¯\\ Envoy DaemonSet: OK \\__/¯¯\\__/ Hubble Relay: disabled \\__/ ClusterMesh: disabled DaemonSet cilium Desired: 1, Ready: 1/1, Available: 1/1 DaemonSet cilium-envoy Desired: 1, Ready: 1/1, Available: 1/1 Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1 Containers: cilium Running: 1 cilium-envoy Running: 1 cilium-operator Running: 1 clustermesh-apiserver hubble-relay Cluster Pods: 2/2 managed by Cilium Helm chart version: 1.18.1 Image versions cilium quay.io/cilium/cilium:v1.18.1@sha256:65ab17c052d8758b2ad157ce766285e04173722df59bdee1ea6d5fda7149f0e9: 1 cilium-envoy quay.io/cilium/cilium-envoy:v1.34.4-1754895458-68cffdfa568b6b226d70a7ef81fc65dda3b890bf@sha256:247e908700012f7ef56f75908f8c965215c26a27762f296068645eb55450bda2: 1 cilium-operator quay.io/cilium/operator-generic:v1.18.1@sha256:97f4553afa443465bdfbc1cc4927c93f16ac5d78e4dd2706736e7395382201bc: 1 Update talosconfig The beginning of your talosconfig file will start with something like this:\ncontext: sisyphus contexts: sisyphus: endpoints: - 10.0.1.51 ca: Update it to include a nodes entry\ncontext: sisyphus contexts: sisyphus: endpoints: - 10.0.1.51 nodes: - 10.0.1.51 ca: This will keep you from contantly having to specify --nodes for your talosctl commands.\nConfirm that the cluster is showing healthy talosctl health Check external connectivity to cluster services First, test a LoadBalancer service Make a playground directory and put the following files in it.\nCreate the playground namespace # 01-create-namespace.yaml apiVersion: v1 kind: Namespace metadata: labels: kubernetes.io/metadata.name: playground name: playground Create an IP Pool and Announcement Policy # 02-cilium-setup.yaml # Create our list of IPs apiVersion: \u0026#34;cilium.io/v2alpha1\u0026#34; kind: CiliumLoadBalancerIPPool metadata: name: \u0026#34;default-pool\u0026#34; spec: blocks: - start: \u0026#34;10.0.1.160\u0026#34; # Use IPs that are outside of your DHCP range but on stop: \u0026#34;10.0.1.170\u0026#34; # the same /24 as your talos VM. --- apiVersion: \u0026#34;cilium.io/v2alpha1\u0026#34; kind: CiliumL2AnnouncementPolicy metadata: name: l2-announcement-policy spec: serviceSelector: {} nodeSelector: {} # On a multi-node cluster, you may not want the control-plane nodes # making arp announcements. Uncomment the nodeSelector stanza here # to disable that. # nodeSelector: # matchExpressions: # - key: node-role.kubernetes.io/control-plane # operator: DoesNotExist externalIPs: true loadBalancerIPs: true # Different hardware will show different network device names. # This list of regexes (in Golang format) will find all the common # naming schemes I\u0026#39;ve seen for network devices so that Cilium can # find a network interface to make arp announcements. interfaces: - ^eth+ - ^enp+ - ^ens+ - ^wlan+ - ^vmbr+ - ^wlp+ Create a deployment and service in the playground Talos is focused on giving you a default secure cluster out of the box, so you can\u0026rsquo;t just use kubectl create deployment hello-server --image=gcr.io/google-samples/hello-app:1.0 - talos requires you to configure the securityContext. Here\u0026rsquo;s an example deployment for nginx that runs as a non-root user and specifies the pod\u0026rsquo;s resource requirements to satisfy the Pod Security Admission configuration that ships with talos. More info about that here.\n# 03-playground-nginx.yaml apiVersion: v1 kind: Service metadata: name: playground-nginx-service namespace: playground annotations: # Tells Cilium to manage this IP via LB IPAM cilium.io/lb-ipam-pool-name: \u0026#34;default-pool\u0026#34; # Optional: For L2/BGP to announce this IP cilium.io/assign-internal-ip: \u0026#34;true\u0026#34; # Or use externalIPs: # If using externalIPs: # kubernetes.io/ingress.class: \u0026#34;cilium\u0026#34; # For Ingress # For a specific IP # lbipam.cilium.io/ips: \u0026#34;192.168.1.50\u0026#34; # The specific IP you want Cilium to answer on spec: type: LoadBalancer selector: app: playground-nginx-pod ports: - name: http port: 80 targetPort: 8080 --- apiVersion: apps/v1 kind: Deployment metadata: name: playground-nginx-app namespace: playground spec: selector: matchLabels: app: playground-nginx-pod # \u0026lt;-- must match pod labels exactly replicas: 1 template: metadata: labels: app: playground-nginx-pod spec: # Talos is very security oriented, so we have to set up the # security context explicitly # ---------- Pod‑level security settings ---------- securityContext: runAsNonRoot: true runAsUser: 101 # non‑root UID that the image can run as seccompProfile: type: RuntimeDefault # Uncomment if you need a shared FS group for volume writes # fsGroup: 101 # ---------- Containers ---------- containers: - name: nginx-playground-container image: nginxinc/nginx-unprivileged:latest # Alpine, but we force a non‑root UID imagePullPolicy: IfNotPresent ports: - containerPort: 8080 # talos requires us to specify our resources instead of # letting k8s YOLO them resources: requests: memory: \u0026#34;64Mi\u0026#34; cpu: \u0026#34;250m\u0026#34; limits: memory: \u0026#34;128Mi\u0026#34; cpu: \u0026#34;500m\u0026#34; securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL Deploy the playground If you pass a directory name to kubectl with -f, it will apply (or delete) all resources found in the .yaml files in that directory.\nkubectl apply -f playground See what IP the playground is using It will almost certainly be the first IP address in your IP Pool, but confirm that.\nkubectl get service -n playground You\u0026rsquo;ll see something like:\nNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE playground-nginx-service LoadBalancer 10.111.180.4 10.0.1.160 80:30597/TCP 1m15s Confirm Connectivity curl http://THE_EXTERNAL_IP You should see the nginx default page! Don\u0026rsquo;t delete the playground yet, we\u0026rsquo;re going to use it to confirm that the Cilium Gateway API is working.\nTest Cilium\u0026rsquo;s Gateway API We configured Cilium to provide Gateway API services to the cluster when we installed it. Let\u0026rsquo;s confirm that it\u0026rsquo;s working correctly.\nMake a new gateway-tests directory.\nCreate a Gateway Create gateway-tests/01-create-gateway.yaml. For ease of testing we\u0026rsquo;re going to configure the gateway to allow it to be used by services in any namespace. We\u0026rsquo;re also assigning it a specific IP address so we can give it a stable FQDN. We don\u0026rsquo;t want that changing.\napiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: playground-gateway spec: gatewayClassName: cilium addresses: - type: IPAddress value: 10.0.1.160 listeners: - name: http protocol: HTTP # Case matters! port: 80 allowedRoutes: namespaces: from: All Update the playground-nginx-service Before we can add a HTTPRoute, we\u0026rsquo;re going to need to update the playground-nginx-service so it isn\u0026rsquo;t a LoadBalancer, so create gateway-tests/02-playground-service.yaml with the following contents:\napiVersion: v1 kind: Service metadata: name: playground-nginx-service namespace: playground spec: selector: app: playground-nginx-pod ports: - name: http port: 80 # This is what the service is listening on, and what will be routed to targetPort: 8080 # Port the pods are listening on, don\u0026#39;t route directly here! Create a HTTPRoute Create gateway-tests/03-http-route.yaml to route all incoming requests for ip-160.mydomain.com to our playground-nginx-service.\napiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: playground-http-route namespace: playground spec: parentRefs: - name: playground-gateway namespace: default hostnames: - \u0026#34;ip-160.mydomain.com\u0026#34; rules: - backendRefs: - name: playground-nginx-service port: 80 Create the Gateway test resources kubectl apply -f gateway-tests Confirm it worked curl http://ip-160.mydomain.com It should display a \u0026ldquo;Welcome to nginx!\u0026rdquo; document that looks like this:\n\u0026lt;!DOCTYPE html\u0026gt; \u0026lt;html\u0026gt; \u0026lt;head\u0026gt; \u0026lt;title\u0026gt;Welcome to nginx!\u0026lt;/title\u0026gt; \u0026lt;style\u0026gt; html { color-scheme: light dark; } body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } \u0026lt;/style\u0026gt; \u0026lt;/head\u0026gt; \u0026lt;body\u0026gt; \u0026lt;h1\u0026gt;Welcome to nginx!\u0026lt;/h1\u0026gt; \u0026lt;p\u0026gt;If you see this page, the nginx web server is successfully installed and working. Further configuration is required.\u0026lt;/p\u0026gt; \u0026lt;p\u0026gt;For online documentation and support please refer to \u0026lt;a href=\u0026#34;http://nginx.org/\u0026#34;\u0026gt;nginx.org\u0026lt;/a\u0026gt;.\u0026lt;br/\u0026gt; Commercial support is available at \u0026lt;a href=\u0026#34;http://nginx.com/\u0026#34;\u0026gt;nginx.com\u0026lt;/a\u0026gt;.\u0026lt;/p\u0026gt; \u0026lt;p\u0026gt;\u0026lt;em\u0026gt;Thank you for using nginx.\u0026lt;/em\u0026gt;\u0026lt;/p\u0026gt; \u0026lt;/body\u0026gt; \u0026lt;/html\u0026gt; Congrats, you have a working talos cluster.\nI cover setting up SSL by creating certificates with cert-manager, LetsEncrypt and Route 53 in Part 3 - Set up Secret Management with SOPS.\nAdding a worker node I originally planned to make that another post, but it\u0026rsquo;s only two steps.\nCreate another VM The talos website recommends at least 2 CPUs and 2 GB RAM. I set mine to 16G of disk. You can use the same ISO you used when creating the control plane node.\nMake sure it has a static IP assignment in DHCP.\nAdd it to the cluster Create a patch file with the node name you want\n# worker-1-patch.yaml machine: network: hostname: clustername-worker-1 When the worker node console gets to Maintenance, you can add it with a one line talosctl command\ntalosctl apply-config --insecure --nodes \u0026#34;$WORKER_IP\u0026#34; --file worker.yaml --config-patch @worker-1-patch.yaml My proxmox cluster isn\u0026rsquo;t on beefy hardware so it took a couple of minutes for the new node to join the cluster and start accepting workload.\nProblems I ran into When I was making this post, I ran into a couple of problems because I was redoing everything to run on a single node blog cluster and made some mistakes tidying things up for a post.\ncurl fails to connect While testing the LoadBalancer service, even though the service LoadBalancer shows that it has an external IP, when you test with curl, it gives an error message similar to this:\n$ curl http://10.0.1.160/ curl: (7) Failed to connect to 10.0.1.160 port 80 after 16 ms: Could not connect to server And when you check the pods\nkubectl get pods -n playground it shows the worker pod as pending, not crash loop backoff, not creating, just pending.\nNAMESPACE NAME READY STATUS RESTARTS AGE playground pod/playground-nginx-app-6d7ddb5b95-lv82x 0/1 Pending 0 12s When I ran into this with the a blog cluster while writing this post, it was because I made the test cluster a single node cluster and forgot to set allowSchedulingOnControlPlanes to true, so there was no place to schedule the pods. You can fix this by applying an updated configuration.\ntalosctl apply-config --insecure \\ --nodes $(CONTROL_PLANE_IP) \\ --file controlplane.yaml \\ --config-patch @controlplane-1-patch.yaml curl connects, but you get a 404 While testing the gateway, curl connects to the IP but you get a 404 error.\ncurl -iv http://ip-160.mydomain.com/index.html * Host ip-160.mydomain.com:80 was resolved. * IPv6: (none) * IPv4: 10.0.1.160 * Trying 10.0.1.160:80... * Established connection to ip-160.mydomain.com (10.0.1.160 port 80) from 10.0.1.121 port 61150 * using HTTP/1.x \u0026gt; GET /index.html HTTP/1.1 \u0026gt; Host: ip-160.mydomain.com \u0026gt; User-Agent: curl/8.17.0 \u0026gt; Accept: */* \u0026gt; * Request completely sent off \u0026lt; HTTP/1.1 404 Not Found \u0026lt; date: Sun, 04 Jan 2026 00:36:07 GMT \u0026lt; server: envoy \u0026lt; content-length: 0 \u0026lt; * Connection #0 to host ip-160.mydomain.com:80 left intact I ran into this during testing because I forgot to update the gateway yaml file to allow all namespaces after I moved all the test manifests into the playground namespace.\nTalos reboots so fast that when I make settings changes to a single node cluster I always reboot the control plane node with talosctl reboot -n $CONTROL_PLANE_IP.\nYou changed cilium settings but they don\u0026rsquo;t appear to have any affect Some changes to cilium settings require you to restart cilium pods to pick them up.\nkubectl -n kube-system rollout restart ds/cilium kubectl -n kube-system rollout restart ds/cilium-envoy kubectl -n kube-system rollout restart deployment/cilium-operator Tips If you plan on standing up and tearing down VMs, copy the MAC of the first one. Go to the proxmox datacenter UI, select the VM, then select Hardware and double click Network Device for details) and set each replacement to that MAC. Your DHCP server uses a machine\u0026rsquo;s MAC to determine if it should get a static assignment, so recycling the MAC keeps you from having to update DHCP each time you bring up a new VM.\nThis is one of the few times it\u0026rsquo;s a good idea to reuse a MAC - having two VMs or physical machines with the same MAC running simultaneously will cause problems with on your network.\n","permalink":"https://unixorn.github.io/post/homelab/k8s/01-talos-with-cilium-cni-on-proxmox/","summary":"\u003cp\u003eI\u0026rsquo;ve been meaning to set up a \u003ca href=\"https://www.siderolabs.com/talos-linux/\"\u003eTalos\u003c/a\u003e linux kubernetes cluster in my homelab for a while and set one up over the holiday break. Here\u0026rsquo;s how I did it.\u003c/p\u003e","title":"Creating a Talos kubernetes cluster with a Cilium CNI on Proxmox"},{"content":"After 19 years of hosting email for my domains on Google Workspace (I was an early internal tester when it was still Google Apps) I finally moved my domains to proton.me last month when they had a Cyber Monday sale.\nThere were a lot of reasons that boiled down to:\nI no longer trust Google to not use my data to train Gemini. I pay less for 500GB worth of storage for proton services than I did for 30GB on Google. And proton includes a VPN as part of my package\u0026rsquo;s services. Proton is a non-profit, so I don\u0026rsquo;t have to worry about them deciding to sell my data to prop up the stock price in a down quarter. If you have a proton email account but you haven\u0026rsquo;t already started importing email into proton, don\u0026rsquo;t kick that off until you read this - it\u0026rsquo;s what I wish I had known when I was setting up my account. If you don\u0026rsquo;t have one but are considering one, here\u0026rsquo;s a referral link that will get you two weeks for free and $20 off of your first bill. Disclaimer - I get $20 in credit too.\nGetting Started Don\u0026rsquo;t start importing your email right away. Instead, work in this order:\nSet up your domain\u0026rsquo;s DNS to deliver to proton Create filters, labels and folders based on your incoming messages. If you want a lot more control of your filters than gMail allows, take a look at the sieve filter options. One of their first examples is using sieve to create a filter that works based on what contact group the sender is in. I found it a lot easier to have a single ephemeral political label that I filter most fundraising emails into based on adding their sender address to a political contact group, same for tech-ephemeral and shopping-notices You can turn phone notification on or off on a per-folder basis, so create at least two folders - one for messages you want to filter and label, but don\u0026rsquo;t want them making your phone ping, and one for ones you do actually want notifications. I have filters that label messages and move them to a Notify folder, and other filters that label them and move them to a Filtered folder that doesn\u0026rsquo;t notify me. The only messages in my Inbox proper are either spam or stuff I haven\u0026rsquo;t created a filter for. Once you like what your filters are doing, then you kick off the mail ingestion so that they\u0026rsquo;ll be applied to messages as they get imported. I made the mistake of kicking off the ingestion first, so not all filters got applied to the messages. I had 19 years worth of email, so it took three-ish days to ingest everything. Nicer things about Proton Mail Filters can apply multiple labels to a message, unlike gmail - I filed an internal feature request about that in 2006 and it still isn\u0026rsquo;t in gMail. You\u0026rsquo;re not stuck using the gui to make filters. You can also make more complex filters by writing them in sieve. And you can start a filter using the gui, then edit the generated sieve code. The sieve editor won\u0026rsquo;t let you save unless it\u0026rsquo;s valid sieve syntax. It may not act how you\u0026rsquo;re expecting, but at least it doesn\u0026rsquo;t break filtration entirely. Not terrible but not so nice either Once you modify a filter by editing its sieve representation, you can easily get to a point where you can only edit it via sieve. Which is fair, the gui only understands the sieve it generates. My first modification is usually to make it work on contact groups and that always roach motels it to the sieve editor You have to enable downloading metadata in the mail search dialog to be able to do full-text searches, and depending on how much mail you have that download could take a while. On the other hand, they aren\u0026rsquo;t doing it server side because the email is encrypted at rest. When you edit a filter, there\u0026rsquo;s an option to apply it to all your existing mail. They only allow a limited number of those backfills to be running at once, so you have to wait for one to finish before you can kick off another. I have 19 years of email, so the backfills can take some time. I think I also ran into a limit on how many can be kicked off in a single day. Finally So far I\u0026rsquo;m a lot happier using Proton to host my domains than I was with hosting them on Google Workspace. I\u0026rsquo;m paying less and getting more functionality and more privacy. I had stayed on Google mainly out of inertia and thinking it\u0026rsquo;d be a huge pain to move, and they made that really easy.\n","permalink":"https://unixorn.github.io/post/2025-12-26-advice-for-switching-to-proton-hosted-email/","summary":"\u003cp\u003eAfter 19 years of hosting email for my domains on Google Workspace (I was an early internal tester when it was still Google Apps) I finally moved my domains to \u003ca href=\"https://proton.me\"\u003eproton.me\u003c/a\u003e last month when they had a Cyber Monday sale.\u003c/p\u003e\n\u003cp\u003eThere were a lot of reasons that boiled down to:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eI no longer trust Google to not use my data to train Gemini.\u003c/li\u003e\n\u003cli\u003eI pay less for 500GB worth of storage for proton services than I did for 30GB on Google. And proton includes a VPN as part of my package\u0026rsquo;s services.\u003c/li\u003e\n\u003cli\u003eProton is a non-profit, so I don\u0026rsquo;t have to worry about them deciding to sell my data to prop up the stock price in a down quarter.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIf you have a proton email account \u003cem\u003ebut\u003c/em\u003e you haven\u0026rsquo;t already started importing email into proton, don\u0026rsquo;t kick that off until you read this - it\u0026rsquo;s what I wish I had known when I was setting up my account. If you don\u0026rsquo;t have one but are considering one, here\u0026rsquo;s a \u003ca href=\"https://pr.tn/ref/3E8ZRJVR\"\u003ereferral link\u003c/a\u003e that will get you two weeks for free \u003cem\u003eand\u003c/em\u003e $20 off of your first bill. Disclaimer - I get $20 in credit too.\u003c/p\u003e","title":"Switching to Proton.me eMail Advice"},{"content":"I\u0026rsquo;ve been experimenting with running Talos in my home lab. I really like the idea of an immutable OS layer under Kubernetes and wanted to stand up a cluster to run some of my services that are currently run with docker-compose.\nI decided to use my Synology to store k8s volumes, here\u0026rsquo;s how I set that up.\nI have a Talos cluster running in my proxmox cluster. Because it\u0026rsquo;s immutable and is managed via API calls with talosctl, using the Synology isn\u0026rsquo;t as simple as sshing to a kubernetes node and configuring it to automount a filesystem.\nPre-requisites helm \u0026amp; talosctl. I installed them on my M3Book with brew, but you can get other install instructions at helm.sh and the Sidero Labs website An NFS server. I\u0026rsquo;m using my Synology for this post and including setup instructions for it. There are are many tutorials out there for setting up NFS on Linux so that\u0026rsquo;s out of scope for this post. A working Talos cluster. I used a single-node cluster to validate these instructions. I wrote Setting up Talos with a Cilium CNI on proxmox with instructions to set one up. The NFS server on my Synology grants access based on the IP of the client machine. I recommend that you either configure your DHCP server to always assign your talos cluster nodes the same IP or assign a static address in your Talos cluster configuration. I prefer to do it on my DHCP server so there\u0026rsquo;s a single source of truth for IP address assignments in my homelab.\nSoftware Versions I\u0026rsquo;ve found it frustrating when I go find a technical post and can\u0026rsquo;t tell what versions of software they were using, so here are the versions of the software I used while writing this post.\nSoftware Version helm 4.0.1 kubectl 1.34 talosctl 1.11.5 talos 1.11.5 proxmox 9.0.3 Synology DSM 7.2.2 Set up a NFS share On my Synology, I went to Control Panel -\u0026gt; Shared Folder and created a new shared folder talos-nfs, and enabled NFS on it. Keep track of the complete path to the folder (in my case /volume2/talos-nfs), you\u0026rsquo;re going to need it later.\nI also created a second share, talos-nfs-retains that I\u0026rsquo;m going to use for a separate storageClass with a reclaimPolicy of Retains. This one\u0026rsquo;s path is /volume2/talos-nfs-retains.\nGo to the NFS Permissions tab and create an access rule that allows connections from your talos node\u0026rsquo;s IP address.\nIt\u0026rsquo;ll look something like this:\nPut in the IP address of your talos node. I checked all three of the optional boxes - Enable asynchronous, Allow connections from unprivileged ports and Allow users to access mounted subfolders.\nYou could put in the entire /24 (10.1.2.0/24 in my example) but I don\u0026rsquo;t recommend it for security reasons. Yes, it\u0026rsquo;s a homelab and you should trust the other machines on your network, but assigning wide-open access to things \u0026ldquo;just to get it running\u0026rdquo; is a really bad habit to get into. Do it right, not right now.\nInstall nfs-subdir-external-provisioner We\u0026rsquo;re going to use helm to install nfs-subdir-external-provisioner. I prefer using NFS to iSCSI because then any files the pods write are regular files in a directory tree on the server. This lets me use restic to back them up like I do with my other shares, and I can also move files in or out of the NFS volume without having to crack open an iSCSI volume.\nDepending on your workload it can be a little slower than iSCSI, but my homelab services only have to support two users. Ease of backup is more of a priority for me than performance is.\nAdd the helm repository We\u0026rsquo;re going to use helm to install the provisioner, so to get started, install the helm repository and run helm repo update.\nhelm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner \\ \u0026amp;\u0026amp; helm repo update Install the provisioners My Synology has two bonded ports on 10.0.1.108. nfs.path should be the path being exported from the nfs server, you don\u0026rsquo;t have to care where talos mounts it.\nThe default storage class name is nfs-client). I\u0026rsquo;m using --set storageClass.name=different-nfs-client-name in the helm command below to specify names that are clear about the reclaim policy.\nI don\u0026rsquo;t want to accidentally create any volume claims locally if I forget to specify a storageClass, so I made this the default storageClass for my cluster. Remove the --set storageClass.defaultClass=true argument if you want something else for your default.\nexport NFS_SERVER_IP=YOUR_SERVERS_IP helm install nfs-provisioner-deletes \\ nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \\ --set nfs.server=$NFS_SERVER_IP \\ --set nfs.path=/volume2/talos-nfs \\ --set storageClass.reclaimPolicy=Delete \\ --set storageClass.defaultClass=true \\ --set storageClass.archiveOnDelete=false \\ --set storageClass.name=nfs-client-deletes \\ --set storageClass.onDelete=delete I also added a second storageClass, this time with the reclaimPolicy set to Retains\nhelm install nfs-provisioner-retains \\ nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \\ --set nfs.server=$NFS_SERVER_IP \\ --set nfs.path=/volume2/talos-nfs-retains \\ --set storageClass.name=nfs-client-retains \\ --set storageClass.reclaimPolicy=Retain Confirm that the install succeeded Check your pods and storage classes with kubectl.\n$ kubectl get pod NAME READY STATUS RESTARTS AGE nfs-provisioner-retains-nfs-subdir-external-provisioner-bc7f6tp 1/1 Running 0 11m nfs-subdir-external-provisioner-745688c67-c27mg 1/1 Running 0 21m $ kubectl get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE nfs-client-deletes (default) cluster.local/nfs-subdir-external-provisioner Delete Immediate true 22m nfs-client-retains cluster.local/nfs-provisioner-retains-nfs-subdir-external-provisioner Retain Immediate true 12m It only took a few seconds for the provisioner pod to come ready. If the provisioner pod never comes ready, check that you have the right IP address in your NFS permissions on the server side, not just the cluster - when I first set this up I put a .15 where it should have had a .51.\nOnce the nfs-subdir-external-provisioner pod is ready, we can start testing it.\nTest NFS on talos Create a dynamic PVC Create the a yaml file describing the test pvc, debian-tester-pvc-deletes.yaml\n# debian-tester-pvc-deletes.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: debian-test-pvc spec: accessModes: - ReadWriteOnce storageClassName: nfs-client-deletes resources: requests: storage: 5Gi Create the pvc with kubectl.\nkubectl apply -f debian-tester-pvc-deletes.yaml Create a debian deployment that uses the new PVC Create a yaml file for a debian test deployment that attaches the test pvc, debian-pvc-test-deployment.yaml\n# Toy debian deployment apiVersion: apps/v1 kind: Deployment metadata: name: debian-tester-deployment spec: replicas: 1 selector: matchLabels: app: debian-tester template: metadata: labels: app: debian-tester spec: containers: - name: debian-tester-container # Use a standard Debian image image: debian:stable command: [\u0026#34;/bin/bash\u0026#34;, \u0026#34;-c\u0026#34;, \u0026#34;echo \u0026#39;Data created in PVC\u0026#39; \u0026gt; /mnt/data/testfile.txt; while true; do sleep 30; done;\u0026#34;] # Mount the volume named \u0026#39;persistent-storage\u0026#39; to the container path \u0026#39;/mnt/data\u0026#39; volumeMounts: - name: persistent-storage mountPath: /mnt/data volumes: - name: persistent-storage persistentVolumeClaim: claimName: debian-test-pvc And deploy it with\nkubectl apply -f debian-pvc-test-deployment.yaml Confirm our deployment succeeded We should now be able to see our deployment:\n$ kubectl get deployment NAME READY UP-TO-DATE AVAILABLE AGE debian-tester-deployment 1/1 1 1 25s A running pod for it:\nkubectl get pod NAME READY STATUS RESTARTS AGE debian-tester-deployment-77ddd68557-qvcrk 1/1 Running 0 2m24s And finally get a shell in the running pod:\n# Use the pod name from kubectl get pod, in this case DEBIAN_P=debian-tester-deployment-77ddd68557-qvcrk kubectl exec $DEBIAN_P -it -- bash root@debian-tester-deployment-77ddd68557-qvcrk:/# Inside the pod, we can now cd /mnt/data and create some files as a test\ncd /mnt/data echo \u0026#34;Success\u0026#34; \u0026gt; pvc-test root@debian-tester-deployment-77ddd68557-qvcrk:/mnt/data# ls pvc-test testfile.txt Confirm it on the NFS server side ssh into your NFS server and cd into the directory you shared. In my case, the NVS volume is /volume2/talos-nfs and we can see a directory for the pvc.\ncd /volume2/talos-nfs ls #recycle @eaDir default-debian-test-pvc-pvc-e6cf5df1-90c8-4962-97bb-91ae795945bc /volume2/talos-nfs $ cat default-debian-test-pvc-pvc-e6cf5df1-90c8-4962-97bb-91ae795945bc/pvc-test Success /volume2/talos-nfs $ Persistence test Delete the deployment.\nkubectl delete deployment debian-tester-deployment Create the deployment again.\nkubectl apply -f debian-pvc-test-deployment.yaml Find the new pod with kubectl get pod, and get a shell in it. You\u0026rsquo;ll see the files you created in /mnt/data\nClean up Delete the deployment\nkubectl delete deployment debian-tester-deployment Delete the pvc\nkubectl delete pvc debian-test-pvc The subdirectory on your NFS server should be gone - ssh in and confirm\ncd /volume2/talos-nfs ls #recycle @eaDir /volume2/talos-nfs $ And with that, NFS setup is done.\n","permalink":"https://unixorn.github.io/post/homelab/talos-nfs-provisioner/","summary":"\u003cp\u003eI\u0026rsquo;ve been experimenting with running \u003ca href=\"https://www.talos.dev/\"\u003eTalos\u003c/a\u003e in my home lab. I really like the idea of an immutable OS layer under Kubernetes and wanted to stand up a cluster to run some of my services that are currently run with \u003ccode\u003edocker-compose\u003c/code\u003e.\u003c/p\u003e\n\u003cp\u003eI decided to use my Synology to store k8s volumes, here\u0026rsquo;s how I set that up.\u003c/p\u003e","title":"Using a NFS Provisioner with Talos"},{"content":"So here\u0026rsquo;s a fun macOS weirdness I ran into this weekend where I couldn\u0026rsquo;t connect to a port on another machine from a shell session inside of iTerm, even though I was able to ssh to other hosts.\nI wanted to experiment with Talos so I stood up a Talos VM in my homelab proxmox cluster. I happened to be doing the setup steps from a jump VM in the cluster.\nInitial setup worked fine. Then I decided to try and run some talosctl commands from my M3Book Air, and it gave me a no route to host error. I could ping the VM just fine from the M3Book, and I confirmed the proxmox firewall was turned off for the talos VM. The laptop was on the same /24 as the VMs (both the jump and the talos one).\nI wanted to confirm that it was reachable from a machine that wasn\u0026rsquo;t hosted on the proxmox cluster, so I tried running talosctl on one of my Raspberry PIs, and there were no issues.\nThen I tried running talosctl from Terminal on the M3Book instead of iTerm, and it worked! WTF!\nLooked in System Settings -\u0026gt; Privacy \u0026amp; Security -\u0026gt; Local Network, and iTerm did have that permission enabled. Turned Local Network off for iTerm, quit iTerm, turned it back on and restarted iTerm. Suddenly talosctl and kubectl worked from inside an iTerm window.\n","permalink":"https://unixorn.github.io/post/homelab/weird-shit-is-afoot-with-macos-and-iterm/","summary":"\u003cp\u003eSo here\u0026rsquo;s a fun macOS weirdness I ran into this weekend where I couldn\u0026rsquo;t connect to a port on another machine from a shell session inside of iTerm, even though I was able to \u003ccode\u003essh\u003c/code\u003e to other hosts.\u003c/p\u003e","title":"Weird shit is afoot with macOS and iTerm"},{"content":"Here\u0026rsquo;s the current (as of 2025-09-30) version of my shrimp food recipe.\nI posted my shrimp food recipe a few months ago here. I\u0026rsquo;ve made several batches since then, tweaking it every time. The most recent batch has powdered floating plants from my tanks.\nHere\u0026rsquo;s V2 of the shrimp cubes recipe. My colonies love this stuff - here\u0026rsquo;s a picture two minutes after I added it to my blue diamond tank:\nIngredients Substitute and/or add any vegetables that you know your colony likes. You want to make sure that you don\u0026rsquo;t end up with so much mix that it takes more than three months for your colony to finish it all, otherwise don\u0026rsquo;t worry if you don\u0026rsquo;t have the exact measurements of each ingredient.\nOne inch long section of peeled cucumber, cut into 1/4 to 1/2 inch slices. I get organic cucumber, but I still peel it to make sure there\u0026rsquo;s nothing on the skin that can harm my shrimp. Blanch it in boiling water for five minutes to soften it.\nHalf a cup of powdered floating plants (see below). You can substitute spirolina powder (or add some as an extra ingredient)\nOptionally add blanched squash or any other vegetable your colony likes.\nOne egg + one or two extra yolks, scrambled well.\nOne cup of water. If you\u0026rsquo;re blanching th\n1 tsp garlic powder\n1 tsp paprika\n1 tablespoon of egg shell powder (see below)\nAgar Agar\nA flat takeout container like the ones grocery store sushi come in\nThere are a couple of ingredients in this recipe that you\u0026rsquo;re going to have to make before you can make the shrimp cubes. Fortunately, they\u0026rsquo;re easy to make.\nEgg Shell Powder Two egg shells make enough powder for several batches of shrimp cubes. I keep it in an old vitamin bottle.\nCook something that uses a couple of eggs and save the shells Boil the empty shells for 10-15 minutes to sterilize them Remove the membrane from the shell (optional), and rinse the shells clean Bake the shells in the oven on a baking sheet at 250F / 125C for 25 minutes After the shells cool, grind them with a mortar and pestle until it\u0026rsquo;s a fine powder. You can probably blitz the shells in a food processor, but I haven\u0026rsquo;t tried that. You want a fine powder so there aren\u0026rsquo;t any sharp edges that could harm the shrimp. It\u0026rsquo;s much easier to grind the shell in batches of small pieces than to try and grind an entire eggshell at once. Floater Powder I have Red Root Floaters, Salvinia Minima and Duckweed in my shrimp tanks. They do a great job using up the nitrogen compoumds in the water and my shrimp like to graze on the biofilm on their roots.\nThey grow so fast that I have to pull handfuls of plants out of my tanks every week or two or they completely cover the surface and shade out the other plants in the tank, so I wanted to see if my shrimp would eat them and put them (in powdered form) in my latest batch of food.\nLine a baking sheet (one of the ones with raised edges) with parchment or aluminum foil and put a wire cooling rack in it. This lets the hot air in the oven hit all sides of the plants Spread the floating plants in a single layer on top of the rack. If you make the layer too thick, air won\u0026rsquo;t get at all parts of the plants and it\u0026rsquo;ll take longer to dry them out. Set the oven to 250F/120C and bake the plants until they\u0026rsquo;re dry and crumbly. I meant to bake mine for 20m but I got distracted and they ended up drying for an hour. You could put them outside to dry, but that takes a couple of days and I don\u0026rsquo;t have that kind of patience. Crumble the dried plants to powder. My mortar and pestle were in the dishwasher so I used my fingers - if they don\u0026rsquo;t crumble easily, they aren\u0026rsquo;t dry enough - put them back in the oven for another 5-10 minutes. Instructions Add 1 tsp Agar Agar powder to a cup of water and stir constantly while bringing it to a boil. If your Agar Agar has different directions on it, follow whatever it says to get the consistency of jello. If you blanched vegetables to add into the food, use the blanching water, it has nutrients that were leached out of the vegetables during blanching Once the water is boiling, stir in all the ingredients. Be thorough, you don\u0026rsquo;t want them clumping up Keep simmering the liquid for 1-2 minutes, stirring constantly, until it starts to thicken. If you have an ice cube mold that makes small cubes (I found one at Amazon that makes cubes that each hold roughly one tablespoon), pour it into the tray, otherwise pour it into a plastic to go container. Don\u0026rsquo;t fill it more than 1/2 inch (roughly 12 mm) deep. Let it set at room temperature. It usually takes 30-60 minutes to firm up enough to unmold the pieces or cut the sheet of food If you used a to go container, cut it into cubes roughly 1/2 inch square. I like to make smaller cubes. If the colony is big enough you can feed it multiple cubes - that will let more shrimp get at the food without squabbling.\nNotes Don\u0026rsquo;t worry about exact measurements of any of the ingredients except the Agar Agar. Too much Agar Agar is better than too little. You can add fish food to the mix, just powder it first. If you do add fish food, either don\u0026rsquo;t add something with bloodworm in it or use a dedicated pan. You can either keep it in the refrigerator for a few weeks or freeze it for at least three months, that\u0026rsquo;s how long it takes for my colonies to go through a batch. Never put bloodworms into anything you plan on using to prepare food for people later. Not your blender, not your ice tray, not the pan, nothing. 20% of people can develop severe allergies to bloodworms from even a small exposure, sometimes severe enough that it triggers a shellfish allergy as well.\n","permalink":"https://unixorn.github.io/post/aquarium/shrimp-food-recipe-v2/","summary":"\u003cp\u003eHere\u0026rsquo;s the current (as of 2025-09-30) version of my shrimp food recipe.\u003c/p\u003e","title":"Shrimp Food Recipe V2"},{"content":"Building a Debian 13 (Trixie) LXC Template for Proxmox Debian 13 (trixie) was released, but (at least as of 2025-08-12) there isn\u0026rsquo;t a LXC template available for it on proxmox. I wanted a Debian 13 LXC container, so I made a template of my own.\nInstall tooling We\u0026rsquo;re going to use Debian Appliance Builder to create a Debian 13 LXC template.\nFirst, install the tooling:\napt update apt install -y dab wget Download Debian 13 configuration mkdir dab cd dab # Get the debian 13 dab configuration wget -O dab.conf \u0026#34;https://git.proxmox.com/?p=dab-pve-appliances.git;a=blob_plain;f=debian-13-trixie-std-64/dab.conf;hb=HEAD\u0026#34; # Get a Makefile to make builds easier wget -O Makefile \u0026#34;https://git.proxmox.com/?p=dab-pve-appliances.git;a=blob_plain;f=debian-13-trixie-std-64/Makefile;hb=HEAD\u0026#34; Build the template Now we can build a template.\ndab init dab bootstrap (or dab bootstrap --minimal to get a smaller template) dab finalize --compressor zstd-max It\u0026rsquo;s easier to use the Makefile - run\nmake all and make will run all that for you and create a zst file, debian-13-standard_13.0.20250715-1_amd64.tar.zst (as of 2025-08-12). Copy it to the proxmox cache directory, and you can now make Debian 13 LXC containers in the GUI.\nmv *.zst /var/lib/vz/template/cache ","permalink":"https://unixorn.github.io/post/homelab/2025-08-13-building-debian-13-lxc-template-for-proxmox/","summary":"\u003ch1 id=\"building-a-debian-13-trixie-lxc-template-for-proxmox\"\u003eBuilding a Debian 13 (Trixie) LXC Template for Proxmox\u003c/h1\u003e\n\u003cp\u003eDebian 13 (trixie) was released, but (at least as of 2025-08-12) there isn\u0026rsquo;t a LXC template available for it on proxmox. I wanted a Debian 13 LXC container, so I made a template of my own.\u003c/p\u003e\n\u003ch2 id=\"install-tooling\"\u003eInstall tooling\u003c/h2\u003e\n\u003cp\u003eWe\u0026rsquo;re going to use \u003ca href=\"https://pve.proxmox.com/wiki/Debian_Appliance_Builder\"\u003eDebian Appliance Builder\u003c/a\u003e to create a Debian 13 LXC template.\u003c/p\u003e\n\u003cp\u003eFirst, install the tooling:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sh\" data-lang=\"sh\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003eapt update\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003eapt install -y dab wget\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch2 id=\"download-debian-13-configuration\"\u003eDownload Debian 13 configuration\u003c/h2\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sh\" data-lang=\"sh\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003emkdir dab\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nb\"\u003ecd\u003c/span\u003e dab\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e# Get the debian 13 dab configuration\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003ewget -O dab.conf \u003cspan class=\"s2\"\u003e\u0026#34;https://git.proxmox.com/?p=dab-pve-appliances.git;a=blob_plain;f=debian-13-trixie-std-64/dab.conf;hb=HEAD\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e# Get a Makefile to make builds easier\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003ewget -O Makefile \u003cspan class=\"s2\"\u003e\u0026#34;https://git.proxmox.com/?p=dab-pve-appliances.git;a=blob_plain;f=debian-13-trixie-std-64/Makefile;hb=HEAD\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch2 id=\"build-the-template\"\u003eBuild the template\u003c/h2\u003e\n\u003cp\u003eNow we can build a template.\u003c/p\u003e","title":"Building a Debian 13 LXC Template for Proxmox"},{"content":"Nodered, Home Assistant and Tailscale I\u0026rsquo;m moving my Home Assistant from a docker container to a proxmox VM running HAOS, and as part of that I\u0026rsquo;m moving Node-RED to its own container so I can move it to other proxmox hosts independently of HAOS.\nI\u0026rsquo;m setting up a new Node-RED instance as part of moving Home Assistant out of docker and onto an HAOS VM. My requirements were:\nRun Node-RED in a separate VM so I can move it to other proxmox hosts if there is resource contention or I need to fail over Proper SSL certificate Easy access via my tailnet Pre-requisites To follow these instructions, you will need:\nYour own Tailnet. You can get a free tailnet at Tailscale.com. A server with docker installed that you can run docker compose on. I\u0026rsquo;m using a LXC container in my proxmox cluster, but you can run it on a standalone linux server. A working Home Assistant server. Installation Installing docker and Home Assistant are out of scope for this post. We will only be covering how to get Node-RED working with an existing Home Assistant server.\nGet a Tailscale Auth key Log into the Tailscale admin page. Click the Settings tab. On the bottom left, you\u0026rsquo;ll see a Personal Settings, and under that, Keys. Click Generate Auth Key. Give it a description, and then click Generate Key Copy the key to someplace secure, it won\u0026rsquo;t be displayed again Create a configuration file for tailscale serve By default, node-red runs on port 1880. Because we\u0026rsquo;re running tailscale and node-red in the same docker network, the tailscale container will be able to connect to port 1880 on the node-red container and act as a proxy.\nSave this as ts-config/nodered.json.\n{ \u0026#34;TCP\u0026#34;: { \u0026#34;443\u0026#34;: { \u0026#34;HTTPS\u0026#34;: true } }, \u0026#34;Web\u0026#34;: { \u0026#34;${TS_CERT_DOMAIN}:443\u0026#34;: { \u0026#34;Handlers\u0026#34;: { \u0026#34;/\u0026#34;: { \u0026#34;Proxy\u0026#34;: \u0026#34;http://127.0.0.1:1880\u0026#34; } } } }, \u0026#34;AllowFunnel\u0026#34;: { \u0026#34;${TS_CERT_DOMAIN}:443\u0026#34;: false } } Docker Compose Now that we have an auth token and a configuration file for tailscale, we\u0026rsquo;re going to use docker compose to start up node-red and a ts-nodered containers. The ts-nodered sidecar will connect to tailscale, register itself as nodered.your-tailnet.ts.net, use Lets Encrypt to create a SSL certificate that is trusted by your browser, then finally start proxying SSL traffic from nodered.your-tailnet.ts.net into the node-red container.\nThis docker-compose file is based on the Node-RED Getting Started with Docker page. I added a ts-nodered container to act as a tailscale sidecar and made some other minor changes. I prefer to mount a local directory instead of using a docker volume to make backups easier. I can copy the directory into a .tar.bz2 or .zip file, and it\u0026rsquo;s also easier to edit the configuration files since I can do that from the host OS.\nSave this as docker-compose.yaml\n################################################################################ # Node-RED Stack or Compose ################################################################################ version: \u0026#34;3.7\u0026#34; services: ts-nodered: image: tailscale/tailscale:latest # The hostname specified here will be registered in your tailnet. You will # be able to access Node-RED by entering nodered.your-tailnet.ts.net into # your browser hostname: nodered environment: # Set TS_AUTHKEY in the environment, you don\u0026#39;t want it saved in # cleartext here - TS_AUTHKEY=${TS_AUTHKEY} - TS_STATE_DIR=/var/lib/tailscale - TS_SERVE_CONFIG=/config/nodered.json - TS_USERSPACE=true volumes: - ./ts-config:/config - ./ts-state:/var/lib/tailscale restart: unless-stopped nodered: container_name: node-red image: nodered/node-red:latest-debian # If you don\u0026#39;t set the network_mode to the name of the tailscale sidecar # container, tailscale won\u0026#39;t be able to connect to Node-RED and you will # have 500 errors network_mode: service:ts-nodered restart: unless-stopped environment: - TZ=America/Denver # We specify the port in nodered.json so tailscale serve knows what port to # proxy to. We don\u0026#39;t actually activate external access to these ports since # everything gets proxied by Tailscale. # # ports: # - \u0026#34;1880:1880\u0026#34; volumes: - ./nodered:/data - ./externals:/externals - /etc/hostname:/etc/hostname:ro - /etc/localtime:/etc/localtime:ro - /etc/machine-id:/etc/machine-id:ro - /etc/timezone:/etc/timezone:ro Start Node-RED Prepare Your Node-RED Directory First, prepare your directory. You need to make a few subdirectories and move tailscale\u0026rsquo;s configuration file into the ts-config subdirectory.\nmkdir ts-config \u0026amp;\u0026amp; mkdir ts-state Move the nodered.json file into the ts-config directory. Your directory structure should look like this:\n./docker-compose.yaml ./ts-config/nodered.json ./ts-state Start the Node-RED containers We\u0026rsquo;re going to pass the Tailscale auth token as an environment variable so we don\u0026rsquo;t have it visible in the configuration file. Run docker compose using the command below\nexport TS_AUTHKEY=\u0026#39;yourReallyLongTailscaleAuthKey\u0026#39; docker compose up -d ; docker compose logs -f In a minute or two depending on your network connection, you\u0026rsquo;ll see a bunch of logs start scrolling. Eventually you should see a line like:\nts-nodered-1 | 2025/07/14 00:54:52 serve: creating a new proxy handler for http://127.0.0.1:1880 to show that the ts-nodered proxy is up and running, and another line similar to\nnode-red | 13 Jul 18:54:52 - [info] Server now running at http://127.0.0.1:1880/ when the node-red container is ready. The containers start in parallel, so the order of those lines isn\u0026rsquo;t guaranteed. Once you\u0026rsquo;ve seen both lines, you can go to nodered.your-tailnet.ts.net in your browser and access the Node-RED web interface. The first time you do this, it\u0026rsquo;ll take a little longer while tailscale generates a SSL certificate for the host with Lets Encrypt.\nThere\u0026rsquo;s one last thing to do before we can consider this operational.\nMake Node-RED require a login Create your password hash Run docker exec -it node-red bash to get a shell inside the running node-red container Run node-red admin hash-pw. It will ask you for a password that will be used to log into your node-red, then print out a hash. Save that so we can update the node-red container\u0026rsquo;s configuration. Update settings.js Use your editor of choice to edit ./nodered/settings.js. You will find a commented out block that looks like\n//adminAuth: { // type: \u0026#34;credentials\u0026#34;, // users: [{ // username: \u0026#34;admin\u0026#34;, // password: \u0026#34;$2a$08$zZWtXTja0fB1pzD4sHCMyOCMYz2Z6dNbM6tl8sJogENOMcxWV9DN.\u0026#34;, // permissions: \u0026#34;*\u0026#34; // }] //}, Replace that with:\nadminAuth: { type: \u0026#34;credentials\u0026#34;, users: [{ username: \u0026#34;admin\u0026#34;, password: \u0026#34;TheHashFromRunning_nNodered_admin_hash-pw\u0026#34;, permissions: \u0026#34;*\u0026#34; }] }, Now we restart Node Red so it picks up the new admin user you created.\ndocker compose restart Give it 15-30 seconds, then refresh your browser. It should now give you a login dialog and make you login as admin with the password you input to the node-red admin hash-pw command.\nConnecting Node-RED to Home Assistant Now that you have a running Node-RED instance, you need to connect it to your Home Assistant server.\nCreate an access token To connect Node-RED to your Home Assistant (HA), we\u0026rsquo;re going to need an access token. To create an access token:\nLog into your HA instance Click on your username at the bottom of the sidebar Select the Security tab Scoll down to the Long Lived Access Tokens Section and click Ceate Troken. Name the new token. Store it someplace safe, it will only be displayed once. Configure the connection between Node-RED and Home Assistant Log into your Node-RED instance Click the burger menu on the right side of the screen, and select Manage Palette Select the Palette tab from the dialog\u0026rsquo;s left side, then Install on the top Install the HA plugin. Make sure you pick the one named node-red-contrib-home-assistant-websocket. You do not want the one named node-red-contrib-home-assistant, it is old and abandoned. You will now see a bunch more HA-related nodes in the left sidebar. Scroll down, find the API node, and drag it onto the Flow palette. Double-click the new node. An Edit API node dialog will appear, click add new server. Keep the protocol as Websocket. Do not check the Using the Home Assistant Add-on, we\u0026rsquo;re running Node-RED outside the HAOS instance. Put in the DNS name or IP address of your Home Assistant server. Hit Done to save. It\u0026rsquo;ll switch to the properties tab, just click Done again. You\u0026rsquo;ll now see a Deploy button at the upper right of the screen. Click that so NR can activate the connection to your HA instance You should be good to start writing Node-RED automations for your Home Assistant, but let\u0026rsquo;s verify that it\u0026rsquo;s working.\nVerifying your Node-RED connection Drag a Current State node onto your flow palette. Double-click it to edit it You should see your Home Assitant connection is already selected in the Server entry Start typing an entity name in the Entity ID field. If things are working correctly, it will start auto-completing the name for you. If it autocompleted, you\u0026rsquo;re good to go. Creating automations in NR is out of scope for this post, but there are a ton of videos on YouTube to show you how.\nCongratulations, you now have Node-RED running independently from Home Assistant.\n","permalink":"https://unixorn.github.io/post/homelab/nodered-homeassistant-and-tailscale/","summary":"\u003ch1 id=\"nodered-home-assistant-and-tailscale\"\u003eNodered, Home Assistant and Tailscale\u003c/h1\u003e\n\u003cp\u003eI\u0026rsquo;m moving my Home Assistant from a docker container to a proxmox VM running HAOS, and as part of that I\u0026rsquo;m moving Node-RED to its own container so I can move it to other proxmox hosts independently of HAOS.\u003c/p\u003e\n\u003cp\u003eI\u0026rsquo;m setting up a new Node-RED instance as part of moving Home Assistant out of docker and onto an HAOS VM. My requirements were:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eRun Node-RED in a separate VM so I can move it to other proxmox hosts if there is resource contention or I need to fail over\u003c/li\u003e\n\u003cli\u003eProper SSL certificate\u003c/li\u003e\n\u003cli\u003eEasy access via my tailnet\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"pre-requisites\"\u003ePre-requisites\u003c/h2\u003e\n\u003cp\u003eTo follow these instructions, you will need:\u003c/p\u003e","title":"Nodered, Home Assistant and Tailscale"},{"content":"Advice for New Neocaridina Keepers Here\u0026rsquo;s my advice for someone starting out with neocaridina shrimp. I don\u0026rsquo;t pretend to be an expert, but my colonies are thriving so I\u0026rsquo;m doing something right.\nTank Setup Add some floating plants like Red Root Floaters or Salvinia - they\u0026rsquo;re easy to maintain (just pull a clump or two out every week or so when the surface of the tanks is more than 50% covered). Shrimp love to hang out in the roots and graze on the biofilm there, and they do a great job of sucking ammonia/nitrate/nitrite out of the water.\nI like having a nursery area in my tank for the babies. Basically throw some moss (any moss but Marimo moss balls - those are too dense for the shrimp to crawl it inside to hide) in a low-flow section of the tank. It\u0026rsquo;ll make the female shrimp feel a little more secure knowing there\u0026rsquo;s a safe place for their babies to hide in, and for them to hide in after a molt. I use java moss because someone generously gave me some for free when I was starting my tank, but it doesn\u0026rsquo;t really matter what kind.\nIf you\u0026rsquo;re considering an airstone, go with a sponge filter instead. It\u0026rsquo;ll do just as much good as the airstone and the sponge surface develops a lot of biofilm - there are always some babies and a half dozen or so adults grazing on mine. My tanks are heavily planted and handled the nitrate/nitrite/ammonia fine before I added filters but I like having something to cause some water flow and give the shrimp a grazing area.\nGive them plenty of hiding places. They know they\u0026rsquo;re on the bottom of the food chain, and even in a single-species tank they\u0026rsquo;ll feel more secure if they have places to hide. Moss, dense plants like a carpet of short to medium plants, piles of little rocks with nooks and crannies, these are all good. Small chunks of lava rock in piles will not only provide hiding places, they will accumulate a lot of biofilm \u0026amp; algae in the pores for shrimp to graze on\nAdd some botanicals like Catappa, Mulberry or Oak Leaves, and/or Alder cones. Cholla wood is also good - my colony loves to graze on it, and I frequently see smaller shrimp hanging out inside the hollow core.\nFeeding While neocaridinas can live on just algae and biofilm, they will breed faster and have bigger clutches if you feed them high protein food occasionally. It takes a month or two for enough biofilm to build up in a new tank to feed more than a half dozen shrimp, so feed them a couple of times a week until then. I only clean the sides of my tank that face the room and leave the other two sides to develop a layer of algae and/or biofilm for the colony to graze on.\nVariety is the spice of life. They don\u0026rsquo;t eat the same thing every day in the wild, so I don\u0026rsquo;t feed them the same food every time either. I prefer to put their food in a feeding dish to make it easier to suck out uneaten food with a turkey baster after a couple of hours. If you leave a lot of uneaten food in the tank it can start to decay and foul the water, or the snails in your tank will gobble it down and you\u0026rsquo;ll have to deal with a snail population explosion.\nNot all colonies like the same foods. Here are some that my colonies enjoy.\nBacter AE I add Bacter AE to my tanks twice a week and since I started doing that, I\u0026rsquo;ve noticed more of the babies surviving to the point they\u0026rsquo;re large enough to see and start exploring the tank. I don\u0026rsquo;t use nearly as much as the instructions on the jar say to though, I put about half a rice grain\u0026rsquo;s worth per 5 gallons of tank in an old vitamin bottle half filled with tank water, shake it till it\u0026rsquo;s mixed well and doesn\u0026rsquo;t have any clumps, then pour it across the top of the tank, with a little extra over the moss patch where the babies are usually hiding.\nEgg Yolk My colony really likes hard boiled egg yolk - I put in about a quarter yolk for every 20 shrimp and it rarely takes more than an hour to be devoured. Put it in a feeding dish though, it breaks apart quickly and it\u0026rsquo;s a lot easier to pull the dish out if they don\u0026rsquo;t eat it than to chase all the debris with a turkey baster. It\u0026rsquo;s a treat food - even in a feeding dish it can make a mess.\nVegetables You can feed them vegetables too. Blanch them for about 3 minutes in boiling water, then dunk them in ice water for 30 seconds to stop them from cooking any further. This softens them so they\u0026rsquo;re easier to eat, and helps keep them from floating. I\u0026rsquo;ve fed my colony cucumber and zucchini (peeled in case there are any fertilizers or pesticide contaminating the peel) in slices just thick enough that they can go onto a skewer, lettuce and kale. I have heard people talk about feeding their colonies squash, baby carrots, cauliflower and broccoli too, but I haven\u0026rsquo;t personally tried those yet. Put the vegetable on a skewer so it\u0026rsquo;s easier to pull it out after 24 hours before it starts to rot.\nI make home-made frozen food for my colonies. Any snello recipe you find online will do, my personal recipe is here. I\u0026rsquo;ve updated the recipe here to include powdered plants from my tanks. I give it to them at least once a week.\nCommercial Food Options As far as commercial food goes, I feed my colony Fluval Bug Bites (I use both the bottom feeder formula and the shrimp formula) one day a week. There are plenty of other good food options out there. You can get them food designed for shrimp, but they\u0026rsquo;ll happily eat flake or bottom feeder food too. The main thing is to make sure that the first ingredient is either something like krill, some sort of insect or fish meal. You don\u0026rsquo;t want something with wheat or even corn as the first ingredient. I give mine roughly 1 pellet of the bug bites per ten shrimp in the colony.\nSoy Hulls / Snowflake Food Soy Hull pellets are also good. They\u0026rsquo;re usually called Snowflake food. It\u0026rsquo;s something like 11% protein and it breaks down into white flakes (hence the name) that can be in your tank for a fairly long time without fouling the water. Buy Soy Hull Pellets instead of ones labeled as shrimp food. Get organic ones meant for mushroom cultivation. Why? Snowflake food ranges from $3.70 an ounce to $12 an ounce on Amazon. Soy Hulls (at least the ones I bought) come in a ten pound bag, and cost $0.13 an ounce. It\u0026rsquo;s 28x cheaper than the cheapest Snowflake food I found. Buy the ten pound bag, then trade one pound bags of it to other shrimpkeepers for plants or shrimp so the pellets don\u0026rsquo;t go bad on you.\nBloodworms My colony likes bloodworms. I prefer freeze dried to frozen for a couple of reasons. First, it\u0026rsquo;s easier to portion them out since you don\u0026rsquo;t have to give them a whole cube. Secondly, I\u0026rsquo;ve seen multiple videos and posts where people talked about getting parasites from frozen worms, but no complaints about freeze-dried.\nOne warning about bloodworms - some people (I\u0026rsquo;ve read 20%) can develop allergies after frequent contact with frozen or freeze-dried bloodworms. It can sometimes cause people to develop allergies to eating shrimp as well. I only handle it with tweezers. If you do have an allergic reaction, it typically gets more severe with each exposure, so if you do develop an allergy, be safe and don\u0026rsquo;t try to use gloves or tweezers with them, just get rid of the bloodworms.\nHow much should you feed the colony? You want to feed them no more than they can finish in an hour or two. You should break up the pellets of food (you don\u0026rsquo;t need to bother with this with soy hulls, they break up as they expand in water) - that will let more of the shrimp get at the food without getting into shoving matches.\nIf you over feed your tank, the snail population will boom. It\u0026rsquo;s only really an aesthetic problem, and you can solve it long term by reducing the amount you\u0026rsquo;re feeding your colony. In the short term, use aquarium tweezers to crush them and feed them to your colony. They\u0026rsquo;ll appreciate the high protein, high calcium snack.\nWhen you see shrimp molts, don\u0026rsquo;t bother to remove them from the tank unless they\u0026rsquo;ve got something growing on them. The shrimp will eat them and it\u0026rsquo;ll help them make their next exoskeleton. Same for dead shrimp - unless there are a lot of them at once, they were killed by disease, or they have something growing on them, it\u0026rsquo;s safe to let the colony will recycle them. I\u0026rsquo;ve never seen a body last more than a couple of hours, and once my colony hit 40ish shrimp, they get eaten so fast I almost never spot a body.\nWater You should be testing for GH and KH in addition to ammonia/nitrite/nitrate. My tanks are heavily planted so I only test once or twice a week.\nDon\u0026rsquo;t chase \u0026ldquo;perfect\u0026rdquo; water parameters. Neocaridina are tolerant of a pretty wide range of water parameters, and it\u0026rsquo;s far more important for the parameters to be stable than that they be at a specific \u0026ldquo;perfect\u0026rdquo; level. You will kill more shrimp chasing perfect water than if you leave things where they\u0026rsquo;re stable.\nBreeding Ideally you start with a ratio of one male to three females. I recommend starting with a minimum of 10 and preferably 20 shrimp. Smaller groups sometimes end up all female because they\u0026rsquo;re bigger and more colorful.\nFeed the colony high protein foods to promote egg production. Most of the stuff I listed in the food section will do.\nDon\u0026rsquo;t be worried if you have a berried shrimp appear one day with no eggs, but you don\u0026rsquo;t see any babies. They\u0026rsquo;re ridiculously small - less than 2mm long, mostly transparent the first few molts, and they know they\u0026rsquo;re on the bottom of the food chain and hide. On top of that, baby shrimp don\u0026rsquo;t move very far the first week after hatching, so make sure there\u0026rsquo;s biofilm everywhere by dosing the tank with BacterAE or powdered fish food. That will increase the percentage that survive long enough to grow big enough for you to spot them.\n","permalink":"https://unixorn.github.io/post/aquarium/2025-05-31-advice-for-new-neocaridina-keepers/","summary":"\u003ch1 id=\"advice-for-new-neocaridina-keepers\"\u003eAdvice for New Neocaridina Keepers\u003c/h1\u003e\n\u003cp\u003eHere\u0026rsquo;s my advice for someone starting out with neocaridina shrimp. I don\u0026rsquo;t pretend to be an expert, but my colonies are thriving so I\u0026rsquo;m doing something right.\u003c/p\u003e","title":"Advice for New Neocaridina Keepers"},{"content":"It seems like everyone has their own recipe for home made food to feed their shrimp. Here\u0026rsquo;s mine.\nUnixorn\u0026rsquo;s Shrimp Food Recipe, v1 I\u0026rsquo;m still trying to work out exactly how much agar agar to use, but here\u0026rsquo;s my snello recipe in progress (Edit - I posted an updated recipe here)\nIngredients Substitute and/or add any vegetables that you know your colony likes. You want to make sure that you don\u0026rsquo;t end up with so much mix that it takes more than three months for your colony to finish it all, otherwise don\u0026rsquo;t worry if you don\u0026rsquo;t have the exact measurements of each ingredient.\nOne two inch long section of peeled cucumber, cut into 1/4 to 1/2 inch slices. I get organic cucumber, but I still peel it to make sure there\u0026rsquo;s nothing on the skin that can harm my shrimp.\nHalf a cup of organic spinach leaves. You can substitute salvinia powder, or just add it as an extra ingredient. You can add blanched squash or any other vegetable your colony likes.\nOne hard boiled egg, peeled + one or two extra hard boiled egg yolks\n1 tsp garlic powder\n1 tsp paprika\n1 tablespoon of egg shell powder (see below)\nAgar Agar\nA flat takeout container like the ones grocery store sushi come in\nInstructions Egg Shell Powder Boil and eat a couple of eggs Boil the empty shells for another 10-15 minutes to sterilize them Remove the membrane from the shell, and rinse the shells clean Bake in the oven at 250F / 125C for 25 minutes After the shells cool, grind them with a mortar and pestle until it\u0026rsquo;s a fine powder. You don\u0026rsquo;t want any sharp edges that could harm the shrimp. It\u0026rsquo;s easier to grind a small amount of shell at a time than to try and grind an entire eggshell at once. Shrimp Jello Blanch the cucumber and spinach leaves and any other vegetables you\u0026rsquo;re using for 5 minutes in boiling water until they\u0026rsquo;re all soft. Save the water afterward to use for the agar agar and to help get the ingredient mix blended to a liquid.\nThrow all the ingredients except the Agar Agar in a blender until it\u0026rsquo;s a liquid. Add water if you need to to get a liquid consistency. I find it works better if it\u0026rsquo;s thicker than a broth but not as thick as a smoothie - it\u0026rsquo;ll get thicker when you add the agar agar.\nFollow the directions on the Agar Agar to get the water/agar measurements so you can make a half a cup of Agar Agar.\nMix the blender contents with the cooked Agar Agar liquid and pour it into a takeout container and let it set. You don\u0026rsquo;t need to put it in the freezer, Agar Agar will set at room temperature.\nOnce it\u0026rsquo;s set, cut it into cubes. I cut it into cubes roughly 1/2 inch on a side to make it easier to portion out into the tank.\nNotes None of the ingredients have to be exact measurements. Don\u0026rsquo;t put bloodworms into anything you plan on using for food for people later. Not your blender, not your ice tray, nothing. 20% of people can develop severe allergies to bloodworms, sometimes severe enough that it triggers a shellfish allergy as well. Putting the food in the freezer changes the texture from jello/custard like to ice cubes, but the shrimp don\u0026rsquo;t mind and it keeps much longer I eventually bought a dedicated silicone ice cube mold from Amazon that I use instead of pouring a sheet in a takeout container so I don\u0026rsquo;t have to cut the shrimp food into cubes. The mold also gives more consistent amounts per portion. Make a small enough batch so that even with feeding the tank a cube or two every couple of days it runs out in less than 3 months. I\u0026rsquo;m sure it\u0026rsquo;ll keep longer than that in the freezer, but why take chances? I prefer to feed my tank several small cubes instead of one big one. That way the shrimp aren\u0026rsquo;t all swarming on the same cube and more of them can get at the food at once. If there is anything left after 48 hours remove it from the tank to prevent it starting to rot and cause an ammonia spike. ","permalink":"https://unixorn.github.io/post/aquarium/shrimp-food-recipe/","summary":"\u003cp\u003eIt seems like everyone has their own recipe for home made food to feed their shrimp. Here\u0026rsquo;s mine.\u003c/p\u003e","title":"Shrimp Food Recipe"},{"content":"Released ha-mqtt-discoverable 0.19.1\nA lot of code cleanups Dependency updates Thanks for your contributions and maintenance work, @kratz00\nWhat\u0026rsquo;s Changed MegaLinter: fix linter errors by @kratz00 in https://github.com/unixorn/ha-mqtt-discoverable/pull/323 Update Pydantic to version 2.10.6 by @kratz00 in https://github.com/unixorn/ha-mqtt-discoverable/pull/322 Update pyaml to version 25.1.0 by @kratz00 in https://github.com/unixorn/ha-mqtt-discoverable/pull/324 [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/unixorn/ha-mqtt-discoverable/pull/326 Miscellaneous fixes and improvements by @kratz00 in https://github.com/unixorn/ha-mqtt-discoverable/pull/325 ruff: Simplifying Python linter usage by @kratz00 in https://github.com/unixorn/ha-mqtt-discoverable/pull/327 SensorInfo: Add \u0026lsquo;suggested_display_precision\u0026rsquo; (fixes #179) by @kratz00 in https://github.com/unixorn/ha-mqtt-discoverable/pull/328 [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/unixorn/ha-mqtt-discoverable/pull/330 Misc2 by @kratz00 in https://github.com/unixorn/ha-mqtt-discoverable/pull/329 Package Updates by @kratz00 in https://github.com/unixorn/ha-mqtt-discoverable/pull/334 [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/unixorn/ha-mqtt-discoverable/pull/336 Update ruff to version 0.11.4 by @kratz00 in https://github.com/unixorn/ha-mqtt-discoverable/pull/339 Update ruff to version 0.11.5 by @kratz00 in https://github.com/unixorn/ha-mqtt-discoverable/pull/340 Update pydantic to version 2.11.3 by @kratz00 in https://github.com/unixorn/ha-mqtt-discoverable/pull/341 Update to v0.19.0 for release by @unixorn in https://github.com/unixorn/ha-mqtt-discoverable/pull/342 Full Changelog: https://github.com/unixorn/ha-mqtt-discoverable/compare/v0.18.0...v0.19.0\n","permalink":"https://unixorn.github.io/post/hass/ha-mqtt-discoverable-0.19.1/","summary":"\u003cp\u003eReleased ha-mqtt-discoverable 0.19.1\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eA lot of code cleanups\u003c/li\u003e\n\u003cli\u003eDependency updates\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThanks for your contributions and maintenance work, @kratz00\u003c/p\u003e\n\u003ch2 id=\"whats-changed\"\u003eWhat\u0026rsquo;s Changed\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eMegaLinter: fix linter errors by @kratz00 in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/323\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/323\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eUpdate Pydantic to version 2.10.6 by @kratz00 in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/322\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/322\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eUpdate pyaml to version 25.1.0 by @kratz00 in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/324\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/324\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e[pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/326\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/326\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eMiscellaneous fixes and improvements by @kratz00 in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/325\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/325\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eruff: Simplifying Python linter usage by @kratz00 in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/327\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/327\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eSensorInfo: Add \u0026lsquo;suggested_display_precision\u0026rsquo; (fixes #179) by @kratz00 in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/328\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/328\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e[pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/330\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/330\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eMisc2 by @kratz00 in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/329\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/329\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ePackage Updates by @kratz00 in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/334\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/334\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e[pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/336\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/336\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eUpdate ruff to version 0.11.4 by @kratz00 in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/339\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/339\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eUpdate ruff to version 0.11.5 by @kratz00 in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/340\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/340\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eUpdate pydantic to version 2.11.3 by @kratz00 in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/341\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/341\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eUpdate to v0.19.0 for release by @unixorn in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/342\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/342\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u003cstrong\u003eFull Changelog\u003c/strong\u003e: \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/compare/v0.18.0...v0.19.0\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/compare/v0.18.0...v0.19.0\u003c/a\u003e\u003c/p\u003e","title":"Released ha-mqtt-discoverable 0.19.1"},{"content":"Released version 1.5.0 of https://github.com/unixorn/lima-xbar-plugin\nlima-xbar-plugin works with both xbar and SwiftBar to add a macos menubar item to control your lima VMs and containers.\nThanks to new contributors pythoninthegrass and deepjia.\n","permalink":"https://unixorn.github.io/post/open-source/lima-xbar-plugin-1.5.0/","summary":"\u003cp\u003eReleased version 1.5.0 of \u003ca href=\"https://github.com/unixorn/lima-xbar-plugin\"\u003ehttps://github.com/unixorn/lima-xbar-plugin\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003elima-xbar-plugin works with both \u003ca href=\"https://xbarapp.com/\"\u003exbar\u003c/a\u003e and \u003ca href=\"https://github.com/swiftbar/SwiftBar\"\u003eSwiftBar\u003c/a\u003e to add a macos menubar item to control your lima VMs and containers.\u003c/p\u003e\n\u003cp\u003eThanks to new contributors \u003ca href=\"https://github.com/pythoninthegrass\"\u003epythoninthegrass\u003c/a\u003e and \u003ca href=\"https://github.com/deepjia\"\u003edeepjia\u003c/a\u003e.\u003c/p\u003e","title":"Lima Xbar Plugin 1.5.0"},{"content":"Nanotank Setup Advice You\u0026rsquo;ve seen some nice videos on YouTube of small tanks with fresh water shrimp, and now you want one for yourself. Here are some tips on setting a shrimp tank up for yourself.\nBackground Anything smaller than 15 gallons is considered a nanotank. If you search for \u0026ldquo;nanotank stocking\u0026rdquo;, you\u0026rsquo;ll find many lists of fish and shrimp that will do well in that small of a tank.\nI recommend going with a planted or Walstad-style tank. Waste in the tank will break down into nitrite, nitrate and ammonia, and those are bad for fish and shrimp. They are also fertilizer, so plants will help keep the levels down to safe (sometimes not even detectable) levels. The faster growing a plant is, the better it is at cleaning up nitrogen compounds.\nTerms Hardscape - Rocks, wood and other decorations. Livestock - Anything alive in the tank that isn\u0026rsquo;t a plant Substrate - The dirt and sand in the bottom of the tank Requirements Sources for these are in the Shopping List section below.\nA five to twenty gallon tank. If you have the space, I recommend at least a ten gallon. Because a larger tank has more water, the water parameters (more on that later) will change more slowly and help keep the water safe for your livestock. A leveling mat if your tank didn\u0026rsquo;t come with one. Basically, this is a thin layer of neoprene that the tank sits on to protect the glass from stress - once your tank is full, even something as small as a little bit of sand underneath it can cause stress points that can eventually make the glass crack or even shatter. Five gallons of water can make a huge mess, so either invest in a mat, or cut your own from a used yoga mat or roll of neoprene. Make sure it\u0026rsquo;s a quarter to half inch (5 to 10 mm) thick. A bag of potting soil or topsoil. Organic is best. Try to find something without extra fertilizer in it. A bag of sand. You can buy it at a local fish store and pay $20 for five pounds, or you can buy playground sand at a garden center and pay about $10 for 50 pounds. Some people use pool filter sand, but I prefer playground sand because it\u0026rsquo;s meant for children to play in and theoretically won\u0026rsquo;t have any noxious chemicals in it. Dechlorinator. Unless you have well water, the water from your tap will have chlorine and/or chloramine in it to keep bacteria from growing in it, and it will kill the good bacteria in your tank and also harm your fish, shrimp or snails. I use API AquaSafe because it was in stock at my local pet store, but there are a lot of other options out there. You\u0026rsquo;re only going to be using small (like fractions of a cap full per gallon of water) so you don\u0026rsquo;t need to buy a huge bottle. Some people will say you can \u0026ldquo;age\u0026rdquo; tap water and let the chlorine evaporate out. That doesn\u0026rsquo;t work with chloramine. Be safe, treat the water. A water test kit. This is optional - most local pet stores will test your water for free if you bring in a sample, but it\u0026rsquo;s less hassle to do it at home and save yourself a trip Rocks and/or driftwood to put in the tank. It\u0026rsquo;ll make the tank look more interesting, give the livestock places to hide that will make them feel more secure and it provides a lot of surface area for good bacteria and biofilm to form. I don\u0026rsquo;t like the decorations at pet stores, but it\u0026rsquo;s your tank, add whatever makes you happy. I recommend adding at least a few lava rocks since they\u0026rsquo;re porous and provide a lot of surface area for biofilm to grow. A new five gallon bucket. You want one that\u0026rsquo;s never had anything but water in it - fish and shrimp are incredibly sensitive to some chemicals and you don\u0026rsquo;t want to accidentally poison them. Plants. The more, the better. I recommend a mix of floating plants and rooted plants. There\u0026rsquo;s a section below with some species suggestions that are easy to keep alive. A bacteria starter. I use Seachem Stability but there are a ton of other alternatives Do not buy any fish or shrimp yet. The tank will take three to six weeks to get a nitrogen cycle going and biofilm growing in the tank. Adding livestock to a tank too early is likely to kill them, and it\u0026rsquo;s such a common mistake that there\u0026rsquo;s a name for it - New Tank Syndrome.\nTank Setup Before you set up the tank, decide where you\u0026rsquo;re going to want to put it Water weighs about 8.3 pounds (3.8 kg) a gallon, so a five gallon tank will weigh 40 pounds just for the water - the tank, substrate and plants will add at least another 15-20%. You definitely will not want to have to move a 50 pound fragile glass tank later.\nDo a water leak test before you set the tank up Find a spot where water can\u0026rsquo;t hurt anything, and put the leveling pad on top of some paper towels. You want the paper towels to extend out from under the pad all around the bottom edges of the tank so you can see if there are any slow leaks. Put the tank on the pad and fill it with water. I like to wait at least 24 hours so I can find any slow leaks. Do not try this in a bathtub - the bottom of the tub is sloped to help with drainage and that will stress the bottom of your aquarium and potentially cause future leaks.\nOk, we\u0026rsquo;re ready to start setting up the tank Ok, now we\u0026rsquo;re ready to get started. Any time these directions call for water, I mean dechlorinated water. If you soak the substrate with chlorinated water, it will make it harder for the beneficial bacteria you need to establish a colony.\nSift some of the topsoil/potting soil through a screen to get the big chunks out. If you don\u0026rsquo;t have a screen you can pick it out with your fingers but that\u0026rsquo;ll take a little longer. You should put about half an inch to an inch (12.5mm to 25mm) of this in the bottom of your tank as the first layer of your substrate Gently wet it down with water until it\u0026rsquo;s wet but not mud. You don\u0026rsquo;t want to disturb it too much, but you also want to get as many air pockets out of it as you can. I just press it with my hands until it\u0026rsquo;s level, but I\u0026rsquo;ve seen videos where people poke it with toothpicks or bamboo skewers. If any of the soil got on the glass where you\u0026rsquo;re going to add sand, clean it off now - once the sand layer is in you won\u0026rsquo;t be able to get at it. Wash your sand before you add it to your tank. Put it in a bucket and rinse it with a hose (turn the water on, poke the hose into the bucket and work it down to the bottom of the bucket, then let the water work it\u0026rsquo;s way up through the sand and start overflowing the bucket) until the water runs clear or your tank will be cloudy for several days to a week. Put a layer of sand on top of the potting soil until it\u0026rsquo;s at least twice as thick as the dirt layer. Slowly add about two inches (5cm) of water into the tank. To keep it from scattering sand everywhere, I put a plastic bag into the tank and then pour the water slowly onto the bag. This will make adding your plants easier. If you\u0026rsquo;re using a sponge filter, put it in the tank now so you can place the rocks \u0026amp; driftwood around it. If you\u0026rsquo;re adding rocks, driftwood or other decorations, now\u0026rsquo;s the time. If your driftwood isn\u0026rsquo;t waterlogged enough, it\u0026rsquo;ll float. You can use superglue to glue it to some of your rocks to hold it down. The easiest way to do this is to put some glue on the rock where the wood is going to rest, add a very small wad of tissue, then more superglue, and then hold the driftwood against the glue-soaked tissue until it sets. Make sure to leave enough room around the sponge filter that you can pull it out to clean it every month or two without disturbing the hardscape you just spent time arranging to look just how you want it. Most of the rooted plants you buy will come in big clumps - separate them gently into smaller clumps before planting so you can cover more of the substrate. They\u0026rsquo;ll spread as they grow, so more small clumps will cover the bottom faster than a few larger clumps, even if it\u0026rsquo;s the same total amount of plants. Plant all your rooted plants. I found it easier to use a set of aquarium tongs to wiggle the roots into the sand, but if bare fingers work better for you, go for it. Again, leave a little room around the filter. If you\u0026rsquo;re adding moss or epiphytes like Java Fern, you can either tie it to your hardscape with thread, or use a few drops of superglue. You don\u0026rsquo;t need to use a lot, it\u0026rsquo;s just to hold the moss or plant in place until it can get established and stick to the wood/rock on its own. Epiphytes like Java Fern or Anubias have a rhizome in addition to roots. If the rhizome is completely covered in sand or super glue, it will eventually suffocate the rhizome and kill the plant. Now that all your plants are in place, slowly fill the tank the rest of the way up. This is where the plastic bag trick really helps - it will keep the water from stirring up the sand layer of your substrate and knocking your plants loose. Ironically, most floating plants don\u0026rsquo;t do well if the tops of their leaves are wet, so wait for the tank to fill before adding them. When the tank is full, add your floating plants. Follow the instructions on your bacteria starter to kick start the beneficial bacteria colony in your tank. The bacteria colony and your plants will take care of cleaning nitrates, nitrites and ammonia produced by animal waste, decomposing plants and excess food that will harm your shrimp and fish. Cycling the Tank Now you wait for the tank to establish a nitrogen cycle and season. It can take anywhere from three to eight weeks if you don\u0026rsquo;t use a bacteria starter, or two to three if you use the starter. I put a single Nerite snail in my tank after about a week so it\u0026rsquo;d produce waste to help feed the cycle. I recommend dosing the tank every few days with Bacter AE (see blow) to help encourage biofilm to grow in the tank so when you do add the shrimp, they\u0026rsquo;ve got something to graze on.\nCycling Shortcuts If you have a friend who keeps fish, see if you can get a used filter from them. Just squeeze it onto your new sponge filter or into the tank - it\u0026rsquo;ll look like filthy brown liquid but that filth is full of the bacteria you want converting your nitrogen compounds into safe ones your plants will use as fertilizer. Some fish stores will sell you their used filters.\nIf your friend is willing, see if you can put a rock or other bit of hardscape in their tank for a week - make sure you keep it submerged in tank water when you bring it home so it doesn\u0026rsquo;t dry out and kill the bacteria that colonized it from their tank.\nShopping list Everything here will definitely be fine for a 5 or 10 gallon tank. You may need to upsize the pump and filters if you have a bigger tank. They will have a gallon range in their descriptions.\nHardware A 5 or 10 gallon tank. If you\u0026rsquo;re in the US, Petco frequently has 50% off sales. I recommend buying from a fish or pet store over Amazon because you can inspect the tank for chips or cracks. An air pump - I bought a Pawfly air pump and it\u0026rsquo;s good for up to a 20 gallon tank. It\u0026rsquo;s very quiet. If they\u0026rsquo;re in stock, get the bundle that includes a sponge filter, it\u0026rsquo;ll also include enough air line and assorted valves to get you set up. If the pump + filter bundle isn\u0026rsquo;t in stock, I recommend a sponge filter so that the baby shrimp don\u0026rsquo;t get pulled into the filter and killed. I got an Aquaneat filter but anything sized for a 10 gallon tank will do. Get some tools. At a minimum you are going to want tweezers (both straight and ones with a bent end for planting), scissors to trim plants and something to smooth out the substrate when it gets stirred up. There are many bundles out there, but I got a Ohtomber Aquascape Tools Aquarium Kit that comes with aquarium scissors, two different kinds of tweezers for handling food or replanting plants and a tool for tidying up the substrate. Seachem Stability - Use this to kick-start your tank\u0026rsquo;s beneficial bacteria colony. There are many other options, but that\u0026rsquo;s what was in stock at my local pet store and it worked for me. Tetra Aquasafe Plus - You can get the smallest size bottle, 1/2 tsp will treat 5 gallons of water. Six months of water treatment only used a quarter bottle. A turkey baster. They\u0026rsquo;re great for spot cleaning the tank. You can buy something designed for aquariums and pay twice the price if you want. Food You can feed your shrimp just about anything that\u0026rsquo;s meant for bottom dwellers, or even just flake fish food. They can get by on just the algae and biofilm in your tank after it\u0026rsquo;s had a few months to develop, but they will only be getting by. Your colony will breed faster and stay healthier if you supplement their diet with other food.\nI really like GlasGarten Bacter AE - a little goes a very long way. For my five gallon tank, I add about a third of the scoop (roughly the size of a grain or two of rice) that comes with the food once or twice a week. It helps encourage biofilm growth and will really help baby shrimplets to survive. The powder can be a little hard to dissolve, so I fill an old prescription bottle half way with tank water, add the powder, than shake it hard until it looks milky white. Then I can pour the solution across the top of the tank and let it disperse. The first week or so after birth, baby shrimp don\u0026rsquo;t move very far, so if there isn\u0026rsquo;t food right there for them they can starve. Sometimes I\u0026rsquo;ll use the baster to squirt some of the bacter solution directly into the java moss clumps in my tank since that\u0026rsquo;s where I usually see babies first.\nMy colony also likes Fluval Bug Bites.\nShrimp also enjoy blanched vegetables - boil a piece of cucumber, spinach leaves, kale or thinly sliced carrot for two or three minutes until it\u0026rsquo;s soft, then stick it in the tank. I put them on bamboo skewers to make it easier to pull out the scraps later - I don\u0026rsquo;t leave them in the tank more than 24 hours to keep them from starting to rot and polluting the tank water.\nI\u0026rsquo;ve also fed them hard boiled egg yolk, roughly a quarter yolk for every 30 shrimp, and my shrimp really like it. The females need protein to make eggs, so it encourages them to breed.\nPlants Here are some species of plants that I\u0026rsquo;ve found do well in my tank with a minimal amount of effort on my part. If you want some more suggestions, search for \u0026ldquo;low-tech\u0026rdquo; friendly plants - those tend not to need CO2 supplementation or regular doses of fertilizer and are less hassle long term.\nFloating Plants The only thing you need to do for these plants is make sure there isn\u0026rsquo;t a lot of surface agitation in the tank. They don\u0026rsquo;t do well if the tops of the leaves get wet for long periods of time.\nRed Root Floaters I like these because they grow pretty quickly and the shrimp like to hang out in their hanging roots and eat the biofilm growing there. The only maintenance I have to do is pull a couple of fistfuls out of the tank every week or two to keep them from over-shading the rooted plants in the bottom of the tank.\nSalvinia Minima These are similar to RRFs and have the same requirements. I like having both to make the surface of the tank look a little more interesting.\nDuckweed You\u0026rsquo;ll hear a lot of opinions about duckweed. Some people hate it and call it the glitter of the aquarium hobby. It\u0026rsquo;s ridiculously hard to get it out of a tank once it has infested established itself, but it grows fast and sucks a lot of waste out of the water.\nOther Plants Java Ferns These are epiphytes. The TL;DR is that if you bury the rhizome (the bit the roots grow out of), they will die. You can either tie the rhizome to rocks/driftwood/decorations with thread (which I found to be a pain in the ass) or just superglue it. If you do glue it, use the smallest possible drops - they tolerate the glue very well, but if you cover the whole rhizome it will kill the plant. If you glue them to a small rock it makes it easy to move them around the tank if you want to later.\nJava Moss This grows very easily and is a great hiding place for baby shrimp and fish. Mother shrimp feel more secure and breed more quickly when they see that there are good hiding places for their babies, so I recommend adding Java Moss, or really any kind of moss except Marimo to your tank. Marimo isn\u0026rsquo;t bad, it\u0026rsquo;s just very dense and the shrimplets won\u0026rsquo;t be able to hide in the center like that can in other mosses.\nDwarf Sagitarrius This is a grass-like plant that will spread out to cover the bottom of your tank by sending out runner roots. It grows relatively quickly, I started with one small plant and two months later it had formed a nice carpet. The only maintenance I do on it is to cut the leaves short every few weeks since I prefer the way that looks. I leave it tall enough for the shrimp to hide in it if they want, but short enough the the middle depth of the tank is clear so I can see the shrimp moving around.\nPogostemon There are many varieties of this stem plant and will grow moderately quickly. When it gets longer than you want, you can cut it and replant the trimmings in the tank to help it spread faster.\nPearlweed This doesn\u0026rsquo;t require a lot of care and grows quickly. Some would say too quickly - if you don\u0026rsquo;t trim it back regularly (like at most every two weeks) it will take over the tank to the point that all you can see is a thick clump of pearlweed. It\u0026rsquo;s great for outcompeting algae and providing hiding places for your shrimp, but will easily grow to a point where you can barely see any of the shrimp or fish in the tank.\nLivestock Shrimp Neocaridina Davidii (aka Cherry Shrimp) I like neocaridina shrimp because they come in a wide variety of color strains and are hardy enough that they thrive in a wide range of water conditions. You can have multiple colors (aka a skittles tank) but the different color strains will interbreed, and after a few generations you will end up with mostly wild-type neos and not the vivid colors of the original strains.\nThe females are larger (up to 1.25 inches) and usually more colorful than the males. Ideally you will start your colony with 10-20 shrimp in a 1 male to 3 female ratio.\nAmano Shrimp These are also hardy. They grow larger than neocaridinas (up to two inches). They are great algae eaters and are very hardy. They may lay eggs and hatch larvae, but unlike neocaridinas, their larvae will not survive for more than a couple of days in fresh water.\nFish As far as fish are concerned, anything smaller than their mouth is food. And if they can tear it apart into pieces smaller than their mouth easily, things larger than their mouth are food too. If you decide to add fish to your shrimp tank, I recommend that you wait until the colony is well established with at least 50 adults.\nIf you\u0026rsquo;re planning to add fish to the tank, I recommend you add some small piles of lava rocks to the tank and/or cholla wood to give the shrimp more places to hide than just in the plants. And feed the fish regularly so they aren\u0026rsquo;t hungry. Your choice is to feed them fish food, or feed them your shrimp and shrimplets.\nChili Rasboras These are small colorful fish with small mouths. They may eat a few of your baby shrimp when they are very small, but will leave larger juveniles and the adults alone. They\u0026rsquo;re social fish and will be nervous if you have less than five or six in the tank. Edit: I\u0026rsquo;ve had them in one of my tanks for six months and they\u0026rsquo;ve never bothered a shrimp. The colony population is growing fast so I\u0026rsquo;m pretty sure they haven\u0026rsquo;t eaten any baby shrimp either.\nCorydoras Cories will only eat dead or very sick shrimp. They\u0026rsquo;re social fish, so if you get them, get at least three of them.\nBettas Bettas can be aggressive hunters and not only will they go after and decimate your baby shrimp, they have been known to kill adult shrimp as well. Avoid, especially if the tank is smaller than ten gallons.\nGoldfish I only mention them because of how bad an idea it is to add them to a nanotank. Contrary to popular belief, they grow very large, produce a lot of waste (more maintenance for you) and if that wasn\u0026rsquo;t bad enough, will happily eat your shrimp.\nGuppies Not only will they go after your baby shrimp, they breed ridiculously fast. If you do decide to add guppies, I recommend you only add males.\nMaintenance The joy of a planted tank is that you don\u0026rsquo;t actually have to do all that much maintenance. If you have enough plants, you won\u0026rsquo;t need to do water changes very often if at all.\nWater When water evaporates from your tank (and it will evaporate surprisingly quickly, even with a lid), it leaves any dissolved solids or minerals behind. If you top off with conditioned tap water, the concentration of dissolved solids will slowly creep up, and that could lead to a colony crash when the levels finally build up high enough that the shrimp can no longer tolerate them.\nYou could do regular water changes, but that\u0026rsquo;s a fair bit of work, and then you\u0026rsquo;re going to have big swings in the water parameters after water changes, and shrimp like stability. The easiest solution is to just top off with distilled or reverse osmosis water that is at roughly the same temperature as your tank. I keep the gallon jugs of distilled water in the same room as the tank so it is roughly the same temperature as the tank. When I need to top off, I slowly add a cup or so to the tank every 30-40 minutes until it\u0026rsquo;s back at the proper water level.\nYou really don\u0026rsquo;t need to do water changes unless you test the water and see a nitrate, nitrite or ammonia spike.\nPlants Floating Plants The floating plants are easy. I set up a small plastic bin, and fill it half way with tank water. Then as I pull each clump of plants out, I swirl them through the water in the bin so any shrimplets clinging to their roots let go, and then put them in the trash.\nStemmed Plants I trim the stemmed plants. When I\u0026rsquo;m first setting up a tank, I\u0026rsquo;ll replant the cutting in a sparsely planted area of the tank, but if the tank doesn\u0026rsquo;t have bare patches any more and I\u0026rsquo;m going to toss them or give them away, I\u0026rsquo;ll give them a swirl in the plastic bin just like with floaters to shake loose any shrimplets.\nFinally After I\u0026rsquo;ve pulled everything out that I\u0026rsquo;m going to, I pour the water back into the tank - more often than not this saves two or three shrimplets from going in the garbage.\n","permalink":"https://unixorn.github.io/post/aquarium/nanotank-setup/","summary":"\u003ch1 id=\"nanotank-setup-advice\"\u003eNanotank Setup Advice\u003c/h1\u003e\n\u003cp\u003eYou\u0026rsquo;ve seen some nice videos on YouTube of small tanks with fresh water shrimp, and now you want one for yourself.  Here are some tips on setting a shrimp tank up for yourself.\u003c/p\u003e","title":"Nanotank Setup"},{"content":"I recently replaced a 2019 Intel MacBook Pro with a M3 Macbook Air, so I decided to wipe the MBP and install proxmox on it.\nIt wasn\u0026rsquo;t as straightforward as installing on non-Apple hardware, so I\u0026rsquo;m documenting what I had to do here.\nNote that this post only covers getting things working on a MacBook Pro - look at the many online tutorials for what you should do once your node is up and running.\nI\u0026rsquo;ve been wanting to experiment with proxmox in my homelab for a while, and when I replaced my old Intel MacBook Pro with a M3 Macbook Air, I decided it was a good time to start.\nWhen I did this install, the current version of Proxmox was 8.3.0.\nPre-requisites Make a backup The install is going to wipe your Mac\u0026rsquo;s disk, so make sure you\u0026rsquo;ve backed up any files you care about.\nA boot stick As with a normal proxmox install, I downloaded a installer from https://proxmox.com/en/downloads then burned it to a thumb drive with Etcher, but use whatever tool you\u0026rsquo;re comfortable with.\nNetwork setup Proxmox doesn\u0026rsquo;t do DHCP during install, so you\u0026rsquo;re going to need to collect your network information. I recommend you pick an address outside your DHCP pool so you don\u0026rsquo;t have to worry about IP address collisions. While you\u0026rsquo;re still running macOS, open System Settings, then select Network so you can collect the current working IP address, router, and DNS servers.\nHardware Once you boot into the Proxmox installer it doesn\u0026rsquo;t recognize a lot of the MacBook Pro\u0026rsquo;s hardware, so you\u0026rsquo;ll need some additional hardware to get through installation. Once the install is complete, everything can get done through the webUI or via ssh.\nYou\u0026rsquo;re going to need:\nUSB hub USB keyboard and mouse. Proxmox 8.3.0 doesn\u0026rsquo;t recognize the MacBook Pro\u0026rsquo;s keyboard or trackpad A USB ethernet dongle. Running a server node over WIFI is a bad idea to begin with, and a stock install of Proxmox 8.3.0 doesn\u0026rsquo;t recognize the MBP\u0026rsquo;s WIFI either so you don\u0026rsquo;t reallhy get a choice. Installing Disable Boot Security By default, MacBooks with a T2 chip won\u0026rsquo;t boot off external devices, or boot unsigned operating systems.\nTo unlock the security restrictions:\nReboot into recovery mode by holding Command-R while rebooting Once you\u0026rsquo;re in recovery mode, open Startup Security Utility - it\u0026rsquo;s in Utilities -\u0026gt; Startup Security Utility Set Secure Boot to No Security Set Allow booting from external or removable media You\u0026rsquo;re set to start the proxmox install\nProxmox install Connect the install USB drive, keyboard, mouse and ethernet dongles. Reboot while holding the Option key. You should see a menu of drives come up. If not, you didn\u0026rsquo;t press the Option key fast enough, reboot and try again. You\u0026rsquo;ll see your Mac\u0026rsquo;s drive, a UEFI option, and a GRUB option. On my 2019 MacBook Pro, I had to pick UEFI. The Proxmox splash screen will appear. Pick your install option. I picked graphical because when I tried the terminal option the text was so small it was hard to read. Read and accept the EULA. Choose the drive you want to install on. Click the Options button if you want to change from the default ext4 filesystem. Set your country and time zone. Set a password and email address. Set up the network with the address, router, and DNS information you collected earlier. Choose your FQDN wisely - it\u0026rsquo;s a pita to change the name once you\u0026rsquo;ve created VMs or LXC containers. It\u0026rsquo;ll present a summary. Make sure everything is correct, especially the network information, then click install. On my MBP, it took less than ten minutes to install proxmox and get to the prompt for me to remove the install disk and let it reboot. You\u0026rsquo;ll see a login screen, with instructions on what URL to use to manage the instance.\nGotchas As of 2025-02-01, I ran into the following issues after installing proxmox 8.3.0 on a 2019 MacBook Pro.\nThe node goes to sleep when the lid is closed If you\u0026rsquo;re going to use this as a proxmox cluster node, you\u0026rsquo;re not going to want it going to sleep just because you closed the laptop lid.\nTo disable sleeping when the lid is closed, edit /etc/systemd/logind.conf as root.\nChange the HandleLidSwitch entries to ignore - they should look like\nHandleLidSwitch=ignore HandleLidSwitchExternalPower=ignore HandleLidSwitchDocked=ignore Then\nsudo systemctl reload logind You should now be able to close the lid without the node going to sleep.\nNetworking not coming up automatically during boot I installed on both an old Mac Mini and a MacBook Pro. On the Mac Mini, networking (using the built in ethernet) worked fine, but on the MacBook Pro it failed to bring up the network.\nOnce I logged into the console, I did some poking, and it looks like what\u0026rsquo;s happening is that systemd-udev-settle fails during boot, probably because my ethernet adapter is USB-C, and the networking systemd unit won\u0026rsquo;t start. But after boot, systemctl restart networking detects the adapter and brings it up.\nI don\u0026rsquo;t want to have to log into the console every boot of course, so I wrote a helper script that restarts networking if it\u0026rsquo;s down.\nDownload from kick-mbp-networking\n#!/usr/bin/env bash # # kick-mbp-networking # # This script is Apache 2.0 licensed. # # Networking on proxmox on my 2019 MacBook Pro stubbornly fails to come up # during boot, but a systemctl networking restart will fix it. # # Stick this into root\u0026#39;s crontab as a @reboot item and it will fix the # damned networking # # The entry should look like: # m h dom mon dow command #@reboot /usr/local/sbin/kick-mbp-networking # # Copyright 2025, Joe Block \u0026lt;jpb@unixorn.net\u0026gt; set -o pipefail if [[ -n \u0026#34;$DEBUG\u0026#34; ]]; then # shellcheck disable=SC2086 if [[ \u0026#34;$(echo $DEBUG | tr \u0026#39;[:upper:]\u0026#39; \u0026#39;[:lower:]\u0026#39;)\u0026#34; == \u0026#34;verbose\u0026#34; ]]; then set -x fi fi function debug() { if [[ -n \u0026#34;$DEBUG\u0026#34; ]]; then echo \u0026#34;$@\u0026#34; fi } function echo-stderr() { echo \u0026#34;$@\u0026#34; 1\u0026gt;\u0026amp;2 ## Send message to stderr. } function fail() { printf \u0026#39;%s\\n\u0026#39; \u0026#34;$1\u0026#34; \u0026gt;\u0026amp;2 ## Send message to stderr. Exclude \u0026gt;\u0026amp;2 if you don\u0026#39;t want it that way. exit \u0026#34;${2-1}\u0026#34; ## Return a code specified by $2 or 1 by default. } function has() { # Check if a command is in $PATH which \u0026#34;$@\u0026#34; \u0026gt; /dev/null 2\u0026gt;\u0026amp;1 } function get-settings() { LOG_F=${LOG_F:-\u0026#39;/root/mbp-network-weirdness.log\u0026#39;} } function check-dependencies() { # Confirm the stuff we need is in $PATH debug \u0026#34;Checking dependencies...\u0026#34; # shellcheck disable=SC2041 for p in \u0026#39;ip\u0026#39; \u0026#39;systemctl\u0026#39; do if ! has $p; then fail \u0026#34;Can\u0026#39;t find $p in your $PATH\u0026#34; else debug \u0026#34;- Found $p\u0026#34; fi done } function path-exists() { local file=\u0026#34;${1}\u0026#34; [[ -s \u0026#34;${file}\u0026#34; ]] || fail \u0026#34;$1 is not valid\u0026#34; [[ -d \u0026#34;${file}\u0026#34; ]] \u0026amp;\u0026amp; return [[ -f \u0026#34;${file}\u0026#34; ]] \u0026amp;\u0026amp; return fail \u0026#34;$1 is not a directory or file\u0026#34; } function restart-networking-if-down() { debug \u0026#34;$LOG_F\u0026#34; # Check if the vmbr interface is present if [[ $(ip link show | grep -c vmbr) != 0 ]]; then debug \u0026#34;ip link show = $(ip link show)\u0026#34; | tee -a \u0026#34;$LOG_F\u0026#34; debug \u0026#34;Networking is up\u0026#34; | tee -a \u0026#34;$LOG_F\u0026#34; else debug \u0026#34;Networking was down after boot, restarting at $(date)\u0026#34; | tee -a \u0026#34;$LOG_F\u0026#34; echo \u0026#34;Restarting networking after failed to start during boot at $(date)\u0026#34; | tee -a \u0026#34;$LOG_F\u0026#34; time systemctl restart networking | tee -a \u0026#34;$LOG_F\u0026#34; fi } check-dependencies get-settings restart-networking-if-down Put the script in /usr/local/sbin/kick-mbp-networking, then add it as a cron reboot job. As root, run crontab -e, then add a line to run the script. The crontab entry should look like\n# m h dom mon dow command @reboot /usr/local/sbin/kick-mbp-networking Test it by rebooting. You should now be able to access proxmox\u0026rsquo;s web ui without having to enable networking manually in a console session.\nFinally You should now have a working proxmox node you can safely use, either as a standalone machine or as part of a cluster.\n","permalink":"https://unixorn.github.io/post/homelab/2025-02-01-install-proxmox-on-macbook-pro/","summary":"\u003cp\u003eI recently replaced a 2019 Intel MacBook Pro with a M3 Macbook Air, so I decided to wipe the MBP and install \u003ca href=\"https://www.proxmox.com\"\u003eproxmox\u003c/a\u003e on it.\u003c/p\u003e\n\u003cp\u003eIt wasn\u0026rsquo;t as straightforward as installing on non-Apple hardware, so I\u0026rsquo;m documenting what I had to do here.\u003c/p\u003e\n\u003cp\u003eNote that this post only covers getting things working on a MacBook Pro - look at the many online tutorials for what you should do once your node is up and running.\u003c/p\u003e","title":"How to install Proxmox on a 2019 Macbook Pro"},{"content":"Released ha-mqtt-discoverable v0.16.2 and ha-mqtt-discoverable-cli v0.16.2\nWhat\u0026rsquo;s Changed Added missing last_reset to sensor for metering (electricity,gas, water etc.) by @unl0ck in https://github.com/unixorn/ha-mqtt-discoverable/pull/286 New Contributors @unl0ck made their first contribution in https://github.com/unixorn/ha-mqtt-discoverable/pull/286 ","permalink":"https://unixorn.github.io/post/hass/ha-mqtt-discoverable-0.16.2/","summary":"\u003cp\u003eReleased ha-mqtt-discoverable \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/releases/tag/v0.16.2\"\u003ev0.16.2\u003c/a\u003e and ha-mqtt-discoverable-cli \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable-cli/releases/tag/v0.16.2\"\u003ev0.16.2\u003c/a\u003e\u003c/p\u003e\n\u003ch2 id=\"whats-changed\"\u003eWhat\u0026rsquo;s Changed\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eAdded missing last_reset to sensor for metering (electricity,gas, water etc.) by @unl0ck in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/286\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/286\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"new-contributors\"\u003eNew Contributors\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e@unl0ck made their first contribution in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/286\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/286\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e","title":"Released ha-mqtt-discoverable v0.16.2"},{"content":"I run PiHole for ad-blocking on my home network. I\u0026rsquo;m an SRE in my day job, so of course I\u0026rsquo;m not running a single instance of something as important as DNS. I also don\u0026rsquo;t want to have to update things like local DNS entries or blocklists in multiple places, that will cause weird and annoying inconsistencies in my DNS.\nEnter orbital-sync.\nInstallation docker-compose I\u0026rsquo;m running orbital-sync via docker-compose with this docker-compose.yaml file.\n# docker-compose.yaml version: \u0026#39;3\u0026#39; services: orbital-sync: image: mattwebbio/orbital-sync:1 environment: PRIMARY_HOST_BASE_URL: $PRIMARY_HOST_BASE_URL PRIMARY_HOST_PASSWORD: $PRIMARY_HOST_PASSWORD SECONDARY_HOSTS_1_BASE_URL: $SECONDARY_HOSTS_1_BASE_URL SECONDARY_HOSTS_1_PASSWORD: $SECONDARY_HOSTS_1_PASSWORD SECONDARY_HOSTS_2_BASE_URL: $SECONDARY_HOSTS_2_BASE_URL SECONDARY_HOSTS_2_PASSWORD: $SECONDARY_HOSTS_2_PASSWORD # I only have two secondary hosts, but you could sync a third one # SECONDARY_HOSTS_3_BASE_URL: \u0026#39;http://server:8080\u0026#39; # SECONDARY_HOSTS_3_PASSWORD: \u0026#39;your_password4\u0026#39; # SECONDARY_HOSTS_3_PATH: \u0026#39;/apps/pi-hole\u0026#39; INTERVAL_MINUTES: $INTERVAL_MINUTES # Run it in the same docker network I run the master pihole in for simplicity. # You can comment this stanza out if you\u0026#39;re not using a ssl proxy. networks: default: external: name: ssl_proxy_network Configuration I have a .env file in the same directory as docker-compose.yaml where I set the configuration variables.\n# I run orbital-sync on the same host as the master pihole in the same # docker network I use for my SSL nginx proxy, so I can refer to it here # by container name. PRIMARY_HOST_BASE_URL=http://pihole PRIMARY_HOST_PASSWORD=YOUR_MASTER_PIHOLE_ADMIN_PASSWORD SECONDARY_HOSTS_1_BASE_URL=https://dns-secondary-one.example.com SECONDARY_HOSTS_1_PASSWORD=SECONDARY_ONE_PIHOLE_ADMIN_PASSWORD SECONDARY_HOSTS_2_BASE_URL=https://dns-secondary-two.example.com SECONDARY_HOSTS_2_PASSWORD=SECONDARY_TWO_PIHOLE_ADMIN_PASSWORD # I sync the primary to the secondaries every ten minutes. INTERVAL_MINUTES=10 # I only run two secondaries, but I could sync a third if I wanted #SECONDARY_HOSTS_3_BASE_URL=https://dns-secondary-three.example.com #SECONDARY_HOSTS_3_PASSWORD=SECONDARY_THREE_PIHOLE_ADMIN_PASSWORD The SSL proxy setup is documented at Set up nginx-proxy-manager with LetsEncrypt SSL certificates\nRunning Now you can run docker-compose up -d and your settings will be synced from your master pihole to its secondaries.\nHere\u0026rsquo;s a sanitized example docker log:\n$ docker-compose logs orbital-sync_1 | 12/6/2024, 4:44:28 PM: ➡️ Signing in to http://pihole/admin... orbital-sync_1 | 12/6/2024, 4:44:28 PM: ✔️ Successfully signed in to http://pihole/admin! orbital-sync_1 | 12/6/2024, 4:44:28 PM: ➡️ Downloading backup from http://pihole/admin... orbital-sync_1 | 12/6/2024, 4:44:28 PM: ✔️ Backup from http://pihole/admin completed! orbital-sync_1 | 12/6/2024, 4:44:28 PM: ➡️ Signing in to https://dns-s1.example.com/admin... orbital-sync_1 | 12/6/2024, 4:44:28 PM: ✔️ Successfully signed in to https://dns-s1.example.com/admin! orbital-sync_1 | 12/6/2024, 4:44:28 PM: ➡️ Uploading backup to https://dns-s1.example.com/admin... orbital-sync_1 | 12/6/2024, 4:44:46 PM: ✔️ Backup uploaded to https://dns-s1.example.com/admin! orbital-sync_1 | 12/6/2024, 4:44:46 PM: ➡️ Updating gravity on https://dns-s1.example.com/admin... orbital-sync_1 | 12/6/2024, 4:44:49 PM: ✔️ Gravity updated on https://dns-s1.example.com/admin! orbital-sync_1 | 12/6/2024, 4:44:49 PM: ✔️ Success: 1/1 hosts synced. orbital-sync_1 | 12/6/2024, 4:44:49 PM: Waiting 10 minutes...``` ","permalink":"https://unixorn.github.io/post/homelab/orbital-sync/","summary":"\u003cp\u003eI run \u003ca href=\"https://pi-hole.net/\"\u003ePiHole\u003c/a\u003e for ad-blocking on my home network. I\u0026rsquo;m an SRE in my day job, so of course I\u0026rsquo;m not running a single instance of something as important as DNS. I also don\u0026rsquo;t want to have to update things like local DNS entries or blocklists in multiple places, that will cause weird and annoying inconsistencies in my DNS.\u003c/p\u003e\n\u003cp\u003eEnter \u003ca href=\"https://github.com/mattwebbio/orbital-sync\"\u003eorbital-sync\u003c/a\u003e.\u003c/p\u003e","title":"Synchronizing Multiple Piholes with orbital-sync"},{"content":"Got my first neocaridinas yesterday!\nThe new tank has been cycling for six weeks and the water parameters have been good for the last three weeks, so I decided it was past time to add more livestock to the tank so the Nerite doesn\u0026rsquo;t get lonely.\nI decided on Yellow Golden Backs because I thought they\u0026rsquo;d stand out in my tank, and ordered 20 from LVexoticfish.com. I have to give them props - as advertised, all the shrimp arrived alive.\nThey were very well packaged - they had a regular fish bag with the shrimp and a bit of mesh for them to hold on to, inside a heat-sealed outer clear bag to contain any leaks (there weren\u0026rsquo;t any though) from the inner bag, and that bag was inside the silver thermal bag here. They shipped me 22 instead of the 20 I\u0026rsquo;d ordered. This was great because while I was temp acclimating them to my tank, one little guy managed to crawl part way into the knot in the bag and I accidentally crushed him while trying to untie the knot. Totally my fault.\nI\u0026rsquo;ve included a couple of pictures of the shrimp from this morning, but I took them with a phone and they don\u0026rsquo;t do the color justice. They\u0026rsquo;re already brighter than when they were in the bag and I\u0026rsquo;m looking forward to seeing them color up as they get used to the new tank.\nI will definitely order from LV Exotic Fish again if I decide on a different color morph for my next tank.\n","permalink":"https://unixorn.github.io/post/aquarium/first-neos/","summary":"\u003cp\u003eGot my first neocaridinas yesterday!\u003c/p\u003e\n\u003cp\u003eThe new tank has been cycling for six weeks and the water parameters have been good for the last three weeks, so I decided it was past time to add more livestock to the tank so the Nerite doesn\u0026rsquo;t get lonely.\u003c/p\u003e\n\u003cp\u003eI decided on Yellow Golden Backs because I thought they\u0026rsquo;d stand out in my tank, and ordered 20 from \u003ca href=\"https://LVexoticfish.com\"\u003eLVexoticfish.com\u003c/a\u003e. I have to give them props - as advertised, all the shrimp arrived alive.\u003c/p\u003e","title":"First Neos"},{"content":"Just started a new 5 gallon tank. Haven\u0026rsquo;t had fish tanks in years, and someone local generously gave me some plant trimmings when cleaning his tanks, and I forgot what species they are.\nWhat are these? Moss I think this might be Java moss, but am not sure. Stem One Java Moss Here\u0026rsquo;s a mystery stem plant Java Moss that\u0026rsquo;s growing pretty well Stem Two Pogostemon Another mystery plant This turned out to be Pogostemon Stem three Dwarf Sagitarius And finally, This turned out to be Dwarf sagitarius ","permalink":"https://unixorn.github.io/post/aquarium/plant-question/","summary":"\u003cp\u003eJust started a new 5 gallon tank. Haven\u0026rsquo;t had fish tanks in years, and someone local generously gave me some plant trimmings when cleaning his tanks, and I forgot what species they are.\u003c/p\u003e","title":"What are these plants?"},{"content":"I was a guest on Pagerduty\u0026rsquo;s Page it to the Limit podcast in the July 2024 podcast book club episode about Andy Weir\u0026rsquo;s novel, Project Hail Mary.. You can find the episode here.\n","permalink":"https://unixorn.github.io/post/hail-mary-podcast/","summary":"\u003cp\u003eI was a guest on Pagerduty\u0026rsquo;s Page it to the Limit podcast in the July 2024 podcast book club episode about Andy Weir\u0026rsquo;s novel, \u003ca href=\"https://andyweirauthor.com/\"\u003eProject Hail Mary.\u003c/a\u003e. You can find the episode \u003ca href=\"https://www.pageittothelimit.com/project-hail-mary/\"\u003ehere\u003c/a\u003e.\u003c/p\u003e","title":"Hail Mary Podcast"},{"content":"Released ha-mqtt-discoverable v0.14.0.\nNew Contributors @dianlight made their first contribution in https://github.com/unixorn/ha-mqtt-discoverable/pull/167 @mmaeusezahl made their first contribution in https://github.com/unixorn/ha-mqtt-discoverable/pull/178 @chripede made their first contribution in https://github.com/unixorn/ha-mqtt-discoverable/pull/182 @meowmeowahr made their first contribution in https://github.com/unixorn/ha-mqtt-discoverable/pull/197 What\u0026rsquo;s Changed Features Add via_device property to DeviceInfo by @dianlight in https://github.com/unixorn/ha-mqtt-discoverable/pull/167 Fixing a typo in README.md by @mmaeusezahl in https://github.com/unixorn/ha-mqtt-discoverable/pull/178 Fix closed -\u0026gt; opening by @chripede in https://github.com/unixorn/ha-mqtt-discoverable/pull/182 Un-private Binary Sensor update_state by @meowmeowahr in https://github.com/unixorn/ha-mqtt-discoverable/pull/197 Dependency updates build(deps): Bump gitlike-commands from 0.2.1 to 0.3.0 by @dependabot in https://github.com/unixorn/ha-mqtt-discoverable/pull/173 build(deps-dev): Bump pytest from 7.4.4 to 8.0.0 by @dependabot in https://github.com/unixorn/ha-mqtt-discoverable/pull/171 build(deps-dev): Bump flake8 from 6.1.0 to 7.0.0 by @dependabot in https://github.com/unixorn/ha-mqtt-discoverable/pull/168 build(deps-dev): Bump pre-commit from 3.6.0 to 3.6.1 by @dependabot in https://github.com/unixorn/ha-mqtt-discoverable/pull/180 build(deps-dev): Bump pre-commit from 3.6.1 to 3.6.2 by @dependabot in https://github.com/unixorn/ha-mqtt-discoverable/pull/184 build(deps-dev): Bump black from 23.12.1 to 24.2.0 by @dependabot in https://github.com/unixorn/ha-mqtt-discoverable/pull/183 build(deps-dev): Bump black from 24.2.0 to 24.3.0 by @dependabot in https://github.com/unixorn/ha-mqtt-discoverable/pull/192 build(deps-dev): Bump pytest from 8.0.0 to 8.1.1 by @dependabot in https://github.com/unixorn/ha-mqtt-discoverable/pull/190 build(deps-dev): Bump pre-commit from 3.6.2 to 3.7.0 by @dependabot in https://github.com/unixorn/ha-mqtt-discoverable/pull/195 build(deps-dev): Bump black from 24.3.0 to 24.4.0 by @dependabot in https://github.com/unixorn/ha-mqtt-discoverable/pull/198 build(deps-dev): Bump black from 24.4.0 to 24.4.2 by @dependabot in https://github.com/unixorn/ha-mqtt-discoverable/pull/200 Bump pydantic version to ^2.0.0 by @mmaeusezahl in https://github.com/unixorn/ha-mqtt-discoverable/pull/191 Fix poetry lock by @unixorn in https://github.com/unixorn/ha-mqtt-discoverable/pull/206 Full Changelog: https://github.com/unixorn/ha-mqtt-discoverable/compare/v0.13.1...v0.14.0\n","permalink":"https://unixorn.github.io/post/hass/ha-mqtt-discoverable-0.14.0/","summary":"\u003cp\u003eReleased \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/releases/tag/v0.14.0\"\u003eha-mqtt-discoverable v0.14.0\u003c/a\u003e.\u003c/p\u003e\n\u003ch2 id=\"new-contributors\"\u003eNew Contributors\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e@dianlight made their first contribution in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/167\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/167\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e@mmaeusezahl made their first contribution in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/178\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/178\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e@chripede made their first contribution in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/182\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/182\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e@meowmeowahr made their first contribution in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/197\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/197\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"whats-changed\"\u003eWhat\u0026rsquo;s Changed\u003c/h2\u003e\n\u003ch3 id=\"features\"\u003eFeatures\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eAdd via_device property to DeviceInfo by @dianlight in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/167\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/167\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eFixing a typo in README.md by @mmaeusezahl in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/178\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/178\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eFix closed -\u0026gt; opening by @chripede in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/182\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/182\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eUn-private Binary Sensor update_state by @meowmeowahr in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/197\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/197\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"dependency-updates\"\u003eDependency updates\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003ebuild(deps): Bump gitlike-commands from 0.2.1 to 0.3.0 by @dependabot in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/173\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/173\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ebuild(deps-dev): Bump pytest from 7.4.4 to 8.0.0 by @dependabot in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/171\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/171\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ebuild(deps-dev): Bump flake8 from 6.1.0 to 7.0.0 by @dependabot in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/168\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/168\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ebuild(deps-dev): Bump pre-commit from 3.6.0 to 3.6.1 by @dependabot in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/180\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/180\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ebuild(deps-dev): Bump pre-commit from 3.6.1 to 3.6.2 by @dependabot in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/184\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/184\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ebuild(deps-dev): Bump black from 23.12.1 to 24.2.0 by @dependabot in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/183\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/183\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ebuild(deps-dev): Bump black from 24.2.0 to 24.3.0 by @dependabot in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/192\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/192\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ebuild(deps-dev): Bump pytest from 8.0.0 to 8.1.1 by @dependabot in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/190\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/190\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ebuild(deps-dev): Bump pre-commit from 3.6.2 to 3.7.0 by @dependabot in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/195\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/195\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ebuild(deps-dev): Bump black from 24.3.0 to 24.4.0 by @dependabot in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/198\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/198\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ebuild(deps-dev): Bump black from 24.4.0 to 24.4.2 by @dependabot in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/200\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/200\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eBump pydantic version to ^2.0.0 by @mmaeusezahl in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/191\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/191\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eFix poetry lock by @unixorn in \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/pull/206\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/pull/206\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u003cstrong\u003eFull Changelog\u003c/strong\u003e: \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable/compare/v0.13.1...v0.14.0\"\u003ehttps://github.com/unixorn/ha-mqtt-discoverable/compare/v0.13.1...v0.14.0\u003c/a\u003e\u003c/p\u003e","title":"Released ha-mqtt-discoverable v0.14.0"},{"content":"I ordered a second UPS because thanks to PeaNUT, I saw that my current UPS is only providing 20 minutes of uptime. Unfortunately, as of this post, PeaNUT will only read one UPS\u0026rsquo; metrics from Network UPS Tools per PeaNUT instance, so I\u0026rsquo;m going to start scraping the metrics into Prometheus and displaying them with Grafana. Here\u0026rsquo;s how I did it.\nPrerequisites A working Network UPS Tools (aka NUT) instance. I documented setting one up in Set up nut-upsd and peanut in your homelab A Linux server with docker-compose and docker, podman or containerd set up. Working Prometheus and Grafana servers. I documented setting them up with docker-compose here. Setup Configure the NUT exporter for Prometheus I decided to use the hon95/prometheus-nut-exporter container to proxy metrics from NUT to Prometheus.\nHere\u0026rsquo;s a docker-compose.yaml file for starting nut-upsd and the prometheus nut exporter. For simplicity, I\u0026rsquo;m not including how to make it accessible via https. If you do want to secure the exporter with SSL, I documented how to use nginx-proxy-manager as an SSL reverse proxy here.\nversion: \u0026#39;3.9\u0026#39; services: # Details on nuts-upsd configuration are at https://unixorn.github.io/post/homelab/homelab-nut-upsd nut-upsd: image: instantlinux/nut-upsd container_name: nut environment: - API_PASSWORD=${API_PASSWORD:-\u0026#39;aPasswordForAPIaccess\u0026#39;} - TZ=${TZ:-America/Denver} # Driver found with this tool https://networkupstools.org/stable-hcl.html - DRIVER=usbhid-ups # If you want to have the UPS show up in grafana named something other than \u0026#39;ups\u0026#39; set name # - NAME=cp800avr devices: # Device numbers are subject to change, so map in the whole bus so nuts-upsd can find your UPS - /dev/bus/usb:/dev/bus/usb ports: - \u0026#34;3493:3493\u0026#34; restart: unless-stopped prometheus-nut-exporter: image: hon95/prometheus-nut-exporter:stable container_name: prometheus-nut-exporter environment: - HTTP_PATH=/nut - TZ=${TZ:-America/Denver} # Defaults #- RUST_LOG=info #- HTTP_PORT=9995 #- HTTP_PATH=/nut #- LOG_REQUESTS_CONSOLE=false #- PRINT_METRICS_AND_EXIT=false ports: - \u0026#34;9995:9995\u0026#34; # Don\u0026#39;t start until nut-upsd is running so we don\u0026#39;t serve garbage data # to prometheus when it scrapes us depends_on: - nut-upsd restart: unless-stopped Run docker-compose up -d and you should be able to go to http://yourserver.example.com:9995/nut?target=nut-upsd:3493 and see output similar to this:\n# TYPE nut_exporter_info info # UNIT nut_exporter_info # HELP nut_exporter_info Metadata about the exporter. nut_exporter_info{version=\u0026#34;1.2.1\u0026#34;} 1 # TYPE nut_server_info info # UNIT nut_server_info # HELP nut_server_info Metadata about the NUT server. nut_server_info{version=\u0026#34;2.8.1\u0026#34;} 1 # TYPE nut_ups_info info # UNIT nut_ups_info # HELP nut_ups_info Metadata about the UPS. nut_ups_info{ups=\u0026#34;ups\u0026#34;,description=\u0026#34;UPS\u0026#34;,device_type=\u0026#34;ups\u0026#34;,manufacturer=\u0026#34;CPS\u0026#34;,model=\u0026#34;CP800AVRa\u0026#34;,battery_type=\u0026#34;PbAcid\u0026#34;,driver=\u0026#34;usbhid-ups\u0026#34;,driver_version=\u0026#34;2.8.1\u0026#34;,driver_version_internal=\u0026#34;0.52\u0026#34;,driver_version_data=\u0026#34;CyberPower HID 0.8\u0026#34;,usb_vendor_id=\u0026#34;0764\u0026#34;,usb_product_id=\u0026#34;0501\u0026#34;,type=\u0026#34;ups\u0026#34;,nut_version=\u0026#34;2.8.1\u0026#34;} 1 # TYPE nut_info info # A lot more information snipped for brevity The example URL assumes you\u0026rsquo;re running nut-upsd out of the same docker-compose.yaml file you\u0026rsquo;re using to run the exporter. If not, replace nut-upsd in the link with an ip or dns name so it looks like http://yourserver.example.com:9995/nut?target=nut-upsd.example.com:3493\nAdd NUT scrape configuration to Prometheus Here\u0026rsquo;s the nut scrape job I\u0026rsquo;m using, with my hostnames stripped. More detailed configuration instructions available at github.com/hon95/prometheus-nut-exporter site.\nglobal: scrape_interval: 15s scrape_timeout: 10s scrape_configs: - job_name: \u0026#34;nut\u0026#34; scrape_interval: 30s static_configs: # Insert NUT server address here - targets: [\u0026#34;nut-upsd:3493\u0026#34;] metrics_path: /nut relabel_configs: - source_labels: [__address__] target_label: __param_target - source_labels: [__param_target] target_label: instance - target_label: __address__ # Insert NUT exporter address here replacement: your.exporter.server.example.com:9995 Add a nut dashboard to Grafana Finally, add a dashboard to your grafana instance to display your nut metrics. I\u0026rsquo;m using the one here - the ID is 14371 if you want to directly import it to grafana.\nHere\u0026rsquo;s what the resulting dashboard looks like in Grafana for one of my UPSes.\n","permalink":"https://unixorn.github.io/post/homelab/homelab-nut-prometheus-grafana/","summary":"\u003cp\u003eI ordered a second UPS because thanks to \u003ca href=\"https://github.com/Brandawg93/PeaNUT\"\u003ePeaNUT\u003c/a\u003e, I saw that my current UPS is only providing 20 minutes of uptime. Unfortunately, as of this post, PeaNUT will only read one UPS\u0026rsquo; metrics from \u003ca href=\"https://networkupstools.org/\"\u003eNetwork UPS Tools\u003c/a\u003e per PeaNUT instance, so I\u0026rsquo;m going to start scraping the metrics into Prometheus and displaying them with Grafana. Here\u0026rsquo;s how I did it.\u003c/p\u003e","title":"Store NUT metrics in Prometheus so we can display them with Grafana"},{"content":"I wanted nice graphs for the various metrics I collect in my homelab and from my Home Assistant server.\nHere\u0026rsquo;s how I installed Prometheus and Grafana in my homelab to get them.\nPrerequisites Server running docker, podman or containerd Install Instructions For the sake of brevity, I\u0026rsquo;m not going to go into detail on how to make it accessible via https. If you do want to secure these containers with SSL, I\u0026rsquo;ve documented how to use Nginx Proxy Manager as an SSL proxy here.\nPrometheus Here\u0026rsquo;s the docker-compose.yaml I\u0026rsquo;m using to start prometheus. We\u0026rsquo;re specifying a local directory to map in as /prometheus so we don\u0026rsquo;t lose all our history every time we restart the container. I prefer to use a local directory instead of a named volume to make it easier to back up data from my prometheus server to b2.\nversion: \u0026#39;3\u0026#39; services: prometheus: image: prom/prometheus:latest container_name: prometheus # Uncomment if you want prometheus accessible outside the docker network # created by docker-compose # ports: # - \u0026#34;9090:9090\u0026#34; volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml - ./prometheus/data:/prometheus - /etc/hostname:/etc/hostname:ro - /etc/localtime:/etc/localtime:ro - /etc/machine-id:/etc/machine-id:ro - /etc/timezone:/etc/timezone:ro restart: unless-stopped command: - \u0026#34;--config.file=/etc/prometheus/prometheus.yml\u0026#34; - \u0026#39;--web.enable-lifecycle\u0026#39; Don\u0026rsquo;t start prometheus yet - we want to get metric collection running on some servers first so it has something to ingest.\nCollect metrics on your servers If you don\u0026rsquo;t put data into Prometheus, Grafana will have nothing to graph.\nAll of my homelab machines run docker to provide services, so in addition to node_exporter, I run a cadvisor container to collect container data.\nHere\u0026rsquo;s the docker-compose.yaml I run on all my homelab instances. I mount several of the host machine\u0026rsquo;s interesting directory trees as read-only to allow node_exporter and cadvisor to create useful metrics for prometheus \u0026amp; grafana.\nversion: \u0026#39;3\u0026#39; services: node_exporter: image: quay.io/prometheus/node-exporter:latest container_name: node_exporter command: - \u0026#39;--path.rootfs=/host\u0026#39; - \u0026#39;--collector.textfile.directory=/promfiles\u0026#39; pid: host ports: - 9100:9100 restart: unless-stopped volumes: - \u0026#39;/:/host:ro,rslave\u0026#39; - /etc/hostname:/etc/hostname:ro - /etc/localtime:/etc/localtime:ro - /etc/machine-id:/etc/machine-id:ro - /etc/timezone:/etc/timezone:ro - /etc/node_exporter/promfiles:/promfiles:ro - /run/udev/data:/run/udev/data:ro - /sys:/sys:ro cadvisor: image: zcube/cadvisor:v0.45.0 container_name: cadvisor ports: - \u0026#34;9119:8080\u0026#34; volumes: - /:/rootfs:ro - /var/run:/var/run:ro - /sys:/sys:ro - /var/lib/docker/:/var/lib/docker:ro - /dev/disk/:/dev/disk:ro - /etc/hostname:/etc/hostname:ro - /etc/localtime:/etc/localtime:ro - /etc/machine-id:/etc/machine-id:ro - /etc/timezone:/etc/timezone:ro devices: - /dev/kmsg restart: unless-stopped I like to keep that as a separate docker-compose.yaml file than the other ones on a given server to make it easier to keep the metrics collection consistent across my servers.\nPut that file into a prometheus_collector directory, then run it with docker-compose up -d.\nYou can confirm that the exporters are running correctly with\n# node exporter curl http://localhost:9100/metrics # cadvisor curl http://localhost:9119/metrics You should see a lot of text that looks similar to this snippet:\n# A lot of lines snipped # HELP process_max_fds Maximum number of open file descriptors. # TYPE process_max_fds gauge process_max_fds 1.048576e+06 # HELP process_open_fds Number of open file descriptors. # TYPE process_open_fds gauge process_open_fds 17 # HELP process_resident_memory_bytes Resident memory size in bytes. # TYPE process_resident_memory_bytes gauge process_resident_memory_bytes 1.38641408e+08 # HELP process_start_time_seconds Start time of the process since unix epoch in seconds. # TYPE process_start_time_seconds gauge process_start_time_seconds 1.69881316144e+09 # More snipped lines Ingest server metrics into prometheus Prometheus stores its configuration in prometheus.yml. This example configuration assumes you\u0026rsquo;re running node_exporter and cadvisor on the same server you\u0026rsquo;re running prometheus on.\nglobal: scrape_interval: 15s # Scrape targets every 15 seconds unless otherwise specified query_log_file: /prometheus/data/query.log # Attach these labels to any time series or alerts when communicating with # external systems (federation, remote storage, Alertmanager). # external_labels: # monitor: \u0026#39;codelab-monitor\u0026#39; # A scrape configuration containing exactly one endpoint to scrape: # Here it\u0026#39;s Prometheus itself. scrape_configs: # The job name is added as a label `job=\u0026lt;job_name\u0026gt;` to any timeseries scraped from this config. - job_name: \u0026#39;prometheus\u0026#39; # Override the global default and scrape targets from this job every 5 seconds. scrape_interval: 5s static_configs: - targets: [\u0026#39;localhost:9090\u0026#39;] # Example job for node_exporter - job_name: \u0026#39;node_exporter\u0026#39; scrape_interval: 10s static_configs: - targets: - \u0026#39;localhost:9100\u0026#39; # Assumes we\u0026#39;re running node_exporter on our prometheus server - \u0026#39;foo.example.com:9100\u0026#39; - \u0026#39;bar.example.com:9100\u0026#39; # Example job for cadvisor - job_name: \u0026#39;cadvisor\u0026#39; scrape_interval: 10s static_configs: - targets: - \u0026#39;localhost:8080\u0026#39; - \u0026#39;foo.example.com:9119\u0026#39; - \u0026#39;bar.example.com:9119\u0026#39; Grafana Now that prometheus is working and collecting data, time to visualize the metrics with Grafana.\nAdd the following service stanza to your docker-compose.yaml\ngrafana: image: grafana/grafana:latest container_name: grafana ports: - \u0026#34;3000:3000\u0026#34; volumes: - ./grafana/data:/var/lib/grafana - /etc/hostname:/etc/hostname:ro - /etc/localtime:/etc/localtime:ro - /etc/machine-id:/etc/machine-id:ro - /etc/timezone:/etc/timezone:ro restart: unless-stopped environment: - GF_INSTALL_PLUGINS=grafana-clock-panel,natel-discrete-panel,grafana-piechart-panel Configure Grafana You can now log into grafana at http://hostname:3000 with username admin, password admin. You\u0026rsquo;ll be prompted to set a new admin password.\nNow, connect grafana to your prometheus. Click Connections -\u0026gt; Add new connection on the left side of the window, then search for prometheus, then add prometheus (not prometheus alertmanager!)\nSet the connection to http://prometheus:9090 - since you\u0026rsquo;re running prometheus in the same docker network created by docker-compose, you can refer to it by its container_name field. That\u0026rsquo;s also why we didn\u0026rsquo;t specify ports in the prometheus docker-compose stanza - we don\u0026rsquo;t want it accessed by anything but grafana, and any containers running in the same docker network can access any ports on other containers in that network.\nAdd Dashboards There are a multitude of dashboards to download at grafana.com/grafana/dashboards.\nLet\u0026rsquo;s start by adding the node-exporter-full dashboard. Go to the page and copy it\u0026rsquo;s dashboard ID (1860 as of this post). Then go back to your local grafana, click the blue New button on the right hand side, and select Import.\nEnter the dashboard ID and then click Load. It\u0026rsquo;ll present you a new dialog, click the Prometheus box and select the default Prometheus, and Import.\nYou\u0026rsquo;ll see a nice new dashboard similar to this\nRepeat for the docker dashboard.\nGrafana will automatically populate the host menu with all the hosts your prometheus is scraping.\nThere are many, many dashboards you can experiment with, and of course you can create your own.\nFinally Now you have a working prometheus and grafana stack you can use to monitor the machines in your homelab.\nCombined YAML Here\u0026rsquo;s the combined docker-compose.yaml to start prometheus and grafana.\nversion: \u0026#39;3\u0026#39; services: prometheus: image: prom/prometheus:latest container_name: prometheus # Uncomment only if you want access without the SSL proxy # ports: # - \u0026#34;9090:9090\u0026#34; volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml - ./prometheus/data:/prometheus - /etc/hostname:/etc/hostname:ro - /etc/localtime:/etc/localtime:ro - /etc/machine-id:/etc/machine-id:ro - /etc/timezone:/etc/timezone:ro restart: unless-stopped command: - \u0026#34;--config.file=/etc/prometheus/prometheus.yml\u0026#34; - \u0026#39;--web.enable-lifecycle\u0026#39; grafana: image: grafana/grafana:latest container_name: grafana # Uncomment only if you want access without the SSL proxy # ports: # - \u0026#34;3000:3000\u0026#34; volumes: - ./grafana/data:/var/lib/grafana - /etc/hostname:/etc/hostname:ro - /etc/localtime:/etc/localtime:ro - /etc/machine-id:/etc/machine-id:ro - /etc/timezone:/etc/timezone:ro restart: unless-stopped environment: - GF_INSTALL_PLUGINS=grafana-clock-panel,natel-discrete-panel,grafana-piechart-panel # If you aren\u0026#39;t using SSL you can remove this section networks: default: external: name: ssl_proxy_network PS - Enabling SSL I go into a lot more detail on using nginx-proxy-manager as an SSL proxy here, but the TL;DR is:\nYou need to own a domain If you haven\u0026rsquo;t already done so, create an nginx ssl proxy using the blog post instructions Create new CNAME entries in your domain for grafana.yourdomain.com and prometheus.yourdomain.com that point at the docker server you\u0026rsquo;re hosting prometheus and grafana on Add the ssl_proxy network to the end of your docker-compose.yaml file networks: default: external: name: ssl_proxy_network Delete or comment out all the port stanzas from docker-compose.yaml. This will prevent the containers from being accessed except through the SSL proxy Add a proxy host to your nginx-proxy-manager for grafana.yourdomain.com and set its destination to http://grafana:3000 Add a proxy host to nginx-proxy-manager for prometheus.yourdomain.com and set its destination to http://prometheus:9090. Prometheus doesn\u0026rsquo;t have authentication built in, so add a basic auth to it in nginx-proxy-manager (read the other blog post for details) ","permalink":"https://unixorn.github.io/post/homelab/homelab-setup-prometheus-and-grafana/","summary":"\u003cp\u003eI wanted nice graphs for the various metrics I collect in my homelab and from my Home Assistant server.\u003c/p\u003e\n\u003cp\u003eHere\u0026rsquo;s how I installed Prometheus and Grafana in my homelab to get them.\u003c/p\u003e","title":"Setup Prometheus and Grafana in Your Homelab"},{"content":"Set up nut-upsd and peanut in your homelab I run my Synology server and network switches off a UPS. I decided I wanted a dashboard for the UPS, here\u0026rsquo;s how I did set up Network UPS Tools, aka NUT, and the PeaNUT dashboard.\nTable of Contents Prerequisites Setup Connect the UPS to your server Configure NUT Confirm the driver you need is present Create a docker-compose.yaml for nut-upsd Confirm UPS was detected by NUT Set up Peanut webui Prerequisites A UPS compatible with Network UPS Tools, aka NUT. A linux box with a free UPS port. I\u0026rsquo;m using my Orange Pi 5, but this would work on a Raspberry Pi or any other linux server. A USB-A to USB-B cable to connect the UPS to the linux box Setup Connect the UPS to your server Connect the UPS to your linux server. Check which USB device it\u0026rsquo;s showing up as by running lsusb.\nOn my machine, it appears on bus 2. Note how it shows up on the bus, we\u0026rsquo;ll need it to determine which nut driver to use.\n$ lsusb Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 003 Device 002: ID 0764:0501 Cyber Power System, Inc. CP1500 AVR UPS Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub $ Configure NUT I looked up the CP1500 on the NUTS Hardware Compatibility List, and that says to use the usbhid-ups driver.\nConfirm the driver you need is present docker run --rm --entrypoint /bin/ls instantlinux/nut-upsd /usr/lib/nut | grep YOURDRIVER Create a docker-compose.yaml for nut-upsd Now we can create a docker-compose.yaml file for nut.\nversion: \u0026#39;3.9\u0026#39; services: nut-upsd: image: instantlinux/nut-upsd container_name: nut environment: - TZ=${TZ:-America/Denver} - API_PASSWORD=${API_PASSWORD:-\u0026#39;YourPasswordForAPIaccess\u0026#39;} # Look up your UPS here https://networkupstools.org/stable-hcl.html to # determine which driver to use. - DRIVER=usbhid-ups # If you want the ups to be named something other than \u0026#39;ups\u0026#39;, set the NAME env variable # -NAME=cp800avr devices: # Device numbers are subject to change, so map in the whole bus. NUT # will automatically find the UPS for you. - /dev/bus/usb:/dev/bus/usb ports: - \u0026#34;3493:3493\u0026#34; restart: unless-stopped Start the container with docker-compose up -d\nConfirm UPS was detected by NUT Confirm that nut detected your UPS. telnet YOURHOST 3493, then enter LIST UPS\nIt should look similar to\n$ telnet localhost 3493 Trying ::1... Connected to localhost. Escape character is \u0026#39;^]\u0026#39;. list ups BEGIN LIST UPS UPS ups \u0026#34;UPS\u0026#34; END LIST UPS Now confirm that it is reporting reasonable looking metrics by entering list var ups.\nHere\u0026rsquo;s what NUT reported about my UPS:\nlist var ups BEGIN LIST VAR ups VAR ups battery.charge \u0026#34;100\u0026#34; VAR ups battery.charge.low \u0026#34;10\u0026#34; VAR ups battery.charge.warning \u0026#34;20\u0026#34; VAR ups battery.mfr.date \u0026#34;CPS\u0026#34; VAR ups battery.runtime \u0026#34;1226\u0026#34; VAR ups battery.runtime.low \u0026#34;300\u0026#34; VAR ups battery.type \u0026#34;PbAcid\u0026#34; VAR ups battery.voltage \u0026#34;13.3\u0026#34; VAR ups battery.voltage.nominal \u0026#34;12\u0026#34; VAR ups device.mfr \u0026#34;CPS\u0026#34; VAR ups device.model \u0026#34;CP800AVRa\u0026#34; VAR ups device.serial \u0026#34;CXPHX2000916\u0026#34; VAR ups device.type \u0026#34;ups\u0026#34; VAR ups driver.debug \u0026#34;0\u0026#34; VAR ups driver.flag.allow_killpower \u0026#34;0\u0026#34; VAR ups driver.name \u0026#34;usbhid-ups\u0026#34; VAR ups driver.parameter.pollfreq \u0026#34;30\u0026#34; VAR ups driver.parameter.pollinterval \u0026#34;2\u0026#34; VAR ups driver.parameter.port \u0026#34;auto\u0026#34; VAR ups driver.parameter.synchronous \u0026#34;auto\u0026#34; VAR ups driver.state \u0026#34;quiet\u0026#34; VAR ups driver.version \u0026#34;2.8.1\u0026#34; VAR ups driver.version.data \u0026#34;CyberPower HID 0.8\u0026#34; VAR ups driver.version.internal \u0026#34;0.52\u0026#34; VAR ups driver.version.usb \u0026#34;libusb-1.0.26 (API: 0x1000109)\u0026#34; VAR ups input.voltage \u0026#34;117.0\u0026#34; VAR ups input.voltage.nominal \u0026#34;120\u0026#34; VAR ups output.voltage \u0026#34;117.0\u0026#34; VAR ups ups.beeper.status \u0026#34;enabled\u0026#34; VAR ups ups.delay.shutdown \u0026#34;20\u0026#34; VAR ups ups.delay.start \u0026#34;30\u0026#34; VAR ups ups.load \u0026#34;38\u0026#34; VAR ups ups.mfr \u0026#34;CPS\u0026#34; VAR ups ups.model \u0026#34;CP800AVRa\u0026#34; VAR ups ups.productid \u0026#34;0501\u0026#34; VAR ups ups.realpower.nominal \u0026#34;450\u0026#34; VAR ups ups.serial \u0026#34;CXPHX2000916\u0026#34; VAR ups ups.status \u0026#34;OL\u0026#34; VAR ups ups.test.result \u0026#34;No test initiated\u0026#34; VAR ups ups.timer.shutdown \u0026#34;-60\u0026#34; VAR ups ups.timer.start \u0026#34;-60\u0026#34; VAR ups ups.vendorid \u0026#34;0764\u0026#34; END LIST VAR ups Set up Peanut webui Edit: I now scrape the NUT data into Prometheus and view it with a Grafana dashboard. Details are here.\nIt\u0026rsquo;d be nice to have a pretty web dashboard for the UPS. Enter peanut.\nNow to add the peanut service to our docker-compose.yaml. Because it\u0026rsquo;s in the same docker-compose.yaml file (and therefore by default, the same docker network), we can use the nut container name as the NUT_HOST. We\u0026rsquo;re also going to use depends_on to make it start after the nut container is up and running.\nFor simplicity, I\u0026rsquo;m not including how to make PeaNUT accessible via https. If you do want to secure PeaNUT with SSL, I\u0026rsquo;ve documented how to use nginx-proxy-manager as an SSL reverse proxy here.\nAdd a PeaNUT stanza to docker-compose.yaml.\n# Monitor our UPS version: \u0026#39;3.9\u0026#39; services: nut-upsd: image: instantlinux/nut-upsd container_name: nut environment: - TZ=${TZ:-America/Denver} - API_PASSWORD=${API_PASSWORD:-\u0026#39;aPasswordForAPIaccess\u0026#39;} # Driver found with this tool https://networkupstools.org/stable-hcl.html - DRIVER=usbhid-ups # If you want the ups to be named something other than \u0026#39;ups\u0026#39;, set the NAME env variable # -NAME=cp800avr devices: # Device numbers are subject to change, so map in the bus - /dev/bus/usb:/dev/bus/usb ports: - \u0026#34;3493:3493\u0026#34; restart: unless-stopped peanut: image: brandawg93/peanut:latest container_name: peanut restart: unless-stopped # Don\u0026#39;t start peanut until after nut-upsd is running depends_on: - nut-upsd ports: - 8080:8080 environment: - NUT_HOST=nut - NUT_PORT=3493 - WEB_PORT=8080 Do a docker-compose down \u0026amp;\u0026amp; docker-compose up -d to force docker-compose to recognize the new service, and now you can open https://yourserver:8080 and see a nice dashboard.\nHere\u0026rsquo;s the dashboard for the UPS that drives my network stack \u0026amp; Synology:\nLooks like it\u0026rsquo;s time to get a second UPS and split the load - I\u0026rsquo;d like to have at least 45 minutes of network \u0026amp; Home Asssitant run time if the power goes out.\nEdit: Add link to nginx-proxy-manager post documenting how to put http services like PeaNUT behind an SSL reverse proxy.\n","permalink":"https://unixorn.github.io/post/homelab/homelab-nut-upsd/","summary":"\u003ch1 id=\"set-up-nut-upsd-and-peanut-in-your-homelab\"\u003eSet up nut-upsd and peanut in your homelab\u003c/h1\u003e\n\u003cp\u003eI run my Synology server and network switches off a UPS. I decided I wanted a dashboard for the UPS, here\u0026rsquo;s how I did set up \u003ca href=\"https://networkupstools.org/\"\u003eNetwork UPS Tools\u003c/a\u003e, aka NUT, and the \u003ca href=\"https://github.com/Brandawg93/PeaNUT\"\u003ePeaNUT\u003c/a\u003e dashboard.\u003c/p\u003e","title":"Set up nut-upsd and peanut in your homelab"},{"content":"Released ha-mqtt-discoverable 0.12.0\nNew features since 0.10 Light entities, thanks to ha-enthus1ast Connect to a TLS enabled MQTT Broker using username and password authentication, thanks to trunet Dependency updates to pull in security updates It\u0026rsquo;s been a while since I announced a new release, so I\u0026rsquo;d like to thank the new contributors since my last release announcement.\nDavidMikeSimon ha-enthus1ast kratz00 ti-mo trunet ","permalink":"https://unixorn.github.io/post/hass/ha-mqtt-discoverable-0.12.0/","summary":"\u003cp\u003eReleased ha-mqtt-discoverable 0.12.0\u003c/p\u003e","title":"Released ha-mqtt-discoverable 0.12.0"},{"content":"I ran into a weird Home Assistant scripts issue when I upgraded to 2023.11 from 2023.06 (I know, but I didn’t need the new features and had higher priority stuff to deal with) last week.\nAfter the update, when I went to add or edit a script, I started seeing 500s.\nAfter some messing around, I eventually moved the couple scripts I had from scripts.yaml as inline scripts in configuration.yaml, restarted, and when I went to edit them, HA offered to migrate them into scripts.yaml. Annoyingly, then when I went to create a new script, HA didn’t return 500s, but the new script didn’t show up in the script list either. I could see it in scripts.yaml, though, and configuration.yaml had the expected script: !include scripts.yaml line in it.\nWhat finally fixed it was to make an empty scripts.yaml, restart HA, and then recreate the few scripts I cared about, which was a good excuse to update them.\nBlogging this in case someone else runs into this, I couldn\u0026rsquo;t find a solution on Google.\n","permalink":"https://unixorn.github.io/post/hass/2023-11-weird-ha-scripts-bug/","summary":"\u003cp\u003eI ran into a weird Home Assistant scripts issue when I upgraded to 2023.11 from 2023.06 (I know, but I didn’t need the new features and had higher priority stuff to deal with) last week.\u003c/p\u003e\n\u003cp\u003eAfter the update, when I went to add or edit a script, I started seeing 500s.\u003c/p\u003e\n\u003cp\u003eAfter some messing around, I eventually moved the couple scripts I had from \u003ccode\u003escripts.yaml\u003c/code\u003e as inline scripts in \u003ccode\u003econfiguration.yaml\u003c/code\u003e, restarted, and when I went to edit them, HA offered to migrate them into \u003ccode\u003escripts.yaml\u003c/code\u003e. Annoyingly, \u003cem\u003ethen\u003c/em\u003e when I went to create a new script, HA didn’t return 500s, but the new script didn’t show up in the script list either. I could see it in \u003ccode\u003escripts.yaml\u003c/code\u003e, though, and \u003ccode\u003econfiguration.yaml\u003c/code\u003e had the expected \u003ccode\u003escript: !include scripts.yaml\u003c/code\u003e line in it.\u003c/p\u003e","title":"Weird Home Assistant Scripts Bug"},{"content":"Last Tuesday night, my USG Pro died. I got it secondhand and I got almost five years use out of it, so it was past time for a more performant replacement.\nAfter several factory resets the USG still wouldn\u0026rsquo;t respond to network at all, which is, shall we say, problematic when a device is the security gateway that routes all of my home\u0026rsquo;s network traffic through it. I have Unifi switches and Unifi wireless access points, so I wanted to continue being able to use one controller interface for everything, so Wednesday morning I ordered a Unifi Dream Machine Pro to replace the security gateway. I couldn\u0026rsquo;t bring myself to pay $50 for overnighting it, so we made do with the C3000A modem Centurylink provided to terminate our gigabit drop - its wifi is awful though, so I only put our laptops and the Roku on its wifi network. It was struggling with 5 devices and I didn\u0026rsquo;t want to throw another 40 at it - we needed to be able to work until the replacement arrived.\nUPS Ground was faster than I expected and the UDM Pro arrived at around 7pm Friday. After a couple of days of the house feeling lobotomized because the automation was offline, I started the replacement right away.\nFirst impressions: packaged very well, and they included a little foam strip that they embedded all the mounting screws and cage nuts that was a nice touch - I could pull them out one by one and not worry about the cat supervising my work knocking them all onto the basement floor and scattering them.\nIt would have been nicer if it was recyclable, but it was convenient. The IOS application for setting it up was good, and it connected via Bluettoth so I didn\u0026rsquo;t have to hassle with fping or nmap to find out what IP the UDM Pro was assigned by the router, and I was able to use it to kick off software updates.\nWhile I was waiting for the UDM to arrive, I had imported a backup from my old controller (yes, I should have updated it long ago, but it never got up to the top of my backlog since it was working just fine) running 5.10.17 into a new controller running in the latest docker container so I could export from the new container into a file with the latest format. Even after updating the UDM, it claimed that the one I’d exported from network 7.4.162 running in docker was too new to import but at least it was happy to read the 5.10.17 backup. The one thing I didn’t have documented about my network were the MAC addresses tied to all my static DHCP assignments, so that’s another project. I want to give Unifi kudos for importing a backup that was two major revisions old with absolutely no hassle - I had done that ahead of time because I was dreading having to load old controller versions sequentially to get the backup updated to the present, but it turned out to be a non-issue.\nIn hindsight the backup file version issue was my fault - I had updated the base UnifiOS for the device, and didn\u0026rsquo;t realize that the Network application needed a separate update until after I\u0026rsquo;d already just imported the older backup version.\nAfter restoring from the backup file, both of my switches both were immediately adopted as soon as I plugged their uplink connections into the UDM, but my APs both needed an advanced adoption before they\u0026rsquo;d join to the controller. Fortunately I\u0026rsquo;d enabled SSH in the device settings when I initially set them up, so it only took a couple of minutes to get them adopted as well.\nWith the exception of my N2 running armbian, everything in the rack just came up and quietly started working again as soon as the switches were connected to the UDM and they could get DHCP leases. In particular, my moosefs cluster came back online with no issues - even after being unable to talk to the master for 3 days the chunkservers all just reconnected to the master as soon as they got a DHCP lease. The master started some replication of blocks until all the chunkservers had connected and it realized all the files had hit their replication goals. I didn\u0026rsquo;t even have to remount the filesystem on the client machines.\n","permalink":"https://unixorn.github.io/post/2023-08-homelab-upgrade/","summary":"\u003cp\u003eLast Tuesday night, my USG Pro died. I got it secondhand and I got almost five years use out of it, so it was past time for a more performant replacement.\u003c/p\u003e","title":"Homelab Upgrade from USG-Pro to UDM-Pro"},{"content":"In the next few posts, I\u0026rsquo;m going to document how to set up Home Assistant (HA) from scratch. We\u0026rsquo;re going to want to protect the admin UI interfaces for HA and its support services with SSL, and add authentication to services that don\u0026rsquo;t provide it themselves.\nWe\u0026rsquo;re going to do this with Nginx Proxy Manager because it has built in support for using LetsEncrypt to obtain free SSL certificates, supports adding authentication to services that don\u0026rsquo;t do it themselves, and is overall easy to use.\nBefore I start writing more Home Assitant articles, let\u0026rsquo;s set up a SSL proxy server to keep everything secure.\nWe\u0026rsquo;re going to run the proxy server and the services behind it in containers with docker-compose to make things easier.\nPre-requisites A linux machine with docker installed that doesn\u0026rsquo;t already have something running on port 80 A domain that is using Route 53 for DNS A server on your network with a static IP, for example 10.1.2.3 A DNS entry pointing at your server, like demo.yourdomain.com Setup All screen shots and other instructions are valid as of 2023-07-22 when I wrote this post.\nRather than set up a web server and expose it to the internet so that LetsEncrypt can validate that you own the domain, we\u0026rsquo;re going to configure nginx-proxy-manager to use a DNS01 challenge. This lets LetsEncrypt validate ownership of your domain by special records that will be added to your domain\u0026rsquo;s DNS entries during certificate creation / renewal by our nginx-proxy-manager container.\nnginx-proxy-manager supports many DNS providers. I use Route 53 so I\u0026rsquo;ll use that for this article.\nSet up a domain in AWS Route 53 If you already have a domain, you can transfer it to Route 53. I think I pay about $1.50 a month for the domains I host there. And that\u0026rsquo;s in total, not per domain. My homelab doesn\u0026rsquo;t get a ton of DNS queries each month.\nIf you don\u0026rsquo;t own a domain or don\u0026rsquo;t want to transfer one you own to R53, you can use Amazon for your registrar. They support many tlds, and some of the ones they support cost less than $13 a year - in my opinion it\u0026rsquo;s worth a dollar a month to have a separate domain for a homelab.\nSet up Route 53 You don\u0026rsquo;t want to use your root IAM credentials with nginx-proxy-manager. It is never a good idea to use root IAM credentials for anything. Instead, we\u0026rsquo;re going to create an IAM user that only has privileges to affect your Route53-hosted domains.\nThis is pretty tedious, so it\u0026rsquo;s a good thing you only need to do it once. Log in to the AWS console and select Identity and Access Management (IAM).\nCreate an IAM policy We\u0026rsquo;re going to start by creating a policy that grants control over Route53, so select Policies from the sidebar, then hit the Create Policy button.\nCreating the policy first makes it easier to attach to the IAM group when we create it in the next step.\nI\u0026rsquo;ve already created a working policy, so instead of manually adding permissions, click JSON to the right of where it says Policy Editor and paste in this JSON snippet\n{ \u0026#34;Version\u0026#34;: \u0026#34;2012-10-17\u0026#34;, \u0026#34;Statement\u0026#34;: [ { \u0026#34;Effect\u0026#34;: \u0026#34;Allow\u0026#34;, \u0026#34;Action\u0026#34;: \u0026#34;route53:ListHostedZonesByName\u0026#34;, \u0026#34;Resource\u0026#34;: \u0026#34;*\u0026#34; }, { \u0026#34;Effect\u0026#34;: \u0026#34;Allow\u0026#34;, \u0026#34;Action\u0026#34;: [ \u0026#34;route53:ChangeResourceRecordSets\u0026#34;, \u0026#34;route53:ListResourceRecordSets\u0026#34;, \u0026#34;route53:GetHostedZone\u0026#34; ], \u0026#34;Resource\u0026#34;: [ \u0026#34;arn:aws:route53:::hostedzone/THX1138XYZZY\u0026#34; ] } ] } If you want to restrict it to only controlling a specific domain, update the \u0026quot;Resource\u0026quot;: \u0026quot;*\u0026quot; line - that\u0026rsquo;s out of scope for this post, though.\nAs of 2023-07-22, it should look like this:\nClick Next, and give your new policy a name and a description, for example blog-letsencrypt.\nCreate an IAM group Now you need to create a group, so click User Groups in the side bar, then hit the Create Group button. Give it a name like blog-le-users\nUnder Attach permissions policy* select the blog-letsencrypt policy you just created.\nHit the Create Group button on the bottom right of the page.\nCreate an IAM user Now that the group and policy are created, you can create your r53 IAM user. Click Users in the side bar, then click the Add Users button on the right side of the screen.\nGive it a name like r53-acme-user and click Next.\nDon\u0026rsquo;t bother to check the Provide user access to the AWS Management Console checkbox, this user will only be used by your proxy manager. Select the blog-le-users group\nClick Next again.\nYou\u0026rsquo;ll see a review and create page:\nClick Create user on the bottom right of the page.\nOne last thing - you\u0026rsquo;ll need to create access credentials for the user. Click on Users in the side bar again, then your brand new r53-acme-user user.\nYou\u0026rsquo;ll see an info page about the user, click the Security Credentials tab\nScroll down to Access Keys, and click Create access key.\nClick Other on the next page\nClick Next. Put in letsencrypt for the description,and click Create access key again.\nYou\u0026rsquo;ll see something like\nYou only get one chance to see the secret key, so store it in your password manager. It isn\u0026rsquo;t a huge deal if you lose it, you can always create another access key for your user. Copy the access key \u0026amp; secret keys into your password manager for later use.\nInstall nginx-proxy-manager Now that you have the DNS domain on Route 53 and have created an IAM user with rights to update it, you can set up nginx-proxy-manager and have it generate LetsEncrypt SSL certificates.\nFor the SSL proxy, I like to set it to use an external docker network. This lets you run other docker containers behind it easily, without having to cram them all into the same docker-compose.yaml file.\nCreating the network is easy - docker network create ssl_proxy_network and we\u0026rsquo;re good to go.\nHere\u0026rsquo;s the docker-compose.yaml file I use to start nginx-proxy-manager. As of 2023-07-21, there\u0026rsquo;s a bug getting Route 53 to work with the latest tag, but it does work with the docker image with github-pr-2971 tag. I\u0026rsquo;ll update this post when that fix gets merged upstream.\nConfigure docker-compose version: \u0026#39;3\u0026#39; services: nginx-proxy-manager: # image: \u0026#39;jc21/nginx-proxy-manager:latest\u0026#39; # Use github-pr-2971 until the fix is merged image: jc21/nginx-proxy-manager:github-pr-2971 restart: unless-stopped container_name: nginx-proxy-manager ports: - \u0026#39;80:80\u0026#39; - \u0026#39;81:81\u0026#39; - \u0026#39;443:443\u0026#39; volumes: - ./nginx/data:/data - ./nginx/letsencrypt:/etc/letsencrypt - /etc/hostname:/etc/hostname:ro - /etc/localtime:/etc/localtime:ro - /etc/machine-id:/etc/machine-id:ro - /etc/timezone:/etc/timezone:ro # environment: # DISABLE_IPV6: \u0026#39;true\u0026#39; networks: default: external: name: ssl_proxy_network You can download this here and put it on your server.\nNow you can run docker-compose up -d and docker-compose will download the image, create the nginx/data and nginx/letsencrypt directories, and start the proxy manager. The image is roughly 700 megs, and on my test Raspberry Pi, it took a couple of minutes to download and start. Run docker-compose logs -f to see what\u0026rsquo;s happening inside the container as it does initial setup. On slower hardware, it can take over a minute to start up and get the service running, especially on the first run, so be patient.\nUsage Set up a demo container to proxy I\u0026rsquo;m going to use a simple nginx demo server to demo SSL proxying. We don\u0026rsquo;t need anything fancy, just something that will serve a web page so we can confirm the proxy is working. Make a demo directory, and put the following snippet into its docker-compose.yaml file.\nI deliberately did not set up port forwarding in this configuration file because I don\u0026rsquo;t want it to be accessible from outside our ssl_proxy_network docker network - that\u0026rsquo;s what the proxy is for.\nversion: \u0026#39;3\u0026#39; services: demo: image: nginxdemos/hello restart: unless-stopped container_name: demo volumes: - /etc/hostname:/etc/hostname:ro - /etc/localtime:/etc/localtime:ro - /etc/machine-id:/etc/machine-id:ro - /etc/timezone:/etc/timezone:ro networks: default: external: name: ssl_proxy_network Start it up with docker-compose up -d. Because you\u0026rsquo;re using the same ssl_proxy_network that you created earlier, and your nginx-proxy-manager is also using that network, the two containers will be able to communicate with each other. Open yourserver.example.com in your browser, it should give you an error message about being unable to connect. You want to see that because you don\u0026rsquo;t want the service accessible except through the proxy.\nConfigure the Proxy Server Now that you have the demo backend and the nginx-proxy-manager proxy running, let\u0026rsquo;s set up proxying.\nFirst, log into the proxy manager at https://yourserver.example.com:81. The default username is admin@example.com, and the password is changeme. It\u0026rsquo;ll make you change those when you first log in.\nAdd an SSL certificate. To make things easier later, you\u0026rsquo;re going to create a wildcard SSL certificate for your domain. This will let you run as many services as you like as servicename.example.com on the server, and not have to specify port numbers, just add new DNS entries pointing at your server.\nSelect SSL Certificates. You\u0026rsquo;ll see\nClick Add SSL certificate. You should see something like this:\nPut *.yourdomain.com in as the domain name. Select Use DNS Challenge Put in your email address so LetsEncrypt can send you notifications if there\u0026rsquo;s an issue with your certificate later. Select Route 53 for the DNS provider Set the AWS access key to the one you created for your r53-acme-user IAM user earlier Set the AWS secret key Agree to the terms of service, and hit save. This can take a few minutes, especially if your machine is low powered or the LetsEncrypt backend is under load.\nIf everything went according to plan, you should see something like this:\nAdd a proxy host Click hosts, then proxy host from the submenu.\nYou\u0026rsquo;ll see a dialog like this, hit Add Proxy Host\nYou could have several different DNS names point at the same backend, but for this example, you only want demo.yourdomain.com, so stick that in the Domain Names field. The hostname should be the name of the container we\u0026rsquo;re proxying - the demo service we set up earlier uses http on port 80, so enter those here. Even though there isn\u0026rsquo;t a port entry in the docker-compose file, the ports on the backend will be accessible from other containers on the same docker network, and this is why you configured the nginx-proxy-manager and demo services to both use the external ssl_proxy_network network. I haven\u0026rsquo;t encountered problems with any backends using them, so I turn on Cache Assets, Websockets Support and Block Common Exploits since nginx-proxy-manager can add them. Don\u0026rsquo;t hit save yet, you still need to attach an SSL certificate. Click on the SSL tab at the top of the dialog.\nSelect the *.yourdomain.com certificate you created earlier\nFinally, turn on Force SSL and HTTP/2 Support\nConfirm things are working Go to yourserver.yourdomain.com and you should see the demo page\nAnd there should be a lock icon in your browser.\nAdd authentication Some services you\u0026rsquo;re going to want to run (like our demo) don\u0026rsquo;t have any user authentication. Fortunately, nginx-proxy-manager lets you insert a basic auth login step before a connection is created to the service it\u0026rsquo;s proxying.\nClick Access Lists in the main menu, and Add Access List.\nPut in a name - I used demo-acl\nClick on Authorization\nPut in a username and password. I used demo and demo here.\nI don\u0026rsquo;t care where the service is accessed from as long as they have a valid username \u0026amp; password, so click on the Access tab, so I\u0026rsquo;m allowing from 0.0.0.0/8 - you could put in your lan\u0026rsquo;s network and netmask if you wanted to be more strict.\nSave. Go back to Hosts -\u0026gt; Proxy Hosts, and use the \u0026hellip; menu on the right of your demo proxy entry to Edit.\nSelect your new ACL and Save\nNow when you go to the service, it should prompt you with a basic auth dialog.\nLock down the proxy manager management UI Now that you have SSL proxying working, there\u0026rsquo;s one last thing to do - putting the management interface behind itself, so that it is protected by SSL too.\nIn this example, I\u0026rsquo;m running nginx-proxy-manager on cthulhu, one of the HC2s in my homelab. The container is unimaginatively named nginxproxymanager, so I set up the proxy manager to route cthulhu.miniclusters.rocks to port 81 of the nginxproxymanager as seen below.\nAfter you confirm it\u0026rsquo;s working, do a docker-compose down in your proxy manager directory, delete or comment out the - '81:81' line in docker-compose.yaml, and restart it with docker-compose up -d. If you\u0026rsquo;ve done it correctly, if you try to access port 81 on your host you\u0026rsquo;ll get a connection refused, but if you connect to it with https://yourmachine.example.com, you\u0026rsquo;ll see the admin interface come up and it\u0026rsquo;ll have a lock icon to show it\u0026rsquo;s an SSL connection.\nUpdate 2023-07-30: Add the instructions on securing the management interface with SSL that I forgot to commit.\n","permalink":"https://unixorn.github.io/post/hass/2023-07-09-set-up-nginx-proxy-manager/","summary":"\u003cp\u003eIn the next few posts, I\u0026rsquo;m going to document how to set up Home Assistant (HA) from scratch. We\u0026rsquo;re going to want to protect the admin UI interfaces for HA and its support services with SSL, and add authentication to services that don\u0026rsquo;t provide it themselves.\u003c/p\u003e\n\u003cp\u003eWe\u0026rsquo;re going to do this with \u003ca href=\"https://nginxproxymanager.com/\"\u003eNginx Proxy Manager\u003c/a\u003e because it has built in support for using \u003ca href=\"https://letsencrypt.org\"\u003eLetsEncrypt\u003c/a\u003e to obtain free SSL certificates, supports adding authentication to services that don\u0026rsquo;t do it themselves, and is overall easy to use.\u003c/p\u003e\n\u003cp\u003eBefore I start writing more Home Assitant articles, let\u0026rsquo;s set up a SSL proxy server to keep everything secure.\u003c/p\u003e","title":"Set up nginx-proxy-manager with LetsEncrypt SSL certificates"},{"content":"Some of my posts assume that the user already has ESPHome installed, so I\u0026rsquo;m documenting how to install it here so I don\u0026rsquo;t have to repeat it everywhere.\nInstallation This assumes you already have a machine with docker and docker-compose installed on it. If not, go install that first. You don\u0026rsquo;t need to run the ESPHome server all of the time, just when you\u0026rsquo;re configuring/adding devices.\nYou\u0026rsquo;re going to need a USB-\u0026gt;UART adapter to reflash devices - mine is a DollaTek 3.3V / 5V USB to TTL Converter CH340G UART Serial Adapter Module. If you don\u0026rsquo;t already have an adapter, make sure that the one you get works with both 3.3 volt and 5 volt devices for flexibility later.\nPrep your ESPHome directory First, make a directory to put a docker-compose.yml file in. Create a config subdirectory in it so that ESPHome has someplace to put device configuration YAML files. You\u0026rsquo;re going to want to keep those files to make it easier to update your devices later.\nCreate a docker-compose.yml file with the following contents:\nversion: \u0026#39;3\u0026#39; services: esphome: container_name: esphome environment: - TZ=America/Denver image: ghcr.io/esphome/esphome ports: - \u0026#34;6052:6052/tcp\u0026#34; volumes: - $PWD/config:/config - /etc/localtime:/etc/localtime:ro privileged: true restart: unless-stopped You\u0026rsquo;re also going to need a file to store secrets like your WIFI network name and password. Create config/secrets.yml with the following contents:\nwifi_ssid: \u0026#34;WIFI_NETWORK_TO_JOIN\u0026#34; wifi_password: \u0026#34;WIFI_PASSWORD\u0026#34; api_password: \u0026#34;AnAPIPassword\u0026#34; This will let you use secrets in your device configurations as !secretname.\nRun it I only run ESPHome when I\u0026rsquo;m adding a new device or updating an existing one, so I usually run it on my laptop. It\u0026rsquo;s more convenient this way since I don\u0026rsquo;t have to go downstairs to the server rack to connect whatever device I\u0026rsquo;m working on.\nRun docker-compose up -d on your laptop and you\u0026rsquo;ll see the ESPHome UI at http://localhost:6532.\nYou need a browser that supports Webserial to be able to flash devices directly from the ESPHome webui - on my Mac, I use Chrome for this since Firefox doesn\u0026rsquo;t support it.\n","permalink":"https://unixorn.github.io/post/hass/2023-05-21-install-esphome/","summary":"\u003cp\u003eSome of my posts assume that the user already has \u003ca href=\"https://esphome.io/\"\u003eESPHome\u003c/a\u003e installed, so I\u0026rsquo;m documenting how to install it here so I don\u0026rsquo;t have to repeat it everywhere.\u003c/p\u003e","title":"Installing ESPHome"},{"content":"As part of moving from Twitter to Mastodon I decided to add comments to the blog using Fediverse posts. Fortunately, Carl Schwan showed how he does it on his blog here.\nHere are the exact tweaks to his post I did to get it working with the papermod theme I\u0026rsquo;m using on this blog.\nBackground I\u0026rsquo;ve been meaning to add comments to this blog for a while, and migrating to mastodon from twitter made me think that there had to way to use mastodon toots as comments. I don\u0026rsquo;t have a lot of javascript experience since I only work with back-end services and don\u0026rsquo;t do any front-end work so I hadn\u0026rsquo;t gotten around to it, then I saw Carl\u0026rsquo;s post. Veronica Berglyd Olsen extended Carl\u0026rsquo;s post to make the comments display threaded on her blog.\nCarl did all the heavy lifting and Veronica extended his work. I did have to do some minor tweaks both to merge Veronica\u0026rsquo;s update and because I\u0026rsquo;m running mainline Hugo instead of a variant build with SCSS support, make everything work with raw CSS instead of SCSS .\nInstallation Carl is kind enough to publish his blog\u0026rsquo;s source in a public repository on GitLab.\nAny broken bits (like not realizing there was more to getting this to css than stripping the var statements out of it in the original version of this post) here are errors made by me when I modified Carl \u0026amp; Veronica\u0026rsquo;s code - look at their blog posts if you run into any issues.\nInstead of directly modifying the theme files in place, we\u0026rsquo;re going to override them. This will make it easier to update the theme without breaking all your comments.\nFirst, make the override directories you\u0026rsquo;re going to need with mkdir -p layouts/default layouts/partials/mastodon static/css static/assets/js at the root level of your blog repository.\nNow that the directories are in place, we\u0026rsquo;re going to make a partial with Carl\u0026rsquo;s comment code (as modified by Veronica) in it. Put this combined code in layouts/partials/mastodon/mastodon.html.\n{{ with .Params.comments }} \u0026lt;div class=\u0026#34;article-content\u0026#34;\u0026gt; \u0026lt;h2\u0026gt;Comments\u0026lt;/h2\u0026gt; \u0026lt;p\u0026gt;You can use your Mastodon account to reply to this\u0026lt;a class=\u0026#34;button\u0026#34; href=\u0026#34;https://{{ .host }}/@{{ .username }}/{{ .id }}\u0026#34;\u0026gt;post\u0026lt;/a\u0026gt;.\u0026lt;/p\u0026gt; \u0026lt;p\u0026gt;\u0026lt;button class=\u0026#34;button\u0026#34; id=\u0026#34;replyButton\u0026#34; href=\u0026#34;https://{{ .host }}/@{{ .username }}/{{ .id }}\u0026#34;\u0026gt;Reply\u0026lt;/button\u0026gt;\u0026lt;/p\u0026gt; \u0026lt;dialog id=\u0026#34;toot-reply\u0026#34; class=\u0026#34;mastodon\u0026#34; data-component=\u0026#34;dialog\u0026#34;\u0026gt; \u0026lt;h3\u0026gt;Reply to {{ .username }}\u0026#39;s post\u0026lt;/h3\u0026gt; \u0026lt;p\u0026gt; With an account on the Fediverse or Mastodon, you can respond to this post. Since Mastodon is decentralized, you can use your existing account hosted by another Mastodon server or compatible platform if you don\u0026#39;t have an account on this one. \u0026lt;/p\u0026gt; \u0026lt;p\u0026gt;Copy and paste this URL into the search field of your favourite Fediverse app or the web interface of your Mastodon server.\u0026lt;/p\u0026gt; \u0026lt;div class=\u0026#34;copypaste\u0026#34;\u0026gt; \u0026lt;input type=\u0026#34;text\u0026#34; readonly=\u0026#34;\u0026#34; value=\u0026#34;https://{{ .host }}/@{{ .username }}/{{ .id }}\u0026#34;\u0026gt; \u0026lt;button class=\u0026#34;button\u0026#34; id=\u0026#34;copyButton\u0026#34;\u0026gt;Copy\u0026lt;/button\u0026gt; \u0026lt;button class=\u0026#34;button\u0026#34; id=\u0026#34;cancelButton\u0026#34;\u0026gt;Close\u0026lt;/button\u0026gt; \u0026lt;/div\u0026gt; \u0026lt;/dialog\u0026gt; \u0026lt;p id=\u0026#34;mastodon-comments-list\u0026#34;\u0026gt;\u0026lt;button id=\u0026#34;load-comment\u0026#34; class=\u0026#34;button\u0026#34;\u0026gt;Load comments\u0026lt;/button\u0026gt;\u0026lt;/p\u0026gt; \u0026lt;noscript\u0026gt;\u0026lt;p\u0026gt;You need JavaScript to view the comments.\u0026lt;/p\u0026gt;\u0026lt;/noscript\u0026gt; \u0026lt;script src=\u0026#34;/assets/js/purify.min.js\u0026#34;\u0026gt;\u0026lt;/script\u0026gt; \u0026lt;script type=\u0026#34;text/javascript\u0026#34;\u0026gt; const dialog = document.querySelector(\u0026#39;dialog\u0026#39;); document.getElementById(\u0026#39;replyButton\u0026#39;).addEventListener(\u0026#39;click\u0026#39;, () =\u0026gt; { dialog.showModal(); }); document.getElementById(\u0026#39;copyButton\u0026#39;).addEventListener(\u0026#39;click\u0026#39;, () =\u0026gt; { navigator.clipboard.writeText(\u0026#39;https://{{ .host }}/@{{ .username }}/{{ .id }}\u0026#39;); }); document.getElementById(\u0026#39;cancelButton\u0026#39;).addEventListener(\u0026#39;click\u0026#39;, () =\u0026gt; { dialog.close(); }); dialog.addEventListener(\u0026#39;keydown\u0026#39;, e =\u0026gt; { if (e.key === \u0026#39;Escape\u0026#39;) dialog.close(); }); const dateOptions = { year: \u0026#34;numeric\u0026#34;, month: \u0026#34;numeric\u0026#34;, day: \u0026#34;numeric\u0026#34;, hour: \u0026#34;numeric\u0026#34;, minute: \u0026#34;numeric\u0026#34;, }; function escapeHtml(unsafe) { return unsafe .replace(/\u0026amp;/g, \u0026#34;\u0026amp;amp;\u0026#34;) .replace(/\u0026lt;/g, \u0026#34;\u0026amp;lt;\u0026#34;) .replace(/\u0026gt;/g, \u0026#34;\u0026amp;gt;\u0026#34;) .replace(/\u0026#34;/g, \u0026#34;\u0026amp;quot;\u0026#34;) .replace(/\u0026#39;/g, \u0026#34;\u0026amp;#039;\u0026#34;); } document.getElementById(\u0026#34;load-comment\u0026#34;).addEventListener(\u0026#34;click\u0026#34;, function() { document.getElementById(\u0026#34;load-comment\u0026#34;).innerHTML = \u0026#34;Loading\u0026#34;; fetch(\u0026#39;https://{{ .host }}/api/v1/statuses/{{ .id }}/context\u0026#39;) .then(function(response) { return response.json(); }) .then(function(data) { if(data[\u0026#39;descendants\u0026#39;] \u0026amp;\u0026amp; Array.isArray(data[\u0026#39;descendants\u0026#39;]) \u0026amp;\u0026amp; data[\u0026#39;descendants\u0026#39;].length \u0026gt; 0) { document.getElementById(\u0026#39;mastodon-comments-list\u0026#39;).innerHTML = \u0026#34;\u0026#34;; data[\u0026#39;descendants\u0026#39;].forEach(function(reply) { reply.account.display_name = escapeHtml(reply.account.display_name); reply.account.reply_class = reply.in_reply_to_id == \u0026#34;{{ .id }}\u0026#34; ? \u0026#34;reply-original\u0026#34; : \u0026#34;reply-child\u0026#34;; reply.account.emojis.forEach(emoji =\u0026gt; { reply.account.display_name = reply.account.display_name.replace(`:${emoji.shortcode}:`, `\u0026lt;img src=\u0026#34;${escapeHtml(emoji.static_url)}\u0026#34; alt=\u0026#34;Emoji ${emoji.shortcode}\u0026#34; height=\u0026#34;20\u0026#34; width=\u0026#34;20\u0026#34; /\u0026gt;`); }); mastodonComment = `\u0026lt;div class=\u0026#34;mastodon-wrapper\u0026#34;\u0026gt; \u0026lt;div class=\u0026#34;comment-level ${reply.account.reply_class}\u0026#34;\u0026gt;\u0026lt;svg viewBox=\u0026#34;0 0 32 32\u0026#34; xmlns=\u0026#34;http://www.w3.org/2000/svg\u0026#34; fill=\u0026#34;none\u0026#34; transform=\u0026#34;rotate(180)\u0026#34;\u0026gt;\u0026lt;g id=\u0026#34;SVGRepo_bgCarrier\u0026#34; stroke-width=\u0026#34;0\u0026#34;\u0026gt;\u0026lt;/g\u0026gt;\u0026lt;g id=\u0026#34;SVGRepo_tracerCarrier\u0026#34; stroke-linecap=\u0026#34;round\u0026#34; stroke-linejoin=\u0026#34;round\u0026#34;\u0026gt;\u0026lt;/g\u0026gt;\u0026lt;g id=\u0026#34;SVGRepo_iconCarrier\u0026#34;\u0026gt; \u0026lt;path stroke=\u0026#34;#535358\u0026#34; stroke-linecap=\u0026#34;round\u0026#34; stroke-linejoin=\u0026#34;round\u0026#34; stroke-width=\u0026#34;2\u0026#34; d=\u0026#34;M5.608 12.526l7.04-6.454C13.931 4.896 16 5.806 16 7.546V11c13 0 11 16 11 16s-4-10-11-10v3.453c0 1.74-2.069 2.65-3.351 1.475l-7.04-6.454a2 2 0 010-2.948z\u0026#34;\u0026gt;\u0026lt;/path\u0026gt; \u0026lt;/g\u0026gt;\u0026lt;/svg\u0026gt;\u0026lt;/div\u0026gt; \u0026lt;div class=\u0026#34;mastodon-comment\u0026#34;\u0026gt; \u0026lt;div class=\u0026#34;avatar\u0026#34;\u0026gt; \u0026lt;img src=\u0026#34;${escapeHtml(reply.account.avatar_static)}\u0026#34; height=60 width=60 alt=\u0026#34;\u0026#34;\u0026gt; \u0026lt;/div\u0026gt; \u0026lt;div class=\u0026#34;content\u0026#34;\u0026gt; \u0026lt;div class=\u0026#34;author\u0026#34;\u0026gt; \u0026lt;a href=\u0026#34;${reply.account.url}\u0026#34; rel=\u0026#34;nofollow\u0026#34;\u0026gt; \u0026lt;span\u0026gt;${reply.account.display_name}\u0026lt;/span\u0026gt; \u0026lt;span class=\u0026#34;disabled\u0026#34;\u0026gt;${escapeHtml(reply.account.acct)}\u0026lt;/span\u0026gt; \u0026lt;/a\u0026gt; \u0026lt;a class=\u0026#34;date\u0026#34; href=\u0026#34;${reply.uri}\u0026#34; rel=\u0026#34;nofollow\u0026#34;\u0026gt; ${reply.created_at.substr(0, 10)} \u0026lt;/a\u0026gt; \u0026lt;/div\u0026gt; \u0026lt;div class=\u0026#34;mastodon-comment-content\u0026#34;\u0026gt;${reply.content}\u0026lt;/div\u0026gt; \u0026lt;/div\u0026gt; \u0026lt;/div\u0026gt; \u0026lt;/div\u0026gt;`; document.getElementById(\u0026#39;mastodon-comments-list\u0026#39;).appendChild(DOMPurify.sanitize(mastodonComment, {\u0026#39;RETURN_DOM_FRAGMENT\u0026#39;: true})); }); } else { document.getElementById(\u0026#39;mastodon-comments-list\u0026#39;).innerHTML = \u0026#34;\u0026lt;p\u0026gt;No comments found\u0026lt;/p\u0026gt;\u0026#34;; } }); }); \u0026lt;/script\u0026gt; \u0026lt;/div\u0026gt; {{ end }} If you want to change the threading icon, find another svg and replace the \u0026lt;svg\u0026gt;....\u0026lt;/svg\u0026gt; tag in the comment-level div.\nThe code needs a css file to define how the comments are going to look. I defined the background-color, padding, and border-radius inline and compiled Carl\u0026rsquo;s original scss to css with scss-to-css instead of using SCSS variables. I\u0026rsquo;m running stock hugo and didn\u0026rsquo;t want to switch to the variant supporting SCSS since I\u0026rsquo;m not using it anywhere else.\nCarl\u0026rsquo;s original SCSS code is here.\nHere\u0026rsquo;s a compiled version (with Veronica\u0026rsquo;s additions merged in) below in static/css/mastodon.css.\n.mastodon-wrapper { display: flex; gap: 3rem; flex-direction: row; } .comment-level { max-width: 3rem; min-width: 3rem; } .reply-original { display: none; } #article-comments div.reply-original { display: none; } #article-comments div.reply-child { display: block; flex: 0 0 1.75rem; text-align: right; } .mastodon-comment { background-color: #e9e5e5; border-radius: 10px; padding: 30px; margin-bottom: 1rem; display: flex; gap: 1rem; flex-direction: column; flex-grow: 2; } .mastodon-comment .comment { display: flex; flex-direction: row; gap: 1rem; flex-wrap: true; } .mastodon-comment .comment-avatar img { width: 6rem; } .mastodon-comment .content { flex-grow: 2; } .mastodon-comment .comment-author { display: flex; flex-direction: column; } .mastodon-comment .comment-author-name { font-weight: bold; } .mastodon-comment .comment-author-name a { display: flex; align-items: center; } .mastodon-comment .comment-author-date { margin-left: auto; } .mastodon-comment .disabled { color: #34495e; } .mastodon-comment-content p:first-child { margin-top: 0; } .mastodon { --dlg-bg: #282c37; --dlg-w: 600px; --dlg-color: #9baec8; --dlg-button-p: 0.75em 2em; --dlg-outline-c: #00D9F5; } .copypaste { display: flex; align-items: center; gap: 10px; } .copypaste input { display: block; font-family: inherit; background: #17191f; border: 1px solid #8c8dff; color: #9baec8; border-radius: 4px; padding: 6px 9px; line-height: 22px; font-size: 14px; transition: border-color 0.3s linear; flex: 1 1 auto; overflow: hidden; } .copypaste .button { border: 10px; border-radius: 4px; box-sizing: border-box; color: #fff; cursor: pointer; display: inline-block; font-family: inherit; font-size: 15px; font-weight: 500; letter-spacing: 0; line-height: 22px; overflow: hidden; padding: 7px 18px; position: relative; text-align: center; text-decoration: none; text-overflow: ellipsis; white-space: nowrap; width: auto; background-color: #232730; } .copypaste .button:hover { background-color: #16181e; } Update: Stewart Wright posted an article based on this, but his css works in dark mode too - add this to mastodon.css to enable dark mode.\n.dark .mastodon-comment { background-color: #36383d; } .dark .mastodon-comment .disabled { color: #ad55fd; } It also needs a local copy of DOMPurify - curl https://raw.githubusercontent.com/cure53/DOMPurify/main/dist/purify.min.js \u0026gt; static/assets/js/purify.min.js\nFinally, create layouts/default/single.html\nFirst we need to add a link to the style sheet we added - I added a link to the stylesheet at the beginning \u0026lt;link rel=\u0026quot;stylesheet\u0026quot; type=\u0026quot;text/css\u0026quot; href=\u0026quot;{{.Site.BaseURL}}css/mastodon.css\u0026quot; /\u0026gt;\nWe also need to add the mastodon.html partial file to present the Load Commants and Reply buttons.\nRight after the post-content div, I added\n\u0026lt;div\u0026gt; {{ partial \u0026#34;mastodon/mastodon.html\u0026#34; .}} \u0026lt;/div\u0026gt; Here\u0026rsquo;s the full version of my modified copy of the papermod layout file so you don\u0026rsquo;t have to edit it yourself.\n{{- define \u0026#34;main\u0026#34; }} \u0026lt;link rel=\u0026#34;stylesheet\u0026#34; type=\u0026#34;text/css\u0026#34; href=\u0026#34;{{.Site.BaseURL}}css/mastodon.css\u0026#34; /\u0026gt; \u0026lt;article class=\u0026#34;post-single\u0026#34;\u0026gt; \u0026lt;header class=\u0026#34;post-header\u0026#34;\u0026gt; {{ partial \u0026#34;breadcrumbs.html\u0026#34; . }} \u0026lt;h1 class=\u0026#34;post-title\u0026#34;\u0026gt; {{ .Title }} {{- if .Draft }}\u0026lt;sup\u0026gt;\u0026lt;span class=\u0026#34;entry-isdraft\u0026#34;\u0026gt;\u0026amp;nbsp;\u0026amp;nbsp;[draft]\u0026lt;/span\u0026gt;\u0026lt;/sup\u0026gt;{{- end }} \u0026lt;/h1\u0026gt; {{- if .Description }} \u0026lt;div class=\u0026#34;post-description\u0026#34;\u0026gt; {{ .Description }} \u0026lt;/div\u0026gt; {{- end }} {{- if not (.Param \u0026#34;hideMeta\u0026#34;) }} \u0026lt;div class=\u0026#34;post-meta\u0026#34;\u0026gt; {{- partial \u0026#34;post_meta.html\u0026#34; . -}} {{- partial \u0026#34;translation_list.html\u0026#34; . -}} {{- partial \u0026#34;edit_post.html\u0026#34; . -}} {{- partial \u0026#34;post_canonical.html\u0026#34; . -}} \u0026lt;/div\u0026gt; {{- end }} \u0026lt;/header\u0026gt; {{- $isHidden := .Params.cover.hidden | default site.Params.cover.hiddenInSingle | default site.Params.cover.hidden }} {{- partial \u0026#34;cover.html\u0026#34; (dict \u0026#34;cxt\u0026#34; . \u0026#34;IsHome\u0026#34; false \u0026#34;isHidden\u0026#34; $isHidden) }} {{- if (.Param \u0026#34;ShowToc\u0026#34;) }} {{- partial \u0026#34;toc.html\u0026#34; . }} {{- end }} {{- if .Content }} \u0026lt;div class=\u0026#34;post-content\u0026#34;\u0026gt; {{- if not (.Param \u0026#34;disableAnchoredHeadings\u0026#34;) }} {{- partial \u0026#34;anchored_headings.html\u0026#34; .Content -}} {{- else }}{{ .Content }}{{ end }} \u0026lt;/div\u0026gt; \u0026lt;div\u0026gt; {{ partial \u0026#34;mastodon/mastodon.html\u0026#34; .}} \u0026lt;/div\u0026gt; {{- end }} \u0026lt;footer class=\u0026#34;post-footer\u0026#34;\u0026gt; {{- $tags := .Language.Params.Taxonomies.tag | default \u0026#34;tags\u0026#34; }} \u0026lt;ul class=\u0026#34;post-tags\u0026#34;\u0026gt; {{- range ($.GetTerms $tags) }} \u0026lt;li\u0026gt;\u0026lt;a href=\u0026#34;{{ .Permalink }}\u0026#34;\u0026gt;{{ .LinkTitle }}\u0026lt;/a\u0026gt;\u0026lt;/li\u0026gt; {{- end }} \u0026lt;/ul\u0026gt; {{- if (.Param \u0026#34;ShowPostNavLinks\u0026#34;) }} {{- partial \u0026#34;post_nav_links.html\u0026#34; . }} {{- end }} {{- if (and site.Params.ShowShareButtons (ne .Params.disableShare true)) }} {{- partial \u0026#34;share_icons.html\u0026#34; . -}} {{- end }} \u0026lt;/footer\u0026gt; {{- if (.Param \u0026#34;comments\u0026#34;) }} {{- partial \u0026#34;comments.html\u0026#34; . }} {{- end }} \u0026lt;/article\u0026gt; {{- end }}{{/* end main */}} Usage Now that all the files are in place, all you have to do to add comments to a post is add a comments stanza to the front matter in your post file to let the comment load know what toot to look at.\nHere\u0026rsquo;s an example of the comments stanza for this post:\ncomments: host: hachyderm.io username: unixorn id: 110149495764332469 The only awkward bit is that first you need to create a toot pointing at your blog post, then get the id, then update the post front matter to include the comment section.\nUpdates 2023-03-06 - Compiled the SCSS to CSS. Thanks for the catch, Carl. 2023-03-06 - Added indented comments based on Veronica Berglyd Olsen\u0026rsquo;s post 2023-04-24 - Fix typo in file paths - /partial/ should have been /partials/ - thanks for the catch, Stewart. Also updated mastodon.css to include Stewart\u0026rsquo;s dark css section to make the comments visible when the blog is in dark mode. ","permalink":"https://unixorn.github.io/post/2023-03-blog-comments-via-mastodon/","summary":"\u003cp\u003eAs part of moving from Twitter to Mastodon I decided to add comments to the blog using Fediverse posts. Fortunately, Carl Schwan showed how he does it on his blog \u003ca href=\"https://carlschwan.eu/2020/12/29/adding-comments-to-your-static-blog-with-mastodon/\"\u003ehere\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eHere are the exact tweaks to his post I did to get it working with the \u003ca href=\"https://github.com/adityatelange/hugo-PaperMod\"\u003epapermod\u003c/a\u003e theme I\u0026rsquo;m using on this blog.\u003c/p\u003e","title":"Blog Comments via Mastodon"},{"content":"Just got an Orange Pi 5. I couldn\u0026rsquo;t find a simple set of instructions on how to boot it off the M.2 NVMe slot, so I\u0026rsquo;m documenting it here.\nBackground I didn\u0026rsquo;t find simple instructions for getting the Orange Pi 5 to boot off of NVMe. It\u0026rsquo;s not difficult, just not documented in one place that I could find.\nParts List An Orange Pi 5. I got a 16GB one from Amazon for $149 A microSD card. I used a Sandisk 32GB card because they\u0026rsquo;re reliable, quick, and cost effective. You don\u0026rsquo;t need something that fast or reliable, you\u0026rsquo;re only going to use it long enough for imaging. An NVMe drive. If you want it to fit neatly without sticking out from under the Orange Pi, you need a 2230 or 2242 form factor card. I got a SAMSUNG 512GB M.2 2242 model MZALQ512HALU) for $49. Unfortunately the Orange Pi 5 doesn\u0026rsquo;t come with an M.2 mounting screw, so I bought a bag full from Amazon for $5.97 Instructions Preparatiion Download Joshua Riek\u0026rsquo;s ubuntu build from Joshua-Riek/ubuntu-orange-pi5. Here\u0026rsquo;s a direct link to the releases. I used Burn it to a microSD card. I use Balena Ether because I can use it on my Mac or Linux boxes, and it validates the results automatically for you. Install the NVMe drive. Update the SPI firmware Boot with the microSD card. I didn\u0026rsquo;t bother to do any more configuration than it forced me to since I knew I was going to have to do it again after switching to the NVMe drive Flash the SPI bootloader by running sudo dd if=/lib/u-boot-orangepi-rk3588/rkspi_loader.img of=/dev/mtdblock0 conv=notrunc Prep the NVMe drive Run sudo lsblk to see what devices the drives on your machine are recognized as. It should be easy to tell them apart by their size. On my machine, the NVMe drive was /dev/nvme0n1 and my microSD card was /dev/mmcblk1. This step will nuke any data on the NVMe drive and it will be difficult to retrieve the data if it\u0026rsquo;s even possible at all. sudo cat /dev/mmcblk1 \u0026gt; /dev/nvme0n1. It won\u0026rsquo;t print anything, and may take 20-30 minutes depending on the size of your drive. Be patient. Once you\u0026rsquo;re done, shut down the system with sudo shutdown -h now, remove the microSD card and reboot.\nThe first boot from NVMe may take a few minutes - Joshua\u0026rsquo;s ubuntu build is automagically extending the filesystem to consume the entire NVMe drive.\nBenchmarks I ran the storage benchmark from PiBenchmarks.\nCategory Test Result HDParm Disk Read 367.77 MB/s HDParm Cached Disk Read 362.99 MB/s DD Disk Write 232 MB/s FIO 4k random read 87521 IOPS (350085 KB/s) FIO 4k random write 40235 IOPS (160943 KB/s) IOZone 4k read 72318 KB/s IOZone 4k write 105149 KB/s IOZone 4k random read 38021 KB/s IOZone 4k random write 69923 KB/s Score: 20187 For comparison, when I ran it on my Odroid N2 with a USB 3.0 SSD, the Odroid wasn\u0026rsquo;t just embarrassed, it was humiliated. M.2 for the win!\nCategory Test Result HDParm Disk Read 237.62 MB/s HDParm Cached Disk Read 133.27 MB/s DD Disk Write 20.4 MB/s FIO 4k random read 5017 IOPS (20068 KB/s) FIO 4k random write 2258 IOPS (9034 KB/s) IOZone 4k read 36002 KB/s IOZone 4k write 8283 KB/s IOZone 4k random read 14312 KB/s IOZone 4k random write 7929 KB/s Score: 2443 Update: For the curious, here\u0026rsquo;s the uname output: Linux medusa.example.com 5.10.110-rockchip-rk3588 #1 SMP Wed Mar 29 08:28:12 BST 2023 aarch64 aarch64 aarch64 GNU/Linux\n","permalink":"https://unixorn.github.io/post/2023-03-boot-orange-pi-5-from-nvme/","summary":"\u003cp\u003eJust got an Orange Pi 5. I couldn\u0026rsquo;t find a simple set of instructions on how to boot it off the M.2 NVMe slot, so I\u0026rsquo;m documenting it here.\u003c/p\u003e","title":"Booting an Orange Pi 5 from NVMe"},{"content":"We\u0026rsquo;re doing a kitchen \u0026amp; bathrooms renovation, and the construction is generating a lot of dust, to the point that I was changing the HVAC filter twice a week. I wanted to see just how much garbage is in the air, so I started looking around for air quality sensors. Naturally I wanted one that I could integrate into my Home Assistant so I could generate notifications if the air was extra filthy.\nMost of the sensors I found were either over priced, didn\u0026rsquo;t expose their data via a local API, or both.\nIkea has a very cheap ($15 US) VINDRIKTNING sensor that measures 2.5 µm particles in the air, and after looking around online, I realized that I could splice an ESP8266 or ESP32 into it and transform it from something that just displays relatively useless green, red \u0026amp; orange LED lights to something I could scrape more meaningful data into my Home Assistant installation and potentially trigger automations.\nI decided to use Tasmota because it already has support for the VINDRIKTNING\u0026rsquo;s air sensor baked into the all-sensors version of the firmware, and also supports i2c so that in addition to the VINDRIKTNING\u0026rsquo;s 2.5 µm sensor, I added a BME680 from my parts box and get temperature, humidity, dew point and VOC measurements too.\nParts List Small Phillips screwdriver Wemos D1 Mini - Any ESP32 or ESP8266 will do, this is what I had in my parts bin Ikea VINDRIKTNING air sensor USB C cable \u0026amp; power brick to power the VINDRIKTNING, it doesn\u0026rsquo;t come with them BME688 (optional) qwiic cable for cannibalization (optional) Pre-requisites A MQTT server Home Assistant configured to use MQTT Software Setup First, flash the D1 - it\u0026rsquo;s easier to do when it isn\u0026rsquo;t wired into the Ikea unit. I used Tasmota\u0026rsquo;s online tool to flash my board and chose the Tasmota all-sensors build - make sure you use all-sensors, not the sensors build at the top of the popup menu or it won\u0026rsquo;t have the VINDRIKTNING support baked in.\nNext, make sure it\u0026rsquo;s connecting to your WIFI and you can reach the web UI panel. Get this working now so that you don\u0026rsquo;t have to open up the case later to reflash it. You want to be sure the board is working before you do all the soldering to embed it into the VINDRIKTNING.\nConfiguration Tasmota defaults to sending metrics to MQTT every 600 seconds. Given that the VINDRIKTNING is powered by a USB brick and I don\u0026rsquo;t have to care about battery life, I set mine to report every 60 seconds so my graphs will be smoother. To do that, connect to the Tasmota\u0026rsquo;s web ui, select the console, and enter TelePeriod 60 (or whatever interval in seconds you prefer).\nSet up MQTT Go to the web ui and go into the configuration menu, then MQTT. The topic you specify will be the device name in Home Assistant.\nSet up pins Next, go into the configuration menu and configure what pins are going to be used for what. Here\u0026rsquo;s what my working setup looks like:\nHardware Setup We\u0026rsquo;re not going to replace the Ikea\u0026rsquo;s electronics. Instead, we\u0026rsquo;re going to tap into it like a symbiote so that the VINDRIKTNING supplies power and data to our 8266, and the original board remains fully functional to drive the LED display.\nSteps Open up the VINDRIKTNING Unfortunately the screws holding it together are in annoyingly deep holes, so you\u0026rsquo;ll need a eyeglass-style screwdriver like this one that came with a fan kit for one of my Raspberry Pis (AirPods Pro for scale).\nOpen it gently - the screws are tapped into the plastic case and it is easy to strip the holes.\nOnce it\u0026rsquo;s open, carefully unsnap the two cables connecting the sensor and the motherboard. It\u0026rsquo;ll look like this when you\u0026rsquo;re done:\nI didn\u0026rsquo;t bother to unscrew the IKEA motherboard from the case, there aren\u0026rsquo;t that many connections to make to it and they\u0026rsquo;re pretty easily accessible.\nWire up the ESP8266 Connect a QWIIC cable to the Wemos I2C Bus (Optional) If you\u0026rsquo;re not going to use the i2c bus to connect another sensor, you can skip this step.\nI happened to have a BME688 in my parts bin that I could add to the sensor, so I cut a QWIIC cable in half and soldered that onto the Wemos before I connected that to the Ikea.\nConnect the wires as follows:\nRed wire -\u0026gt; 3v3 Black wire -\u0026gt; Ground G Yellow -\u0026gt; D1 Blue -\u0026gt; D2 Here\u0026rsquo;s a picture of the D1 with the QWIIC cable - ignore the green wire, I forgot to take a picture before starting the next step\nConnect the ESP 8266 to the VINDRIKTNING You only need to connect 3 wires for this.\nConnect D5 on the D1 to REST on the VINDRIKTNING (Blue wire in pic) Connect G on the D1 to GND on the VINDRIKTNING (Green wire in pic) Connect 5V on the D1 to +5V on the VINDRIKTNING (Red wire in pic) Now you can reconnect the cables from the VINDRIKTNING sensor to its motherboard, and it should look very similar to this image.\nReassembly Plug in the USB-C power cable and make sure that you can access the Tasmota web ui and that it is reading the sensors before you put any screws back in.\nIf you have a BME688 in yours, it\u0026rsquo;ll look like:\notherwise it\u0026rsquo;ll only have the VINDRIKTNING reading.\nNow you can carefully put the D1 (and the BME 688 if you added one) into the empty space in the case, then screw everything shut.\nConnecting to Home Assistant If you don\u0026rsquo;t already have the Tasmota integration, you\u0026rsquo;ll need to add it. Go into Settings, Devices \u0026amp; Services, then Integrations. Click Add Integration, pick the Tasmota Integration and configure it. It\u0026rsquo;ll ask for the MQTT server, user and password.\nHome Assistant should detect your new sensor and let you add the device.\nCongratulations, you\u0026rsquo;ve successfully DIYed an air sensor!\n","permalink":"https://unixorn.github.io/post/add-smarts-to-ikea-vindriktning-air-sensors/","summary":"\u003cp\u003eWe\u0026rsquo;re doing a kitchen \u0026amp; bathrooms renovation, and the construction is generating a lot of dust, to the point that I was changing the HVAC filter twice a week. I wanted to see just how much garbage is in the air, so I started looking around for air quality sensors. Naturally I wanted one that I could integrate into my Home Assistant so I could generate notifications if the air was extra filthy.\u003c/p\u003e","title":"Add Smarts to IKEA Vindriktning Sensors"},{"content":"Released version 0.8.0 of ha-mqtt-discoverable and version 0.2.0 of ha-mqtt-discoverable-cli today.\nha-mqtt-discoverable is a python module that allows python programs to create sensor and device entities on an mqtt server that will be automagically recognized by Home Assistant.\nha-mqtt-discoverable-cli installs command line scripts for device and sensor creation.\nThere is also a docker image, unixorn/ha-mqtt-discoverable-cli, that contains the command line tools.\n","permalink":"https://unixorn.github.io/post/ha-mqtt-discoverable-0.8.0/","summary":"\u003cp\u003eReleased version 0.8.0 of ha-mqtt-discoverable and version 0.2.0 of ha-mqtt-discoverable-cli today.\u003c/p\u003e","title":"Released ha-mqtt-discoverable 0.8.0"},{"content":"I\u0026rsquo;ve been backing my homelab up with duplicacy (See Backing Up the Cluster Using Duplicacy), but I\u0026rsquo;m fed up with it returning a 0 exit code even if there\u0026rsquo;s a problem with the backup. This makes me have to do a lot of annoying rummaging through log output to be sure that a backup actually worked, so I decided to switch to restic.\nIn this blog entry, I\u0026rsquo;m going to explain how to create a jail in TrueNAS, mount directories you want to back up into the jail, install restic, and how to use it to back up to Backblaze b2.\nBackground I\u0026rsquo;m tired of worrying about whether I found all the possible duplicacy error messages, or made a mistake in one of the regexes for the errors I have seen. I want my backups nice and boring so I don\u0026rsquo;t go to do a restore and find out \u0026ldquo;Oops! We had a new failure mode you hadn\u0026rsquo;t seen before, so the backups haven\u0026rsquo;t been working for three months, too bad about your data!\u0026rdquo;\nMy work had a holiday shutdown, so I decided it was past time to switch to a better backup system. Of course I\u0026rsquo;m going to keep running the old crappy one for a couple of months in parallel because I don\u0026rsquo;t want to find a problem with the new one after three months of usage when I do my next restore test. You do test your restores periodically, I hope.\nI\u0026rsquo;ve been running TrueNAS Core (formerly FreeNAS) long enough that I\u0026rsquo;ve upgraded to larger drives in the zpool three times. I picked it because FreeNAS fully supported ZFS, I could buy a chassis with 4 hot-swap bays, I could buy it from the company that makes the software, both to support the open source project and to ensure that I had fully supported hardware.\nCan I build a server from parts? Sure, I\u0026rsquo;ve done it before. Do I want to build a server from parts and have to fight with a bunch of different vendors if something goes awry? Oh hell no, especially for the server with all my photos on it. I opted to minimize my hassle factor, support the FreeNAS project, and only have to deal with one vendor in case of any hardware issues. I bought a 4 bay FreeNAS mini, and have been happy with it for 7+ years.\nAnyway, enough background on the server I\u0026rsquo;m backing up.\nWhy restic? restic is:\nOpen Source Works on macOS, Windows, Linux and (most importantly for TrueNAS) FreeBSD Written in go, so it\u0026rsquo;s a single binary blob and we don\u0026rsquo;t have to worry about installing a bunch of dependencies inside the jail Does data deduplication and compression on your backups. Since we pay Backblaze based on space usage, this reduces the backup cost Why Jails? TrueNAS Core is based on FreeBSD, so the native way of running software in an isolated environment are jails (one of the inspirations for docker containers). I could run it in a docker container, but that would burn more resources because I\u0026rsquo;d have to run it in a docker vm.\nWhy B2? I\u0026rsquo;ve been using Backblaze b2 for years. It has a s3-compatible API so a lot of s3-aware applications work with it easily, Backblaze only does storage so I don\u0026rsquo;t have to worry about it going away, and last but definitely not least, it only costs 20% of what the same amount of data would cost to store in S3.\nrestic supports a lot of other storage back ends (including local disk), but I\u0026rsquo;m only going to discuss b2 here.\nAll that said, let\u0026rsquo;s get down to it.\nSet up B2 First, create an account at Backblaze.com.\nNow that you have an account, you\u0026rsquo;ll need to create a bucket to store your backups in. Log into your account and click Buckets in the sidebar on the left, then Create a Bucket. Give your new bucket a unique name, make sure it\u0026rsquo;s set to private (which should be set by default), and click create.\nYou\u0026rsquo;ll also need an application key that restic can use to connect to b2.\nDo not use your master application key! You should restrict restic to just the bucket you want it using - don\u0026rsquo;t give it power over your entire backblaze account. Click App Keys at the bottom of the sidebar, then Add a New Application Key, give it a name (I used restic-key), select the bucket you just created from the Allow access to buckets dropdown menu, and create the key. Copy the key into your password manager, it\u0026rsquo;ll only be displayed once. Copy the keyID too, you\u0026rsquo;ll need both of them when we configure restic.\nCreate a jail for restic Use the TrueNAS wizard to create your restic jail.\nLog into your TrueNAS webui Click Jails. Click add. You should see something similar to Give it a name (make life easy for yourself and don\u0026rsquo;t include spaces or any special characters other than -), leave the jail type as default, select whatever the highest FreeBSD version is showing available in the release drop-down menu, then next It\u0026rsquo;s going to need network access, so click DHCP Autoconfigure, and if it doesn\u0026rsquo;t automatically populate the VNET checkbox, enable that too, then next It\u0026rsquo;ll show you a summary of your settings that should look something like click submit to create the new jail. Add Mount Points By default, your jail can\u0026rsquo;t see any directory trees outside itself. That\u0026rsquo;s great for security, but not so great if we want to back up files outside the jail, so we\u0026rsquo;re going to add some mount points.\nAdd the mount points before you start the new jail - you can\u0026rsquo;t add them to a running jail. You can stop the jail, add/remove mount points, then restart it, but it\u0026rsquo;s easier to set them up before starting the jail.\nSelect your new jail, and select add mountpoint .\nYou can either make individual mounts for the directories you want to back up, or you can make one for your whole zpool. Whichever you pick, I recommend setting them read-only so you can\u0026rsquo;t accidentally restore an older version of a file over the current version.\nMake a directory outside the jail to store configuration files and to restore files to, and don\u0026rsquo;t set it read-only. Segregating the config/restore directory to a separate mount point will make it easier to examine restored files before putting them back where they belong.\nNow that you have your mount points added to your jail, go ahead and start your jail. It should take less than a minute to start.\nInstall restic Now that the jail is running, you\u0026rsquo;ll have to install restic into it.\nssh into your TrueNAS server Enter the jail so you can install software by running sudo iocage console YOURJAILNAME pkg install ca_root_nss restic Configure restic \u0026amp; do your first backup For convenience, I store the various settings as environment variable exports in a shell script that I can source for a couple of reasons:\nThis makes it easy to do the backups via cron and also to do test backups and restores from within the jail. I don\u0026rsquo;t have to write a config file parser and overcomplicate things Install the driver script and config files ssh into your TrueNAS server and stick the following files in a directory visible inside your backups jail.\nHere\u0026rsquo;s the restic-driver script I use to run restic and trim snapshots once they age out.\n#!/usr/bin/env bash # # restic-driver-script # # License: Apache 2.0 # Copyright 2023 Joe Block \u0026lt;jpb@unixorn.net\u0026gt; set -o pipefail if [[ -n \u0026#34;$DEBUG\u0026#34; ]]; then set -x fi function debug() { if [[ -n \u0026#34;$DEBUG\u0026#34; ]]; then echo \u0026#34;$@\u0026#34; fi } function fail() { printf \u0026#39;%s\\n\u0026#39; \u0026#34;$1\u0026#34; \u0026gt;\u0026amp;2 ## Send message to stderr. Exclude \u0026gt;\u0026amp;2 if you don\u0026#39;t want it that way. exit \u0026#34;${2-1}\u0026#34; ## Return a code specified by $2 or 1 by default. } function has() { # Check if a command is in $PATH which \u0026#34;$@\u0026#34; \u0026gt; /dev/null 2\u0026gt;\u0026amp;1 } function show_params() { debug \u0026#34;BACKUP_PATHS: $BACKUP_PATHS\u0026#34; debug \u0026#34;EXCLUDE_FILE: $EXCLUDE_FILE\u0026#34; debug \u0026#34;DRY_RUN: $DRY_RUN\u0026#34; debug \u0026#34; \u0026#34; debug \u0026#34;Retention settings:\u0026#34; debug \u0026#34;Minimum snapshots $MINIMUM_SNAPSHOTS_RETAINED\u0026#34; debug \u0026#34;Hourly snapshots: $HOURS_RETAINED\u0026#34; debug \u0026#34;Daily snapshots: $DAYS_RETAINED\u0026#34; debug \u0026#34;Weekly snapshots: $WEEKS_RETAINED\u0026#34; debug \u0026#34;Monthly snapshots: $MONTHS_RETAINED\u0026#34; debug \u0026#34;Yearly snapshots: $YEARS_RETAINED\u0026#34; } if ! has restic; then fail \u0026#34;Can\u0026#39;t find restic in $PATH!\u0026#34; fi # Our first argument is the settings file to source to get our backup # parameters, so peel it off - we\u0026#39;ll pass all the other arguments directly # to restic PREFS_F=\u0026#34;$1\u0026#34; shift if [[ ! -r \u0026#34;$PREFS_F\u0026#34; ]]; then fail \u0026#34;Can\u0026#39;t load $PREFS_F\u0026#34; fi source \u0026#34;$PREFS_F\u0026#34; show_params # If you\u0026#39;re backing up a filesystem that you\u0026#39;re mounting by FUSE, the inode # information is misleading at best, so add --ignore-inode. restic backup --verbose=2 \\ --exclude=.duplicacy \\ --exclude=.DS_Store \\ --tag periodic \\ -o b2.connections=15 \\ $EXCLUDE_FILE $DRY_RUN $BACKUP_PATHS $@ if [[ $? != 0 ]]; then fail \u0026#34;restic backup failed\u0026#34; # We don\u0026#39;t want to prune any snapshots if this backup failed fi # Prune backup snapshots restic forget --verbose \\ --tag periodic \\ --group-by \u0026#34;paths,tags\u0026#34; \\ --keep-last $MINIMUM_SNAPSHOTS_RETAINED \\ --keep-hourly $HOURS_RETAINED \\ --keep-daily $DAYS_RETAINED \\ --keep-weekly $WEEKS_RETAINED \\ --keep-monthly $MONTHS_RETAINED \\ --keep-yearly $YEARS_RETAINED if [[ $? != 0 ]]; then fail \u0026#34;restic snapshot cleanup failed\u0026#34; fi Here\u0026rsquo;s an example settings file for it:\n#!/usr/bin/env bash # # Here are our backup settings # export B2_ACCOUNT_ID=\u0026#39;your_b2_account_id\u0026#39; export B2_ACCOUNT_KEY=\u0026#39;your_b2_key\u0026#39; # Use different directory prefixes for each backup repo # so they can share a bucket without interference export RESTIC_REPOSITORY=\u0026#39;b2:your-restic-backups-bucket:dir-prefix\u0026#39; # This is used as the encryption key for your backups. If you lose it, # you won\u0026#39;t be able to restore anything. I keep a copy of mine in my # 1Password vault export RESTIC_PASSWORD=\u0026#39;your-encryption-key\u0026#39; # If you want to exclude some directories from your backups, list # them in an exclude file and set EXCLUDE_FILE export EXCLUDE_FILE=\u0026#34;--exclude-file=example-excludes\u0026#34; # Uncomment if you only want to dry-run and not actually write any data # to the backup repository #export DRYRUN=\u0026#39;--dry-run\u0026#39; # What paths do we want to back up? Remember to use the paths as seen inside # the jail, not the paths as seen in your TrueNAS environment export BACKUP_PATHS=\u0026#34;/mnt/path-inside-jail/share /mnt/path-inside-jail/anothershare\u0026#34; # How many snapshots do we want to keep around? export MINIMUM_SNAPSHOTS_RETAINED=4 export HOURS_RETAINED=48 export DAYS_RETAINED=14 export WEEKS_RETAINED=8 export MONTHS_RETAINED=12 export YEARS_RETAINED=5 And here\u0026rsquo;s an example excludes file - it\u0026rsquo;s a list of paths to directories and files we don\u0026rsquo;t want to back up - I don\u0026rsquo;t back up my downloads or installers directories since that\u0026rsquo;s all stuff I can re-download later if necessary, and I\u0026rsquo;m better off getting the current version anyway.\n/mnt/path-inside-jail/Downloads /mnt/path-inside-jail/Installers/*.dmg Test your backups You\u0026rsquo;ll have to do this from inside the backups jail you created. ssh to your server, then run sudo iocage console YOURJAILNAME\nI recommend you set BACKUPS_PATH in your settings file to a single small directory to make your testing faster. You can change it to the path to your full directory tree once you confirm your backups and restores are working as expected.\nInitialize your backup repository Now that you\u0026rsquo;re inside the jail, you\u0026rsquo;re going to have to initialize the repository directory in your B2 bucket. FreeBSD/TrueNAS uses csh as the default root shell, but our settings file is set up for bash, so start by running exec bash to get a decent shell.\nLoad the configuration file you created by running source /path/to/yoursettingsfile.sh, then run restic init.\nYou\u0026rsquo;re ready to do a backup.\nRun a backup Now that your repository has been initialized, run /path/to/restic-driver /path/to/yoursettingsfile.sh\nIt shouldn\u0026rsquo;t take long if you used a small test subdirectory.\nTest a restore First, let\u0026rsquo;s look at the list of snapshots with restic snapshots.\nNow we can list the files in the snapshot with either restic ls SNAPSHOTID or restic ls latest\nPick a file and restore it.\nMake a directory to restore to - mkdir ./restores Restore the file. We don\u0026rsquo;t want to restore the entire snapshot, just a specific file/directory, so we\u0026rsquo;ll use --include and run restic restore latest --target ./restores --include /mnt/path/to/file It put the whole directory path for that file inside ./restores, so we\u0026rsquo;ll do a simple md5 check with md5sum /path/to/test/file ./restores/path/to/test/file Run restic out of cron Great, you\u0026rsquo;re almost done. All we have to do now is add it to cron so it runs at least once a day. You need to add it to the crontab inside the jail though, or TrueNAS won\u0026rsquo;t find your script.\nI stored my restic backup script and configuration in a directory that appears as /mnt/alcatraz inside my backups jail. Change the path to match wherever you stored yours.\nRun crontab -e and add the following:\n# minute\thour\tmday\tmonth\twday\tcommand 13 */4 * * * /mnt/alcatraz/restic-driver /mnt/alcatraz/backup-settings.sh | logger -t backups I tagged all the log output with backups to make it easier to grep out of /var/log/messages, and I run mine four times a day. You may prefer a different cadence, if so set the hour field accordingly. restic locks the repository when it starts, so you don\u0026rsquo;t have to worry about repository corruption if you accidentally try to run more than one backup at a time, or one runs so long that it isn\u0026rsquo;t complete before the next run starts.\nUpdate: Jesse Alter wrote a perl version of this script, you can find it on GitHub at jessealter/restickit.\n","permalink":"https://unixorn.github.io/post/restic-backups-on-truenas/","summary":"\u003cp\u003eI\u0026rsquo;ve been backing my homelab up with \u003ccode\u003eduplicacy\u003c/code\u003e (See \u003ca href=\"https://unixorn.github.io/post/backing-up-the-cluster-with-duplicacy/\"\u003eBacking Up the Cluster Using Duplicacy\u003c/a\u003e), but I\u0026rsquo;m fed up with it returning a \u003ccode\u003e0\u003c/code\u003e exit code \u003cem\u003eeven if there\u0026rsquo;s a problem with the backup\u003c/em\u003e. This makes me have to do a lot of annoying rummaging through log output to be sure that a backup actually worked, so I decided to switch to \u003ca href=\"https://restic.net/\"\u003erestic\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eIn this blog entry, I\u0026rsquo;m going to explain how to create a jail in TrueNAS, mount directories you want to back up into the jail, install \u003ca href=\"https://restic.net/\"\u003erestic\u003c/a\u003e, and how to use it to back up to \u003ca href=\"https://www.backblaze.com/b2/cloud-storage.html\"\u003eBackblaze b2\u003c/a\u003e.\u003c/p\u003e","title":"Restic Backups on TrueNAS"},{"content":"I considered a switch to the Congo theme for Hugo. The one thing I didn\u0026rsquo;t like in it was the way it handles inline backticks - it displays them, in addition to formatting the enclosed text as code.\nTo fix it, create assets/css/custom.css with the following contents:\n.prose code::before { content: \u0026#39;\u0026#39; } .prose code::after { content: \u0026#39;\u0026#39; } ","permalink":"https://unixorn.github.io/post/fix-congo-backticks/","summary":"\u003cp\u003eI considered a switch to the \u003ca href=\"https://hugothemesfree.com/a-simple-lightweight-theme-for-hugo-built-with-tailwind-css/\"\u003eCongo\u003c/a\u003e theme for \u003ca href=\"https://gohugo.io/\"\u003eHugo\u003c/a\u003e. The one thing I didn\u0026rsquo;t like in it was the way it handles inline backticks - it displays them, in addition to formatting the enclosed text as code.\u003c/p\u003e\n\u003cp\u003eTo fix it, create \u003ccode\u003eassets/css/custom.css\u003c/code\u003e with the following contents:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-css\" data-lang=\"css\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"nc\"\u003eprose\u003c/span\u003e \u003cspan class=\"nt\"\u003ecode\u003c/span\u003e\u003cspan class=\"p\"\u003e::\u003c/span\u003e\u003cspan class=\"nd\"\u003ebefore\u003c/span\u003e \u003cspan class=\"p\"\u003e{\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"k\"\u003econtent\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;\u0026#39;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"nc\"\u003eprose\u003c/span\u003e \u003cspan class=\"nt\"\u003ecode\u003c/span\u003e\u003cspan class=\"p\"\u003e::\u003c/span\u003e\u003cspan class=\"nd\"\u003eafter\u003c/span\u003e \u003cspan class=\"p\"\u003e{\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"k\"\u003econtent\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;\u0026#39;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e","title":"Fix Congo Backticks"},{"content":"I\u0026rsquo;m moving from Twitter to Mastodon. Specifically, I\u0026rsquo;m @unixorn@hachyderm.io. Here are my first impressions.\nIt\u0026rsquo;s refreshing to use social media without the distorting effects caused by selling ads. No algorithm tuning to keep you doomscrolling, you just see posts in reverse chronological order, from the people and hashtags you follow. No promoted posts, no \u0026ldquo;here\u0026rsquo;s a tweet that other people like, so we\u0026rsquo;re going to jam it into your feed ahead of content from accounts you\u0026rsquo;re actually following\u0026rdquo;, just posts from your follows, and all the posts from your follows, in reverse chronological order.\nAnd when you\u0026rsquo;re done reading all that\u0026rsquo;s new, it doesn\u0026rsquo;t puke up posts selected by the algorithm to make you stick around, you can close it and come back in a few hours and see only new content you care about. It reminds me a lot of USENET that way.\nGetting Started Because the fediverse is composed of many interconnected servers, you don\u0026rsquo;t have to put your account on the big popular ones that are currently being swanped with new users. And you shouldn\u0026rsquo;t. There\u0026rsquo;s no disadvantage to using different servers, you can still follow and interact with people in the rest of the fediverse. Think of it like email - if you\u0026rsquo;re on gmail, you can still send and recieve mail from people on other services like O365, on Mastodon you can post on one server and have those seen by your followers on other servers.\nJoinMastodon.org has a list of servers sorted by topics and locations. You\u0026rsquo;re better off getting an account on a smaller server that aligns with your interests so the local feed will be more interesting to you and help you find similar-minded people to follow. And smaller servers are generally faster.\nNorms Mastodon isn\u0026rsquo;t just a Twitter clone. It has been around for 5+ years, and the culture is different in a good way. Here are some of the norms - I\u0026rsquo;ve only been on Mastodon a few days and won\u0026rsquo;t pretend to have a full grasp of it.\nIn no particular order\nThere are no ads on Mastodon. Which is awesome, but it means no ad revenue. Consider donating to your server via patreon or paypal. There\u0026rsquo;s no algorithm prioritizing tweets. Favoriting something shows appreciation to the poster, but it doesn\u0026rsquo;t make their post more visible to others. If you want to help other people see it you have to boost (the equivalent of a retweet) it. DMs are not end to end encrypted. Admins can read them if necessary. If you want to really privately talk, use Signal Mastodon is far more into letting people consent to read your content over letting you just jam it into your followers\u0026rsquo; eyeballs. If something is at all likely to disturb a reader, use the content warning. It will let you mention why there\u0026rsquo;s a content warning. You can also use it for media spoilers. Unlike the bird site, Mastodon doesn\u0026rsquo;t do whole-toot searches of toots, only hashtag ones. Partly because it\u0026rsquo;s computationally expensive, but mostly to prevent pile-ons by assholes searching for posts with a word like LGBT to find people to abuse. Because of this, hashtagging your posts is crucial if you want non-followers to see them. Be screen-reader friendly and camel cap your hashtags - most screenreaders will use that to properly pronounce them, so use#HomeAssistant instead of #homeassistant. While I\u0026rsquo;m talking about screen readers, please use alt text for images you post. Alt text is far more prevalent on Mastodon than it was in the posts I saw on Twitter. The guides to Mastodon suggest that you post the first post in a thread as public, then make your replies to that post unlisted. That will let anyone interested see the replies, but not spam your followers\u0026rsquo; feeds with a lot of messages they may not be interested in. Tools I found some really helpful tools for the migration\na.gup.pe Guppe is a bot that brings social groups to the fediverse. You follow @TOPIC@a.gup.pe, then if you mention @TOPIC@a.gup.pe in a toot, the bot will boost (Mastodon\u0026rsquo;s version of Twitter\u0026rsquo;s retweet) your post, so everyone else who follows the topic will see it in their feed. You don\u0026rsquo;t have to do anything special to create a topic, just mention it in a post or follow it.\nDebirdify Debirdify will look at your follows (or followers), and scrape their profiles to find Mastodon-style identifiers in the format @username@server.tld. You can then export those lists as CSV files, which you can import into Mastodon by going into Settings -\u0026gt; Import. Make sure you pick MERGE so that it only merges any new handles instead of replacing your existing list.\ntwitter-archive-parser Transforms a downloaded Twitter archive into markdown format, including expanding all t.co URLs with their original versions.\n","permalink":"https://unixorn.github.io/post/switching-to-mastodon/","summary":"\u003cp\u003eI\u0026rsquo;m moving from Twitter to Mastodon. Specifically, I\u0026rsquo;m @unixorn@hachyderm.io. Here are my first impressions.\u003c/p\u003e","title":"Switching to Mastodon"},{"content":"Some tips about setting up Zigbee or Z-wave mesh networks.\nI\u0026rsquo;ve had a few people new to Home Assistant ask about how to best set up their mesh networks. Here\u0026rsquo;s my take on what works best.\nBefore you start using Zigbee, make sure your Zigbee and 2.4GHz WIFI aren\u0026rsquo;t stepping on each other See https://www.youtube.com/watch?v=t-gw7kURXCk for details, but the TL;DR is that Zigbee uses the same 2.4GHz radio frequency band that 2.4GHz WIFI does, so choose your channels wisely to avoid overlap.\nYou probably already know that wifi channels interfere with each other, so the best channels to use to avoid interference are 1, 6 and 11. Once you put Zigbee into the mix, you need to also know that Wifi channel 1 interferes with zigbee channels 11-17, wifi channel 6 interferes with zigbee channels 13-23, and wifi channel 11 interferes with zigbee channels 18-26. Some zigbee devices don\u0026rsquo;t work with channel 26, so even though it\u0026rsquo;ll get the least interference from wifi, you\u0026rsquo;ll want to rule it out.\nIf you\u0026rsquo;re using multiple wifi access points, you\u0026rsquo;ll want to set them to use different wifi channels so they don\u0026rsquo;t interfere with each other. Your best choices are using wifi channels 1 \u0026amp; 6 with zigbee channel 24, wifi 6 \u0026amp; 11 with zigbee channel 11, or wifi 1 and 11 with zigbee channel 18.\nChanging the Zigbee channel forces you to re-pair every Zigbee device. This sucks, so plan ahead. Changing the 2.4GHz WIFI channel on your access points is a lot less painful. You don\u0026rsquo;t have to worry about this with Z-Wave since it uses 800-900Mhz depending on the country.\nFinding out what zigbee channel zigbee2mqtt is using Go to Settings, Advanced, and then you\u0026rsquo;ll see your zigbee channel.\nUse a USB extension cable Computers can spew out a lot of radio interference, especially if you\u0026rsquo;re using a single-board computer like a Raspberry Pi or ODROID that has a cheap plastic case that is effectively unshielded. A 15 foot USB extension cable will make a lot of signal issues with your Zigbee or Z-Wave mesh disappear, with the added bonus that you can put the dongle in a location that\u0026rsquo;s not convenient to put a computer. I\u0026rsquo;ve got my servers racked in my basement, but the dongles are on the basement ceiling directly under powered Zwave and Zigbee switches so my meshes are very strong. Keep the dongles away from your 2.4GHz WIFI access points if possible to minimize interference.\nAdd your powered devices first Zigbee and Z-Wave both are mesh networks. If you\u0026rsquo;re doing a new setup, start with your powered devices since they can act as routers. Start by adding the ones closest to your coordinator, then work your way outward. This way, when you start adding your battery devices the mesh will already have routers in place and the battery devices won\u0026rsquo;t all try to connect directly to the coordinator.\nDon\u0026rsquo;t bother buying range extenders I made this mistake early on in my Zwave mesh. You\u0026rsquo;re going to have to plug the extender in anyway, so you might as well just buy a smart plug. They\u0026rsquo;ll also extend your range, and when I last compared prices, the price difference was less than five dollars. You may not think of a need for the plug now, but you almost certainly will want its functionality later.\nAdd devices where you\u0026rsquo;re going to use them, not while next to your coordinator In the past, I\u0026rsquo;ve had new Zigbee and Z-Wave devices sometimes show spotty performance. After some experimentation, the problem seemed to be based on where the new device was when I added it to the mesh. When you add a device when it\u0026rsquo;s right next to the mesh coordinator attached to your Home Assistant machine, it will query all the routers it can see, see that the coordinator is available, and to try to minimize hops to the coordinator it will connect directly. This would be no big deal except that when you move it somewhere else (like when I moved a new smart plug from the basement next to my Home Assistant machine to the second floor), it will still try to talk directly to the coordinator rather than any adjacent routers and have sub-optimal performance.\nInstead, take it to wherever you\u0026rsquo;re going to use it and plug it in there and pair it there. This will let it detect which of your routers has the strongest signal and connect to that instead of trying to connect directly to the coordinator. If you\u0026rsquo;re adding a new router like a smart plug, this will immediately strengthen your mesh.\nIt\u0026rsquo;s not a huge deal if you don\u0026rsquo;t do it this way. With Zigbee the coordinator will eventually get around to rebalancing the routing connections. Z-Wave will require you to heal the mesh, it doesn\u0026rsquo;t do it automagically in the background like Zigbee does.\nYou should heal your Z-Wave network every time you add a device. When you first add a device to the Z-Wave nework, only the device it initially connects to will know it exists and be able to route messages to the new device. Healing the network makes all the devices in your mesh learn about all the other devices - until they learn about the other devices, you will end up with some wonky message routing paths.\nTL;DR - Add powered devices to your mesh first. Add new devices to your mesh where you\u0026rsquo;re actually going to use them. If you\u0026rsquo;re using Z-Wave, you should periodically heal your Z-Wave mesh so it can re-calculate optimum routing. Heal the Z-Wave mesh even if you haven\u0026rsquo;t added, moved or removed any devices - sometimes even moving furniture around can affect the signal between devices, so give it a chance to recalculate optimum routing.\nEdit: Added instructions for finding the zigbee channel used by zigbee2mqtt, included optimal wifi-zigbee channel combinations, added note on range extenders.\n","permalink":"https://unixorn.github.io/post/zigbee-and-zwave-setup-tip/","summary":"\u003cp\u003eSome tips about setting up Zigbee or Z-wave mesh networks.\u003c/p\u003e\n\u003cp\u003eI\u0026rsquo;ve had a few people new to Home Assistant ask about how to best set up their mesh networks. Here\u0026rsquo;s my take on what works best.\u003c/p\u003e","title":"Zigbee and Zwave Setup Tips"},{"content":"I wanted to switch my new Home Assistant (HA) installation to write data to PostgreSQL instead of SQLite for a variety of reasons. Here\u0026rsquo;s how I did it.\nHere\u0026rsquo;s why I decided to switch:\nResilience. If you\u0026rsquo;re running Home Assistant on a Raspberry Pi\u0026rsquo;s SD card, the constant churn of history updates will eventually destroy the card. The more entities you have, the faster HA will grind your SD card to failure. Writing all that to another server that is writing to a real SSD or spinning disk eliminates that problem. Convenience. I can back up the postgres database without having to stop HA. I don\u0026rsquo;t even have to run the backup on the HA server. Speed. Using a real database will speed up history display, especially once you have a large number of entities. Pre-requisites Note - I did this on a fresh server with no history I wanted to preserve. What I\u0026rsquo;m describing here will discard all your old history data. If you do have history you want to preserve, there\u0026rsquo;s a forum post here that explains how to load your history from SQLite and into postgres with pgloader.\nA PostgreSQL server. I recommend that you configure your router to assign your server a static IP address so when you reboot it it doesn\u0026rsquo;t get a different IP from the DHCP pool and force you to update your HA server\u0026rsquo;s configuration.yml.\nSetting up postgres Here\u0026rsquo;s an example docker-compose.yaml file that will start postgres for you. Rather than use a docker volume and then have to save and restore that, this configuration will mount /path/to/postgres/data into the container so you can safely destroy and recreate your docker environment without having to restore your database from a backup.\nversion: \u0026#39;3\u0026#39; services: postgres: container_name: postgres image: postgres:14.5 restart: always network_mode: host ports: - \u0026#34;5234:5234\u0026#34; environment: POSTGRES_USER: postgresadmin POSTGRES_PASSWORD: \u0026lt;redacted\u0026gt; volumes: - /path/to/postgres/data:/var/lib/postgresql/data - /etc/localtime:/etc/localtime:ro The first time you run this image, it\u0026rsquo;ll generate the database files, create a user with admin privileges named POSTGRES_USER with password POSTGRES_PASSWORD. Now that it\u0026rsquo;s running, we should create a user just for Home Assistant.\nI set this up on one of my Odroid HC2s so I could keep the data directory on the hard drive there for ease of backup.\nUpdating Home Assistant to use PostgreSQL Now that the postgres server is running, we\u0026rsquo;re going to use psql (the postgres command line client) to set up a Home Assistant user and database. You can install psql on your machine, but I prefer to use a container.\nRun docker run -it --rm postgres:14.5 bash to get a shell running inside a postgres container. It\u0026rsquo;ll have the tools you need to create an account for your Home Assistant instance.\nHere\u0026rsquo;s the commands you\u0026rsquo;re going to need:\npsql CREATE USER homeassistant WITH PASSWORD 'yourHomeAsssistantPassword'; CREATE DATABASE homeassistant_db WITH OWNER homeassistant ENCODING 'utf8' TEMPLATE template0; Now that the postgres server is set up, you can configure your HA server to use it instead of SQLite.\nHere\u0026rsquo;s a snippet from my configuration.yaml\n# Database recorder: db_url: !secret psql_connector_string db_retry_wait: 10 # Wait 10 seconds before retrying exclude: domains: - automation - updater entity_globs: - sensor.weather_* entities: - sun.sun # Don\u0026#39;t record sun data - sensor.last_boot # Comes from \u0026#39;systemmonitor\u0026#39; sensor platform - sensor.date event_types: - call_service # Don\u0026#39;t record service calls To save space, I\u0026rsquo;ve disabled storing changes to sun.sun, sensor.date and sensor.last_boot, along with weather information and calls to services. Tune yours as you see fit.\nI\u0026rsquo;m using a secret for the db_url, and here\u0026rsquo;s a redacted example from my secrets.yaml file showing the proper format:\npsql_connector_string: \u0026#34;postgresql://DATABSE_USERNAME:DATABASE_PASSWORD@DNSNAME_OR_IP_OF_POSTGRES_SERVER/DATABASE_NAME\u0026#34; So in my case, the redacted string is postgresql://hassuser:hasspassword@postgres.example.com/homeassistant_db\nNow that you\u0026rsquo;ve updated the server configuration, confirm that you don\u0026rsquo;t have any typos and your configuration is valid by going to http://yourHA:8123/developer-tools/yaml and clicking CHECK CONFIGURATION. If it reports it valid, you can safely restart HA and you\u0026rsquo;ll be storing all your history data in Postgres.\nBacking Up Your Database Now that we\u0026rsquo;re writing data to postgres, time to take advantage of not having to shut it down for backups and start backing things up.\nI wrote a simple shell script for backing up your postgres database, ha-postgresql-backup. It has reasonable default settings, which you can override for your environment by setting environment variables.\nCOMPRESSOR - If you set this, also set EXTENSION. Defaults looking for bzip2 and gzip, and will use bzip2 if both are found. DUMP_D - What directory to write the dump file to. If you don\u0026rsquo;t set this, it will use the current directory HASS_D - What postgres database to back up. Defaults to homeassistant_db PG_PASSWORD - The ha user\u0026rsquo;s password PG_SERVER - What server to connect to PG_USERNAME - Used to log into your postgres server. Defaults to homeassistant. You should use the same username \u0026amp; password your Home Assistant is using. POSTGRESQL_IMAGE - What docker image to use. Defaults to postgres:14.5 Example usage: PG_PASSWORD=your_ha_pg_password PG_USER=your_ha_pg_user PG_SERVER=pg.example.com HASS_DB=homeassistant_db DUMP_D=/path/to/directory hass-postgresql-backup\nYou don\u0026rsquo;t need to stop Home Assistant to pull a backup, and you also don\u0026rsquo;t need to run it on the same server you\u0026rsquo;re running the postgres server on.\nUpdate @myoung34 has shared their Argo Workflow for backing up Postgres\n","permalink":"https://unixorn.github.io/post/hass-using-postgresql-instead-of-sqlite/","summary":"\u003cp\u003eI wanted to switch my new Home Assistant (HA) installation to write data to \u003ca href=\"https://www.postgresql.org/\"\u003ePostgreSQL\u003c/a\u003e instead of SQLite for a variety of reasons. Here\u0026rsquo;s how I did it.\u003c/p\u003e\n\u003cp\u003eHere\u0026rsquo;s why I decided to switch:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eResilience\u003c/strong\u003e. If you\u0026rsquo;re running Home Assistant on a Raspberry Pi\u0026rsquo;s SD card, the constant churn of history updates will eventually destroy the card. The more entities you have, the faster HA will grind your SD card to failure. Writing all that to another server that is writing to a real SSD or spinning disk eliminates that problem.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eConvenience\u003c/strong\u003e. I can back up the postgres database without having to stop HA. I don\u0026rsquo;t even have to run the backup on the HA server.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eSpeed\u003c/strong\u003e. Using a real database will speed up history display, especially once you have a large number of entities.\u003c/li\u003e\n\u003c/ul\u003e","title":"Switch Home Assistant to Use PostgreSQL Instead of SQLite"},{"content":"Securifi Peanut plugs have issues with zigbee2mqtt.\nI have several Securifi Peanut Zigbee switches. Overall, they\u0026rsquo;re nice little smart plugs and make good Zigbee routers to strengthen your Zigbee mesh, but they have one annoying issue - zigbee2mqtt doesn\u0026rsquo;t recognize them perfectly, though there\u0026rsquo;s a simple fix that I\u0026rsquo;m going to document here.\nAdding Peanut Smart Plugs to zigbee2mqtt The Peanut Smart Plug does not provide a modelId in its database entry, so zigbee2mqtt can\u0026rsquo;t identify it to know how to handle it. Fortunately, it\u0026rsquo;s an easy fix, though you\u0026rsquo;ll have to do it every time you add a new Peanut plug.\nPair your new Peanut(s) to your zigbee2mqtt instance Once you\u0026rsquo;ve added all the peanut plugs, stop zigbee2mqtt. We\u0026rsquo;re going to need to do some surgery on its database.db file and that can\u0026rsquo;t be done with the service running. Backup your database.db file. If you mess up the edit, you\u0026rsquo;ll want to be able to revert and try again easily. Edit your database.db file. Add a \u0026quot;modelId\u0026quot;:\u0026quot;PP-WHT-US\u0026quot; to each of your Peanut entries. For example, change \u0026quot;\u0026quot;manufId\u0026quot;:4098, to \u0026quot;manufId\u0026quot;:4098,\u0026quot;modelId\u0026quot;:\u0026quot;PP-WHT-US\u0026quot;, Once you\u0026rsquo;ve finished editing database.db, restart the zigbee2mqtt service. You should now see proper entries with capabilities in zigbee2mqtt and be able to turn the switches on and off, both from zigbee2mqtt and from Home Assistant. Now would be a good time to go to zigbee2mqtt\u0026rsquo;s OTA tab and check if your Peanut plug(s) have any firmware updates.\n","permalink":"https://unixorn.github.io/post/securifi-peanut-gotcha/","summary":"\u003cp\u003eSecurifi Peanut plugs have issues with zigbee2mqtt.\u003c/p\u003e\n\u003cp\u003eI have several Securifi \u003ca href=\"https://smile.amazon.com/gp/product/B00TC9NC82\"\u003ePeanut\u003c/a\u003e Zigbee switches. Overall, they\u0026rsquo;re nice little smart plugs and make good Zigbee routers to strengthen your Zigbee mesh, but they have one annoying issue - \u003ca href=\"https://zigbee2mqtt.io\"\u003ezigbee2mqtt\u003c/a\u003e doesn\u0026rsquo;t recognize them perfectly, though there\u0026rsquo;s a simple fix that I\u0026rsquo;m going to document here.\u003c/p\u003e","title":"Fix Securifi Peanut issue with zigbee2mqtt"},{"content":"I wanted my Home Assistant to be able to send me alerts when Bad Things are detected like water on my basement floor. I\u0026rsquo;m an SRE, and have been using PagerDuty for years, so I decided to set up a personal PagerDuty account and connect it to my Home Assistant.\nWhy PagerDuty and not Twilio SMS? When I was working at Twilio I wrote the blog post on their site about using their API with Home Assistant. The down side of using Twilio to notify is that their API doesn\u0026rsquo;t have a simple way to mark messages as potential duplicates, so if you have an automation that triggers for some potentially flappy state like motion in the yard you might get 20 messages for the same problem, which is not awesome in the middle of the night.\nWith PagerDuty, you can specify a dedupe_key and it will automatically ignore new alerts on that key until you resolve the incident, without you having to add any complicated logic to your automations or notify service.\nPrerequisites A working Home Assistant installation. Initial set up of Home Assistant is out of scope for this post - presumably you\u0026rsquo;re reading this because you already have Home Assistant running and want to enable it to send you notifications A PagerDuty account. They have a free tier that is probably more than adequate for most people\u0026rsquo;s needs. Setup PagerDuty Create your PagerDuty account at PagerDuty.com and create a new service. I unimaginatively named mine \u0026ldquo;Home Assistant\u0026rdquo;. You\u0026rsquo;ll also need to configure your notification settings in your profile. If you don\u0026rsquo;t have their app installed, you can install it now, but PagerDuty can send you SMS messages and you can interact with it that way if you prefer.\nIn the new service, click the Integrations tab. We\u0026rsquo;re going to want to use v2 of the Events API to create alerts, so click on the down arrow to the right of Events API V2 to reveal details. We\u0026rsquo;re going to need both the Integration Key and the Integration URL (Alert Events), so copy them into a scratch file for later.\nSchedules PagerDuty will have created a default schedule for your service, which is great. What\u0026rsquo;s not great is that it defaults to alerting 24 hours a day - I don\u0026rsquo;t want to get woken up because of something minor like a network issue, especially on weekends, so I edited my schedule to only be active between 10 am and 10 pm.\nHome Assistant Now we\u0026rsquo;ll connect PagerDuty to Home Assistant. I didn\u0026rsquo;t want to install a helper script into my HA system, and conveniently enough you can create alerts using PagerDuty\u0026rsquo;s API directly, so we\u0026rsquo;re going to create a rest_command to make the API calls for us.\nTo keep our configuration.yaml easier to share, we want to use !secret, so first, edit your secrets.yaml file and add a pd_integration_key entry with the integration key you copied from your service\u0026rsquo;s Integrations tab.\nAnnoyingly, you can\u0026rsquo;t directly embed a !secret xyz call inside the rest_command\u0026rsquo;s payload, so we\u0026rsquo;re going to have to work around it by creating a sensor (where we can use !secret) and then read that value into the rest_command\u0026rsquo;s payload template. This is ugly but it works.\nAdd a pd_routing_key sensor to your configuration.yaml file\nsensor: - platform: template sensors: pd_routing_key: value_template: !secret pd_integration_key Now that we\u0026rsquo;ve defined our sensor, we can refer to its value inside of the payload template in our rest_command, so define a pagerduty_message command by adding the following yaml snippet to your configuration.yaml. I created the payload in this snippet from the examples in PagerDuty\u0026rsquo;s Incident API documentation.\nIf the URL you copied for Integration URL (Alert Events) is different than the url key in the snippet, change the url value accordingly.\nrest_command: pagerduty_message: url: https://events.pagerduty.com/v2/enqueue method: POST payload: \u0026gt;- { \u0026#34;routing_key\u0026#34;:\u0026#34;{{ states(\u0026#39;sensor.pd_routing_key\u0026#39;) }}\u0026#34;, \u0026#34;dedup_key\u0026#34;:\u0026#34;{{ dedup_key }}\u0026#34;, \u0026#34;event_action\u0026#34;:\u0026#34;trigger\u0026#34;, \u0026#34;payload\u0026#34;: { \u0026#34;summary\u0026#34;:\u0026#34;{{ message }}\u0026#34;, \u0026#34;source\u0026#34;:\u0026#34;{{ source }}\u0026#34;, \u0026#34;severity\u0026#34;:\u0026#34;{{ severity }}\u0026#34;, \u0026#34;custom_details\u0026#34;: { \u0026#34;{{custom_title}}\u0026#34;: \u0026#34;{{ custom_details }}\u0026#34; } } } Confirm that you don\u0026rsquo;t have any typos and your configuration is valid by going to http://yourHA:8123/developer-tools/yaml and clicking CHECK CONFIGURATION. If it reports it valid, you can safely restart HA.\nTesting Click on the services tab on your HA\u0026rsquo;s developer tools page. Select RESTful Command: pagerduty_message, and then paste the following snippet in.\nmessage value is the subject of the alert. dedup_key is used to prevent you from getting spammed with multiple alerts for the same problem. Until you mark an incident as resolved, any new alerts with the same dedup_key value will not generate new alerts. I have a different unique dedup_key for each of my water sensor automations so that if water is detected under one of my sinks, I\u0026rsquo;ll only get one alert per sink instead of getting a new alert every five minutes for the same problem. severity- must be critical, error, warning, or info custom_title - This will be displayed in the Details section of alerts. custom_details - This will be in the Details section of alerts. service: rest_command.pagerduty_message data: message: This is the subject line dedup_key: a_unique_string severity: info source: Test Source custom_title: Issue custom_details: Test alert to PagerDuty Click the CALL SERVICE button. You should get an alert almost immediately, depending on your personal notification settings. Once that\u0026rsquo;s working, you can use your new pagerduty_message service in your scripts and automations.\n","permalink":"https://unixorn.github.io/post/use-pagerduty-with-home-assistant/","summary":"\u003cp\u003eI wanted my Home Assistant to be able to send me alerts when Bad Things are detected like water on my basement floor. I\u0026rsquo;m an SRE, and have been using PagerDuty for years, so I decided to set up a personal \u003ca href=\"https://www.pagerduty.com/\"\u003ePagerDuty\u003c/a\u003e account and connect it to my Home Assistant.\u003c/p\u003e","title":"Use Pagerduty With Home Assistant"},{"content":"I signed up for the free personal edition of Autodesk Fusion 360 to get started with 3d printing, and the first time I tried to launch it on macOS, it wouldn\u0026rsquo;t let me in because it refused to create a new team for me to join.\nThe solution ended up being to quit Fusion 360, open the Fusion Team Signup page in a browser, create a team there, then restart Fusion 360.\nI found the solution in a forum post from 2019, so I\u0026rsquo;m not holding my breath waiting for Autodesk to fix it.\n","permalink":"https://unixorn.github.io/post/autodesk-fusion-signup/","summary":"\u003cp\u003eI signed up for the free personal edition of Autodesk Fusion 360 to get started with 3d printing, and the first time I tried to launch it on macOS, it wouldn\u0026rsquo;t let me in because it refused to create a new team for me to join.\u003c/p\u003e\n\u003cp\u003eThe solution ended up being to quit Fusion 360, open the \u003ca href=\"http://login.autodesk360.com/login/signup?product=fusion\u0026amp;edition=business\"\u003eFusion Team Signup\u003c/a\u003e page in a browser, create a team there, then restart Fusion 360.\u003c/p\u003e","title":"Fixing Autodesk Fusion 360 First Time Launch on macOS Problems"},{"content":"TL;DR - SMR drives can take thirteen to sixteen times as long to resilver in your ZFS raid than CMR drives. If they even succeed. This wouldn\u0026rsquo;t be a big deal, except that Western Digital started using SMR technology in their WD-Red drives that are marketed toward SOHO and small business raid, without any warnings about the RAID performance implications.\nI got lucky when I bought my last batch of Reds, they were all CMR, but it was pure luck - I bought them based on Western Digital\u0026rsquo;s reputation and because I\u0026rsquo;ve seen multiple NAS vendors recommend WD-REDs in the past.\nHere\u0026rsquo;s some articles with more details.\nSurreptitiously Swapping SMR into Hard Drive Lines Must Stop IXSystems (the TrueNAS/FreeNAS vendor)\u0026rsquo;s statement that they will no longer ship SMR drives in their products WD has since forked off a RED PLUS product line with CMR.\nAnyway, check your spares bin for SMR drives, and check the drives in your NAS - if you have SMR drives in the NAS, pull a backup and start swapping in CMR replacements.\n","permalink":"https://unixorn.github.io/post/western-digital-red-smr-fiasco/","summary":"\u003cp\u003eTL;DR - SMR drives can take thirteen to sixteen times as long to resilver in your ZFS raid than CMR drives. If they even succeed. This wouldn\u0026rsquo;t be a big deal, except that Western Digital started using SMR technology in their WD-Red drives that are marketed toward SOHO and small business raid, without any warnings about the RAID performance implications.\u003c/p\u003e\n\u003cp\u003eI got lucky when I bought my last batch of Reds, they were all CMR, but it was pure luck - I bought them based on Western Digital\u0026rsquo;s reputation and because I\u0026rsquo;ve seen multiple NAS vendors recommend WD-REDs in the past.\u003c/p\u003e","title":"Western Digital Red drive SMR Fiasco"},{"content":"I wrote two articles in sysadvent 2021\nBaking Multi-Architecture Docker Images Setting up k3s in your home lab ","permalink":"https://unixorn.github.io/post/sysadvent-2021-article/","summary":"\u003cp\u003eI wrote two articles in sysadvent 2021\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://sysadvent.blogspot.com/2021/12/day-7-baking-multi-architecture-docker.html\"\u003eBaking Multi-Architecture Docker Images\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://sysadvent.blogspot.com/2021/12/day-16-setting-up-k3s-in-your-home-lab.html\"\u003eSetting up k3s in your home lab\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e","title":"Sysadvent 2021 Articles"},{"content":"For a variety of reasons, I needed to enable some EC2 instances to write/update a single EC2 tag, but the instaces needed to only be able to tag themselves.\nThis was more annoying than I expected, so I\u0026rsquo;m documenting the IAM policy here.\n{ \u0026#34;Version\u0026#34;: \u0026#34;2012-10-17\u0026#34;, \u0026#34;Statement\u0026#34;: [ { \u0026#34;Effect\u0026#34;: \u0026#34;Allow\u0026#34;, \u0026#34;Action\u0026#34;: [ \u0026#34;ec2:DeleteTags\u0026#34;, \u0026#34;ec2:CreateTags\u0026#34;, \u0026#34;ec2:DescribeInstances\u0026#34; ], \u0026#34;Resource\u0026#34;: \u0026#34;*\u0026#34;, \u0026#34;Condition\u0026#34;: { \u0026#34;StringEquals\u0026#34;: { \u0026#34;aws:ARN\u0026#34;: \u0026#34;${ec2:SourceInstanceARN}\u0026#34; }, \u0026#34;ForAllValues:StringEquals\u0026#34;: { \u0026#34;aws:TagKeys\u0026#34;: \u0026#34;THAT_ONE_ALLOWED_TAG\u0026#34; } } } ] } Some notes:\nThe AWS IAM editor in the webui will (as of June 2021) complain about SourceInstanceARN. Ignore it and click next. Then it will complain that the policy doesn\u0026rsquo;t add any permissions. It lies. Ignore it and save the policy. You can attach this policy to an IAM role and the instances will then be able to tag themselves, but only with the THAT_ONE_ALLOWED_TAG tag.\n","permalink":"https://unixorn.github.io/post/iam-self-tagging/","summary":"\u003cp\u003eFor a variety of reasons, I needed to enable some EC2 instances to write/update a single EC2 tag, but the instaces needed to only be able to tag themselves.\u003c/p\u003e","title":"AWS IAM Self Tagging EC2 Instances"},{"content":"I wanted to set up a security camera outside, but I didn\u0026rsquo;t want to be dependent on an outside cloud service - if my internet goes out, I don\u0026rsquo;t want to lose my ability to record video.\nWyze cameras are nice and cheap, and you can reflash them to support RTSP in addition to streaming to the Wyze cloud.\nSetup Prerequisites A camera that supports the RTSP protocol (I\u0026rsquo;m using a Wyze G2) A spare microSD card for reflashing the Wyze G2 camera An x86 machine running docker. As of 2021-03-14, Shinobi only publishes an amd64 version of the shinobisystems/shinobi docker image. A reasonable amount of disk space - the Wyze G2 I\u0026rsquo;m using generates around 330 megs per hour of stored 1080p video. Camera Setup I wanted a camera that supported the Real Time Streaming Protocol (RTSP) because that is an open standard which works with a wide variety of tooling, both Open Source and commercial.\nI looked at a variety of camera options, and Wirecutter\u0026rsquo;s Best Wifi Home Security Camera listed the Wyze G2 as runner-up. It and the first choice (Eufy 2K Indoor cam) both support RTSP, but the Wyze was in stock (and half the price at $26) so I went with it.\nI did have to reflash the Wyze G2 to enable a beta firmware that supports both Wyze\u0026rsquo;s cloud and RTSP. Conveniently, it can stream to both simultaneously, so I can watch the streams with the Wyze app when away from home and still record everything to my homelab cluster.\nReflash the Wyze Camera Wyze now has a beta firmware that simultanously supports both their cloud offering and RTSP. Note that they\u0026rsquo;ve explicitly stated that the RTSP branch will get features later than the mainline firmware. I personally don\u0026rsquo;t care, but it is something to consider if you\u0026rsquo;re going to want to use bleeding edge features.\nThe official instructions for reflashing the G2 camera are on the Wyze Cam RTSP page and clear, so I\u0026rsquo;m not going to rehash them here. You\u0026rsquo;ll need a FAT32 formatted microSD card to do the firmware reflash.\nAfter you reflash the camera, you\u0026rsquo;ll need to configure a username/password combination for the camera stream using the Wyze phone app.\nBefore you configure the camera, I recommend that you go into your router\u0026rsquo;s configuration and assign the camera a static IP so that your DVR doesn\u0026rsquo;t lose the stream connection when the camera or router are rebooted. You can also hardcode an IP address into the G2 camera\u0026rsquo;s configuration, but I prefer to keep all the static IP assignments for my network in one place, the DHCP configuration on my router.\nYou\u0026rsquo;ll end up with a rtsp url that looks like rtsp://username:password@192.168.1.101/live.\nDVR Setup I don\u0026rsquo;t use any IOT devices that require a cloud service to function. In this case, I especially do not want to be unable to record security footage just because the internet is down, so I set up shinobi as a local DVR to record my security footage.\nStart shinobi I\u0026rsquo;m running shinobi in a docker container. As of 2021-03-14, there is only an AMD64 build of this docker image so I\u0026rsquo;m running it on the Intel machine in my homelab.\nHere\u0026rsquo;s a shinobi-start script:\n#!/usr/bin/env bash # # Start shinobi # # Copyright 2021, Joe Block \u0026lt;jpb@unixorn.net\u0026gt; # # License: Apache 2.0 SHINOBI_D=${SHINOBI_D:-\u0026#39;/data/shinobi\u0026#39;} set -o pipefail if [[ -n \u0026#34;$DEBUG\u0026#34; ]]; then set -x fi for dvr_d in config customAutoLoad database plugins video do mkdir -p \u0026#34;$SHINOBI_D/$dvr_d\u0026#34; done exec docker run -d -p 8080:8080 \\ --name=\u0026#39;shinobi\u0026#39; \\ -v ${SHINOBI_D}/config:/config:rw \\ -v ${SHINOBI_D}/customAutoLoad:/home/Shinobi/libs/customAutoLoad:rw \\ -v ${SHINOBI_D}/database:/var/lib/mysql:rw \\ -v ${SHINOBI_D}/plugins:/plugins:rw \\ -v ${SHINOBI_D}/videos:/home/Shinobi/videos:rw \\ -v /dev/shm/Shinobi/streams:/dev/shm/streams:rw \\ -v /etc/localtime:/etc/localtime:ro \\ -v /etc/timezone:/etc/timezone:ro \\ --restart always \\ shinobisystems/shinobi Run this with SHINOBI_D=/path/to/local/dvr/files shinobi_start and it will create any missing required directories for you and start shinobi.\nConfigure shinobi Set up a new admin account Login at http://your.shinobi.server:8080/super with username admin@shinobi.vido and password admin Create a new admin account Don\u0026rsquo;t forget to reset the password for the admin@shinobi.video account! Add the camera Login at http://your.shinobi.server:8080 Click on the + icon in the toolbar at the top of the page Set mode to record Change the name to something human friendly like \u0026ldquo;Mailbox Camera\u0026rdquo; Set input type (in the connection section) to H.264 / H.265 / H.265+ Set the full URL path to the rtsp stream url you got from the camera Optionally set Skip Ping to Yes Set Stream Type to HLS (includes audio) Set Record File Type to MP4 Set Video codec to copy Set Audio Codec to Auto Save You can optionally set retention times for the camera data.\nIt took about 30-45 seconds before my camera stream was visible in shinobi.\n","permalink":"https://unixorn.github.io/post/setting-up-shinobi-and-a-wyze-g2-camera/","summary":"\u003cp\u003eI wanted to set up a security camera outside, but I didn\u0026rsquo;t want to be dependent on an outside cloud service - if my internet goes out, I don\u0026rsquo;t want to lose my ability to record video.\u003c/p\u003e\n\u003cp\u003eWyze cameras are nice and cheap, and you can reflash them to support RTSP in addition to streaming to the Wyze cloud.\u003c/p\u003e","title":"Setting up Shinobi and a Wyze G2 Camera"},{"content":"I\u0026rsquo;ve got an old HP laser printer in my basement. We barely print 10 pages a month between the two of us, so we only turn it on when we\u0026rsquo;re going to print. That\u0026rsquo;s a hassle though, because inevitably we forget to shut it off sometimes and it stays on overnight or even for days, and while it has a powersave mode, the 4050N is so old that even that burns a good amount of power.\nEnter Home Assistant.\nPrerequisites You have HA configured to connect to a MQTT server The watcher script and associated tooling all presume that we can send messages to a MQTT topic that HA is watching.\nYour printer is connected to a cupsd server running in a container Your computers should be configured to print to the cupsd server instead of directly to the printer.\nI run cupsd in a container on one of my Odroids. I could run it on the same Odroid HC2 that I run Home Assistant (HA) on, but there\u0026rsquo;s no compelling reason to do so and I\u0026rsquo;m reserving that node for strictly HA containers like Home Assistant itself and my MQTT server. I picked an Odroid because it has a SATA drive attached and my /var/lib/docker is on the hard drive and not an microSD card - there\u0026rsquo;s no reason you can\u0026rsquo;t run it on a Raspberry Pi other than to prevent excessive wear on the microSD card.\nYou could modify the watcher script if you\u0026rsquo;re running cupsd directly instead of in a container, but I run my cupsd in a container, so that\u0026rsquo;s what the script is designed for.\nThere are plenty of articles about setting up cupsd, but I wrote about setting up cupsd here.\nYour printer is plugged into an outlet controlled by HA We want to be able to toggle the power from Home Assistant.\nPrinter Power Control mosquitto helper script I don\u0026rsquo;t like to install anything more on my docker hosts than I absolutely have to, so instead of installing mosquitto directly on the printserver machine, I run mosquitto_pub inside a container with the following c-mosquitto_pub helper script. You can download it from github here. Put this in /usr/local/bin.\n#!/usr/bin/env bash # # Use docker to run mosquitto_pub # # Copyright 2019, Joe Block \u0026lt;jpb@unixorn.net\u0026gt; # # Licensed under the Apache License, Version 2.0 (the \u0026#34;License\u0026#34;); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an \u0026#34;AS IS\u0026#34; BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. exec docker run -t --rm eclipse-mosquitto mosquitto_pub $@ cupsd Watcher Once I had cupsd configured to share the printer (as Franklin), I wrote a quick script that checks the print queue to see if it is empty or not. If there are jobs in the queue, it writes ON to an MQTT topic, hass/printers/franklin. If the queue is empty, it writes OFF. The examples here all assume your printer is named Franklin, replace Franklin with your printer\u0026rsquo;s name.\nActually, I lied. When there are jobs, it writes OFF and then ON.\nWhy? Because I don\u0026rsquo;t want HA to switch the printer off immediately once the queue drains - the printer has enough RAM that there may still be several pages left to print when it has accepted all of the job from the server.\nInstead, I\u0026rsquo;ve configured HA to restart a timer every time it sees the MQTT topic hass/printers/franklin switch from OFF to ON, and only turn the printer off after the queue has been empty for five continuous minutes.\nHere\u0026rsquo;s the ha-check-for-print-jobs script source - you can download it from github here.\nPut the script in /usr/local/bin on the same server you\u0026rsquo;re running the cupsd container on - it is designed to run a tool inside that container.\n#!/usr/bin/env bash # # ha-check-for-print-jobs # # Check if there are print jobs on $PRINT_Q. If there are, write # MQTT messages to a watched topic so HA knows to turn on the power # to the printer. # # Copyright 2021, Joe Block \u0026lt;jpb@unixorn.net\u0026gt; # # License: Apache 2 set -o pipefail # Make all these overridable easily in your cron setup PRINT_Q=${PRINT_Q:-\u0026#39;Franklin\u0026#39;} CONTAINER=${CONTAINER:-\u0026#39;cupsd-server\u0026#39;} MQTT_HOST=${MQTT_HOST:-\u0026#39;mqtt.example.com\u0026#39;} MQTT_TOPIC=${MQTT_TOPIC:-\u0026#39;hass/printers/franklin\u0026#39;} # We are run out of cron every minute, but I don\u0026#39;t want it to take an # entire minute to turn on the power because I\u0026#39;m impatient and the printer # takes a bit to start up. When we print and walk downstairs, I want it # to have already started printing by the time I get there. If I was # patient, I wouldn\u0026#39;t have bothered to write this tool :-) # # So, when we get run by cron, we check the queue CHECK_COUNT times, with # CHECK_DELAY seconds between each run. CHECK_COUNT=${CHECK_COUNT:-\u0026#39;11\u0026#39;} CHECK_DELAY=${CHECK_DELAY:-\u0026#39;5\u0026#39;} export PATH=\u0026#34;$PATH:/usr/local/bin:/usr/local/sbin\u0026#34; if [[ -f /tmp/printerdebug ]]; then DEBUG=\u0026#39;true\u0026#39; fi # Only spam syslog when DEBUG is set debugout() { if [[ -n \u0026#34;$DEBUG\u0026#34; ]]; then echo \u0026#34;$@\u0026#34; fi } validate-settings(){ debugout \u0026#34;CONTAINER: $CONTAINER\u0026#34; debugout \u0026#34;PRINT_Q: $PRINT_Q\u0026#34; debugout \u0026#34;MQTT_HOST: $MQTT_HOST\u0026#34; debugout \u0026#34;MQTT_TOPIC: $MQTT_TOPIC\u0026#34; valid=\u0026#39;true\u0026#39; if [[ -z \u0026#34;$CONTAINER\u0026#34; ]]; then echo \u0026#34;CONTAINER is unset\u0026#34; valid=\u0026#39;false\u0026#39; fi if [[ -z \u0026#34;$PRINT_Q\u0026#34; ]]; then echo \u0026#34;PRINT_Q is unset\u0026#34; valid=\u0026#39;false\u0026#39; fi if [[ -z \u0026#34;$MQTT_HOST\u0026#34; ]]; then echo \u0026#34;MQTT_HOST is unset\u0026#34; valid=\u0026#39;false\u0026#39; fi if [[ -z \u0026#34;$MQTT_TOPIC\u0026#34; ]]; then echo \u0026#34;MQTT_TOPIC is unset\u0026#34; valid=\u0026#39;false\u0026#39; fi if [[ \u0026#34;$valid\u0026#34; == \u0026#34;false\u0026#34; ]]; then echo \u0026#34;Configure your settings.\u0026#34; exit 1 fi } print-job-checker() { printjobs=$(docker exec -t \u0026#34;$CONTAINER\u0026#34; lpq -P \u0026#34;$PRINT_Q\u0026#34; | grep -c \u0026#39;no entries\u0026#39;) if [[ \u0026#34;$printjobs\u0026#34; == \u0026#39;1\u0026#39; ]]; then debugout \u0026#34;No jobs in print queue, notifying HA\u0026#34; c-mosquitto_pub -h \u0026#34;$MQTT_HOST\u0026#34; -t \u0026#34;$MQTT_TOPIC\u0026#34; -m OFF else echo \u0026#34;jobs found in print queue, notifying HA\u0026#34; docker exec -t \u0026#34;$CONTAINER\u0026#34; lpq -P \u0026#34;$PRINT_Q\u0026#34; # Set the status off, then back to on, so that the HA timer restarts # and HA doesn\u0026#39;t turn off the printer in the middle of a job c-mosquitto_pub -h \u0026#34;$MQTT_HOST\u0026#34; -t \u0026#34;$MQTT_TOPIC\u0026#34; -m OFF c-mosquitto_pub -h \u0026#34;$MQTT_HOST\u0026#34; -t \u0026#34;$MQTT_TOPIC\u0026#34; -m ON debugout \u0026#34;re-enabling printer $PRINT_Q...\u0026#34; docker exec -t \u0026#34;$CONTAINER\u0026#34; lpadmin -p \u0026#34;$PRINT_Q\u0026#34; -o printer-error-policy=retry-current-job fi } validate-settings # We run the print-job-checker every 5 seconds to minimize the UI delay on the # macOs end for i in $(seq $CHECK_COUNT) do print-job-checker debugout \u0026#34;waiting...\u0026#34; sleep $CHECK_DELAY done Home Assistant Setup I configured my HA to watch a MQTT topic as a binary sensor. You can download this snippet here.\nbinary_sensor: - platform: mqtt name: \u0026#34;Franklin Print Queue\u0026#34; payload_on: \u0026#34;ON\u0026#34; state_topic: \u0026#34;hass/printers/franklin\u0026#34; Now, when the watcher writes ON and OFF to the hass/printers/franklin queue, that binary sensor will change status and we can trigger an automation for it.\nThis automation will turn the printer power on every time the binary sensor is turned on, and turn it off five minutes after the last time the binary sensor switched from ON to OFF.\nThe outlet my printer is plugged into is controlled by HA and rather unimaginatively named switch.printerpower.\nAdd this stanza to your automations.yaml file. Download it here.\n# Franklin power is controlled by MQTT - alias: \u0026#39;Turn on Franklin when there are jobs in the queue\u0026#39; trigger: platform: state entity_id: binary_sensor.franklin_print_queue to: \u0026#39;on\u0026#39; action: service: homeassistant.turn_on entity_id: switch.printerpower - alias: \u0026#39;Turn off printer 5 minutes after print queue drains\u0026#39; trigger: platform: state entity_id: binary_sensor.franklin_print_queue to: \u0026#39;off\u0026#39; for: minutes: 5 action: service: homeassistant.turn_off entity_id: switch.printerpower Test the pieces Print server check Confirm that you\u0026rsquo;ve got the print queue configured correctly by running docker exec -it cupsd-server lpq -P Franklin. If there are no jobs, it should print something like\nFranklin is ready no entries Automation test Reload your automations, and you should now be able to test that the automations are correct by running c-mosquitto_pub -h mqtt.yourdomain.com -t hass/printers/franklin -m OFF or -m ON and watch HA turn the power to your printer off and on.\nOnce that is working, print a job, and if you run ha-check-for-print-jobs the printer power should get turned on.\nRun it all automatically Now that you\u0026rsquo;ve confirmed that the power is being cycled properly when the MQTT queue recieves messages and that the print job checker is seeing the printer queue, we can add the checker job to cron.\nAdd\n* * * * * PRINT_Q=Franklin MQTT_HOST=mqtt.example.com MQTT_TOPIC=hass/printers/franklin CONTAINER=cupsd_server /usr/local/bin/ha-check-for-print-jobs | logger -t printserver to your /etc/crontab, and you\u0026rsquo;re good to go. Now every minute, the checker script will get run by cron, and it will check every five seconds for print jobs and exit before the next invocation by cron.\n","permalink":"https://unixorn.github.io/post/home-assistant-printer-power-management/","summary":"\u003cp\u003eI\u0026rsquo;ve got an old HP laser printer in my basement. We barely print 10 pages a month between the two of us, so we only turn it on when we\u0026rsquo;re going to print. That\u0026rsquo;s a hassle though, because inevitably we forget to shut it off sometimes and it stays on overnight or even for days, and while it has a powersave mode, the 4050N is so old that even that burns a good amount of power.\u003c/p\u003e\n\u003cp\u003eEnter Home Assistant.\u003c/p\u003e","title":"Home Assistant Printer Power Management"},{"content":"I have an old HP 4050N. For a variety of reasons, I want to have it behind a print server instead of having my laptops print directly to it. Here\u0026rsquo;s how I set that up.\nPrerequisites A Raspberry Pi (or honestly any linux box) running docker. I like the Raspberry Pi and Odroid HC2 for this sort of thing because they have very low power requirements. A printer supported by the CUPS project. Setup Setup is easy. There are several docker images out there you can use, I made one (the source is on github at unixorn/docker-cupsd) because I wanted one that was multi-architecture - my image has AMD64, ARM7 and ARM64 artchitectures all baked into the same image so you don\u0026rsquo;t have to change the image label based on what system you\u0026rsquo;re running it on. It works fine on (at least) Raspberry Pi, ODROID and Intel servers.\nThe unixorn/cupsd docker image is a bit on the large side because I crammed a lot of printer drivers into it, you may want to look for images that only support single printer families. The username and password in the image default to print and imaginatively, print.\nWe\u0026rsquo;re going to store printers.conf in a directory outside the container so that we don\u0026rsquo;t lose our printer configuration every time we upgrade our container.\nI run the cupsd server on an Odroid HC2 because I have /var/lib/docker on the 2TB drive attached to it. I could have put it on one of the Raspberry Pis in the cluster, but didn\u0026rsquo;t want it spooling print jobs and causing excessive wear on the rPi\u0026rsquo;s microSD card.\nMake a directory to store your printer configuration. We\u0026rsquo;ll use /docker/cupsd/etc export CUPSD_DIR='/docker/cupsd/etc' touch $CUPSD_DIR/printers.conf Run the cupsd server with sudo docker run -d --restart unless-stopped \\ -p 631:631 \\ --privileged \\ -v /var/run/dbus:/var/run/dbus \\ -v /dev/bus/usb:/dev/bus/usb \\ -v $CUPSD_DIR/printers.conf:/etc/cups/printers.conf \\ unixorn/cupsd You can now connect to http://cupsd.example.com:631 and add printers using the web UI.\nWhen adding the printers to your Mac, select Internet Printing Protocol and put in the IP or DNS name of your print server machine.\nWhen entering the printer information into your printer settings, the queues should be entered as printers/printername, not printername.\n","permalink":"https://unixorn.github.io/post/cupsd-setup/","summary":"\u003cp\u003eI have an old HP 4050N. For a variety of reasons, I want to have it behind a print server instead of having my laptops print directly to it. Here\u0026rsquo;s how I set that up.\u003c/p\u003e","title":"Run a CUPSD print server on Raspberry Pi"},{"content":"Sending notifications from Home Assistant via Twilio SMS I got a post published at work about How to Receive Alerts from Home Assistant with Twilio SMS, so I won\u0026rsquo;t replicate it here.\nTL;DR - It is super easy to send SMS messages via Twilio SMS with curl.\nUpdate: I wrote another article, Use PagerDuty with Home Assistant this time for using PagerDuty which allows automatically de-duping notifications.\n","permalink":"https://unixorn.github.io/post/home-assistant-notifications-via-twilio/","summary":"\u003ch2 id=\"sending-notifications-from-home-assistant-via-twilio-sms\"\u003eSending notifications from Home Assistant via Twilio SMS\u003c/h2\u003e\n\u003cp\u003eI got a post published at work about \u003ca href=\"https://www.twilio.com/blog/home-assistant-twilio-sms-alerts\"\u003eHow to Receive Alerts from Home Assistant with Twilio SMS\u003c/a\u003e, so I won\u0026rsquo;t replicate it here.\u003c/p\u003e\n\u003cp\u003eTL;DR - It is super easy to send SMS messages via Twilio SMS with \u003ccode\u003ecurl\u003c/code\u003e.\u003c/p\u003e\n\u003cp\u003eUpdate: I wrote another article, \u003ca href=\"https://unixorn.github.io/post/use-pagerduty-with-home-assistant/\"\u003eUse PagerDuty with Home Assistant\u003c/a\u003e this time for using PagerDuty which allows automatically de-duping notifications.\u003c/p\u003e","title":"Home Assistant Notifications via Twilio SMS"},{"content":"Biden won! Awesome! The fight isn\u0026rsquo;t over though.\nTime to donate to the Georgia Runoff Senate Elections. I\u0026rsquo;ve set up identical fundraisers, this time for all the different Star Trek Series.\nDonate to your favorite series link and we\u0026rsquo;ll see which series has the most fan support.\nTOS - https://secure.actblue.com/donate/ga-tos TNG - https://secure.actblue.com/donate/ga-tng DS9 - https://secure.actblue.com/donate/ga-ds9 Enterprise - https://secure.actblue.com/donate/ga-enterprise Voyager - https://secure.actblue.com/donate/ga-voyager Discovery - https://secure.actblue.com/donate/ga-discovery Lower Decks - https://secure.actblue.com/donate/ga-lowerdecks Picard - https://secure.actblue.com/donate/ga-picard ","permalink":"https://unixorn.github.io/post/2021-runoff-election/","summary":"\u003cp\u003eBiden won! Awesome! The fight isn\u0026rsquo;t over though.\u003c/p\u003e\n\u003cp\u003eTime to donate to the Georgia Runoff Senate Elections. I\u0026rsquo;ve set up identical fundraisers, this time for all the different Star Trek Series.\u003c/p\u003e\n\u003cp\u003eDonate to your favorite series link and we\u0026rsquo;ll see which series has the most fan support.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eTOS - \u003ca href=\"https://secure.actblue.com/donate/ga-tos\"\u003ehttps://secure.actblue.com/donate/ga-tos\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eTNG - \u003ca href=\"https://secure.actblue.com/donate/ga-tng\"\u003ehttps://secure.actblue.com/donate/ga-tng\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eDS9 - \u003ca href=\"https://secure.actblue.com/donate/ga-ds9\"\u003ehttps://secure.actblue.com/donate/ga-ds9\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eEnterprise - \u003ca href=\"https://secure.actblue.com/donate/ga-enterprise\"\u003ehttps://secure.actblue.com/donate/ga-enterprise\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eVoyager - \u003ca href=\"https://secure.actblue.com/donate/ga-voyager\"\u003ehttps://secure.actblue.com/donate/ga-voyager\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eDiscovery - \u003ca href=\"https://secure.actblue.com/donate/ga-discovery\"\u003ehttps://secure.actblue.com/donate/ga-discovery\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eLower Decks - \u003ca href=\"https://secure.actblue.com/donate/ga-lowerdecks\"\u003ehttps://secure.actblue.com/donate/ga-lowerdecks\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ePicard - \u003ca href=\"https://secure.actblue.com/donate/ga-picard\"\u003ehttps://secure.actblue.com/donate/ga-picard\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e","title":"2021 Georgia Runoff Elections"},{"content":"Adding the Plex integration to Home Assistant is pretty straightforward with the exception of finding your Plex token.\nFind Your Plex token: Sign into your Plex.tv account Browse to one of the media files (TV episode, movie) on your connected server. You\u0026rsquo;ll need a Plex pass for this. Select Get info Click View XML Ignore the XML. Look in the URL to the XML page - you\u0026rsquo;ll see Plex-Token\\=XYZZY. That\u0026rsquo;s the token you\u0026rsquo;ll need. Set up the Plex Home Assistant Integration Once you have the token,\nSign into your Home Assistant Go to Configuration -\u0026gt; Integrations Pick Add, and select Plex. Pick the manual setup option Enter your Plex server\u0026rsquo;s address and the token. ","permalink":"https://unixorn.github.io/post/add-plex-to-hass/","summary":"\u003cp\u003eAdding the Plex integration to Home Assistant is pretty straightforward with the exception of finding your Plex token.\u003c/p\u003e\n\u003ch2 id=\"find-your-plex-token\"\u003eFind Your Plex token:\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003eSign into your \u003ca href=\"https://plex.tv\"\u003ePlex.tv\u003c/a\u003e account\u003c/li\u003e\n\u003cli\u003eBrowse to one of the media files (TV episode, movie) on your connected server. You\u0026rsquo;ll need a Plex pass for this.\u003c/li\u003e\n\u003cli\u003eSelect \u003cstrong\u003eGet info\u003c/strong\u003e\u003c/li\u003e\n\u003cli\u003eClick \u003cstrong\u003eView XML\u003c/strong\u003e\u003c/li\u003e\n\u003cli\u003e\u003cem\u003eIgnore the XML.\u003c/em\u003e Look in the URL to the XML page - you\u0026rsquo;ll see \u003ccode\u003ePlex-Token\\=XYZZY\u003c/code\u003e. That\u0026rsquo;s the token you\u0026rsquo;ll need.\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"set-up-the-plex-home-assistant-integration\"\u003eSet up the Plex Home Assistant Integration\u003c/h2\u003e\n\u003cp\u003eOnce you have the token,\u003c/p\u003e","title":"Add Plex to Home Assistant"},{"content":"Before anyone complains about me getting political on what was mainly a tch site, remember that not being political is a luxury that only people not getting screwed by the current political landscape can afford.\nIn no particular order, the attempts to roll back rights for LGBTQ+ citizens, the assaults on womens\u0026rsquo; rights, the handling of the COVID-19 crisis, packing the Supreme Court with a hypocritical last second Justice, putting kids in cages, the ongoing destruction of the political norms that have existed for decades, the ignoring scientific facts, etc, etc all have focused my efforts on supporting one party.\nSince the tech types reading this blog tend to also be Science Fiction/Gaming/Comic book nerds too, I\u0026rsquo;ve set up a contest to see which fandom is most pro-Democrat.\nLinks below (with identical recipient lists) for several geeky fandoms. Donate, and periodically I\u0026rsquo;ll post totals here.\nStar Wars: https://secure.actblue.com/donate/defeatpalpatine Star Trek https://secure.actblue.com/donate/defeattheborg Dr. Who https://secure.actblue.com/donate/defeatthedaleks Supernatural - https://secure.actblue.com/donate/defeatthechuck Marvel - https://secure.actblue.com/donate/defeatthanos DC - https://secure.actblue.com/donate/defeatdarkseid Feel free to share this, and if you want another fandom added, reply to the tweet I originally announced the contest - https://twitter.com/curiousbiped/status/1315650507806060544\nUpdate:\nFinal standings are:\nStar Trek (by a lot) Star Wars Marvel Battlestar Galactica (widening their lead over #3) Supernatural Tie between DC, and Dr. Who. ","permalink":"https://unixorn.github.io/post/2020-elections/","summary":"\u003cp\u003eBefore anyone complains about me getting political on what was mainly a tch site, remember that \u003cem\u003enot\u003c/em\u003e being political is a luxury that only people not getting screwed by the current political landscape can afford.\u003c/p\u003e\n\u003cp\u003eIn no particular order, the attempts to roll back rights for LGBTQ+ citizens, the assaults on womens\u0026rsquo; rights, the handling of the COVID-19 crisis, packing the Supreme Court with a hypocritical last second Justice, putting kids in cages, the ongoing destruction of the political norms that have existed for decades, the ignoring scientific facts, etc, etc all have focused my efforts on supporting one party.\u003c/p\u003e","title":"2020 Politics"},{"content":"I\u0026rsquo;ve got a mix of architectures in my basement cluster - some Odroid HC2s that are arm7, some Raspberry Pi 4s that are arm64, and am soon going to add an Intel node as well. It\u0026rsquo;s more hassle than it\u0026rsquo;s worth to have to specify different images for the different architectures. I already build my own copies of images, so I decided to start building all my images as multiarchitecture images.\nThis turned out to be a lot easier than I was expecting - recent stable builds (2.0.4.0 (33772) or higher) of Docker Desktop can build for other architectures by running virtual machines in QEMU, so I can do the whole build on my MacBook Pro instead of baking each architecture separately and stitching them together with a manifest file.\nInstall the latest Docker Desktop for macOS Enable experimental mode either by setting DOCKER_CLI_EXPERIMENTAL=enabled in your environment or by adding \u0026quot;experimental\u0026quot; : \u0026quot;enabled\u0026quot; to ~/.docker/config.json Do docker buildx ls to see the current builders docker buildx create --name multiarch docker buildx use multiarch docker buildx inspect --bootstrap Now you\u0026rsquo;re ready to build. Go into one of your docker projects, then do docker buildx --platform linux/amd64,linux/arm64,linux/arm/v7 -t username/demo --push .\nIf you\u0026rsquo;re not ready to push your image to docker hub, do --load instead of --push to have it build the image and copy it out of the buildx system and into your local docker.\nEdit - documented enabling Docker\u0026rsquo;s experimental mode ","permalink":"https://unixorn.github.io/post/multi-architecture-images/","summary":"\u003cp\u003eI\u0026rsquo;ve got a mix of architectures in my basement cluster - some Odroid HC2s that are \u003ccode\u003earm7\u003c/code\u003e, some Raspberry Pi 4s that are \u003ccode\u003earm64\u003c/code\u003e, and am soon going to add an Intel node as well. It\u0026rsquo;s more hassle than it\u0026rsquo;s worth to have to specify different images for the different architectures. I already build my own copies of images, so I decided to start building all my images as multiarchitecture images.\u003c/p\u003e","title":"Building Multi Architecture Docker Images with buildx"},{"content":"I wanted a machine with more memory to be the master node for my ARM k3s cluster. I had an Odroid N2 with 4GB of RAM, sitting around, so here\u0026rsquo;s the log of getting it installed and running.\nParts ODROID N2 MMC card with preloaded Ubuntu Installation Base Login as root. Default password is odroid\nChange the root password! passwd Disable XWindow - systemctl set-default multi-user.target Purge XWindow packages - apt purge 'x11-*' \u0026amp;\u0026amp; apt autoremove Now that we\u0026rsquo;ve gotten rid of XWindow, get up to date - apt update \u0026amp;\u0026amp; apt upgrade Install some useful tools apt install -y zsh git htop iotop vim libssl-dev software-properties-common python3-pip Disable Ubuntu motd spam sed -i 's/^ENABLED=.*/ENABLED=0/' /etc/default/motd-news Fix locale complaints during login locale-gen en_US.UTF-8 Docker Add docker repository key - curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -\nAdd the repo:\nsudo add-apt-repository \\ \u0026#34;deb [arch=arm64] https://download.docker.com/linux/ubuntu \\ $(lsb_release -cs) \\ stable\u0026#34; Install docker community edition apt update \u0026amp;\u0026amp; apt install docker-ce docker-ce-cli containerd.io\n#!/usr/bin/env bash # # Put the setup all in one script. Run as root with sudo. # Disable XWindow login systemctl set-default multi-user.target # This is going to sit on a shelf in my rack, so purge xwindow apt remove xorg apt purge \u0026#39;x11-*\u0026#39; apt autoremove # Add docker apt repository add-apt-repository \\ \u0026#34;deb [arch=arm64] https://download.docker.com/linux/ubuntu \\ $(lsb_release -cs) \\ stable\u0026#34; # Now that we\u0026#39;ve purged xwindow, upgrade what\u0026#39;s left apt-get update \u0026amp;\u0026amp; apt-get upgrade apt-get install -y \\ git \\ htop \\ iotop \\ libssl-dev \\ python3-pip \\ vim vim-doc vim-scripts \\ zsh # Install docker community edition apt update \u0026amp;\u0026amp; apt install docker-ce docker-ce-cli containerd.io # Finally do some cleanups # Fix locale whining locale-gen en_US.UTF-8 # Disable Ubuntu motd spam sed -i \u0026#39;s/^ENABLED=.*/ENABLED=0/\u0026#39; /etc/default/motd-news ","permalink":"https://unixorn.github.io/post/setting-up-an-n2/","summary":"\u003cp\u003eI wanted a machine with more memory to be the master node for my ARM k3s cluster. I had an Odroid N2 with 4GB of RAM, sitting around, so here\u0026rsquo;s the log of getting it installed and running.\u003c/p\u003e","title":"Setting up an ODROID N2"},{"content":"Do not use duplicacy! TL;DR: duplicacy is unusable if you\u0026rsquo;re serious about backing up your data. Use restic instead!\nI wanted to ensure any data I put into my ARM k3s cluster is backed up to prevent data loss.\nI no longer recommend duplicacy. Instead, read my article on restic backups on TrueNas instead.\n2025 edit: This post is only here for historical reasons. Do NOT use duplicacy. It does not report backup errors in its exit code. You will have to parse its logs yourself and hope your regex didn\u0026rsquo;t miss an error condition. And every update, you\u0026rsquo;ll have to check and make sure it hasn\u0026rsquo;t changed anything that your regex was catching. Backups are supposed to be something you set up once and ignore other than to do periodic restore tests, and duplicacy fails that simple criteria.\nBackup Contenders I took a look at CrashPlan, restic and Duplicacy.\nCrashPlan - Even though they have a decent linux client, I eliminated Crashplan because:\nThey\u0026rsquo;ve already abandoned the home market. I currently use their CrashPlan for Small Business account for my Mac. I suspect they\u0026rsquo;ll also abandon this market because accounts with a small number of licenses also aren\u0026rsquo;t worth their time. They bill per-machine instead of by the amount of storage used. restic - Open source, which is great, but I ended up eliminating them because their deduplication wasn\u0026rsquo;t as strong as Duplicacy. It also seemed a little awkward to prune snapshots when I experimented with it.\nDuplicacy - I chose Duplicacy because it:\nSupports cross-source deduplication Works well with B2 Runs on Linux, Windows and Mac Allows multiple source directories to be backed up simultaneously to the same B2 bucket. Continues backing up where it leaves off after being interrupted and restarted. This eliminates having to completely restart the backup and re-upload everything. It didn\u0026rsquo;t hurt that I know several people using it with large amounts of data who are happy with it.\nOn to the Backups I made a docker image, thoth-duplicacy, which installs duplicacy and duplicacy-util on top of debian buster-slim, along with some helper scripts to make using it more convenient.\nThe image is published on docker hub, with both an Intel and and ARM7 version - the most current builds are tagged unixorn/thoth-duplicacy:armv7l and unixorn/thoth-duplicacy:x86_64.\nUsage For simplicity, I\u0026rsquo;m running my backups as kubernetes cron jobs. This allows me to easily run backups of multiple directory trees at once, and the kubernetes scheduler will automagically spread them around the cluster to the least loaded nodes.\nPre-requisites Create a Kubernetes Namespace I like my cluster neat and organized, with different services in their own namespaces, so I created a backups namespace by running kubectl create namespace backup.\nSet up a B2 Storage Bucket for Duplicacy Create a B2 Bucket and App Key Create a bucket in B2. Only use this bucket for duplicacy backups.. Do this first so that when you create the app key, using the dropdown menu you can easily restrict its access to only this bucket. Create an app key in B2 that you only use with Duplicacy. Definitely do not use the root account\u0026rsquo;s credentials. When you create it, specify that it\u0026rsquo;s only allowed to use your backups bucket. Make sure to copy the app key information when you create it, it will only be displayed once. Now you\u0026rsquo;re ready to initialize the bucket for the first directory you want to back up. The easiest way to do this is by running duplicacy inside the thoth-duplicacy container with docker-compose.\nSet up thoth-duplicacy container git clone git@github.com:unixorn/thoth-duplicacy.git BACKUP_LOCATION=/that/first/directory docker-compose run thoth-duplicacy bash cd /data mkdir -p .duplicacy Initialize the B2 Bucket. duplicacy init -encrypt -storage-name b2 STORAGEID b2://yourbucket. STORAGEID cannot have spaces or any special characters besides - and _. duplicacy will prompt you for the B2 app ID, app key, and the encryption password for your backups. Store the password in your secure password manager - without it, you can\u0026rsquo;t restore any of your data. Annoyingly you have to also set the password, B2 id and key again after initializing the bucket so that backups won\u0026rsquo;t prompt you for them. Set the B2 ID - duplicacy set -storage b2://net-unixorn-blog-test -key b2_id -value YOUR_APP_ID Set the B2 key - duplicacy set -storage b2://net-unixorn-blog-test -key b2_key -value YOUR_APP_KEY Set the password - duplicacy set -storage b2://net-unixorn-blog-test -key password -value YOURPASSWORD You can now run backup-cronjob and watch the first backup grind.\nAfter I configured duplicacy for the first time, it was much less hassle to copy the .duplicacy/preferences json file to each new directory tree. I wanted to back up the .duplicacy directory and change the id key to a new unique one — don\u0026rsquo;t put spaces or any special characters in the id other than _ and -. You don\u0026rsquo;t have to change the storage key, and actually shouldn\u0026rsquo;t - sharing the same storage bucket is what allows duplicacy to deduplicate your files across multiple source directories, which keeps your storage bill down.\nHere\u0026rsquo;s an example preferences file -\n[ { \u0026#34;name\u0026#34;: \u0026#34;b2\u0026#34;, \u0026#34;id\u0026#34;: \u0026#34;UNIQUE_ID_FOR_YOUR_DIRECTORY\u0026#34;, \u0026#34;repository\u0026#34;: \u0026#34;\u0026#34;, \u0026#34;storage\u0026#34;: \u0026#34;b2://your-backups-bucket\u0026#34;, \u0026#34;encrypted\u0026#34;: true, \u0026#34;no_backup\u0026#34;: false, \u0026#34;no_restore\u0026#34;: false, \u0026#34;no_save_password\u0026#34;: false, \u0026#34;nobackup_file\u0026#34;: \u0026#34;\u0026#34;, \u0026#34;keys\u0026#34;: { \u0026#34;b2_id\u0026#34;: \u0026#34;ROLE_ACCOUNT_B2_ID\u0026#34;, \u0026#34;b2_key\u0026#34;: \u0026#34;ROLE_ACCOUNT_B2_KEY\u0026#34;, \u0026#34;password\u0026#34;: \u0026#34;SUPER_SECRET_ENCRYPTION_PASSWORD_FOR_YOUR_DATA\u0026#34; } } ] Backing up a Directory Tree Here\u0026rsquo;s a sample job that backs up one of my directory trees. I\u0026rsquo;m using the backups namespace that I created earlier to keep things tidy - if you want to use the default namespace instead, delete the namespace entry in the metadata section.\nHere\u0026rsquo;s some things you\u0026rsquo;ll need to customize if you base a job on this example:\nChange the namespace entry in the metadata section to match whichever namespace you decided to use. I run this on Odroid HC2s and Raspberry Pis, which both use ARM CPUs. If you\u0026rsquo;re using x86, change the image entry to unixorn/thoth-duplicacy:x86_64 in the template spec section. I work from home, so I want to restrict the number of upload threads so that running backups don\u0026rsquo;t burn all my upload bandwidth. Change DUPLICACY_BACKUP_THEAD_COUNT in the env section if you want more simultaneous threads. The odroids only have 8 cores, but I had no issues running 12 threads other than gobbling up upstream bandwith. The B2_STORAGE_NAME environment variable is used by the backup-cronjob script to determine where to write the backup, so alter the value according to your setup. I\u0026rsquo;m backing up a moosefs distributed file system. I had already tagged all my chunk servers with kubectl label node NODENAME odroid=true and I use a nodeSelector stanza in the backup cron jobs to restrict the backup to only run on one of the chunk servers where the data resides. The moosefs data is distributed across all the chunk servers and each chunk server in the cluster currently contains 33% of the files, so running the backup on a chunk server maximizes the amount of data that can be local reads and don\u0026rsquo;t have to go across the network. Update or delete the nodeSelector clause to work with your environment. Once you\u0026rsquo;ve updated the file, install the cronjob with kubectl apply -f backup-example-directory-tree.yml.\nYou can download this from backup-example-directory-tree.yml instead of hassling with copy and paste.\napiVersion: batch/v1beta1 kind: CronJob metadata: name: backup-exampledir namespace: backups spec: schedule: \u0026#34;35 */2 * * *\u0026#34; jobTemplate: spec: # Ensure only one copy of the backup is running, even if it takes # so long to run that it is still running when the next cron slot # occurs concurrencyPolicy: Forbid template: spec: containers: - name: backup-exampledir # I\u0026#39;m running this on the odroids in my cluster, so I\u0026#39;m specifying # the ARM7 build image: unixorn/thoth-duplicacy:arm7l # Use the x86_64 tag if you\u0026#39;re on Intel # image: unixorn/thoth-duplicacy:x86_64 args: - /bin/sh - -c - /usr/local/bin/backup-cronjob volumeMounts: - name: data-volume mountPath: /data/ env: # I want to restrict the number of threads used for uploads # so that duplicacy doesn\u0026#39;t consume all my upload bandwidth. # I don\u0026#39;t care if it makes my backups slower. - name: DUPLICACY_BACKUP_THEAD_COUNT value: \u0026#34;3\u0026#34; # backup-cronjob needs to know what defined storage to back up # files to. - name: B2_STORAGE_NAME value: \u0026#34;b2\u0026#34; restartPolicy: OnFailure # Keep it running on a chunkserver so that at least part of the # I/O is to local disk instead of across the network. Remove if # you don\u0026#39;t care what node backups happen on. nodeSelector: odroid: \u0026#34;true\u0026#34; volumes: - name: data-volume hostPath: # This will be remapped to /data which is where duplicacy # expects to find the data it is backing up, and the .duplicacy # directory with its settings. path: /dfs/volumes/exampledir # this field is optional type: Directory Pruning Snapshots I don\u0026rsquo;t want to keep snapshots forever, so I made a kubernetes cron job to clean them up.\nBriefly, you can specify multiple -keep X:Y arguments, where you keep one snapshot for every X days after the snapshots are older than Y days.\nFor example, in the purge-stale-duplicacy-snapshots.yml job below, I have it set with -keep 0:365 -keep 30:90 -keep 7:30 -keep 1:2, which means keep no snapshots more than 365 days old, for snapshots older than 90 days keep one every 30 days, after fourteen days keep one every seven days, and after two days keep one every day.\nWarning: Notice that I specified the expiration rules starting with the longest (365 days) and continuing in descending age order - a minor annoyance with duplicacy is that you have to specify the -keep clauses starting with the longest age threshold and then specify the rules for shorter thresholds, or duplicacy will ignore the rules specified out of order, which could lead to more snapshots being purged than you would expect. Run with -dry-run first so you can see whether all your rules are being applied as you expect.\nBefore using this job definition, at a minimum you should set the namespace for the cron job, update the image if you\u0026rsquo;re running on x86, and update the -keep X:Y statements to correspond with your snapshot retention policy.\nOnce you\u0026rsquo;ve updated the configuration, install the cron job with kubectl apply -f purge-stale-duplicacy-snapshots.yml\nYou can download this from purge-stale-duplicacy-snapshots.yml instead of hassling with copy and paste.\napiVersion: batch/v1beta1 kind: CronJob metadata: name: purge-stale-duplicacy-snapshots namespace: backups spec: schedule: \u0026#34;48 */3 * * *\u0026#34; jobTemplate: spec: concurrencyPolicy: Forbid template: spec: containers: - name: purge-stale-duplicacy-snapshots # I\u0026#39;m running this on the odroids in my cluster, so I\u0026#39;m specifying # the ARM7 build image: unixorn/thoth-duplicacy:arm7l # Use the x86_64 tag if you\u0026#39;re on Intel # image: unixorn/thoth-duplicacy:x86_64 # Make sure we run inside /data so that duplicacy can find # the configuration directory. workingDir: /data # Remember that the -keep arguments must be listed from longest # time frame to shortest, otherwise the disordered ones will be # ignored, which could mean deleting snapshots you want to keep. # # I\u0026#39;m specifying to keep no snapshots more than 365 days old, keep # a single snapshot every 30 days for snapshots older than 90 days, # a single snapshot a week for snapshots older than 30 days, and # finally keep only a single snapshot per day for snapshots # older than 2 days. # # Also note that the duplicacy verb (prune) has to come before # any of the settings command line options. args: - duplicacy - prune - -storage - b2 - -all - -keep 0:365 - -keep 30:90 - -keep 7:14 - -keep 1:2 - -exhaustive volumeMounts: - name: data-volume mountPath: /data/ env: - name: DUPLICACY_BACKUP_THEAD_COUNT value: \u0026#34;3\u0026#34; - name: B2_STORAGE_NAME value: \u0026#34;b2\u0026#34; restartPolicy: OnFailure volumes: - name: data-volume hostPath: # This will be remapped to /data which is where duplicacy # expects to find the data it is backing up, and the .duplicacy # directory with its settings. path: /dfs/volumes/exampledir # this field is optional type: Directory Restoring Files Backups are useless if you can\u0026rsquo;t restore.\nTo restore, use docker-compose and the thoth-duplicacy repository. I only did my test restores with the command line, I haven\u0026rsquo;t bothered experimenting with the GUI from https://duplicacy.com.\nUse git clone git@github.com:unixorn/thoth-duplicacy.git if you didn\u0026rsquo;t keep the checkout when you initialized your B2 bucket Make a directory to restore to, and a .duplicacy subdirectory for the configuration with mkdir -p /path/to/restore/.duplicacy. While you can restore in place over the live directory, I\u0026rsquo;m a bit too cautious to do that especially if I\u0026rsquo;m doing a restore after having already lost files. Copy the preferences file from the directory tree you want to restore to /path/to/restore/.duplicacy. Start a container with BACKUP_LOCATION=/path/to/restore docker-compose run thoth-duplicacy bash Now that you\u0026rsquo;re in a running thoth-duplicacy container with your restore directory mounted as /data, you can restore files. cd /data before running duplicacy commands so it can find its configuration.\nYou can look at the available snapshots with duplicacy list. It will list snapshot name, revision number, and the timestamp when each snapshot was created.\nOnce you know what snapshots are in the bucket, you can examine the files available in a specific snapshot with duplicacy list -files -r REVISION_NUMBER.\nNow you can restore files - if you want to restore just the foo directory, from revision 99, you\u0026rsquo;d run duplicacy restore -r 99 'foo/*'.\n","permalink":"https://unixorn.github.io/post/backing-up-the-cluster-with-duplicacy/","summary":"\u003ch2 id=\"do-not-use-duplicacy\"\u003eDo not use duplicacy!\u003c/h2\u003e\n\u003cp\u003eTL;DR: duplicacy is unusable if you\u0026rsquo;re serious about backing up your data. Use restic instead!\u003c/p\u003e\n\u003cp\u003eI wanted to ensure any data I put into my \u003ca href=\"https://unixorn.github.io/post/k3s-on-arm/\"\u003eARM k3s cluster\u003c/a\u003e is backed up to prevent data loss.\u003c/p\u003e\n\u003cp\u003eI no longer recommend duplicacy. Instead, read my article on \u003ca href=\"https://unixorn.github.io/post/restic-backups-on-truenas/\"\u003erestic backups on TrueNas\u003c/a\u003e instead.\u003c/p\u003e\n\u003cp\u003e2025 edit: This post is only here for historical reasons. \u003cstrong\u003e\u003cem\u003eDo NOT use duplicacy\u003c/em\u003e\u003c/strong\u003e. It does not report backup errors in its exit code. You will have to parse its logs yourself and hope your regex didn\u0026rsquo;t miss an error condition. And \u003cem\u003eevery\u003c/em\u003e update, you\u0026rsquo;ll have to check and make sure it hasn\u0026rsquo;t changed anything that your regex \u003cem\u003ewas\u003c/em\u003e catching. Backups are supposed to be something you set up once and ignore other than to do periodic restore tests, and duplicacy fails that simple criteria.\u003c/p\u003e","title":"Backing Up the Cluster with Duplicacy"},{"content":"Yesterday I had to grow a live filesystem on a server in EC2, without downtime. I do this just infrequently enough to not quite remember all the details without poking around the internet, so I\u0026rsquo;m documenting it all in one place.\nGrow the volume Log into the EC2 console and find your instance. In the description tab, look at the block devices (bottom right as of August 2019) and find the volume you need to grow and get its volume ID. Find that volume in the EBS volumes list. Now is a good time to name it something useful like \u0026ldquo;InstanceName /data01\u0026rdquo; if you haven\u0026rsquo;t already named it. Click Modify Volume, then give it a new size. It may take a minute or two to finish growing the volume, you\u0026rsquo;ll see a percentage displayed. Resize the filesystem Log into the instance and start a tmux or screen session to do all the work in. Getting disconnected in the middle of resizing the filesystem would be bad. Use lsblk to confirm that the EBS block device has increased to the size you expect. If you have partitioned your drive, do sudo growpart /dev/xyz1 0 to grow the partition. Check /etc/fstab to see what format the filesystem is. If you\u0026rsquo;re using xfs, sudo xfs_growfs /dev/DEVICE. If you\u0026rsquo;re using ext2, ext3 or ext4, do sudo resize2fs /dev/DEVICE. If you\u0026rsquo;re using ext2 or ext3, seriously consider replacing this filesystem with an ext4 one during your next downtime window. Wait. Depending on how much larger the EBS volume has become and the instance type, it can take several minutes for the filesystem to finish growing. Confirm the new size with df ","permalink":"https://unixorn.github.io/post/growing-ebs-volumes-in-place/","summary":"\u003cp\u003eYesterday I had to grow a live filesystem on a server in EC2, without downtime. I do this just infrequently enough to not \u003cem\u003equite\u003c/em\u003e remember all the details without poking around the internet, so I\u0026rsquo;m documenting it all in one place.\u003c/p\u003e","title":"Growing EBS Volumes in Place"},{"content":"Why k3s and not stick with k8s? I wanted to experiment with k3s. They package everything you need in a single binary, don\u0026rsquo;t package in deprecated parts of k8s, and it works on Intel, ARMv7 and ARM64. It seemed like it\u0026rsquo;d be a less painful way to runn Kubernetes on my ARM cluster.\nPrerequisites You must have set up DNS entries for the nodes you want to cluster, or update /etc/hosts on all the nodes so they can find each other.\nInstalling k3s I chose to install k3s without the built-in traefik install so I could install that with a custom configuration. I also chose to use docker instead of the baked-in containerd so that I could also run containers outside k3s on my worker nodes without wasting RAM.\nInstalling the master curl -sfL https://get.k3s.io \u0026gt; install-k3s.sh \u0026amp;\u0026amp; chmod 755 ./install-k3s.sh sudo ./install-k3s.sh --no-deploy traefik --docker sudo chgrp docker /etc/rancher/k3s/k3s.yaml sudo chmod g+r /etc/rancher/k3s/k3s.yaml I also updated /etc/systemd/system/k3s.service to add\nAfter=network-online.target cluster-mfsmount.service docker.service\nbecause I don\u0026rsquo;t want k3s to attempt to start until after the docker service has started and the cluster\u0026rsquo;s moosefs distributed filesystem is mounted.\nOnce all that is done, copy the node token from /var/lib/rancher/k3s/server/node-token to each of the worker nodes.\nInstalling the workers Copy /var/lib/rancher/k3s/server/node-token from the server to your worker.\nRun\n./install-k3s.sh --agent --server https://master-server:6443 --kubelet-arg=\u0026#34;address=0.0.0.0\u0026#34; --token \u0026#34;$(cat node-token)\u0026#34; --docker Remove the --docker if you want to use the containerd bundled into k3s - I wanted to be able to also run apps in docker on my nodes and didn\u0026rsquo;t want it using extra RAM for another containerd.\nIf you\u0026rsquo;re using a distributed filesystem like I am, add\nAfter=network-online.target cluster-mfsmount.service docker.service\nto /etc/systemd/system/k3s-agent.service, and\nAfter=network-online.target cluster-mfsmount.service\nto /lib/systemd/system/docker.service to keep docker from starting until after the distributed filesystem is mounted.\nSet up Networking MetallB I wanted to be able to use LoadBalancerIP entries in my cluster services to make using Traefik easier.\nInstalling MetallB On my master node, I ran\nkubectl apply -f https://raw.githubusercontent.com/danderson/metallb/master/manifests/metallb.yaml Configuring MetallB I used the following configuration for metallb (in metallb-conf.yaml)\napiVersion: v1 kind: ConfigMap metadata: namespace: metallb-system name: config data: config: | address-pools: - name: default protocol: layer2 addresses: - 10.0.1.16/28 And applied it with kubectl apply -f metallb-conf.yaml.\nThis allows me to use 10.0.1.17 through 10.0.1.30 as LoadBalancerIP entries in my k8s service configurations. 14 entries should be more than enough for my immediate needs.\nYou will want to change the addresses entry to conform to your own network.\nTraefik Installed traefik with my own configuration, which I have posted on github:\nHere are the configuration files I used - you\u0026rsquo;ll need to tweak them for your own network.\ntraefik-rbac.yaml --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller rules: - apiGroups: - \u0026#34;\u0026#34; resources: - services - endpoints - secrets verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses verbs: - get - list - watch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: traefik-ingress-controller subjects: - kind: ServiceAccount name: traefik-ingress-controller namespace: kube-system traefik-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: traefik-conf namespace: kube-system data: traefik.toml: | defaultEntryPoints = [\u0026#34;http\u0026#34;,\u0026#34;https\u0026#34;] debug = false logLevel = \u0026#34;INFO\u0026#34; # Do not verify backend certificates (use https backends) InsecureSkipVerify = true [entryPoints] [entryPoints.http] address = \u0026#34;:80\u0026#34; compress = true [entryPoints.https] address = \u0026#34;:443\u0026#34; [entryPoints.https.tls] #Config to redirect http to https #[entryPoints] # [entryPoints.http] # address = \u0026#34;:80\u0026#34; # compress = true # [entryPoints.http.redirect] # entryPoint = \u0026#34;https\u0026#34; # [entryPoints.https] # address = \u0026#34;:443\u0026#34; # [entryPoints.https.tls] [web] address = \u0026#34;:8080\u0026#34; [kubernetes] [metrics] [metrics.prometheus] buckets=[0.1,0.3,1.2,5.0] entryPoint = \u0026#34;traefik\u0026#34; [ping] entryPoint = \u0026#34;http\u0026#34; traefik-deployment.yaml --- apiVersion: v1 kind: ServiceAccount metadata: name: traefik-ingress-controller namespace: kube-system --- kind: Deployment apiVersion: extensions/v1beta1 metadata: name: traefik-ingress-controller namespace: kube-system labels: k8s-app: traefik-ingress-lb spec: replicas: 2 selector: matchLabels: k8s-app: traefik-ingress-lb template: metadata: labels: k8s-app: traefik-ingress-lb name: traefik-ingress-lb spec: serviceAccountName: traefik-ingress-controller terminationGracePeriodSeconds: 60 containers: - image: traefik:1.7.9 name: traefik-ingress-lb volumeMounts: - mountPath: /config name: config ports: - name: http containerPort: 80 - name: https containerPort: 443 - name: admin containerPort: 8080 args: - --api - --kubernetes - --configfile=/config/traefik.toml livenessProbe: httpGet: path: /ping port: 80 initialDelaySeconds: 3 periodSeconds: 3 timeoutSeconds: 1 affinity: podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 100 podAffinityTerm: labelSelector: matchExpressions: - key: role operator: In values: - data topologyKey: kubernetes.io/hostname volumes: - name: config configMap: name: traefik-conf traefik-service.yaml --- kind: Service apiVersion: v1 metadata: name: traefik-ingress-service namespace: kube-system labels: k8s-app: traefik-ingress-lb spec: selector: k8s-app: traefik-ingress-lb externalTrafficPolicy: Local ports: - protocol: TCP port: 80 name: web - protocol: TCP port: 443 name: https - protocol: TCP port: 8080 name: admin type: LoadBalancer loadBalancerIP: 10.0.1.20 --- apiVersion: extensions/v1beta1 kind: Ingress metadata: name: traefik-ingress-lb namespace: kube-system spec: rules: - host: traefik.example.com http: paths: - path: / backend: serviceName: traefik-ingress-service servicePort: admin You\u0026rsquo;ll want to change the loadBalancerIP entry and the host entry in the spec section to match your network and DNS configurations.\nfor traefik_yaml in traefik-rbac.yaml traefik-configmap.yaml traefik-deployment.yaml traefik-service.yaml do kubectl apply -f $traefik_yaml done Updates Updated URL for metallb install yaml file to use latest instead of pinning a specific version. k3s is actively being updated and the old version no longer worked. ","permalink":"https://unixorn.github.io/post/k3s-on-arm/","summary":"\u003ch2 id=\"why-k3s-and-not-stick-with-k8s\"\u003eWhy k3s and not stick with k8s?\u003c/h2\u003e\n\u003cp\u003eI wanted to experiment with \u003ca href=\"https://k3s.io\"\u003ek3s\u003c/a\u003e. They package everything you need in a single binary, don\u0026rsquo;t package in deprecated parts of k8s, and it works on Intel, ARMv7 and ARM64. It seemed like it\u0026rsquo;d be a less painful way to runn Kubernetes on my ARM cluster.\u003c/p\u003e","title":"Trying K3s on ARM, Part 1"},{"content":"Get your money from the Equifax settlement regarding the 2017 security breach.\nEligibility check site Settlement Site Your settlement options are to take a few months of their shitty credit monitoring that costs them nothing to provide, or to collect $125 you can spend anywhere on things that are actually of value to you.\nIn addition to the $125, if you spent up to 10 hours doing things like resetting all your passwords, setting up credit freezes, checking your bank accounts, credit cards, stock trading sites \u0026amp; etc for fraudulent transactions, you can get some extra money to compensate you for all that wasted time without having to submit paperwork to them about it. If you wasted more than 10 hours dealing with the fallout from their incompetence, you\u0026rsquo;ll have to submit paperwork for the extra hours.\nWhat I really want is an option of \u0026ldquo;never collect any data about me again you incompetent assholes, and delete all the data you have on me now\u0026rdquo;, but I had to settle for cash.\n","permalink":"https://unixorn.github.io/post/equifax-2017-breach-settlement/","summary":"\u003cp\u003eGet your money from the Equifax settlement regarding the 2017 security breach.\u003c/p\u003e","title":"Equifax 2017 Breach Settlement Information"},{"content":"One of the reasons I set up my cluster was that I\u0026rsquo;m running out of space on my NAS. I don\u0026rsquo;t want to buy a whole new chassis, and while I could have put individual file shares on each cluster node, that would be both inconvenient and not provide any data redundancy without a lot of brittle home-rolled hacks to rsync data from node to node. And since distributed file systems are a thing, I\u0026rsquo;d rather not resort to hacks.\nAlternatives Considered To make a long story short, I considered ceph, glusterfs, lizardfs \u0026amp; moosefs.\nceph Ceph\u0026rsquo;s published specs show that it basically won\u0026rsquo;t fit in my ODROID nodes. They say metadata servers need 1GB, OSDs need another 500MB, and the HC2s in my cluster only have 2GB of onboard RAM, and I\u0026rsquo;m running k8s on them too, so Ceph is out.\nGlusterFS GlusterFS looked nice at first, but it\u0026rsquo;s both less flexible (you can\u0026rsquo;t add single drives to the storage cluster) and less performant (by a factor of 2 compared to MooseFS) on my HC2 hardware.\nLizardFS LizardFS is a fork of MooseFS, and when I was trying to install it in my cluster I had a hard time finding debs for it. When I was looking for them online, I found a lot of complaints about performance issues on ARM, so that eliminated it from consideration.\nMooseFS There were a lot of things to like about MooseFS for my use case on my hardware.\nThere were prebuilt ARM debs on their site, in a handy PPA. It was twice as fast on my hardware as GlusterFS. I can add individual drive bricks to the cluster, even though my storage policies (more on them below) all require multiple replicas of data in the filesystem they\u0026rsquo;re applied to. The memory requirements are small enough that I can run it on the same nodes that I\u0026rsquo;m running kubernetes on. It dynamically balances disk usage across the bricks - when I added a third brickserver to my cluster, moosefs shuffled replica chunks over to it until all three servers had a roughly equal usage percentage. It allows custom storage policies: You can label storage bricks (say SSD as label A, spinning disks as label B) and use the labels in policy definitions. It\u0026rsquo;s flexible - you can create policies with different replication requirements and assign those on a per-directory or even per-file basis. By referring to brick labels, you can do things like create a policy that requires that at file creation, one replica be written to SSD and one to spinning disk, and then after that initial write is complete (so it can report back to the writing process that the write is done), that it then try to make sure there is a third copy so that there are two copies on spinning disks and one on SSD. You can make policies that change replication policies after user-specified amounts of time - so maybe your policy is that new files get one copy on SSD and 2 on spinning disk, but after 30 days, switch to one copy on a regular spinning disk and two on bigger slower drives. Installing You\u0026rsquo;re going to need a moosefs master, chunkservers for each machine that hosts drives, and should run a metadata backup server. You may also want to run the cgi to visualize the cluster status.\nPre-Requisites Static IPs for the cluster nodes, preferably with DNS entries. Setting this up is out of scope for this post. Decide which nodes will just be chunkservers, which will be the master, and optionally which are going to be metadata servers and cgi servers. You can run the master, metadata and cgi servers on machines that are also chunkservers. All servers Add the MooseFS PPA Add a file, /etc/apt/sources.list.d/moosefs.list, with the following contents deb http://ppa.moosefs.com/moosefs-3/apt/raspbian/jessie jessie main Run wget -O - https://ppa.moosefs.com/moosefs.key | sudo apt-key add - to add the moosefs PPA key\nRun apt-get update\nMaster Server Install the master server software on one of your nodes with apt install moosefs-master. Do this first, the chunkservers will need to communicate with it.\nOptionally install the cgi server with apt install moosefs-cgiserv\nConfigure /etc/mfs/mfsmaster.cfg and /etc/mfs/mfsexports.cfg. Start by copying /etc/mfs/mfsmaster.cfg.sample and /etc/mfs/mfsexports.cfg.sample.\nSet the master software to start on boot with systemctl enable moosefs-master. If you installed the cgi server, enable it too with systemctl enable moosefs-cgiserv.\nStart the master and cgi server with systemctl start moosefs-master \u0026amp;\u0026amp; systemctl start moosefs-cgiserv.\nChunkservers For each of your chunkservers, take the following steps:\nInstall moosefs software Install the software with apt install moosefs-chunkserver.\nConfigure which drives to use for storage Make a directory to store the moosefs data. On my HC2 instances, I mount the data drives on /mnt/sata, and keep the raw mfs data in /mnt/sata/moosefs\nConfigure /etc/mfs/mfshdd.cfg. There\u0026rsquo;s an example in /etc/mfs/mfshdd.cfg.sample - add one line per directory to be shared.\nIn my case, I want to keep 50 gigs free on the drive, so my entry in mfshdd.cfg is\n/mnt/sata/moosefs -50GB Configure the chunkserver options Copy /etc/mfs/mfschunkserver.cfg.sample to /etc/mfs/mfschunkserver.cfg and edit it to meet your needs - at a minimum, you\u0026rsquo;ll need to set MASTER_HOST = yourmaster.example.com\nEnable and start the chunkserver Set up the chunkserver to start at boot, and start it now -\nsystemctl enable moosefs-chunkserver \u0026amp;\u0026amp; systemctl start moosefs-chunkserver Mounting the filesystem Now that the chunkservers are talking to the master, you can set up automounting it on your nodes.\nFirst, install the client software - sudo apt install -y moosefs-client\nSecond, make a mountpoint. On my nodes, I\u0026rsquo;m using /data/squirrel, so sudo mkdir -p /data/squirrel\nFinally, create a systemd unit file so that the filesystem mounts every boot. I want to be able to use hostPath directives in my kubernetes deployments, so I want it to start before docker and kubelet. Make a file, /etc/systemd/system/yourcluster-mfsmount.service with the following content (replace /mountpoint with whatever mountpoint you\u0026rsquo;re using):\n# Original source: https://sourceforge.net/p/moosefs/mailman/message/29522468/ [Unit] Description=MooseFS mounts After=syslog.target network.target ypbind.service moosefs-chunkserver.service moosefs-master.service Before=docker.service kubelet.service [Service] Type=forking TimeoutSec=600 ExecStart=/usr/bin/mfsmount /mountpoint -H YOUR_MASTER_SERVER ExecStop=/usr/bin/umount /mountpoint [Install] WantedBy=multi-user.target Enable it so it starts every boot:\nsystemctl enable yourcluster-mfsmount \u0026amp;\u0026amp; systemctl start yourcluster-mfsmount ","permalink":"https://unixorn.github.io/post/adding-a-distributed-filesystem-to-cluster/","summary":"\u003cp\u003eOne of the reasons I set up my cluster was that I\u0026rsquo;m running out of space on my NAS. I don\u0026rsquo;t want to buy a whole new chassis, and while I could have put individual file shares on each cluster node, that would be both inconvenient and not provide any data redundancy without a lot of brittle home-rolled hacks to \u003ccode\u003ersync\u003c/code\u003e data from node to node. And since distributed file systems are a thing, I\u0026rsquo;d rather not resort to hacks.\u003c/p\u003e","title":"Adding a Distributed Filesystem to the Cluster"},{"content":"I had a bunch of cruft in start scripts that would determine the IP of my laptop and set it as an environment variable to pass to docker-compose and it turns out that with Docker for Mac you don\u0026rsquo;t have to do all that work - just use docker.for.mac.localhost as a hostname.\n","permalink":"https://unixorn.github.io/post/til-about-docker-mac/","summary":"\u003cp\u003eI had a bunch of cruft in start scripts that would determine the IP of my laptop and set it as an environment variable to pass to \u003ccode\u003edocker-compose\u003c/code\u003e and it turns out that with Docker for Mac you don\u0026rsquo;t have to do all that work - just use \u003ccode\u003edocker.for.mac.localhost\u003c/code\u003e as a hostname.\u003c/p\u003e","title":"Things I Learned About Docker Mac"},{"content":"I\u0026rsquo;m not sure if this is just macOS Mojave being flaky or some interaction with the security malware installed on my work laptop, but it refuses to connect to some captive portals.\nThe problem is that it refuses to render the captive portal login page, so you can\u0026rsquo;t agree to their TOS, and don\u0026rsquo;t get a DHCP address.\nThere\u0026rsquo;s a workaround, you can open Apple\u0026rsquo;s hotspot detection page with Safari directly by running\nopen -a \u0026quot;Safari\u0026quot; \u0026quot;http://captive.apple.com/hotspot-detect.html\u0026quot;\nin a terminal window.\nThat\u0026rsquo;s annoying to remember (and honestly, most of the time I don\u0026rsquo;t have Safari running) so I\u0026rsquo;ve added it to my tumult zsh plugin for macOS here.\n","permalink":"https://unixorn.github.io/post/fix-macos-captive-portal-issue/","summary":"\u003cp\u003eI\u0026rsquo;m not sure if this is just macOS Mojave being flaky or some interaction with the security malware installed on my work laptop, but it refuses to connect to some captive portals.\u003c/p\u003e","title":"Fix macOS Captive WIFI Portal Issue"},{"content":"I realized I forgot to include a parts list for the cluster in my ARM cluster post (all prices are as of March 3rd, 2019), so here we go.\nThree Odroid HC2 ARM SBCs @ $54.95 each. These have gigabit ethernet ports, a SATA-3 port for 3.5 inch or 2.5 inch HDD/SSD drives, Samsung Exynos5422 Cortex-A15 2Ghz and Cortex-A7 Octa core 32bit ARM CPUs, with four 2GHz cores and four 1.4GHz cores and a UHS-1 compatible microSD slot that supports up to 128GB/SDXC. Three 12V/2A Power supplies @ $5.95 each and three power cords for the Odroids (They\u0026rsquo;re sold separately) @ $1.95 Three microSD cards - I used some I had already - they\u0026rsquo;re cheap, so get at least 32GB though 16GB works too. One 8 port Ubiquiti UniFi Switch 8 60W (US-8-60W) @ $109.59, though really any gigabit switch will do. I didn\u0026rsquo;t bother with cases for mine, I wanted to maximize airflow since they don\u0026rsquo;t have onboard fans.\n","permalink":"https://unixorn.github.io/post/arm_cluster_parts_list/","summary":"\u003cp\u003eI realized I forgot to include a parts list for the cluster in my \u003ca href=\"https://unixorn.github.io/blog/in_the_beginning_there_was_bare_metal/\"\u003eARM cluster post\u003c/a\u003e (all prices are as of March 3rd, 2019), so here we go.\u003c/p\u003e","title":"Parts list for the ARM cluster"},{"content":"I recently decided to set up a Kubernetes cluster in my basement, partly because I\u0026rsquo;d never set a cluster up from scratch by myself, and partly because my existing NAS was beginning to run out of headroom.\nFor a variety of reasons, I decided to use ODROID HC2 boards. They\u0026rsquo;ve got gigabit ethernet, eight CPU cores, 2 GB RAM and a SATA-3 port for directly connecting a hard drive, which I wanted so I could use them as file server bricks. In a future post I will detail how I set up a distributed filesystem across the cluster.\nEDIT - Added a link to the parts list, and added instructions for finding the new machine on your network.\nSetting up an ODROID HC2 cluster These notes should also work on an Odroid HC1 or XU4.\nInstall Debian Stretch I used meveric\u0026rsquo;s debian-stretch ISO from https://oph.mdrjr.net/meveric/images/Stretch/.\nI used Etcher to burn the debian-stretch ISO to an microSD card.\nFlash your microSD card, plug it into the HC2 and attach a SATA drive to your HC2 if you\u0026rsquo;re going to use one, then connect it to your switch and power up. It will get an IP address with DHCP. Since they don\u0026rsquo;t have a video connector, you\u0026rsquo;ll have to scan your network to figure out what IP it got.\nFind the Odroid on your network nmap You can use nmap to find the Odroid machines. Assuming your network is 10.0.0.1-254, you can scan the network with nmap -sP -n 10.0.0.0/24 | grep -v Host. Look for systems that show Wibrain in their MAC Address line.\nAngry IP Scanner If you\u0026rsquo;re not comfortable with nmap, I recommend Angry IP Scanner to find the new machine on your network because you can configure (select Fetchers in the Angry IP Scanner Menu on macOS, then add MAC Vendor to the selected fetchers) it to show the MAC vendor of your ethernet card - the ODroids show will show up as WIBRAIN.\nLogin as root, password odroid.\nChange your root password! Don\u0026rsquo;t skip this just because you\u0026rsquo;re running this on an internal-only network. The default root password for the image is well known, so run passwd root to change it so you\u0026rsquo;re not vulnerable if you accidentally open up your WIFI.\nInstall your updates Install ISOs are inevitably out of date, but that\u0026rsquo;s ok, we\u0026rsquo;ll begin by updating all the installed packages.\napt-get update \u0026amp;\u0026amp; apt-get upgrade \u0026amp;\u0026amp; apt-get dist-upgrade\nInstall useful tooling Let\u0026rsquo;s also add some useful tools to the machine.\napt-get install -y dnsutils git htop lshw man net-tools rsync sudo\nNow we\u0026rsquo;re ready to install Docker and Kubernetes.\nInstall docker-ce Install the docker-ce pre-requisites apt-get install \\ apt-transport-https \\ ca-certificates \\ curl \\ gnupg2 \\ software-properties-common Partition \u0026amp; Format the drive Install pre-requisites Check what\u0026rsquo;s on your drive with lshw -C disk\nFor the sake of these examples, we\u0026rsquo;ll assume the SATA drive is /dev/sda\nFormat the drive If you didn\u0026rsquo;t add a SATA drive to your Odroid, you can skip this section.\nFirst partition it fdisk /dev/sda list all the existing partitions with the p command Remove any existing partitions with the d command Create a new partition with the n command Write the new partition table to disk with the w command Now format it mkfs.ext4 /dev/sda1 Configure the system to automatically mount the drive Get the UUID with blkid | grep /dev/sda. You\u0026rsquo;ll see something like /dev/sda1: UUID=\u0026quot;abcdabcd-abcd-11bb-9343-9089b93bbb72\u0026quot; TYPE=\u0026quot;ext4\u0026quot; PARTUUID=\u0026quot;13371337-abcd-1234-aa00-abcd1234abcd1234\u0026quot; Create a mount point to mount your filesystem. I picked /mnt/sata and created it with mkdir -p /mnt/sata Add an entry to /etc/fstab. Use your editor of choice to add a line UUID=\u0026quot;abcdabcd-abcd-11bb-9343-9089b93bbb72\u0026quot; /mnt/sata ext4 defaults 0 2. Use the UUID from step 1, not the example one here. You should now be able to mount the drive with mount /mnt/sata. If it succesfully mounts, it should show up after a reboot.\nForce a static IP for the node(s) Your best option is to assign your nodes static IPs on your DHCP server. This way you have one place to assign IPs, and if you need to reassign them you can change them all on the DHCP server and within a few minutes (if you\u0026rsquo;re in a rush, reboot your nodes) your nodes will switch to the new addreses.\nIf you can\u0026rsquo;t assign static addresses in your DHCP server on your router, it\u0026rsquo;s garbage and consider replaceing it. Until you do, you\u0026rsquo;ll have to hard code them on the nodes.\nFirst, back up the current network config with cp /etc/network/interfaces /etc/network/interfaces-original\nNow edit /etc/network/interfaces and put in:\n# Ethernet adapter 0 auto eth0 allow-hotplug eth0 #no-auto-down eth0 iface eth0 inet static address 192.168.1.100 netmask 255.255.255.0 gateway 192.168.1.1 dns-nameservers 8.8.8.8 8.8.4.4 # Or use your own by uncommenting below # dns-nameservers 192.168.1.1 Disable swap Kubernetes doesn\u0026rsquo;t like swap, so disable it with swapoff -a.\nInstall Docker and Kubernetes Docker Install the docker apt signing GPG key curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -\necho \u0026quot;deb [arch=armhf] https://download.docker.com/linux/debian $(lsb_release -cs) stable\u0026quot; \u0026gt; /etc/apt/sources.list.d/docker.list\nInstall pip \u0026amp; docker-compose apt-get update \u0026amp;\u0026amp; apt-get install -y python3-pip \u0026amp;\u0026amp; pip3 install setuptools docker-compose\nInstall docker apt-get install -y docker-ce --no-install-recommends\nConfirm Docker is working docker run hello-world\nYou should see something similar to this:\nroot@rodan:~# docker run hello-world Unable to find image \u0026#39;hello-world:latest\u0026#39; locally latest: Pulling from library/hello-world c1eda109e4da: Pull complete Digest: sha256:2557e3c07ed1e38f26e389462d03ed943586f744621577a99efb77324b0fe535 Status: Downloaded newer image for hello-world:latest Hello from Docker! This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the \u0026#34;hello-world\u0026#34; image from the Docker Hub. (arm32v7) 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal. To try something more ambitious, you can run an Ubuntu container with: $ docker run -it ubuntu bash Share images, automate workflows, and more with a free Docker ID: https://hub.docker.com/ For more examples and ideas, visit: https://docs.docker.com/get-started/ Install Kubernetes Install the k8s repository key curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -\nAdd the k8s apt repo cat \u0026lt;\u0026lt;EOF \u0026gt;/etc/apt/sources.list.d/kubernetes.list deb https://apt.kubernetes.io/ kubernetes-xenial main EOF Install kubernetes sudo apt-get update sudo apt-get install -y kubelet kubeadm kubectl sudo apt-mark hold kubelet kubeadm kubectl Uptodate at https://kubernetes.io/docs/setup/independent/\nThis is pretty tedious, is there an easier way? All this is tedious and prone to mistyping, especially if you\u0026rsquo;ve got multiple nodes to make into a cluster, so I\u0026rsquo;ve put everything except the network setup and disk formatting/mounting into a handy helper script, borg-odroid.\nYou can copy borg-odroid to a new machine after the first boot and it will bring the machine\u0026rsquo;s debian install up to date, then install docker \u0026amp; kubernetes and some other handy support tools.\nConfigure your Kubernetes Cluster First - did you configure your kubernetes nodes to use static addresses? You will have issues if you didn\u0026rsquo;t.\nInitialize the cluster on your master node By default, Flannel requires we use 10.244.0.0/16 for our CIDR when initializing the cluster, because the flannel configuration we\u0026rsquo;re going to install later expects that CIDR - sure, we could change all the references to it, but that is just going to give us chances to break it.\nkubeadm init --pod-network-cidr=10.244.0.0/16\nNote: It is normal to see your master node as NotReady if you run kubectl get nodes before setting up networking.\nSetup your config rm -rf ~/.kube/ \u0026amp;\u0026amp; mkdir ~/.kube \u0026amp;\u0026amp; cp /etc/kubernetes/admin.conf $HOME/.kube/config\nSet up Networking Set /proc/sys/net/bridge/bridge-nf-call-iptables to 1 by running sysctl net.bridge.bridge-nf-call-iptables=1 to pass bridged IPv4 traffic to iptables’ chains. Flannel supports the ARM architecture, so kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/bc79dd1505b0c8681ece4de4c0d86c5cd2643275/Documentation/kube-flannel.yml Allow pods to run on master node If you only have one node in the cluster, k8s won\u0026rsquo;t run pods on the master node. Disable tainting with kubectl taint nodes --all node-role.kubernetes.io/master-\nAdd worker nodes On the master, run kubeadm token create --print-join-command to generate an add command, then run that on the worker. It will look something like:\nkubeadm join --token SUPERSECRET 10.9.8.7:6443 --discovery-token-ca-cert-hash sha256:1234567890ABCDEF1234567890ABCDEF1234567890ABCDEF1234567890ABCDEF Install helm helm init --tiller-image=jessestuart/tiller:v2.9.1 --upgrade\nSet up the dashboard DASHSRC=https://raw.githubusercontent.com/kubernetes/dashboard/master curl -sSL $DASHSRC/src/deploy/recommended/kubernetes-dashboard-arm-head.yaml | kubectl apply -f - Updates Updated the docker install to use --no-install-recommends - this keeps it from attempting to install the aufs-dkms package, which causes error messages (but docker still works) on my Raspberry Pi 4 because the necessary kernel module isn\u0026rsquo;t provided. Added instructions for adding new worker nodes and disabling swap ","permalink":"https://unixorn.github.io/post/in_the_beginning_there_was_bare_metal/","summary":"\u003cp\u003eI recently decided to set up a Kubernetes cluster in my basement, partly because I\u0026rsquo;d never set a cluster up from scratch by myself, and partly because my existing NAS was beginning to run out of headroom.\u003c/p\u003e","title":"Getting an ARM kubernetes cluster up and running"},{"content":"I\u0026rsquo;m an SRE in Denver for Twilio.\nI like cooking, photography, and working on distributed infrastructure.\nI\u0026rsquo;m the organizer for Denver CoffeeOps, which currently meets online - the meetup link has the Google Meet details.\nI have a variety of DevOps (and DevOOPs) themed swag on redbubble, if you\u0026rsquo;d like to buy something and contribute to my hardware habit, or you can contribute at https://www.patreon.com/unixorn.\nhttps://patreon.com/unixorn?utm_medium=unknown\u0026amp;utm_source=join_link\u0026amp;utm_campaign=creatorshare_creator\u0026amp;utm_content=copyLink\n","permalink":"https://unixorn.github.io/about-orig/","summary":"\u003cp\u003eI\u0026rsquo;m an SRE in Denver for Twilio.\u003c/p\u003e\n\u003cp\u003eI like cooking, photography, and working on distributed infrastructure.\u003c/p\u003e\n\u003cp\u003eI\u0026rsquo;m the organizer for Denver \u003ca href=\"https://www.meetup.com/Denver-CoffeeOps/\"\u003eCoffeeOps\u003c/a\u003e, which currently meets online - the meetup link has the Google Meet details.\u003c/p\u003e\n\u003cp\u003eI have a variety of DevOps (and DevOOPs) themed swag on \u003ca href=\"https://www.redbubble.com/people/unixorn/collections/1069603-devoops?asc=u\"\u003eredbubble\u003c/a\u003e, if you\u0026rsquo;d like to buy something and contribute to my hardware habit, or you can contribute at \u003ca href=\"https://www.patreon.com/unixorn\"\u003ehttps://www.patreon.com/unixorn\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003e\u003cscript type=\"text/javascript\" src=\"http://www.redbubble.com/assets/external_portfolio.js\"\u003e\u003c/script\u003e\n\u003cscript id=\"rb-xzfcxvzx\" type=\"text/javascript\"\u003enew RBExternalPortfolio('www.redbubble.com', 'unixorn', 3, 3).renderIframe();\u003c/script\u003e\n\n\u003ca href=\"https://patreon.com/unixorn?utm_medium=unknown\u0026amp;utm_source=join_link\u0026amp;utm_campaign=creatorshare_creator\u0026amp;utm_content=copyLink\"\u003ehttps://patreon.com/unixorn?utm_medium=unknown\u0026amp;utm_source=join_link\u0026amp;utm_campaign=creatorshare_creator\u0026amp;utm_content=copyLink\u003c/a\u003e\u003c/p\u003e","title":""},{"content":"About Me I\u0026rsquo;m an SRE in Denver. Please don\u0026rsquo;t contact me about jobs involving blockchain, I am not interested in working with griftcoins in any way.\nI like cooking, photography, aquariums, home automation and working on distributed infrastructure.\nYou can find me online at:\nMastodon: @unixorn@hachyderm.io Among other projects, I maintain:\nThe awesome-zsh-plugins list of ZSH frameworks, plugins, themes, tab completions and utilities. The git-extra-commands collection of git helper scripts and other resources The ZSH Quickstart Kit an opinionated but fully customizable ZSH setup The Sysadmin Reading List collection of resources for SREs and DevOPs professionals. I\u0026rsquo;m also the organizer for the Denver CoffeeOps meetup, which currently meets online - the meetup link details are posted in the Colorado channels of both the hangops and coffeeops slacks.\nI have a variety of DevOPS (and DevOOPs) themed swag on redbubble and spreadshirt, if you\u0026rsquo;d like to buy something and contribute to me buying new hardware to write about.\nYou can also contribute at https://www.patreon.com/unixorn, or by getting me something from my Amazon Wishlist, or the wish list of IOT devices I want to test and post about.\n","permalink":"https://unixorn.github.io/about/","summary":"\u003ch1 id=\"about-me\"\u003eAbout Me\u003c/h1\u003e\n\u003cp\u003eI\u0026rsquo;m an SRE in Denver. Please don\u0026rsquo;t contact me about jobs involving blockchain, I am not interested in working with griftcoins in any way.\u003c/p\u003e\n\u003cp\u003eI like cooking, photography, aquariums, home automation and working on distributed infrastructure.\u003c/p\u003e\n\u003cp\u003eYou can find me online at:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eMastodon: \u003ca href=\"https://hachyderm.io/@unixorn\"\u003e@unixorn@hachyderm.io\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eAmong other projects, I maintain:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eThe \u003ca href=\"https://github.com/unixorn/awesome-zsh-plugins/\"\u003eawesome-zsh-plugins\u003c/a\u003e list of ZSH frameworks, plugins, themes, tab completions and utilities.\u003c/li\u003e\n\u003cli\u003eThe \u003ca href=\"https://github.com/unixorn/git-extra-commands/\"\u003egit-extra-commands\u003c/a\u003e collection of \u003ccode\u003egit\u003c/code\u003e helper scripts and other resources\u003c/li\u003e\n\u003cli\u003eThe \u003ca href=\"https://github.com/unixorn/zsh-quickstart-kit\"\u003eZSH Quickstart Kit\u003c/a\u003e an opinionated but fully customizable ZSH setup\u003c/li\u003e\n\u003cli\u003eThe \u003ca href=\"https://github.com/unixorn/sysadmin-reading-list\"\u003eSysadmin Reading List\u003c/a\u003e collection of resources for SREs and DevOPs professionals.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eI\u0026rsquo;m also the organizer for the Denver \u003ca href=\"https://www.meetup.com/Denver-CoffeeOps/\"\u003eCoffeeOps\u003c/a\u003e meetup, which currently meets online - the meetup link details are posted in the Colorado channels of both the \u003ca href=\"https://hangops.slack.com\"\u003ehangops\u003c/a\u003e and \u003ca href=\"https://coffeeops.slack.com\"\u003ecoffeeops\u003c/a\u003e slacks.\u003c/p\u003e","title":""},{"content":"About Me I\u0026rsquo;m an SRE in Denver for Charter. Please don\u0026rsquo;t contact me about jobs involving blockchain, I am not interested in working with griftcoins in any way.\nI like cooking, photography, aquariums, home automation and working on distributed infrastructure.\nYou can find me online at:\nMastodon: @unixorn@hachyderm.io I\u0026rsquo;m the organizer for the Denver CoffeeOps meetup, which currently meets online - the meetup link details are posted in the Colorado channels of both the hangops and coffeeops slacks.\nI have a variety of DevOPS (and DevOOPs) themed swag on redbubble and spreadshirt, if you\u0026rsquo;d like to buy something and contribute to me buying new hardware to write about.\nYou can also contribute at https://www.patreon.com/unixorn, or by getting me something from my Amazon Wishlist, or the wish list of IOT devices I want to test and post about.\n","permalink":"https://unixorn.github.io/about.unfucked/","summary":"\u003ch1 id=\"about-me\"\u003eAbout Me\u003c/h1\u003e\n\u003cp\u003eI\u0026rsquo;m an SRE in Denver for Charter. Please don\u0026rsquo;t contact me about jobs involving blockchain, I am not interested in working with griftcoins in any way.\u003c/p\u003e\n\u003cp\u003eI like cooking, photography, aquariums, home automation and working on distributed infrastructure.\u003c/p\u003e\n\u003cp\u003eYou can find me online at:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eMastodon: \u003ca href=\"https://hachyderm.io/@unixorn\"\u003e@unixorn@hachyderm.io\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eI\u0026rsquo;m the organizer for the Denver \u003ca href=\"https://www.meetup.com/Denver-CoffeeOps/\"\u003eCoffeeOps\u003c/a\u003e meetup, which currently meets online - the meetup link details are posted in the Colorado channels of both the \u003ca href=\"https://hangops.slack.com\"\u003ehangops\u003c/a\u003e and \u003ca href=\"https://coffeeops.slack.com\"\u003ecoffeeops\u003c/a\u003e slacks.\u003c/p\u003e","title":""},{"content":"Aquarium Resources Here are some resources I found useful\nPlants Ranking the Best Nutrient Control Plants Tank Setup How To Make Your First Shrimp Tank (EASY STEP BY STEP NEOCARIDINA TUTORIAL) by MD Fish Tanks. He\u0026rsquo;s got a incredible amount of good content ","permalink":"https://unixorn.github.io/aquarium-resources/","summary":"\u003ch1 id=\"aquarium-resources\"\u003eAquarium Resources\u003c/h1\u003e\n\u003cp\u003eHere are some resources I found useful\u003c/p\u003e\n\u003ch2 id=\"plants\"\u003ePlants\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://www.youtube.com/watch?v=YfNU5ZxOqVw\"\u003eRanking the Best Nutrient Control Plants\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"tank-setup\"\u003eTank Setup\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://www.youtube.com/watch?v=F-yywhyv3ho\"\u003eHow To Make Your First Shrimp Tank (EASY STEP BY STEP NEOCARIDINA TUTORIAL)\u003c/a\u003e by MD Fish Tanks. He\u0026rsquo;s got a incredible amount of good content\u003c/li\u003e\n\u003c/ul\u003e","title":""},{"content":"Email me at blog @ unixorn.net.\nYou can find me around the internet at:\n@unixorn@hachyderm.io on Mastodon apeseekingknowledge on Instagram. curiousbiped on Twitter deprecated, use Mastodon instead unixorn.bsky.social on Bluesky unixorn on GitHub I\u0026rsquo;m unixorn on redbubble ","permalink":"https://unixorn.github.io/contact/","summary":"\u003cp\u003eEmail me at blog @ unixorn.net.\u003c/p\u003e\n\u003cp\u003eYou can find me around the internet at:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://hachyderm.io/@unixorn\"\u003e@unixorn@hachyderm.io\u003c/a\u003e on Mastodon\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://www.instagram.com/apeseekingknowledge/\"\u003eapeseekingknowledge\u003c/a\u003e on Instagram.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://twitter.com/curiousbiped\"\u003ecuriousbiped\u003c/a\u003e \u003cdel\u003eon Twitter\u003c/del\u003e deprecated, use Mastodon instead\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://bsky.app/profile/unixorn.bsky.social\"\u003eunixorn.bsky.social\u003c/a\u003e on Bluesky\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/unixorn\"\u003eunixorn\u003c/a\u003e on GitHub\u003c/li\u003e\n\u003cli\u003eI\u0026rsquo;m unixorn on \u003ca href=\"https://www.redbubble.com/people/unixorn?utm_campaign=external-portfolio\u0026amp;utm_medium=embedded\u0026amp;utm_source=unixorn\u0026amp;asc=u\"\u003eredbubble\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e","title":""},{"content":"Holopin test ","permalink":"https://unixorn.github.io/holopinholo/","summary":"\u003ch1 id=\"holopin-test\"\u003eHolopin test\u003c/h1\u003e\n\u003cp\u003e\u003ca href=\"https://holopin.io/@unixorn\"\u003e\u003cimg alt=\"An image of @unixorn\u0026rsquo;s Holopin badges, which is a link to view their full Holopin profile\" loading=\"lazy\" src=\"https://holopin.me/unixorn\"\u003e\u003c/a\u003e\u003c/p\u003e","title":""},{"content":"Ziggy Stardust\n","permalink":"https://unixorn.github.io/jfc/","summary":"\u003ch1 id=\"ziggy\"\u003eZiggy\u003c/h1\u003e\n\u003cp\u003eStardust\u003c/p\u003e","title":""},{"content":"title: \u0026ldquo;Set up samba access to moosefs\u0026rdquo; date: 2023-05-21T18:15:48-06:00 draft: true tags:\nsamba homelab moosefs howto Here\u0026rsquo;s how I set up samba access to my moosefs distributed filesystem\nhttps://www.youtube.com/watch?v=Wj0SsbRbCNo\n","permalink":"https://unixorn.github.io/post/2023-10-16-samba-and-moosefs/","summary":"\u003cp\u003etitle: \u0026ldquo;Set up samba access to moosefs\u0026rdquo;\ndate: 2023-05-21T18:15:48-06:00\ndraft: true\ntags:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003esamba\u003c/li\u003e\n\u003cli\u003ehomelab\u003c/li\u003e\n\u003cli\u003emoosefs\u003c/li\u003e\n\u003cli\u003ehowto\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003cp\u003eHere\u0026rsquo;s how I set up samba access to my moosefs distributed filesystem\u003c/p\u003e","title":""},{"content":"Some projects I maintain and/or created apgar - Apgar is a quick and dirty health-check driver written in Go to prevent dependency conflicts. It is explicitly designed to be simple and not interfere with anything else running on your servers. awesome-zsh-plugins - A list of ZSH frameworks, plugins, themes, tab-completions and tutorials. dotfiles.github.io - The unofficial guide to dotfiles on GitHub. git-extra-commands - A collection of useful extra git scripts I\u0026rsquo;ve discovered or written, packaged for ease of use with shell frameworks. It also includes lists of git tutorials and other git resources. ha-mqtt-discoverable-cli - A set of command line tools usable by shell scripts that uses ha-mqtt-discoverable to make it easy to create MQTT topics that will automatically be detected by Home Asssistant as entities. ha-mqtt-discoverable - A python module that lets you create sensor and device MQTT topics that will automatically be detected by Home Asssistant. Internet-of-Trash - There are many IOT devices out there, and it\u0026rsquo;s hard to know which ones are any good. This list shows you products that are known to not work well with Home Assistant so you can avoid wasting money on them. online-devops-meetups - A list of free online devops meetups. sysadmin-reading-list - A reading/viewing list for larval stage SRE/DevOps engineers. tumult - Tumult is a collection of macOS-specific functions and scripts for your command-line environment. It is packaged as a ZSH plugin, but can be used in bash, fish or other shells as well. Works With Home Assistant - Not all IOT devices that claim to work with HA actually work well. This is a list of stuff that has been vouched to work well with Home Assistant. zsh-quickstart-kit - A simple quick start for switching to ZSH. Includes a curated list of plugins that\u0026rsquo;s easily overridden if you want to change it, is easily customizable without needing to maintain your own fork of the kit, and tweaks the setup to allow history de-duplication, history sharing across shells on the same machine, tab completion for a lot of commands that are not included in a stock ZSH install, and on macOS will load a bunch of command line tools for manipulating your Mac. ","permalink":"https://unixorn.github.io/projects/","summary":"\u003ch1 id=\"some-projects-i-maintain-andor-created\"\u003eSome projects I maintain and/or created\u003c/h1\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/unixorn/apgar\"\u003eapgar\u003c/a\u003e - Apgar is a quick and dirty health-check driver written in Go to prevent dependency conflicts. It is explicitly designed to be simple and not interfere with anything else running on your servers.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/unixorn/awesome-zsh-plugins\"\u003eawesome-zsh-plugins\u003c/a\u003e - A list of ZSH frameworks, plugins, themes, tab-completions and tutorials.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://dotfiles.github.io\"\u003edotfiles.github.io\u003c/a\u003e - The unofficial guide to dotfiles on GitHub.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/unixorn/git-extra-commands\"\u003egit-extra-commands\u003c/a\u003e - A collection of useful extra \u003ccode\u003egit\u003c/code\u003e scripts I\u0026rsquo;ve discovered or written, packaged for ease of use with shell frameworks. It also includes lists of \u003ccode\u003egit\u003c/code\u003e tutorials and other \u003ccode\u003egit\u003c/code\u003e resources.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable-cli\"\u003eha-mqtt-discoverable-cli\u003c/a\u003e - A set of command line tools usable by shell scripts that uses \u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable\"\u003eha-mqtt-discoverable\u003c/a\u003e to make it easy to create MQTT topics that will automatically be detected by \u003ca href=\"https://home-assistant.io\"\u003eHome Asssistant\u003c/a\u003e as entities.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/unixorn/ha-mqtt-discoverable\"\u003eha-mqtt-discoverable\u003c/a\u003e - A python module that lets you create sensor and device MQTT topics that will automatically be detected by \u003ca href=\"https://home-assistant.io\"\u003eHome Asssistant\u003c/a\u003e.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/unixorn/internet-of-trash/\"\u003eInternet-of-Trash\u003c/a\u003e - There are many IOT devices out there, and it\u0026rsquo;s hard to know which ones are any good. This list shows you products that are known to \u003cem\u003enot\u003c/em\u003e work well with \u003ca href=\"https://www.home-assistant.io\"\u003eHome Assistant\u003c/a\u003e so you can avoid wasting money on them.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/unixorn/online-devops-meetups\"\u003eonline-devops-meetups\u003c/a\u003e - A list of free online devops meetups.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/unixorn/sysadmin-reading-list\"\u003esysadmin-reading-list\u003c/a\u003e - A reading/viewing list for larval stage SRE/DevOps engineers.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/unixorn/tumult.plugin.zsh\"\u003etumult\u003c/a\u003e - Tumult is a collection of macOS-specific functions and scripts for your command-line environment. It is packaged as a ZSH plugin, but can be used in \u003ccode\u003ebash\u003c/code\u003e, \u003ccode\u003efish\u003c/code\u003e or other shells as well.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/unixorn/works-with-home-assistant\"\u003eWorks With Home Assistant\u003c/a\u003e - Not all IOT devices that claim to work with HA actually work well. This is a list of stuff that has been vouched to work well with \u003ca href=\"https://www.home-assistant.io\"\u003eHome Assistant\u003c/a\u003e.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/unixorn/zsh-quickstart-kit\"\u003ezsh-quickstart-kit\u003c/a\u003e - A simple quick start for switching to ZSH. Includes a curated list of plugins that\u0026rsquo;s easily overridden if you want to change it, is easily customizable without needing to maintain your own fork of the kit, and tweaks the setup to allow history de-duplication, history sharing across shells on the same machine, tab completion for a lot of commands that are not included in a stock ZSH install, and on macOS will load a bunch of command line tools for manipulating your Mac.\u003c/li\u003e\n\u003c/ul\u003e","title":""}]