Fixes

• [bucket] Fix S3 Manager endpoint mismatch with COSI credentials: The S3 Manager UI previously constructed an s3.<tenant>.<cluster-domain> endpoint even though COSI-issued bucket credentials point to the root-level S3 endpoint. This caused login failures with "invalid credentials" despite valid secrets. The deployment now uses the actual endpoint from BucketInfo, with the old namespace-based endpoint kept only as a fallback before BucketAccess secrets exist (@IvanHunters in #2211, #2215).

• [platform] Fix spurious OpenAPI post-processing errors on cozystack-api startup: The OpenAPI post-processor was being invoked for non-apps.cozystack.io group versions where the base Application* schemas do not exist, producing noisy startup errors on every API server launch. It now skips those non-apps group versions gracefully instead of returning an error (@kvaps in #2212, #2217).

Documentation

• [website] Add troubleshooting for packages stuck in DependenciesNotReady: Added an operations guide that explains how to diagnose missing package dependencies in operator logs and corrected the packages management development docs to use the current make image-packages target (@kvaps in cozystack/website#450).

• [website] Reorder installation docs to install the operator before the platform package: Updated the platform installation guide and tutorial so the setup sequence consistently installs the Cozystack operator first, then prepares and applies the Platform Package, matching the rest of the documentation set (@sircthulhu in cozystack/website#449).

• [website] Add automated installation guide for the Ansible collection: Added a full guide for deploying Cozystack with the cozystack.installer collection, including inventory examples, distro-specific playbooks, configuration reference, and explicit version pinning guidance (@lexfrei in cozystack/website#442).

• [website] Expand monitoring and platform architecture reference docs: Added a tenant custom metrics collection guide for VMServiceScrape and VMPodScrape, and documented PackageSource/Package architecture, reconciliation flow, rollback behavior, and the cozypkg workflow in Key Concepts (@IvanHunters in cozystack/website#444, cozystack/website#445).

• [website] Improve operations guides for CA rotation and Velero backups: Completed the CA rotation documentation with dry-run and post-rotation credential retrieval steps, and expanded the backup configuration guide with concrete examples, verification commands, and clearer operator procedures (@kvaps in cozystack/website#406; @androndo in cozystack/website#440).

Full Changelog: v1.1.1...v1.1.2

Fixes

• [api] Fix spurious OpenAPI post-processing errors for non-apps group versions: The API server no longer logs false errors while generating OpenAPI specs for core and other non-apps.cozystack.io group versions. The post-processor now exits early when the base Application schemas are absent, reducing noisy startup logs without affecting application schema generation (@kvaps in #2212, #2216).

Documentation

• [website] Add DependenciesNotReady troubleshooting and correct packages management build target: Added a troubleshooting guide for packages stuck in DependenciesNotReady, including how to inspect operator logs and identify missing dependencies, and fixed the outdated make image-cozystack command to make image-packages in the packages management guide (@kvaps in cozystack/website#450).

• [website] Clarify operator-first installation order: Reordered the platform installation guide and tutorial so users install the Cozystack operator before preparing and applying the Platform Package, matching the rest of the installation docs and reducing setup confusion during fresh installs (@sircthulhu in cozystack/website#449).

• [website] Add automated installation guide for Ansible: Added end-to-end documentation for deploying Cozystack with the cozystack.installer Ansible collection, including inventory examples, distro-specific playbooks, configuration reference, verification steps, and explicit version pinning guidance to help operators automate installs safely (@lexfrei in cozystack/website#442).

• [website] Expand CA rotation operations guide: Completed the CA rotation documentation with separate Talos and Kubernetes certificate rotation procedures, dry-run preview steps, and post-rotation guidance for fetching updated talosconfig and kubeconfig files after certificate changes (@kvaps in cozystack/website#406).

• [website] Improve backup operations documentation: Enhanced the operator backup and recovery guide with clearer Velero enablement steps, concrete provider and bucket examples, and more useful commands for inspecting backups, schedules, restores, CRD status, and logs (@androndo in cozystack/website#440).

• [website] Add custom metrics collection guide: Added a monitoring guide showing how tenants can expose their own Prometheus exporters through VMServiceScrape and VMPodScrape, including namespace labeling requirements, example manifests, verification steps, and troubleshooting advice (@IvanHunters in cozystack/website#444).

• [website] Document PackageSource and Package architecture: Added a Key Concepts reference covering PackageSource and Package reconciliation flow, dependency handling, update propagation, rollback behavior, FluxPlunger recovery, and the cozypkg CLI for package management (@IvanHunters in cozystack/website#445).

• [website] Refresh v1 application and platform documentation: Fixed the documentation auto-update flow and published a broad v1 documentation refresh covering newly documented applications, updated naming and navigation, virtualization and platform content updates, and reorganized versioned docs pages (@myasnikovdaniil in cozystack/website#439).

Full Changelog: v1.0.4...v1.0.5

Fixes

• [dashboard] Fix hidden MarketplacePanel resources appearing in sidebar menu: The sidebar was generated independently from MarketplacePanels, always showing all resources regardless of their hidden state. Fixed by fetching MarketplacePanels during sidebar reconciliation and skipping resources where hidden=true, so hiding a resource from the marketplace also removes it from the sidebar navigation (@IvanHunters in #2177, #2203).

• [dashboard] Fix disabled/hidden state overwritten on every MarketplacePanel reconciliation: The controller was hardcoding disabled=false and hidden=false on every reconciliation, silently overwriting any user changes made through the dashboard UI. Fixed by reading and preserving the current disabled/hidden values from the existing resource before updating (@IvanHunters in #2176, #2201).

• [dashboard] Fix External IPs factory EnrichedTable rendering: The external-IPs table displayed empty rows because the factory used incorrect EnrichedTable properties. Replaced clusterNamePartOfUrl with cluster and changed pathToItems from array to dot-path string format, consistent with all other working EnrichedTable instances (@IvanHunters in #2175, #2193).

• [platform] Fix VM MAC address not preserved during virtual-machine to vm-instance migration: Kube-OVN reads MAC address exclusively from the pod annotation ovn.kubernetes.io/mac_address, not from the IP resource spec.macAddress. Without the annotation, migrated VMs received a new random MAC, breaking OS-level network configurations that match by MAC (e.g. netplan). Added a Helm lookup for the Kube-OVN IP resource in the vm-instance chart so that MAC and IP addresses are automatically injected as pod annotations when the resource exists (@sircthulhu in #2169, #2190).

• [etcd-operator] Replace deprecated kube-rbac-proxy image: The gcr.io/kubebuilder/kube-rbac-proxy image became unavailable after Google Container Registry was deprecated. Replaced it with quay.io/brancz/kube-rbac-proxy from the original upstream author, restoring etcd-operator functionality (@kvaps in #2181, #2182).

• [migrations] Handle missing RabbitMQ CRD in migration 34: Migration 34 failed with an error when the rabbitmqs.apps.cozystack.io CRD did not exist — which occurs on clusters where RabbitMQ was never installed. Added a CRD presence check before attempting to list resources so that migration 34 completes cleanly on such clusters (@IvanHunters in #2168, #2180).

• [keycloak] Fix Keycloak crashloop due to misconfigured health probes: Keycloak 26.x redirects all HTTP requests on port 8080 to the configured HTTPS hostname; since kubelet does not follow redirects, liveness and readiness probes failed causing a crashloop. Fixed by enabling KC_HEALTH_ENABLED=true, exposing management port 9000, and switching all probes to /health/live and /health/ready on port 9000. Also added a startupProbe for improved startup tolerance (@mattia-eleuteri in #2162, #2179).

Full Changelog: v1.1.0...v1.1.1

Fixes

• [system] Fix Keycloak probe crashloop with management port health endpoints: Fixed a crashloop where Keycloak 26.x was endlessly restarting because liveness and readiness probes were sending HTTP requests to port 8080. Keycloak 26.x redirects all requests on port 8080 to KC_HOSTNAME (HTTPS), and since kubelet does not follow redirects, probes failed, eventually triggering container restarts. The fix switches probes to the dedicated management port 9000 (/health/live, /health/ready) enabled via KC_HEALTH_ENABLED=true, exposes management port 9000, and adds a startupProbe with appropriate failure thresholds for better startup tolerance (@mattia-eleuteri in #2162, #2178).

• [system] Fix etcd-operator deprecated kube-rbac-proxy image: Replaced the deprecated gcr.io/kubebuilder/kube-rbac-proxy:v0.16.0 image with quay.io/brancz/kube-rbac-proxy:v0.18.1 in the vendored etcd-operator chart. The GCR-hosted image became unavailable after March 18, 2025, causing etcd-operator pods to fail on image pull (@kvaps in #2181, #2183).

• [platform] Fix VM MAC address not preserved during virtual-machine to vm-instance migration: During the virtual-machine → vm-instance migration (script 29), VM MAC addresses were not preserved. Kube-OVN reads MAC addresses exclusively from the pod annotation ovn.kubernetes.io/mac_address, not from spec.macAddress of the IP resource. Without this annotation, migrated VMs received a new random MAC address, breaking OS-level network configuration that matches by MAC (e.g., netplan). The fix adds a Helm lookup in the vm-instance chart template to read the Kube-OVN IP resource and automatically inject the MAC and IP addresses as pod annotations (@sircthulhu in #2169, #2191).

• [dashboard] Fix External IPs page showing empty rows: Fixed the External IPs administration page displaying empty rows instead of service data. The EnrichedTable configuration in the external-ips factory was using incorrect property names — replaced clusterNamePartOfUrl with cluster and changed pathToItems from array format to dot-path string format, matching the convention used by all other EnrichedTable instances (@IvanHunters in #2175, #2192).

• [dashboard] Fix disabled/hidden state reset on MarketplacePanel reconciliation: Fixed a bug where the dashboard controller was hardcoding disabled=false and hidden=false on every reconcile loop, overwriting changes made through the dashboard UI. Services disabled or hidden via the marketplace panel now correctly retain their state after controller reconciliation (@IvanHunters in #2176, #2202).

• [dashboard] Fix hidden MarketplacePanel resources appearing in sidebar menu: Fixed the sidebar navigation showing all resources regardless of their MarketplacePanel hidden state. The controller now fetches MarketplacePanels during sidebar reconciliation and filters out resources where hidden=true, ensuring that hiding a resource from the marketplace also removes it from the sidebar navigation. Listing failures are non-fatal — if the configuration fetch fails, no hiding is applied and the dashboard remains functional (@IvanHunters in #2177, #2204).

Documentation

• [website] Add OIDC self-signed certificates configuration guide: Added a comprehensive guide for configuring OIDC authentication with Keycloak when using self-signed certificates (the default in Cozystack). Covers Talos machine configuration with certificate mounting and host entries, kubelogin setup instructions, and a troubleshooting section. The guide is available for both v0 and v1 versioned documentation paths (@IvanHunters in cozystack/website#443).

Full Changelog: v1.0.3...v1.0.4

Cozystack v1.1.0

Cozystack v1.1.0 delivers a major expansion of the managed application catalog with OpenBAO (open-source HashiCorp Vault fork) for secrets management, comprehensive tiered object storage with SeaweedFS storage pools, a new bucket user model with per-user credentials and S3 login support, RabbitMQ version selection, and MongoDB Grafana dashboards. The dashboard gains storageClass dropdowns for all stateful apps. This release also incorporates all fixes from the v1.0.x patch series.

Feature Highlights

OpenBAO: Managed Secrets Management Service

Cozystack now ships OpenBAO as a fully managed PaaS application — an open-source fork of HashiCorp Vault providing enterprise-grade secrets management. Users can deploy OpenBAO instances in standalone mode (single replica with file storage) or in high-availability Raft mode (multiple replicas with integrated Raft consensus), with the mode switching automatically based on the replicas field.

Each OpenBAO instance gets TLS enabled by default via cert-manager self-signed certificates, with DNS SANs covering all service endpoints and pod addresses. The Vault injector and CSI provider are intentionally disabled (they are cluster-scoped components not safe for per-tenant use). OpenBAO requires manual initialization and unsealing by design — no auto-unseal is configured.

A full end-to-end E2E test covers the complete lifecycle: deploy, wait for certificate and API readiness, init, unseal, verify, and cleanup. OpenBAO is available in the application catalog for tenant namespaces.

SeaweedFS Tiered Storage Pools

SeaweedFS now supports tiered storage pools — operators can define separate storage pools per disk type (SSD, HDD, NVMe) in the volume.pools field (Simple topology) or volume.zones[name].pools (MultiZone topology). Each pool creates an additional Volume StatefulSet alongside the default one, with SeaweedFS distinguishing storage via the -disk=<type> flag on volume servers.

Each pool automatically generates its own set of COSI resources: a standard BucketClass, a -lock BucketClass (COMPLIANCE mode, 365-day retention), a read-write BucketAccessClass, and a -readonly BucketAccessClass. This allows applications to place data on specific storage tiers and request appropriate access policies per pool.

In MultiZone topology, pools are defined per zone and each zone × pool combination creates a dedicated StatefulSet (e.g., us-east-ssd, us-west-hdd), with nodes selected via topology.kubernetes.io/zone labels. Existing deployments with no pools defined produce output identical to previous versions — no migration is required.

Bucket User Model with S3 Login

The bucket application introduces a new user model for access management. Instead of a single implicit BucketAccess resource, operators now define a users map where each entry creates a dedicated BucketAccess with its own credentials secret and an optional readonly flag. The S3 Manager UI has been updated with a login screen that uses per-session credentials from the user's own secret, replacing the previous basic-auth approach.

Two new bucket parameters are available: locking provisions from the -lock BucketClass (COMPLIANCE mode, 365-day object lock retention) for write-once-read-many use cases, and storagePool selects a specific pool's BucketClass for tiered storage placement. The COSI driver has been updated to v0.3.0 to support the new diskType parameter.

⚠️ Breaking change: The implicit default BucketAccess resource is no longer created. Existing buckets that relied on the single auto-generated BucketAccess will need to explicitly define users in the users map after upgrading.

RabbitMQ Version Selection

RabbitMQ instances now support a configurable version selector (version field with values: v4.2, v4.1, v4.0, v3.13; default v4.2). The chart validates the selection at deploy time and uses it to pin the runtime image, giving operators control over the RabbitMQ release channel per instance. An automatic migration backfills the version field on all existing RabbitMQ resources to v4.2.

Major Features and Improvements

• [apps] Add OpenBAO as a managed secrets management service: Deployed as a PaaS application with standalone (file storage) and HA Raft modes, TLS enabled by default via cert-manager, injector and CSI provider disabled for tenant safety, and a full E2E lifecycle test (@lexfrei in #2059).

• [seaweedfs] Add storage pools support for tiered storage: Added volume.pools (Simple) and volume.zones[name].pools (MultiZone) for per-disk-type StatefulSets, zone overrides (nodeSelector, storageClass, dataCenter), per-pool COSI BucketClass and BucketAccessClass resources, and bumped seaweedfs-cosi-driver to v0.3.0 (@sircthulhu in #2097).

• [apps][system] Add bucket user model with locking and storage pool selection: Replaced implicit BucketAccess with per-user users map, added locking and storagePool parameters, renamed COSI BucketClass suffix from -worm to -lock, added -readonly BucketAccessClass for all topologies, and updated S3 Manager with login screen using per-user credentials (@IvanHunters in #2119).

• [rabbitmq] Add version selection for RabbitMQ instances: Added version field (v4.2, v4.1, v4.0, v3.13) with chart-level validation, default v4.2, and an automatic migration to backfill the field on existing instances (@myasnikovdaniil in #2092).

• [system] Add MongoDB Overview and InMemory Details Grafana dashboards: Added two comprehensive Grafana dashboards for MongoDB monitoring — Overview (command operations, connections, cursors, query efficiency, write time) and InMemory Details (WiredTiger cache, transactions, concurrency, eviction). Dashboards are registered in dashboards.list for automatic GrafanaDashboard CRD generation (@IvanHunters in #2158).

• [dashboard] Add storageClass dropdown for all stateful apps: Replaced the free-text storageClass input with an API-backed dropdown listing available StorageClasses from the cluster. Affects ClickHouse, Harbor, HTTPCache, Kubernetes, MariaDB, MongoDB, NATS, OpenBAO, Postgres, Qdrant, RabbitMQ, Redis, VMDisk (top-level storageClass), FoundationDB (storage.storageClass), and Kafka (kafka.storageClass, zookeeper.storageClass) (@sircthulhu in #2131).

• [bucket] Add readonly S3 access credentials: Added a readonly BucketAccessClass to the SeaweedFS COSI chart and updated the bucket application to automatically provision two sets of S3 credentials per bucket: read-write (for UI) and readonly (@IvanHunters in #2105).

• [dashboard] Hide sidebar on cluster-level pages when no tenant selected: Fixed broken URLs with double // on the main cluster page (before tenant selection) by clearing CUSTOMIZATION_SIDEBAR_FALLBACK_ID so no sidebar renders when no namespace is selected (@sircthulhu in #2106).

• [cert-manager] Update cert-manager to v1.19.3: Upgraded cert-manager with new CRDs moved into a dedicated CRD package, added global nodeSelector and hostUsers (pod user-namespace isolation), and renamed ServiceMonitor targetPort default to http-metrics (@myasnikovdaniil in #2070).

• [dashboard] Add backupClasses dropdown to Plan/BackupJob forms: Replaced free-text input for backupClass field with an API-backed dropdown populated with available BackupClass resources, making it easier to select the correct backup target (@androndo in #2104).

Fixes

• [platform] Fix package name conversion in migration script: Fixed the migrate-to-version-1.0.sh script to correctly prepend the cozystack. prefix when converting BUNDLE_DISABLE and BUNDLE_ENABLE package name lists, ensuring packages are properly identified during the v0.41→v1.0 upgrade (@myasnikovdaniil in #2144, #2148).

• [backups] Fix RBAC for backup controllers: Updated RBAC permissions for the backup strategy controller to support enhanced backup and restore capabilities, including Velero integration and status management (@androndo in #2145).

• [kubernetes] Set explicit MTU for Cilium in tenant clusters: Set explicit MTU 1350 for Cilium in KubeVirt-based tenant Kubernetes clusters to prevent packet drops caused by VXLAN encapsulation overhead. Cilium's auto-detection does not account for VXLAN overhead (50 bytes) when the VM interface inherits MTU 1400 from the parent OVN/Geneve overlay, causing intermittent connectivity issues and HTTP 499 errors under load (@IvanHunters in #2147).

• [platform] Prevent cozystack-version ConfigMap from deletion: Added resource protection annotations to prevent the cozystack-version ConfigMap from being accidentally deleted, improving platform stability (@myasnikovdaniil in #2112, #2114).

• [installer] Add keep annotation to Namespace and update migration script: Added helm.sh/resource-policy: keep annotation to the cozy-system Namespace in the installer Helm chart to prevent Helm from deleting the namespace and all HelmReleases within it when the installer release is removed. The v1.0 migration script is also updated to annotate the namespace and cozystack-version ConfigMap before migration (@kvaps in #2122, #2123).

• **[dashboard] Add FlowSchema to exempt BFF from API throttl...

Fixes

• [platform] Fix package name conversion in migration script: Fixed the migrate-to-version-1.0.sh script to correctly prepend the cozystack. prefix when converting BUNDLE_DISABLE and BUNDLE_ENABLE package name lists, ensuring packages are properly identified during the v0.41→v1.0 upgrade (@myasnikovdaniil in #2144, #2148).

Documentation

• [website] Add white labeling guide: Added a comprehensive guide for configuring white labeling (branding) in Cozystack v1, covering Dashboard fields (titleText, footerText, tenantText, logoText, logoSvg, iconSvg) and Keycloak fields (brandName, brandHtmlName). Includes SVG preparation workflow with theme-aware template variables, portable base64 encoding, and migration notes from the v0 ConfigMap approach (@lexfrei in cozystack/website#441).

• [website] Actualize backup and recovery documentation: Reworked the backup and recovery docs to be user-focused, separating operator and tenant workflows. Added tenant-facing documentation for BackupJob and Plan resources and status inspection commands, and added a new Velero administration guide for operators covering storage credentials and backup storage configuration (@androndo in cozystack/website#434).

Full Changelog: v1.0.2...v1.0.3

Fixes

• [platform] Suspend cozy-proxy if it conflicts with installer release during migration: Added a check in the v0.41→v1.0 migration script to detect and automatically suspend the cozy-proxy HelmRelease when its releaseName is set to cozystack, which conflicts with the installer release and would cause cozystack-operator deletion during the upgrade (@kvaps in #2128, #2130).

• [platform] Fix off-by-one error in run-migrations script: Fixed a bug in the migration runner where the first required migration was always skipped due to an off-by-one error in the migration range calculation, ensuring all upgrade steps execute correctly (@myasnikovdaniil in #2126, #2132).

• [system] Fix Keycloak proxy configuration for v26.x: Replaced the deprecated KC_PROXY=edge environment variable with KC_PROXY_HEADERS=xforwarded and KC_HTTP_ENABLED=true in the Keycloak StatefulSet template. KC_PROXY was removed in Keycloak 26.x, previously causing "Non-secure context detected" warnings and broken cookie handling when running behind a reverse proxy with TLS termination (@sircthulhu in #2125, #2134).

• [dashboard] Allow clearing instanceType field and preserve newlines in secret copy: Added allowEmpty: true to the instanceType field in the VMInstance form so users can explicitly clear it to use custom KubeVirt resources without a named instance type. Also fixed newline preservation when copying secrets with CMD+C (@sircthulhu in #2135, #2137).

• [dashboard] Restore stock-instance sidebars for namespace-level pages: Restored stock-instance-api-form, stock-instance-api-table, stock-instance-builtin-form, and stock-instance-builtin-table sidebar resources that were inadvertently removed in #2106. Without these sidebars, namespace-level pages such as Backup Plans rendered as empty pages with no interactive content (@sircthulhu in #2136, #2138).

Full Changelog: v1.0.1...v1.0.2

Fixes

• [platform] Prevent cozystack-version ConfigMap from deletion: Added resource protection to prevent the cozystack-version ConfigMap from being accidentally deleted, improving platform stability and reliability (@myasnikovdaniil in #2112, #2114).

• [installer] Add keep annotation to Namespace and update migration script: Added helm.sh/resource-policy: keep annotation to the cozy-system Namespace in the installer Helm chart to prevent Helm from deleting the namespace (and all HelmReleases within it) when the installer release is removed. The v1.0 migration script is also updated to annotate the cozy-system namespace and cozystack-version ConfigMap with this policy before migration (@kvaps in #2122, #2123).

• [dashboard] Add FlowSchema to exempt BFF from API throttling: Added a cozy-dashboard-exempt FlowSchema to exempt the dashboard Back-End-for-Frontend (BFF) service account from Kubernetes API Priority and Fairness throttling. Previously, the BFF fell under the workload-low priority level, causing 429 (Too Many Requests) errors under load, resulting in dashboard unresponsiveness (@kvaps in #2121, #2124).

Documentation

• [website] Replace bundles documentation with variants: Renamed the "Bundles" documentation section to "Variants" to match current Cozystack terminology. Removed deprecated variants (iaas-full, distro-full, distro-hosted) and added new variants: default (PackageSources only, for manual package management via cozypkg) and isp-full-generic (full PaaS/IaaS on k3s, kubeadm, or RKE2). Updated all cross-references throughout the documentation (@kvaps in cozystack/website#433).

• [website] Add step to protect namespace before upgrading: Updated the cluster upgrade guide and v0.41→v1.0 migration guide with a required step to annotate the cozy-system namespace and cozystack-version ConfigMap with helm.sh/resource-policy=keep before running helm upgrade, preventing accidental namespace deletion (@kvaps in cozystack/website#435).

Full Changelog: v1.0.0...v1.0.1