Replaces #484.

As devs we want to optimize the OpenShift certification process, as it is currently a lot of manual work each time.

Scope

This is about generating the OLM manifests automatically, as far as is sensible/practical. It will not and should not be regarded as a 100% solution as some manifests (e.g. ClusterServiceVersion) will need manual checking.

• Using the manifests from the OpenShift repo will be done during certification and is not part of this ticket.
• Deploying against an Openshift cluster is not part of this ticket.

Context

stackable-utils contains two scripts used for manifest preparation and bundle building. Their usage is documented in each script but can be summarised as follows:

build-manifests.sh

• the helm manifests for a specific release are used as a basis for creating Openshift-specific manifests
• some of these can be used 1:1 (CRDs, although they have to be split into separate files), others need slight adaption (e.g. ConfigMap) and other objects are specific to OS (e.g. ClusterServiceVersion)
• the resulting manifests are copied directly to a previously-created branch in the openshift certification respository

build-bundle.sh

• this script takes the manifests created by build-manifests.sh and uses them to build an olm bundle and deploys it against an available Openshift cluster
• the deployment will show up as a Catalog Source in the Openshift console and can be directly deployed from there for testing
• once the Catalog Source has been tested it can be deleted from the cluster and the same manifests used to run the certification pipeline

Tasks

• update: this will not be handled in this issue. consider fixing this issue in the new generation script: Use the namespace stackable-operators in the SecretClass tls in the OLM package #498
• add versions to the CSV file names. This seems to be a requirement for operators to be upgradable. More research is needed.
• fix the spark-k8s/spark confusion. The manifest script expects spark-k8s for the operator, but the bundles use spark (as references to the spark-k8s objects are wrapped in the manifests). This should be made less confusing.
• generate a CSV
  • with the deployment and operator cluster role already embedded (see here)
  • using the correct service account in the CSV
  • adding required metadata annotations (this is a bit of a moving target and will need to be checked against the most recent openshift requirements)
  • adding required namespace settings
  • automatically pull in quay.io images with hashes
• remove helm labels and remove all labels from the cluster role
• 0.0.0-dev versions are now explicitely allowed automatically check for hanging references from previous releases (e.g. 0.0.0-dev, 24.3)
• check the annotations.yaml for the correct Openshift versions
• re-write the manifests script in python as bash is presenting limitations (e.g. stable indenting)
• create an issue template for OLM manifests (and maybe also the certification process): https://github.com/stackabletech/issues/blob/main/.github/ISSUE_TEMPLATE/olm_manifests.md

Testing

1. Install the secret and listener op first from the RH catalog, or using Helm, or by using the existing OLM manifests.
2. For all other operators run the steps below (replace hdfs as needed)

# cd to stackable-utils repo
# generate manifests
./olm/build-manifests.py --openshift-versions 'v4.11-v4.15' --release 24.3.0 --repo-operator ~/repo/stackable/hdfs-operator
# publish operator
./olm/build-bundles.sh -r 24.3.0 -o hdfs -c ~/repo/stackable/openshift-certified-operators -d 
# cd to operator repository
./scripts/run_tests.sh --test-suite openshift --test smoke --skip-release

### Operators
- [x] airflow-operator @adwk67
- [x] commons-operator @razvan
- [x] druid-operator @adwk67
- [x] hbase-operator
- [x] hdfs-operator
- [x] hello-world-operator @adwk67
- [x] hive-operator
- [x] kafka-operator @razvan
- [x] ~listener-operator~: excluded from this issue
- [x] nifi-operator @razvan
- [x] opa-operator @adwk67
- [x] ~secret-operator~: excluded from this issue
- [x] spark-k8s-operator @razvan
- [x] superset-operator @adwk67
- [x] trino-operator @adwk67
- [x] zookeeper-operator

Acceptance Criteria

• A branch should exist in the openshift certification respository for each operator, that refers to a current (or upcoming) release, containing manifests that can be deployed on-demand with build-bundles.sh
• The manifests should be generated in their entirety (as per the task above), but scripts/commands should be available/documented to test their validity. This will not remove the need for an optical inspection!