Skip to content
This repository has been archived by the owner on Jan 23, 2024. It is now read-only.

stolostron/hub-of-hubs-gitops

Hub-of-Hubs GitOps

Go Report Card Go Reference License

The GitOps component of Hub-of-Hubs.

Go to the Contributing guide to learn how to get involved.

Overview

image

The Hub-of-Hubs (HOH) GitOps component shares a volume (persistent storage) with a modified version of the multicloud-operators-subscription operator, where the subscriptions-operator is responsible for syncing Git objects via the ACM Subscriptions mechanism, while the HOH GitOps component watches the files and processes them to provide support for customized gitops / non-k8s gitops.

Disclaimers:

  • The component was implemented to demonstrate the mechanism. It is not fully implemented and is not tested for scale:
    • Syncers should support un-deploying non-k8s resources upon deleting a subscription (e.g., when a repo is found but its subscription is not).
    • Supported non-k8s resource implementations should be completed (currently only supports "INSERT" but not update/delete).
    • Optimizations such as parallelized storage-walking / parallelized & batched DB job handling can be applied.

Prerequisites

Deploying the Shared Volume

  1. Set the GITOPS_NODE_HOSTNAME to the hostname of the node (e.g., ip-10-0-136-193) that the storage, HOH-gitops and the customized operator will run on:

    $ export GITOPS_NODE_HOSTNAME=$(kubectl get node --selector='node-role.kubernetes.io/worker' -o=jsonpath='{.items[0].metadata.labels.kubernetes\.io\/hostname}')
    
  2. Run the following command to deploy the hoh-gitops-pv PersistentVolume and the hoh-gitops-pv-claim PersistentVolumeClaim that claims it to your hub of hubs cluster:

    envsubst < deploy/hub-of-hubs-gitops-pv.yaml | kubectl apply -f -
    

Deploying the customized Subscriptions Operator

Deploying the modified Subscription CRD

kubectl -n open-cluster-management apply -f deploy/customized-subscriptions-operator/apps.open-cluster-management.io_subscriptions_crd_v1.yaml

Creating the namespace for accessible Subscription CRs

kubectl create namespace hoh-subscriptions

Deploying the modified operator

The subscriptions operator deployment is managed by the ACM for Kubernetes Operator. To have the latter deploy the customized version, modify the "multicluster-operators-standalone-subscription" deployment to that present in standalone-subscriptions-operator-deployment.yaml.

The modified code has small modifications of the upstream stable release of the operator in Open Cluster Management organization, therefore it is forked to a personal Git.

  1. Set the MODIFIED_OPERATOR_IMAGE environment variable to hold the URL of the image:
    $ export MODIFIED_OPERATOR_IMAGE=quay.io/maroonayoub/multicloud-operators-subscription@sha256:1c57e1e77ea3c929c7176681d5b64eca43354bbaf00aeb7f7ddb01d3c6d15ad0
    
  2. Patch the ACM for K8s operator:
    kubectl -n open-cluster-management patch ClusterServiceVersion advanced-cluster-management.v2.4.2 --type=merge --patch "$(envsubst < deploy/customized-subscriptions-operator/operators-subscriptions-deployments-patch.yaml)"
    

Visit examples for example Subscription deployments / Git objects

Getting Started

Build and push the image to docker registry

  1. Set the REGISTRY environment variable to hold the name of your docker registry:

    $ export REGISTRY=...
    
  2. Set the IMAGE_TAG environment variable to hold the required version of the image.
    default value is latest, so in that case no need to specify this variable:

    $ export IMAGE_TAG=latest
    
  3. Run make to build and push the image:

    $ make push-images
    

Deploy on the hub of hubs

  1. If the hub-of-hubs-database-transport-bridge-secret does not exist:

    1. Set the DATABASE_URL according to the PostgreSQL URL format: postgres://YourUserName:YourURLEscapedPassword@YourHostname:5432/YourDatabaseName?sslmode=verify-full&pool_max_conns=50. Remember to URL-escape the password, you can do it in bash:

      python -c "import sys, urllib as ul; print ul.quote_plus(sys.argv[1])" 'YourPassword'
      
    2. Create a secret with your database url:

      kubectl create secret generic hub-of-hubs-database-transport-bridge-secret -n open-cluster-management --from-literal=url=$DATABASE_URL
      
  2. Set the REGISTRY environment variable to hold the name of your docker registry:

    $ export REGISTRY=...
    
  3. Set the IMAGE environment variable to hold the name of the image.

    $ export IMAGE=$REGISTRY/$(basename $(pwd)):latest
    
  4. Run the following command to give the hub-of-hubs-gitops service account "privileged" security context constraint permissions:

    oc adm policy add-scc-to-user privileged -z hub-of-hubs-gitops -n open-cluster-management
    
  5. Run the following command to deploy the hub-of-hubs-gitops to your hub of hubs cluster:

    envsubst < deploy/hub-of-hubs-gitops.yaml.template | kubectl apply -f -
    

Cleanup from the hub of hubs

  1. Run the following command to clean hub-of-hubs-gitops from your hub of hubs cluster:

    envsubst < deploy/hub-of-hubs-gitops.yaml.template | kubectl delete -f -
    
  2. Run the following command to remove the "privileged" security context constraint permissions from hub-of-hubs-gitops service account :

    oc adm policy remove-scc-from-user privileged -z hub-of-hubs-gitops -n open-cluster-management
    
  3. If you wish to revert the ACM for K8s operator's customization, run the following:

    kubectl -n open-cluster-management patch ClusterServiceVersion advanced-cluster-management.v2.4.2 \ 
       --type=merge --patch $(cat deploy/customized-subscriptions-operator/revert-operators-subscriptions-deployments-patch.yaml)
    
  4. Finally, delete PV and PVC:

    kubectl -n open-cluster-management delete -f deploy/hub-of-hubs-gitops-pv.yaml