Registration agent stopped updating its lease.

[levinwang88]

The AVAILABLE status started after joining the cluster to the hub is True, the following information:

NAME       HUB ACCEPTED   MANAGED CLUSTER URLS         JOINED   AVAILABLE   AGE
cluster1       true           https://ip:6443          True         True        32s

But after a while, found that the cluster AVAILABLE status automatically changed to Unknown. here is the information:

# kc get managedcluster
NAME       HUB ACCEPTED   MANAGED CLUSTER URLS         JOINED   AVAILABLE   AGE
cluster1   true           https://ip:6443   True     Unknown     36m

#oc describe managedcluster cluster1
Message:               Accepted by hub cluster admin
    Reason:                HubClusterAdminAccepted
    Status:                True
    Type:                  HubAcceptedManagedCluster
    Last Transition Time:  2021-08-18T21:48:26Z
    Message:               Registration agent stopped updating its lease.
    Reason:                ManagedClusterLeaseUpdateStopped
    Status:                Unknown
    Type:                  ManagedClusterConditionAvailable
    Last Transition Time:  2021-08-18T15:25:45Z
    Message:               Managed cluster joined
    Reason:                ManagedClusterJoined
    Status:                True
    Type:                  ManagedClusterJoined

[xauthulei]

you may need to check the klusterlet status on managed cluster cluster1, Thanks

[abdennour]

we got same error after renewing the certificate of api.hubcluster.company.lan (6443)

What should be done ?

Is it ok to update manually the following secret in every managed cluster ?

 oc -n open-cluster-management-agent extract secret/hub-kubeconfig-secret  --to=.

[zhujian7]

@abdennour Has the CA of the hub cluster apiserver changed? you can try to update the bootstrap-hub-kubeconfig secret in the open-cluster-management-agent namespace on the managed cluster, then the registration-agent will rebootstrap.

[abdennour]

Thanks @zhujian7 ! yep it works! Here steps

1. Get Cert from Hub Cluster

#0. Get the new CA certificate 
api_cert_secret_name=$(oc get apiserver cluster -ojsonpath='{.spec.servingCerts.namedCertificates[0].servingCertificate.name}')
## 
crt_api_base64=$(oc -n openshift-config get secret ${api_cert_secret_name} -ojsonpath='{.data.tls\.crt}')
echo $crt_api_base64

2. Apply new Cert in every managed cluster

#1. download current kubeconfig
oc -n open-cluster-management-agent extract secret/bootstrap-hub-kubeconfig --to=.

# Here update kubeconfig and namely "certificate-authority-data" line with the new CA certificate (same value ${crt_api_base64})

#3. Upload new kubeconfig 
oc -n open-cluster-management-agent set data secret/bootstrap-hub-kubeconfig --from-file=kubeconfig

# 4. restart agents 
oc -n open-cluster-management-agent rollout restart deploy