Klusterlet registration pod crashlooping during clusteradm join for Oracle Kubernetes Engine cluster

[mikeshng]

When a user try to get a manage cluster to join, the klusterlet-registration-agent goes into CrashLoopBackOff for an Oracle Kubernetes Engine cluster.

When looking at the logs:

builder.go:230] unable to get owner reference (falling back to namespace): pods is forbidden: User "system:serviceaccount:open-cluster-management-agent:klusterlet-registration-sa" 
cannot list resource "pods" in API group "" in the namespace "open-cluster-management-agent"

It seems like the error might be related to the usage of the code here
https://github.com/openshift/cluster-etcd-operator/blob/release-4.13/vendor/github.com/openshift/library-go/pkg/operator/events/recorder.go#L52-L56

It assumes that the POD_NAME env var is populated and if it isn't then it will try to inspect the POD itself to find whatever information it needs. Since the klusterlet agent rbac does not include pod related accesses so this error occurs and blocks the cluster from joining. The current workaround is adding a new role/rolebinding and the error is bypassed.

kubectl version: v1.23.7+1
clusteradm version:
client          version :v0.3.1
server release  version :v1.23.4
default bundle  version :0.8.0
Kubernetes version: v1.23.4
Distribution: Oracle Kubernetes Engine

Reported by @hyder

[qiujian16]

it will not cause the crashloopback. if the pod name can not be found, namespace based event will be used.

[hyder]

Hi @qiujian16

The crashloopback is actually happening and is only resolved when I add the RBAC recommended by @mikeshng.

[mikeshng]

@hyder if it's not too difficult to reproduce the issue, could you please recreate it and paste the log in here?

The error might not be coming from that library and somewhere else. I forgot how I found that library in the first place.

[hyder]

@mikeshng sure, I'll do that tomorrow.