Rotate encryption (at rest) key (secret box provider)

[crtvmn]

Hello,

I would like to now if it’s possible to rotate the encryption key with the secretbox provider?

I did not use the rke2 secrets-encrypt command because the AES-CBC provider is not recommended for security reasons.

Unfortunately, the official Kubernetes procedure does not work: https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#rotating-a-decryption-key.
Indeed, even if the --encryption-provider-config-automatic-reload=true option is set in the Kube API Server pod, when the rke2-server restarts, the file is overwritten by the bootstrap config stored in ETCD.
I managed to rotate the key without restarting the kube-apiserver pod or the rke2-server service but after a RKE2 restart several hours later, the original file overwrote the new one I had modified. After this restart, I could no longer read secrets in my cluster (the old encryption key was no longer the one used by the kube-apiserver after having rotated it).

Is there a way to update this bootstrap config? Or another solution maybe?

Best regards,

Corentin

[brandond]

You can track this at

• Support secretbox for Secret Data at Rest #3508

[crtvmn]

So today, I am not able to rotate this encryption key

[crtvmn]

I also tried changing the path of the encryption file in the kube-apiserver pod-manifest to point on a new file with the old key and the new one. Everything works fine until restarting the RKE2 server because the kube-apiserver pod-manifest is automatically updated with the old path.

Is there a way to trick the rke2-server to prevent it from updating the kube-apiserver pod-manifest or my custom encryption provider config file?

[brandond]

Don't edit the pod manifests. When I said "point the apiserver at your own config file" I meant something like this in /etc/rancher/rke2/config.yaml:

kube-apiserver-args:
- "encryption-provider-config=/PATH/TO/FILE"

You shouldn't edit the static pod manifests, as you noticed they are managed by rke2 and changes will be reverted on startup.

[crtvmn]

Of course, sorry, I’m st*pid. I don’t know why I edited the pod-manifest. I will try this solution as soon as possible

[crtvmn]

Thank you very much for your help. It works if I change the encryption config file!