v1.4.7

Changes by Kind

Bug or Regression

• Enable nf_conntrack (nf_conntrack_ipv4) module by default on all operating systems. This fixes an issue with pods unable to reach services running on a host on operating systems that are using the NFT backend. (#2283, @xmudrii)

Terraform Integration

AWS

• Remove defaulting for the Flatcar provisioning utility in example Terraform configs for AWS (defaulted to cloud-init by machine-controller) (#2286, @xmudrii)

Checksums

SHA256 checksums can be found in the kubeone_1.4.7_checksums.txt file.

v1.5.0-beta.0

Changelog

The complete changelog since the v1.4.0 release is available in CHANGELOG/CHANGELOG-1.5.md.

Urgent Upgrade Notes

• Automatically apply the node-role.kubernetes.io/control-plane taint to nodes running Kubernetes 1.24. The taint is also applied when upgrading nodes from Kubernetes 1.23 to 1.24. You might need to adjust your workloads to tolerate the node-role.kubernetes.io/control-plane taint (in addition to the node-role.kubernetes.io/master taint). Workloads deployed by KubeOne will be adjusted automatically. (#2019, @xmudrii)
• Kubeadm is now applying the node-role.kubernetes.io/control-plane label for Kubernetes 1.24 nodes. The old label (node-role.kubernetes.io/master) will be removed when upgrading the cluster to Kubernetes 1.24. All addons are updated to use the node-role.kubernetes.io/control-plane label selector instead. All addons now have toleration for node-role.kubernetes.io/control-plane taint in addition to toleration for node-role.kubernetes.io/master taint. If you are overriding addons, make sure to apply those changes before upgrading to Kubernetes 1.24. (#2017, @xmudrii)
• Operating System Manager is enabled by default and is responsible for generating and managing user-data used for provisioning worker nodes
  • Existing worker machines will not be migrated to use OSM automatically. The user needs to manually rollout all MachineDeployments to start using OSM. This can be done by following the steps described in Rolling Restart MachineDeploments document
  • The user can opt-out from OSM by setting .operatingSystemManager.deploy to false in their KubeOneCluster manifest. (#2157, @ahmedwaleedmalik)
• workers_replicas variable has been renamed to initial_machinedeployment_replicas in example Terraform configs for Hetzner (#2115, @adeniyistephen)
• Change default instance size in example Terraform configs for Equinix Metal to c3.small.x86 because t1.small.x86 is not available any longer. If you're using the latest Terraform configs for Equinix Metal with an existing cluster, make sure to explicitly set the instance size (device_type and lb_device_type) in terraform.tfvars or otherwise your instances might get recreated (#2054, @xmudrii)
• Update secret name for backup-restic addon to kubeone-backups-credentials. Manual migration steps are needed for users running KKP on top of a KubeOne installation and using both backup-restic addon from KubeOne and s3-exporter from KKP. Ensure that the s3-credentials Secret with keys ACCESS_KEY_ID and SECRET_ACCESS_KEY exists in kube-system namespace and doesn't have the label kubeone.io/addon:. Remove the label if it exists. Otherwise, s3-exporter won't be functional. (#1880, @ahmedwaleedmalik)

Known Issues

• Calico VXLAN addon has an issue with broken network connectivity for pods running on the same node. If you're using Calico VXLAN, we recommend staying on KubeOne 1.4 until the issue is not fixed. Follow #2192 for updates.

Checksums

SHA256 checksums can be found in the kubeone_1.5.0-beta.0_checksums.txt file.

v1.4.6

Changes by Kind

Feature

• Add missing snapshot controller and webhook for OpenStack Cinder CSI (#2218, @xmudrii)
• Rollout pods that are using kubeone-*-credentials Secrets if credentials are changed (#2216, @xmudrii)

Updates

• Update containerd to v1.5. Escape docker/containerd versions to avoid wildcard matching (#2228, @xmudrii)
• Update Canal to v3.22.4 (#2189, @xmudrii)
• Update OpenStack CCM and Cinder CSI to v1.23.4 for Kubernetes 1.23 clusters (#2186, @xmudrii)
• Update machine-controller to v1.43.6 (#2227, @xmudrii)
• Update machine-controller to v1.43.5 (#2210, @kron4eg)
• Update machine-controller to v1.43.4. This machine-controller release fixes an issue with finding Node objects by ProviderID (#2193, @xmudrii)

Bug or Regression

• Disable --configure-cloud-routes on Azure CCM to fix errors when starting the CCM (#2185, @kubermatic-bot)
• Force regenerating CSRs for Kubelet serving certificates after CCM is deployed. This fixes an issue with Kubelet generating CSRs that are stuck in Pending. (#2204, @xmudrii)
• Properly propagate external cloud provider and CSI migration options to OSM (#2203, @kubermatic-bot)
• Replace operator: Exists toleration with the control plane tolerations for metrics-server. This fixes an issue with metrics-server pods breaking eviction (#2206, @kubermatic-bot)
• Tenant ID or Name is not required when using application credentials (#2201, @ahmedwaleedmalik)

Checksums

SHA256 checksums can be found in the kubeone_1.4.6_checksums.txt file.

v1.4.5

Changes by Kind

Feature

• Add GCP Compute Persistent Disk CSI driver. The CSI driver is deployed by default for all GCE clusters running Kubernetes 1.23 or newer (#2141, @xmudrii)
• Migrate GCE standard default StorageClass to set volumeBindingMode to WaitForFirstConsumer. The StorageClass will be automatically recreated the next time you run kubeone apply (#2141, @xmudrii)

Bug or Regression

• Disable node IPAM in Azure CCM (#2107, @rastislavs)
• Disable preserveUnknownFields in all Canal CRDs. This fixes an issue preventing upgrading Canal to v3.22 for KubeOne clusters created with KubeOne 1.2 and older (#2105, @kubermatic-bot)
• Fix wrong maxPods value on follower control plane nodes and static worker nodes (#2128, @xmudrii)
• Set rp_filter=0 on all interfaces when Cilium is used. This fixes an issue with Cilium clusters losing pod connectivity after upgrading the cluster (#2108, @xmudrii)

Checksums

SHA256 checksums can be found in the kubeone_1.4.5_checksums.txt file.

v1.4.4

Changes by Kind

Feature

• Add MaxPods field to the KubeletConfig used to control the maximum number of pods per node (#2080, @xmudrii)
• Update machine-controller to v1.43.3 (#2080, @xmudrii)
• Add machineObjectAnnotations field to DynamicWorkerNodes used to apply annotations to resulting Machine objects. Add nodeAnnotations field to DynamicWorkerNodes Config as a replacement for deprecated machineAnnotations field (#2077, @xmudrii)
• Update Canal and Calico VXLAN addons to v3.22.2. This allows users to use kube-proxy in IPVS mode on AMD64 clusters running Kubernetes 1.23 and newer. It currently remains impossible to use kube-proxy in IPVS mode on ARM64 clusters running Kubernetes 1.23 and newer. (#2042, @kubermatic-bot)
• Update Terraform integration for Azure with new fields (#2085, @xmudrii)
• Update vSphere CCM to v1.23.0 for Kubernetes 1.23 clusters. Add support for Kubernetes 1.23 on vSphere (#2069, @xmudrii)

Bug or Regression

• Migrate AzureDisk CSIDriver to set fsGroupPolicy to File (#2086, @kubermatic-bot)

Checksums

SHA256 checksums can be found in the kubeone_1.4.4_checksums.txt file.

v1.4.3

Changes by Kind

Bug or Regression

• Add missing VolumeAttachments permissions to machine-controller (#2032, @kubermatic-bot)
• Provide registry configuration to kubeadm when pre-pulling images (#2028, @kron4eg)

Checksums

SHA256 checksums can be found in the kubeone_1.4.3_checksums.txt file.

v1.4.2

Attention Needed

This patch releases updates etcd to v3.5.3 which includes a fix for the data inconsistency issues reported earlier (https://groups.google.com/a/kubernetes.io/g/dev/c/B7gJs88XtQc/m/rSgNOzV2BwAJ). To upgrade etcd for an existing cluster, you need to force upgrade the cluster as described here. If you're running Kubernetes 1.22 or newer, we strongly recommend upgrading etcd as soon as possible.

Changes by Kind

Feature

• Domain is not required when using application credentials (#1938, @ahmedwaleedmalik)

Bug or Regression

• Bump flannel image to v0.15.1 (#1993, @ahmedwaleedmalik)
• Deploy etcd v3.5.3 for clusters running Kubernetes 1.22 or newer. etcd v3.5.3 includes a fix for [the data inconsistency issues announced by the etcd maintainers](https://groups.google.com/a/kubernetes.io/g/dev/c/B7gJs88XtQc/m/rSgNOzV2BwAJ. To upgrade etcd) for an existing cluster, you need to force upgrade the cluster as described here (#1953)
• Fixes containerd upgrade on deb based distros (#1935)
• Show "Ensure MachineDeployments" as an action to be taken only when provisioning a cluster for the first time (#1931)
• Update machine-controller to v1.43.2 (#2001, @kron4eg)
  • Fixes an issue where the machine-controller would not wait for the volumeAttachments deletion before deleting the node
  • Fixes an issue where masked services on Flatcar are not properly stopped when provisioning a Flatcar node

Checksums

SHA256 checksums can be found in the kubeone_1.4.2_checksums.txt file.

v1.3.5

Attention Needed

This patch releases updates etcd to v3.5.3 which includes a fix for the data inconsistency issues reported earlier (https://groups.google.com/a/kubernetes.io/g/dev/c/B7gJs88XtQc/m/rSgNOzV2BwAJ). To upgrade etcd for an existing cluster, you need to force upgrade the cluster as described here. If you're running Kubernetes 1.22 or newer, we strongly recommend upgrading etcd as soon as possible.

Changed

• Upgrade machine-controller to v1.37.3 (#1984)
  • This fixes an issue where the machine-controller would not wait for the volumeAttachments deletion before deleting the node.
• Deploy etcd v3.5.3 for clusters running Kubernetes 1.22 or newer. etcd v3.5.3 includes a fix for [the data inconsistency issues announced by the etcd maintainers](https://groups.google.com/a/kubernetes.io/g/dev/c/B7gJs88XtQc/m/rSgNOzV2BwAJ. To upgrade etcd) for an existing cluster, you need to force upgrade the cluster as described here (#1953)

Checksums

SHA256 checksums can be found in the kubeone_1.3.5_checksums.txt file.

v1.3.4

Attention Needed

This patch release enables the etcd corruption checks on every etcd member that is running etcd 3.5 (which applies to all Kubernetes 1.22+ clusters). This change is a recommendation from the etcd maintainers due to issues in etcd 3.5 that can cause data consistency issues. The changes in this patch release will prevent corrupted etcd members from joining or staying in the etcd ring.

Changed

• Enable the etcd integrity checks (on startup and every 4 hours) for Kubernetes 1.22+ clusters. See the official etcd announcement for more details. (#1928)
• Validate Kubernetes version against supported versions constraints. The minimum supported version is 1.19, and the maximum supported version is 1.22 (#1817)
• Fix AMI filter in Terraform configs for AWS to always use x86_64 images (#1692)

Checksums

SHA256 checksums can be found in the kubeone_1.3.4_checksums.txt file.

v1.4.1

Attention Needed

This patch release enables the etcd corruption checks on every etcd member that is running etcd 3.5 (which applies to all Kubernetes 1.22+ clusters). This change is a recommendation from the etcd maintainers due to issues in etcd 3.5 that can cause data consistency issues. The changes in this patch release will prevent corrupted etcd members from joining or staying in the etcd ring.

Changes by Kind

Bug or Regression

• Regenerate container runtime configurations based on kubeone.yaml during control-plane upgrades on Flatcar Linux nodes, not only on the initial installation. (#1918)
• Approve pending CSRs when upgrading control plane and static worker nodes (#1888)
• Enable the etcd integrity checks (on startup and every 4 hours) for Kubernetes 1.22+ clusters. See the official etcd announcement for more details. (#1909)
• Fix CSR approving issue for existing nodes with already approved and GCed CSRs (#1897)
• Fix missing snapshot CRDs for Openstack CSI (#1913)
• Ensure old machine-controller MutatingWebhookConfiguration is deleted (#1913)
• Fix overwriteRegistry not overwriting the Kubernetes control plane images (#1885)
• Mount /usr/share/ca-certificates to the OpenStack CCM pod to fix the OpenStack CCM pod CrashLooping on Flatcar Linux (#1905)
• Fix the GoBetween script failing to install the zip package on Flatcar Linux (#1905)
• Expand path to SSH private key file (#1859)
• Fix an issue with kubeone config migrate failing to migrate configs with the containerRuntime block (#1861)

Checksums

SHA256 checksums can be found in the kubeone_1.4.1_checksums.txt file.