v2.23.1

Network

• [Cilium] Fix invalid hubble yaml if cilium_hubble_tls_generate is enabled (#10476, @toonalbers)

Feature

• Add hashes for kubernetes 1.27.6 & 1.26.9 (#10443, @bozzo)
• Make kubernetes v1.27.7 default (#10543, @mzaian)
• [etcd] Default version to 3.5.9 for k8s 1.25 , 1.26 , 1.27 (#10483, @mzaian)
• Add crictl 1.26.1 for Kubernetes v1.26 (#10562, @mzaian)
• Change default cri-o versions for Kubernetes 1.25, 1.26 (#10563, @mzaian)
• [ingress-nginx] Fix nginx controller leader election RBAC permissions (#10569, @mzaian)
• Refactor NRI activation for containerd and CRI-O (remove crio_enable_nri and containerd_nri_disable) now only one var nri_enabled default to false (#10496, @fmuyassarov)

Bug or Regression

• Fix get currently configured nameservers error where there are inline comments in /etc/resolv.conf (#10415, @yankay)
• Migrate node-role.kubernetes.io/master to node-role.kubernetes.io/control-plane (#10532, @unai-ttxu)
• [download] Don't fail on 304 Not Modified (#10559, @RomainMou)

v2.23.0

Deprecation / Removal

• Ubuntu 16 and 18 are no longer tested (#10107, @MrFreezeex)
• Drop support for ansible-core 2.11 and update tests dependencies (#10034, @MrFreezeex)
• Drop Kubernetes 1.24 support (#10234, @MrFreezeex)

Feature / Major Changes

• Make kubernetes v1.27.5 default (#10392, @mzaian)
• Add kubernetes v1.27.4 (#10359, @mzaian)
• Add Kubernetes 1.27.2 (#9976, @mzaian)
• Add hashes for 1.27.3 1.26.6, 1.25.11 (#10220, @mzaian)
• Add hashes for 1.27.4 1.26.7, 1.25.12 (#10300, @mzaian)
• Add CPU Management Policies on the Node (#10309, @yankay)
• Add Debian 12(bookworm) support (#10221, @tu1h)
• Add download.timeout to update download timeout value (#10149, @yjqg6666)
• Add corresponding coredns versions to all the supported kubernetes releases. (#10233, @mzaian)
• Add growpart azure enabled (#10241, @pedro-peter)
• Add ingressClass resource for ingress_nginx by default (#10091, @peschmae)
• Add kubelet topology manager policy on the node (kubelet_topology_manager_scope and kubelet_topoloy_manager_policy) (#10370, @tu1h)
• Add labels to kube-vip static pods (#10139, @liupeng0518)
• Add node_taints to aws_inventory script (#10170, @mstoetzer)
• Add option to set SSL_CERT_FILE for offline installation using custom CA for https proxy (#10215, @HappyFX)
• Add terraform support for NIFCLOUD (#10227, @ystkfujii)
• Add the huawei cloud controller as external cloud controller (#10198, @dabeck)
• Show detected ansible version when it isn't compatible with kubespray (#10109, @jcpunk)
• Allow to override etcd listen-metrics-urls configuration (using etcd_listen_metrics_urls variable) (#10332, @forselli-stratio)
• Don't let find search filesystem mounts in docker build run step (#10131, @tomodachi)
• Permit custom names for API server lb/proxy containers (#10166, @jcpunk)
• Permit skipping helm update (#10169, @jcpunk)
• Split defaults main file into 2 files (checksums and version) (#10121, @electrocucaracha)
• System upgrade for Debian-family nodes is available with system_upgrade=true (#10184, @sathieu)
• Update download_hash.sh script (#10120, @electrocucaracha)
• Use a uniform way to get the local path of the binaries (#10211, @ErikJiang)
• Disable fapolicyd service (#10081, @epif4nio)
• Upgrade the load balancer ( nginx and haproxy ) image version to Nginx 1.25, Haproxy 2.8. (#10409, @yankay)
• [etcd] Default version to 3.5.7 for kubernetes 1.27 (#10410, @mzaian)

Applications

• [argocd] update argocd to v2.7.4 (#10226, @mzaian)
• [argocd] update argocd to v2.8.0 (#10364, @mzaian)
• [argocd] Add argocd_install_url option to allow changing argocd url (#10176, @liupeng0518)
• [helm] upgrade to 3.12.1 (#10225, @mzaian)
• [helm] upgrade to 3.12.3 (#10365, @mzaian)
• [helm] add python dependency check for helm-apps (#10192, @palmeXx)
• [krew] add krew_no_upgrade_check (#10175, @liupeng0518)
• [coredns] Bump coredns version to 1.10.1 (#10199, @eminaktas)
• [coredns] Bump nodelocaldns version to 1.22.20 (#10200, @eminaktas)
• [cert-manager] This introduces a new variable for the cert-manager implementation that will allow one to pass in extra arguments to the cert-manager controller.(#10049, @phunyguy)
• Update Helm (v3.12.2) / Skopeo (v1.13.0) and yq (v4.34.2) (#10295, @tu1h)
• Upgrade many tool versions (Helm, crun, kata, youki, gvisor, skopeo, Calico, Cilium etc...) (#9798, @electrocucaracha)
• [local_path_provisioner] Fix invalid podhelper yaml (#10237, @MrFreezeex)
• Update metrics server to v0.6.4 (#10400, @mzaian)

Container-Managers

• [containerd] Make containerd 1.7.5 default (#10397, @mzaian)
• [containerd] Support containerd v1.7.2 (#10219, @Dentrax)
• [containerd] Support containerd 1.7.3 (#10368, @mzaian)
• [containerd] containerd config_path enable mirrors config using new variable containerd_registries_mirrors (deprecate and remove containerd_insecure_registries for containrd and nerdctl_extra_flags and insecure_registry setting for nerdctl (#10196, @yckaolalala)
• [crio] Add crio_insecure_registries option for specifying insecure_registries of crio (#10142, @qlijin)
• [crio] runroot now needs to be setup in storage.conf instead of crio.conf (#10372, @floryut)
• [crio] Fix etcdctl copy operation (#10242, @ErikJiang)
• [Kata] Set/keep owner/group root/root when unarchiving kata-containers (#10338, @rybnico)
• [youki] Fix youki binary download url (not requiring 'v' in version) (#10337, @ErikJiang)

Network

• [calico] Use configmap to configure calico cni config (#10177, @cyclinder)
• [calico] Update calico v3.25.2 (#10414, @mzaian)
• [calico] Add calico version to v3.26.0 (#10224, @mzaian)
• [calico] Add calico version to v3.26.1 (#10235, @mzaian)
• [calico] Clean up calicoctl_alternate_download_url and calicoctl.mirrors (#10271, @yckaolalala)
• [cilium] Add custom rules to clusterrole for cilium operator (#10267, @jeremythuon)
• [cilium] Upgrade to version 1.13.4 (#10269, @yulng)
• [Cilium] Do not mount tls when 'cilium_hubble_tls_generate' is false (#10357, @charlychiu)
• [Cilium] Update cilium to 1.13.3 (#10158, @jcpunk)
• [flannel] Only create /var/lib/calico when needed (#10156, @jcpunk)
• [flannel] Bump flannel version to v0.22.0 and flannel-cni-plugin version to v1.1.2. Also, changes flannel repository from flannelcni to flannel (#10205, @eminaktas)
• [flannel] Remove unused flannel_cni_download_url (#10188, @oomichi)
• [kube-ovn]: update version v1.11.5 (#10125, @yankay)
• [multus] Fix loop_control template error when item is None (#10347, @nicolas-goudry)

API Change

• Unless the pod security standard versions are changed on intentionally, as default it will be the same major version with Kubernetes version. (#10210, @ugur99)
• Upgrade ansible to 7.0 and ansible-core to 2.14.x (#10190, @MrFreezeex) ⚠️ (See Notes 2)

Documentation

• Add github container registry (github_image_repo) to docs/offline-environment.md (#10265, @blackliner)
• Update doc for ansible-core 2.14 support and clarify issues running older python versions (#10261, @MrFreezeex)
• Update links for aws_alb_ingress_controller (#10264, @kundan2707)
• Update links in ingress-controller and kuberentes-apps (#10239, @vaibhav2107)
• Update Calico to lowercase and fix broken calico link in README (#10232, @Xieql)
• Document containerd command to restart nginx-proxy container when adding control plane node (#10406, @nicolas-goudry)

Failing Test

• Increase metallb wait timeout from 30sec to 2min (#10260, @MrFreezeex)
• Update CentOS 7 image and test fedora 37 and 38 instead of fedora 35 and 36 (#10108, @MrFreezeex)

Bug or Regression

• Fix Dockerfile for newest directory layout (#10128, @dabeck)
• Fix Flatcar bootstrap issues (yaml module missing and ntp issue) (#10363, @tenni-paws)
• Fix argocd install not working using the kubespray docker image (#10371, @cortex3)
• Fix correctly mount ssl ca directories (#9794, @maxime1907)
• Fix etcdctl copy operation (#10230, @ErikJiang)
• Fix gce-pd-csi driver (#10208, @ashishsinghdev)
• Fix grep command without -w option causing prefix matched while adding one etcd member (#10291, @yangsenzk)
• Fix hcloud-cloud-controller-manager not working in certain setups (#10297, @cortex3)
• Fix helm (kubelet-csr-approver) installation on redhat distro (#10204, @MrFreezeex)
• Fix kubelet-csr-approver usage with upgrade-cluster.yml and missing package with helm role (#10165, @j4m3s-s)
• Fix nginxingress-class template (missing newline) (#10174, @richard-fairthorne)
• Fix problem migration problem with k8s 1.27 (#10136, @batazor)
• Fix reset_confirmation not working when inputing correct value (#10288, @somewho)
• Fix wrong path in manage-offline-files script (#9886, @Medosopher)
• Fix an issue where using Rocky Linux 8 as OS for Vagrant for testing purposes causing etcd to fail on start. (#10252, @nltimv)
• Fix ansible-lint galaxy rule (#10277, @MrFreezeex)
• Fix ansible-lint key-order error (#10314, @MrFreezeex)
• Fix outdated tag and experimental ansible-lint rules (#10254, @MrFreezeex)
• Fix dockerfile build error (#10127, @yankay)
• Fix metrics-server deployment to run with kubernetes 1.26+ (#10183, @mzaian)
• Fix undefined reset_confirmation_prompt variable in reset play (#10303, @Mishavint)
• Fix CIS Kubernetes V1.23 Benchmark item number 4.1.9 to enhance security (Change kubelet-config.yaml and kubelet.env file permissions from 640 to 600) (#10304, @satandyh)
• Fix parsing of RHSM proxy configuration (#10228, @tmurakam)
• Fix var-spacing ansible rule (#10266, @MrFreezeex)
• Fix specify owner to kube_owner in task of copy cni plugins (#10407, @NierYYDS)
• Fix typo kubelet_topoloy_manager_policy => kubelet_topology_manager_policy (#10384, @hangscer8)
• Fix recover_control_plane playbook (also add debian 12 with cilium as a new nightly test) (#10411, @floryut)
• Fix nameserver inline comments in /etc/resolv.conf (#10415, @yankay)
• Added systemd_resolved_disable_stub_listener variable to disable systemd-resolved's stub listener, defaults to true on Flatcar. (#9875, @cosandr)
• Remove auto_attach and syspurpose in RHEL subscription Organization ID/Activation Key registration. (#10258, @yckaolalala)
• Replace "crio_packages" with "crio_bin_files" (#10182, @yckaolalala)
• Update MetalLB deployment, wait for resource. (#9995, @Jeroen0494)
• Upgrade ansible to 7.0 and ansible-core to 2.14.x in Dockerfile (#10259, @yckaolalala)
• Fix typo kubelet_topoloy_manager_policy => kubelet_topology_manager_policy (#10384, @hangscer8) ⚠️ (See Notes 1)
• Change maximal_ansible_version to 2.15(exclusive) (#10395, @yankay)
• Install etcdutl file by default (#10385, @liupeng0518)

Other (Cleanup or Flake)

• [CI] Add CI VM for debian12 (#10222, @yankay)
• [CI] Removes Ansible reinstall from build pipeline (#10032, @luksi1)
• [CI] cleanup stale packet namespace automatically (#10245, @mrf...

v2.22.1

Bug or Regression

• Don't let find search filesystem mounts in docker build run step (#10131, @tomodachi)
• Fix Dockerfile for newest directory layout (#10128, @dabeck)
• Fix dockerfile build error (#10181, @yankay)
• Fix metrics-server deployment to run with kubernetes 1.26+ (#10183, @mzaian)
• update README for v2.22.0 (#10180, @Payback159)
• Fix Update MetalLB deployment, wait for resource. (#9995, @Jeroen0494)

v2.22.0

Deprecation / Removal

• [Cilium] Delete the probe option of cilium_kube_proxy_replacement (#9929, @XiuguangHuang)
• [Cilium] Remove use_localhost_as_kubeapi_loadbalancer and detect wether we can use localhost apiserver loadbalancer if cilium/calico replace kube-proxy (#9718, @MrFreezeex)
• Drop crun_bin_dir unused variable, now using only bin_dir var (#9845, @electrocucaracha)
• Drop the canal network_plugin support because the network_plugin is unmaintained. (#10100, @oomichi)
• Remove the support of Debian 9 (#10097, @yankay)
• Replaces storage.googleapis.com/kubernetes-release with dl.k8s.io (#10066, @KlwntSingh)

Feature / Major Changes

• Add Kubernetes 1.26.x (#9570, @mzaian ; #9732, @yankay; #9829, @mzaian; #9900, @mzaian)
• Make kubernetes v1.26.5 default (#9983, @mzaian)
• "native" snapshotter of nerdctl config is replaced by new var nerdctl_snapshotter with default "overlayfs" value (#9979, @dmitrytretyakov)
• Support multi-arch using the same image name (#9978, @ErikJiang)
• Add DNS configuration for cert-manager (using new variables cert_manager_dns_policy|config) (#9673, @ErikJiang)
• Add Retry for restart kube-controller-manager (#10013, @hangscer8)
• Add coredns_additional_configuration variable to define extra Coredns configurations (#10025, @navidnabavi)
• Add coredns_rewrite_block to perform internal message rewriting (#10045, @maxime1907)
• Add a new simple network_plugins custom_cni to install user provided manifests (#9819, @MrFreezeex)
• Add back openssh-client to docker image (#9835, @maxime1907)
• Add download retries option download_retries (#9911, @tu1h)
• Add support to install ContainerD on any Linux Distributions using new var allow_unsupported_distribution_setup (#9827, @XDRAGON2002)
• Add the kube-profile config to the kubeadm's kube-scheduler config. (#9993, @yankay)
• Add vim to kubespray docker image (#9805, @XDRAGON2002)
• Adds support for Kubelet-CSR-approver to auto-approve kubelet CSR when kubelet_rotate_server_certificates. (#9877, @j4m3s-s)
• Add dns_cpu_limit value to support large scaled coredns deployments (#10103, @mzaian)
• Add provider meta module_name in Equinix Metal TF configs (#10044, @Vasubabu)
• Allow to configure image garbage collection (using kubelet_image_gc_high_threshold and kubelet_image_gc_low_threshold) (#9832, @zhan9san)
• Apply kubeadm patches during upgrade as recommended by k8s (#9781, @mvandergiesen)
• Cinder-csi: Allow VolumeSnapshotClass' deletionPolicy to be configurable (#9736, @huangkevin404)
• Containerd add containerd_use_config_path config field. (#9770, @lengrongfu)
• Enable control plane load balancing for kube-vip (#9785, @ErikJiang)
• Feat(contrib/terraform): support custom ssh port (#9836, @maxime1907)
• Fix kube-bench 1.2.20 to enhance security (Ensure that the --audit-log-maxbackup argument is set to 10) (#9939, @yankay)
• Fix kube-bench 1.1.19 to enhance security (Change Kubernetes Cert directory and file ownership is set to root:root) (#9937, @yankay)
• Fix kube-bench 4.1.1 to enhance security (Change kubelet systemd init file from 644 to 600) (#9934, @yankay)
• Fix kubernetes-app/argocd: download related things with the download role (#9786, @pli01)
• Kube.py now supports kubeconfig (#9982, @liupeng0518)
• MetricsServer: Add extras nodeselector, affinity, tolerations (using metrics_server_nodeselector, metrics_server_extra_affinity ,metrics_server_extra_tolerations) (#9972, @pli01)
• Refactor Hetzner terraform (fixing flatcar configs and remove deprecated provider) (#10002, @ThisIsQasim)
• Support for MetalLB v0.13.9 with CRD (#9120, @Jeroen0494)
• Throw an error when specifying unsupported os in Vagrant (#9965, @THUzxj)
• Update CoreDNS manifests (remove deprecated annotations) (#9977, @mzaian)
• Update dns-autoscaler configuration and remove deprecated annotations (#9996, @mzaian)
• Update metrics server to v0.6.3 (#10026, @mzaian)
• Upgrade argocd to v2.6.3 (#9848, @panguicai008)
• Upgrades the following Python libraries to their latest available releases (cryptography / jinja2 / jmespath / MarkupSafe/ netaddr / pbr / ruamel.yaml / ruamel.yaml.clib) (#9938, @luksi1)
• Add IPv6 listen directive to haproxy if enable_dual_stack_networks (#9674, @yankay)
• Add support for Ansible collections in Kubespray (⚠️ See notes !) (#9582, @luksi1)
• Support mTLS for Hubble and upgrade backend to v0.11.0 (#9959, @jeremythuon)
• Update nodelocaldns to 1.22.18 (#9800, @sathieu)
• Replace disable_swap variable with kubelet_fail_swap_on (#10036, @Manuelraa)
• Replace nodelocaldns label to k8s-app: node-local-dns (#9745, @stelucz)
• Upgrade rancher local-path-provisioner to v0.0.23 (#9855, @panguicai008)
• Use kube_apiserver_address variable for advertiseAddress (#9967, @liupeng0518)
• Use string for ipv6 forward conf value (#9992, @liupeng0518)
• Update pause image version to v3.9 (#10112, @mzaian)
• Upgrade cni version to v1.3.0 (#10058, @cyclinder)
• [argocd] update argocd to v2.6.7 (#9953, @mzaian)
• [helm] support to 3.11.1 (#9849, @mzaian)
• [helm] support to 3.11.3 (#10022, @mzaian)
• [helm] support to 3.11.2 (#9951, @mzaian)
• [helm] upgrade to 3.12.0 (#10085, @mzaian)
• [UpCloud] Add server group support for vms and target port for loadbalancers (#9831, @robinAwallace)
• [argocd] update argocd to v2.5.10 (#9753, @yanggangtony)
• [cert-manager] Upgrade to v1.11.1 (#9964, @rtsp)
• [flannel] update to v0.21.4 (#10027, @mzaian)
• [nerdctl] support version 1.3.1 (#10024, @mzaian)
• [nerdctl] update to version 1.4.0 (#10119, @mzaian)

Applications

• [kube-vip] Support to v0.5.8 (#9734, @hangscer8)
• [kube-vip] Support kube-vip to v0.5.11 (#9852, @panguicai008)
• [kube-vip] Update default kube-vip to v0.5.12 (#10005, @hangscer8)
• [vSphere-csi] Add resources section to all containers releated to Vsphere CSI driver (#9687, @JRaver)
• [argocd] update argocd to v2.7.2 (#10086, @mzaian)

Container-Managers

• [containerd] Add hashes for containerd version 1.6.19 (#9838, @mzaian)
• [containerd] Add hashes for containerd version 1.6.20 (#9954, @mzaian)
• [containerd] Add hashes for containerd version 1.7.0 (#9892, @mzaian)
• [containerd] Add hashes for containerd versions 1.7.1, 1.6.21 (#10061, @mzaian)
• [containerd] Support version 1.6.16 (#9727, @yanggangtony)
• [cri-o] Bump versions to 1.26.3, 1.25.3, 1.24.5 (#9999, @dkasanic)
• [cri-o] Fix install order -> first runc then crictl (#9780, @mvandergiesen)
• [cri-o] Fix missed double quotes in cri-o config (#10040, @turbosnail)
• [cri-o] Fix CRI-O amd64 v1.26.0 wrong archive checksum (#9872, @panguicai008)
• [cri-o] cri-o restart if config change (#10057, @MrFreezeex)
• [cri-o] Remove deprecated crio_pids_limit (default is now unlimited) (#10056, @j4m3s-s)
• [cri-o] Fix cri-o restart if config change (#10057, @MrFreezeex)
• [runc] Upgrade to v1.1.7 (#10039, @pomland-94)

Network

• [Calico] Add Retry and Ignore Error for Checking calico ready (#9883, @hangscer8)
• [Calico] Add option calico_kubeconfig_wait_timeout (#9994, @tu1h)
• [Calico] Improve version check command (#9861, @zhan9san)
• [Calico] Optimize the detection of calico existence (#9873, @hangscer8)
• [Calico] Support calico version v3.25.0 (#9860, @cyclinder)
• [Calico] upgrade default calico version to v3.25.1 (#9950, @mzaian)
• [Calico] Add missing ipamconfigs resource in RBAC (#9755, @chaunceyjiang)
• [Calico] Fix installation while applying CRD (#10068, @hangscer8)
• [Calico] Add calico version to v3.24.6 (#10113, @mzaian)
• [Cilium] Add and support v1.13.0 (#9879, @utam0k)
• [Cilium] Fix Hubble relay configuration (#9876, @prashantchitta)
• [Cilium] Fix the configuration of TLS for hubble (#9880, @utam0k)
• [Cilium] Remove duplicates in the configuration of tls for hubble (#9932, @CaMoPeZzz)
• [Cilium] Support version above 1.13.x (#9914, @wbh1)
• [Cilium] Updates hubble certgen arguments (wrong since v0.1.7) (#9856, @XDRAGON2002)
• [Cilium] IPAM uses "Cluster Scope" mode by default. Also add the parameters required for this mode (#9443, @dcwbq)
• [flannel] Update image repo from flannelcni to flannel (#10041, @ErikJiang)
• [multus] fix multus include error (#10105, @darkobas2)

API Change

• Openstack cloud controller manager bind address is now configurable using external_openstack_cloud_controller_bind_address (#9958, @dominykasn)

Documentation

• Add a mention for custom_cni in CNI list (#9878, @j4m3s-s)
• ArgoCD no longer uses the pod name as initial password (#9930, @peschmae)
• Drop remaining part for supporting ansible 2.9 and 2.10 (#9842, @oomichi)
• Fix sidebar documentation (#9988, @lijin-union)
• Fixup link in docs/calico.md (#9940, @kundan2707)
• Remove stale contents for cni documention (#9778, @tu1h)
• Reword confusing etcd download url comment when etcd_deployment=host (#9686, @tjanson)
• Suggest to run reset.yml playbook for first-time users (#9865, @kerryeon)
• Update docker tag to v2.21.0 in README.md (#9802, @Payback159)
• Update link for baremetel consideration (#9944, @kundan2707)
• Add port requirements documentation (#9969, @yankay)

Failing Test

• Update Terraform to 1.3.7 and Vagrant to 2.3.4 (#9699, @floryut)
• [CI] Migrate CI_BUILD_ID to CI_JOB_ID and CI_BUILD_REF to CI_COMMIT_SHA following gitlab upgrade (#10063, @floryut)

Bug or Regression

• Add PSS labels to metallb namespace (#9713, @manzsolutions-lpr)
• Add jmespath back to Dockerfile image (#9697, @floryut)
• Add missing krew_download_url to offline.yml (#9788, @jianse)
• Add proxy_env variable to apt_key cleanup task (#9766, @SamuelBECK1)
• Add rsync in Dockerfile (#9839, @zhan9san)
• Add ruamel.yaml back to Dockerfile image (#9707, @floryut)
• Cleanup MetalLB install following update (#10004, @eugene-marchanka)
• Copy contrib/ to Dockerfile (#9774, @oomichi)
• Downgrade the version of CoreDNS to 1.8.6 for co...

v2.21.0

Deprecation / Removal

• Drop calico v3.21 support (#9515, @oomichi)

Feature / Major Changes

• Add Check resolv.conf is empty to avoid CoreDNS crash (#9502, @yankay)
• Add XDG related Helm paths to be removed from reset tasks (#9561, @emiran-orange)
• Add a parameter (disable_host_nameservers) to disable host nameservers (#9357, @eminaktas)
• Add an option (populate_loadbalancer_apiserver_to_hosts_file) to skip adding load balancer name in the hosts file (#9331, @JRaver)
• Add custom options to coredns kubernets plugin (coredns_kubernetes_extra_opts ) (#9608, @mvandergiesen)
• Add docker support for openEuler linux (#9498, @ErikJiang)
• Add support for the OpenEuler Linux (#9494, @ErikJiang)
• Add terraform script for Flatcar Linux on Hetzner (#9618, @florianow)
• Add the ability to define options for DNS upstream servers (using new variable dns_upstream_forward_extra_opts) (#9311, @emiran-orange)
• Add var (ingress_nginx_probe_initial_delay_seconds) for control initialDelaySeconds in ingress-nginx probes (#9405, @zvlb)
• Add variable condition snapshot in vSphere CSI (vsphere_csi_block_volume_snapshot) (#9429, @yanggangtony)
• Add variable in metrics_server deployment (metrics_server_replicas) to enable HA mode (#9539, @ugur99)
• Change dns upstream condition for nodelocaldns when using host_resolvconf (#9378, @unai-ttxu)
• Download coredns image to all hosts in k8s_cluster (#9316, @joes)
• Enable check mode in DNS Cleanup tasks (#9472, @emiran-orange)
• Etcd image has the same tag accross multiple archs (#9516, @hangscer8)
• Fix a pre-upgrade node drain rescue task failure when kube_override_hostname is set (#9556, @chadswen)
• Fix default value for kubelet_secure_addresses (#9355, @willtrnr)
• Provides <kubeadm_init_timeout> to change the timeout of first control-plane initialization (#9617, @tu1h)
• Remove PodSecurityPolicies in MetalLB for kubernetes 1.25 (#9442, @yanggangtony)
• Support Python 3.11 - ruamel.yaml.clib need to be updated to 0.2.7 (#9426, @olivierlemasle)
• Support customize the additional sysctl variables using additional_sysctl (#9351, @yankay)
• Support patches field in kubeadm v1beta3 in both InitConfiguration and JoinConfiguration (using new variable kubeadm_patches) (#9326, @titaneric)
• Switch helm install (from synchronize to copy) to support password authentication (#9343, @ghostloda)
• Update api version for pdb and batch (deprecated in 1.25) (#9369, @yankay)
• Update dashboard image repo to remove arch flag (#9530, @tu1h)
• Update etcd log-level parameter name (new name: ETCD_LOG_LEVEL) (#9540, @ErikJiang)
• Update local-volume-provisioner to 2.5.0 + add documentation (#9463, @olivierlemasle)
• Update the number of nofile limits in containerd to 65535 (#9507, @ErikJiang)
• Upgrade metrics server to v0.6.2 (#9554, @mzaian)
• Upgrade the load balancer ( nginx and haproxy ) image version. (#9506, @yankay)
• Use kube_apiserver_port variable instead of hard-coding 6443 (#9620, @huangkevin404)
• [etcd] Default version to 3.5.5 for k8s 1.25.x (#9419, @mzaian)
• Update CoreDNS version to v1.9.3 (#9503, @yankay)
• Add the possibility to specify extra domains for the coredns kubernets plugin (using coredns_kubernetes_extra_domains) (#9635, @mvandergiesen)
• Streamline ansible_default_ipv4 gathering loop (#9281, @rptaylor)
• Update kubernetes dashboard to 2.7.0 (k8s 1.25 support) (#9425, @mzaian)
• Skip retry operation with containerd when etcd installed on host VM (#9560, @JRaver)
• Update pause image version to v3.8 (#9668, @mzaian)
• Enable kubelet_authorization_mode_webhook back by default and remove extra role (#9662, @MrFreezeex)
• Terraform gcp can now have extra ingress firewall rules, using new variable extra_ingress_firewalls (#9658, @sathieu)
• kubeadm/etcd: use config to download certificate (#9609, @MrFreezeex)

Applications

• [argocd] update argocd to v2.5.5 (#9604, @mzaian)
• Upcloud: Reclaim policy for PV is now delete (#9574, @robinAwallace)
• [Exoscale] Add missing zone input variable (#9495, @ayoubeddafali)
• [MetalLB] Avoid MetalLB speaker image download when MetalLB speaker is disabled (#9248, @unai-ttxu)
• [Openstack] Replace deprecated "template" Terraform provider with supported "cloudinit" Terraform provider (#9536, @inflatador)
• [OpenStack] Updated openstack cloud controller to version v1.25.3 (#9500, @robinAwallace)
• [Openstack] Add bastion_allowed_ports to allow custom security group rules on bastion node (#9336, @bl0m1)
• [Openstack] Upgrade 1.22.0 to 1.23.4 (#9332, @QcFe) (See Notes 1)
• [Openstack] Added override variable, additional server groups and cloudinit config (#9452, @Xartos)
• [cinder-csi-nodeplugin] Remove the pods-cloud-data volume (delete upstream) (#9362, @huangkevin404)
• [vsphere-csi] Add missing defaults for external_vsphere_* variables in the csi_driver/vsphere role (#9664, @rlacko58)
• [hetzner] In config, rename ansible groups to use _ instead of - (#9569, @ym)
• [kube-vip] Minor changes on Kube VIP configuration parameters (and fix wrong properties) (#9414, @woutergd)
• [cert-manager] Upgrade to v1.10.1 (#9512, @rtsp) then v1.11.0 (#9661, @mzaian)
• [helm] upgrade to 3.10.3 (#9605, @mzaian)
• [ingress-nginx] upgrade to 1.5.1 (#9532, @mzaian)
• [vSphere] Removing unneeded terraform dependencie & mark vsphere_password as sensitive (#9672, @sathieu)

Container-Managers

• Optimize cgroups settings for node reserved (using new kube_reserved, see docs for more information) (#9209, @shelmingsong)
• [Docker] Update docker package to 20.10.20 (partial fix for CVE-2022-39253) (#9410, @floryut)
• [containerd] Add support for 1.6.11 (#9544, @yanggangtony)
• [containerd] Added variables for unpriviledged ports and icmp (#9517, @Xartos)
• [containerd] Allow containerd-common to execute multiple times per play (#9543, @chadswen)
• [containerd] Newly started containers will be limited to 16384 open files. To change this number, set containerd_base_runtime_spec_rlimit_nofile, or remove base_runtime_spec from runc runtime to revert to previous behaviour. (#9319, @fungusakafungus)
• [containerd] Support v1.6.13 and v1.6.14 (#9585, @yanggangtony)
• [containerd] Add config_path var in config.toml.j2 file (#9566, @lengrongfu)
• [containerd] Add hashes for containerd versions 1.5.14 , 1.5.15 , 1.5.16 (#9678, @yanggangtony)
• [cri-o] Use cri-o from upstream instead of kubic/OBS (#9374, @cristicalin)
• [nerdctl] upgrade to version 1.0.0 (#9424, @mzaian)

Network

• Bump cni-plugins version to v1.2.0 (#9671, @cyclinder)
• Fix remove Cilium CNI failed because the CNI bin dependency (#9563, @yankay)
• [Calico] Add cni bin when installing (#9367, @ErikJiang)
• [Calico] Add retry for start calico kube controller (#9450, @cleverhu)
• [Calico] Adjust calico-kube-controller pod to non hostNetwork pod (#9465, @cyclinder)
• [Calico] Adjust calico-kube-controller pod to use hostnetwork if using etcd (#9573, @JSpon)
• [Calico] Disable 'Check that IP range is enough for the nodes' (#9491, @mzaian)
• [Calico] Update the tag image to support multiple architectures with the same tag (#9529, @ErikJiang)
• [Calico] remove deprecated PodSecurityPolicy (removed in Kubernetes in v1.25) (#9395, @yankay)
• |Calico] Allow user to set env: FELIX_MTUIFACEPATTERN in calico-node.yml (using calico_felix_mtu_iface_pattern) (#9330, @shelmingsong)
• [Calico] Replace node-role.kubernetes.io/master with control-plane (#9627, @my-git9)
• [Calico] upgrade default calico version to v3.24.5 (#9580, @yankay)
• [Calico] Add vxlan-v6.calico to the list of NetworkManager unmanaged interfaces (#9631, @cyclinder)
• [Calico] Add retry to avoid 'unknown' state for calicoctl (#9633, @tu1h)
• [Calico] Update Calico VXLAN offload docs because Calico changed the default value (#9639, @yankay)
• [Calico] Add possibility to enable calico floatingIPs feature (using calico_felix_floating_ips) (#9680, @MatthieuFin)
• [Cilium] Add download configuration for cilium hubble images (using cilium_enable_hubble variable) (#9376, @ErikJiang)
• [Cilium] Add switch cilium_enable_bandwidth_manager (#9441, @dcwbq)
• [Cilium] Cleanup cilium-init image from cilium template (#9508, @ErikJiang)
• [Cilium] update cilium cli offline download url example (#9458, @cleverhu)
• [Cilium] Install Cilium CLI alongside Cilium (#9436, @dcwbq)
• [flannel] Initcontainer image now correctly support architecture suffix (#9461, @rollandf)
• [flannel] Upgrade version to v0.20.1 (#9528, @ErikJiang)
• [flannel] remove deprecated PodSecurityPolicy (removed in Kubernetes in v1.25) (#9365, @yankay)
• [flannel] Add wireguard encryption backend as option (#9583, @janaurka)
• [flannel] Support dual stack IPv4 & IPv6 networking (#9564, @styshoo)
• [flannel] Allow setting the DirectRouting option on VXLAN (#9438, @willtrnr)
• [flannel] update to v0.20.2 & make it default (#9675, @mzaian)
• [kube-ovn] Update version to v1.10.7 (#9527, @liupeng0518)
• [kube-ovn] Remove kube-ovn log directories when reseting (#9625, @JochenFriedrich)
• [kube-ovn] Remove ovn.kubernetes.io/ovs_dp_type from nodeSelector (#9594, @JochenFriedrich)
• [kube-ovn] Support OVN Interconnect (#9599, @JochenFriedrich)
• [multus] added support for mixed type of container engine (#9224, @mr-yaky)

Bug or Regression

• Change include to import_playbook in recover_control_plane playbook, to support ansible 2.12+ (#9576, @floryut)
• Corrected vsphere directory in docs (#9534, @wojciehm)
• Deleting worker nodes is now skipped if there is no kube_control_plane node. (#9430, @kerryeon)
• Etcd arch can now support arm64 and amd64 (#9421, @yanggangtony)
• Fix cert-manager deployment on hardening environments (#9404, @oomichi)
• Fix checksum of ciliumcli v0.12.5 for arm64 (#9614, @oomichi)
• Fix inconsistent handling of admission plugin list (kube_apiserver_enable_admission_plugins must be ...

v2.20.0

Deprecation / Removal

• Drop Ansible support for v2.9 and v2.10 (#8925, @oomichi)
• Drop support for Fedora 34 (#8967, @floryut)

Feature / Major changes

• Add Rocky Linux 8 support (#8905, @oomichi)
• Add Kylin Linux support. (#9078, @ErikJiang)
• Add Fedora36 support (#8967, @floryut)
• Add 'flush ip6tables' task in reset role (#9168, @GreatLazyMan)
• Add tar in common required package (#9184, @yankay)
• Add support for NTP configuration. (#9027, @yankay)
• Increase ansible fact_caching_timeout (from 2 to 24 hours) (#9059, @rptaylor)
• Add kubelet systemd service hardening option kubelet_systemd_hardening: [true|false] (#9194, @alegrey91)
• Support timezone setting (#9263, @yankay)
• Update deprecated ansible include syntax (#9040, @boeto)
• Update etcd download url in offline.yml to use arch (#8943, @ErikJiang)
• Add Support for Rewrite Plugin to CoreDNS/NodelocalDNS (#9245, @eifelmicha)
• Add SeccompDefault admission plugin for kubelet (using new variable kubelet_seccomp_default) (#9074, @alegrey91)
• Add an optional extra_groups parameter for k8s_nodes (e.g. to configure calico route reflector nodes on Openstack using the calico_rr group) (#9211, @rptaylor)
• Add arm64 Flatcar OS's pypy bootstrapping support (#8959, @kerryeon) (see Notes 1)
• Add docker support for Kylin distributions (#9144, @ErikJiang)
• Add hashes for Kubernetes 1.24.3 , v1.22.12, v1.23.9 (#9092, @marcofortina)
• Add ingress nginx webhook (#9033, @liupeng0518)
• Add manage-offline-files.sh to collect necessary files and provides http file download service for offline deployment. (#8956, @ErikJiang)
• Add missing configuration for extra tolerations (#8908, @smasset)
• Add support for node & pod pid limits (in kubelet-config file) (#9038, @h9-HSFRQDH)
• Add the option to enable default Pod Security Configuration (#9017, @Foxlik)
• Add unsafe_show_logs switch to show more log details (default to false, same as previous behavior) (#9164, @ErikJiang)
• Add variables (delete_node_retries,delete_node_delay_seconds) to tweak remove node process (#9096, @ydFu)
• Added 'avoid-buggy-ips' support of MetalLB (metallb_avoid_buggy_ips for default IP address pool and avoid_buggy_ips for additional IP address pools defined in metallb_additional_address_pools) (#9166, @kerryeon) (see Notes 2)
• Adjust the default value of calico blockSize ipv4 to 26, and ipv6 to 122. (#9055, @cyclinder)
• Make kubernetes owner parametrized (using kube_owner/kube_cert_group/etcd_owner variables) (#8952, @alegrey91)
• Move old etcd backup removal after etcd restart, to prevent removing backup if etcd fail (#9147, @emiran-orange)
• Supports reserve ephemeral-storage (#8895, @Thearas)
• [dev/docs] add support for pre-commit hook (#9158, @cristicalin)
• [etcd] Etcd role won't run on all nodes everytime. (#9173, @liupeng0518)
• [etcd] add 3.5.4 and drop 3.5.1 and 3.5.2 (#9021, @cristicalin)
• [infra] bump pause container to 3.6 (#9024, @cristicalin)
• Update Kubernetes dashboard to 2.6.0 (k8s 1.24 support) (#8906, @floryut)
• [kubernetes] make 1.24.x the new default (#8935, @cristicalin)
• [kubernetes] drop support for 1.21.x (#8935, @cristicalin)
• [kubernetes] drop support for deprecated dynamic_kubelet_configuration (#8935, @cristicalin)
• [offline] Archive offline-files and env NO_HTTP_SERVER to skip Nginx container running. (#9068, @yjqg6666)
• Adds support for multiple architectures to yq (#9288, @ErmalKristo)
• Add variable to tweak the vsphere-csi namespace (vsphere_csi_namespace) (#9278, @MahdiAbbasi95)
• Ensure ping package is installed on the system (#9284, @yankay)
• Add more functionalty to DNS configuration (#9270, @eminaktas)
• Ensure ostree variable has been defined for fcos (#9321, @electrocucaracha)
• Support removing options in resolvconf with tab separator (#9304, @2k0ri)
• preinstall: Add nodelocaldns to supersede_nameserver if enabled (#9282, @azuwis)

Network

• [Calico] calico rr now supports multiple groups (#9134, @liupeng0518)
• [Calico] drop support for 3.19.x and 3.20.x
• [Calico] Make Calico CNI log path configurable and allow disabling this log (#8921, @fungusakafungus)
• [Calico] The NAT (nat_outgoing) would not be disabled automatically when enabling peer_with_router. (#9255, @kerryeon)
• [Calico] The variable calcio_ipam_autoallocateblocks has been renamed to calico_ipam_autoallocateblocks (#9056, @liupeng0518)
• [Calico] calico-typha metrics port are now exposed when metrics are enabled (#8855, @vjacynycz)
• [Calico] Add Wireguard support for Rocky Linux 9 (#9287, @krystianmlynek)
• [Calico] The parameter name calcio_rr_id Is renamed to calico_rr_id for fixing a typo ⚠️ (#9327, @kerryeon)
• [Canal] update templates to work again with both etcd and k8s datastore (#9113, @floryut)
• [Cilium] Add list/watch nodes rules to cilium-operator clusterrole. (#9178, @Thearas)
• [Cilium] Add support for the updated (startup|liveness|readiness)Probe.Port numbers (#9031, @tomberget)
• [Cilium] Update cilium to v1.11.7 (#9119, @dkhachyan)
• [Cilium] Make rolling-restart readiness wait delay and count configurable via cilium_rolling_restart_wait_retries_{count, delay_seconds} (#9176, @Tristan971)
• [Cilium] Upgrades cilium to 1.11.6 and add some default variables. (#9065, @eminaktas) (See Notes 3)
• [Cilium] Update Cilium default to 1.12.x (#9225, @necatican) (See Notes 5)
• [Cilium] Dropped support for < v1.10.0 (#9225, @necatican)
• [Cilium] cilium_ip_masq_agent_enable variable no longer exists. Use enable-ipv4-masquerade and enable-ipv4-masquerade to enable masquerade. (#9225, @necatican)
• [flannel] update to v1.18.1 & make it default (#9104, @mzaian)
• [flannel] update to v1.19.2 & make it default (#9296, @mzaian)
• [Kube-vip] Fail if kube_proxy_strict_arp is set to false in arp mode (#9223, @yankay)
• [Multus] Support multi-architecture installation (#9012, @cyclinder)

Applications

• [Openstack] Add option to use default deny firewall policy and port allowlisting on UpCloud (#9058, @Ajarmar)
• [Openstack] Fix subnet order and number of master nodes (#9159, @robinelastisys)
• [Metallb] Renamed matallb_auto_assign variable to metallb_auto_assign (users disabling 'auto-assign' in metallb must update the variable name) (#8949, @orange-llajeanne)
• [vSphere-csi] Add nodeAffinity to daemonset using vsphere_csi_node_affinity variable (#9293, @dmitrytretyakov)
• [upcload-csi] Bump driver version to v0.3.3 (#9317, @robinAwallace)

Container-Managers

• [containerd] add hashes for 1.5.12, 1.5.13, 1.6.5 and 1.6.6, make 1.6.6 the new default (#8980, @cristicalin)
• [containerd] Add LimitMEMLOCK parameter configuration in containerd.service (using containerd_limit_[proc_num/core/open_file_num/mem_lock) (#9269, @ErikJiang)
• [containerd] Remove duplication in containerd template (#9301, @fungusakafungus)
• [containerd] Allow configuring base_runtime_spec per containerd runtime and supply a default runtime spec (#9302, @fungusakafungus)
• [Docker] use cri-dockerd instead of dockershim by default
• [Docker] Enable cri-dockerd service to prevent issue with reboot (#9201, @mostafaghadimi)
• [cri-o] Add dpkg hold for apt installs (#9075, @SamuelBECK1)
• [cri-o] add support for 1.24.x required by kubernetes 1.24.x (#8935, @cristicalin)
• [runc] update versions for 1.1.x and drop 1.0.x (#9022, @cristicalin)
• [runc] Variable containerd_default_runtime is now undifined by default (but default to runc) (#9026, @rptaylor)
• [crun] add 1.4.5 and drop 1.2 and 1.3 (#9023, @cristicalin)
• [nerdctl] upgrade to 0.20.0 (#8980, @cristicalin) then 0.22.2 (#9180, @panpan0000)

Bug or Regression

• Fix failure to look up user etcd when adding a user (#9016, @yankay)
• Fixing setting up kubespray on Azure with CSI drivers. (#9153, @wayfrro)
• Add --supervisor-fss-namespace=kube-system flag to vcloud-csi installation (#9066, @yasintahaerol)
• Add assertion for IPv4 check in verify settings (to allow IPv6 deployments) (#8946, @Citrullin)
• Add calico-kube-controllers missing verbs (#9032, @ghostloda)
• Allow "openSUSE Tumbleweed" to be run (again) (#9072, @oomichi)
• Apply calico bgp peer definition task to all nodes (#8974, @orange-llajeanne)
• Create snapshot namespace only when needed (#9014, @robinAwallace)
• Disable kubelet_authorization_mode_webhook by default (#9238, @cristicalin)
• Disabled DNSStubListener for Flatcar Linux (#9160, @kerryeon)
• Do not run etcd role in scale.yml playbook when etcd installed by kubeadm (#9210, @LuckySB)
• Fix Hetzner CCM cluster-cidr (wrongly set to a static value) (#9127, @ym)
• Fix calicoctl.sh path error when getting calico configuration (#9217, @tasekida)
• Fix failing tasks when calico_datastore is set to etcd (#9228, @chadswen)
• Fix missing quote in task "See if node is schedulable" (#9146, @emiran-orange)
• Fix number node name can't be added. (#9266, @cleverhu)
• Fix regex for replacing http_proxy host in RedHat Subscription Manager (#8957, @dicksontung)
• Fix some docker reset task (don't remove already uninstalled packages, ignore error on remove docker config files if already removed) (#8966, @orange-llajeanne)
• Fix the Centos/RHEL docker installation issue in ARM64 (#9047, @yankay)
• Fix the kube-vip missed SAN issue (#9099, @yankay)
• Fixed concatenate str & int in auto_renew_certificates_systemd_calendar (#8979, @floryut)
• Fixes the issue when it cannot correctly set the namespace for vphere-csi-driver (#9046, @eminaktas)
• Fixes vSphere CSI for vSphere CSI >= 2.4.0 on vSphere 6.7U3 (#8944, @snowball77)
• No more errors are emitted when attempting to delete worker nodes that do not exist. (#9244, @kerryeon)
• Optimize the format of evictionHard in kubelet-config.yaml template (#9204, @shelmingsong)
• Remove kubeowner different than root condition for user creation (#9125, @alegrey91)
• Remove unneed...

v2.19.1

Feature

• Add missing configuration for extra tolerations (#8999, @smasset)

Bug or Regression

• Allow "openSUSE Tumbleweed" to be run (again) (#9072, @oomichi)
• Disable kubelet_authorization_mode_webhook by default (#9239, @cristicalin)
• Do not run etcd role in scale.yml playbook when etcd installed by kubeadm (#9210, @LuckySB)
• Fix failing tasks when calico_datastore is set to etcd (#9234, @chadswen)
• Set fallback value of kubelet ip6 (#8942, @chinnonae)
• Swap calico download url, as the old primary url was deprecated and artefact no longer published (#8920, @sathieu)

v2.18.2

Feature

• Add missing configuration for extra tolerations (#9000, @smasset)

Bug or Regression

• Disable auth webhook default (#9240, @cristicalin)
• Fix cert-manager unusable due to leader election namespace problem (#8681, @rtsp)
• Removed quotation at nerdctl_extra_flags (#8699, @oomichi)
• Run 0100-dhclient-hooks only if dhcpclient is enabled (#8658, @oomichi)
• Fix image_command_tool var ignored since PR #8601 (#8684, @sathieu)

v2.19.0

Announcements

We are looking for maintainers, reach out in #5432.

Deprecation / Removal

• [metrics server] Remove addon-resizer from image list (no longer in use) (#8566, @cyril-corbon)
• Add kubeadm option to etcd_deployment_type to replace the etcd_kubeadm_enabled variable (#8317, @necatican) (See Notes 3)
• Removes runc-arm64-1.0.3 hash value for non existing binaries (#8391, @Payback159)
• Drop containerd 1.4 support (#8780, @oomichi)

Feature / Major changes

• Add hashes for Kubernetes 1.24.0, 1.24.1, 1.21.12, v1.21.13, 1.22.8, 1.22.9, v1.22.10, 1.21.11, 1.23.5, 1.23.6, v1.23.7 and make kubernetes v1.23.7 default (#8628, #8746, #8783, #8876, #8760, @mzaian, @cristicalin)
• Add youki runtime support to CRI-O (#8411, @electrocucaracha)
• [etcd] add 0 hash for arm v3.5.2 to prevent deployment failures (#8651, @cristicalin)
• [etcd] ensure etcd is properly upgraded when managed by kubeadm (#8722, @cristicalin)
• [etcd] Add etcd_max_request_bytes option to set the request size limit of etcd (#8849, @necatican)
• [etcd] add v3.5.1 for kubernetes 1.22+ (#8588, @mzaian)
• [etcd] Added node label to etcd metrics (#8475, @fungusakafungus)
• [Cilium] Update Cilium manifests and the default version to v1.11.3 (#8717, @necatican)
• [Cilium] Add identity_allocation_mode support (#8430, @necatican)
• [Cilium] Change Cilium setting identity_allocation_mode to cilium_identity_allocation_mode (#8519, @tomberget) (see Notes 1)
• [cilium] Add the cilium ip-masq-agent configuration support (#8893, @mahjonp)
• [docker] add support for cri-dockerd as a replacement for dockershim (#8623, @cristicalin)
• Add dual-stack support to kubelet --node-ip parameter, it works if set ip6 option host vars (#8542, @kakkotetsu)
• Add ppc64le support (#8505, @mgiessing)
• Add runc v1.1.0 hash values to support multi-arch installation. (arm64, amd64) (#8447, @Payback159)
• Add support for EventRateLimit plugin configuration (#8711, @alegrey91)
• Add support for including annotations on aws-ebs-csi-controller (#8779, @dlouks)
• Add support for kube-vip (#8669, @sathieu)
• Add support for service-account-lookup parameter (using kube_apiserver_service_account_lookup) (#8781, @alegrey91)
• [ansible] add support for ansible 5 (ansible-core 2.12) (#8512, @cristicalin)
• [ansible] make ansible 5.x the new default version (#8660, @cristicalin)
• [ansible] update ansible and cryptography requirements (#8826, @cristicalin)
• [cert-manager] Update cert-manager to 1.6.1 (#8377, @electrocucaracha)
• [cert-manager] Update cert-manager to v1.7.2 (#8648, @rtsp)
• [cert-manager] Upgrade to v1.8.0 (#8688, @rtsp)
• Add Ubuntu 22.04 support (#8841, #8795, #8754, @u2216, @arno01, @oomichi)
• Add evictionHard parameter to kubelet config (variables: eviction_hard/eviction_hard_control_plane) (#8421, @cyril-corbon)
• Add hcloud as external cloud provider (#8440, @oujonny)
• Add kube_router_cluster_asn option to set ASN number of the cluster (#8837, @rosskusler)
• Add option to use UpCloud's preconfigured server plans, firewalls and managed load balancers (upgrade to 2.4.0 from 2.0.0) (#8758, @Ajarmar)
• Add possibility to remove ippools from cni config (#8845, @tomcsi)
• Add the ability to set tolerations (cert_manager_tolerations), nodeselector (cert_manager_nodeselector) and affinity (cert_manager_affinity) in cert-manager templates (#8389, @cyril-corbon)
• Add the possibility to use UpCloud csi-driver
  Add the possibility to use ansbile_host as api ip for localhost kubeconfig (#8653, @robinAwallace)
• Add Hardening setup guide (#8868, @alegrey91)
• Add variables to manage kubelet parameters (kubelet_streaming_connection_idle_timeout / kubelet_make_iptables_util_chains) (#8796, @alegrey91)
• Added the optional prompt or delay before uncordoning nodes after upgrades (see variable upgrade_node_post_upgrade_confirm). (#8530, @mac-chaffee)
• Allow installation of a cluster using external CAs (kubernetes-ca, etcd-ca, kubernetes-front-proxy-ca) (#8620, @julienlefur)
• Allow the customization of snapshot controller namespace using snapshot_controller_namespace (#8305, @liupeng0518)
• Allow to change cert-manager leader election namespace for GKE Autopilot support (#8424, @rtsp)
• Allow to choose image pull commands based on container manager or override them (#8380, @sathieu)
• Allow to specify CA data for webhooks (using kube_webhook_token_auth_url_skip_tls_verify / kube_webhook_token_auth) (#8777, @dlouks)
• Assert that IP range is enough for the nodes (#8720, @eakyildirim)
• Bastion support now works for remove-node.yml (#8504, @roedie)
• Bump upcloud csi-driver to v0.2.1 (#8784, @robinAwallace)
• Change default kube_encryption_algorithm to "secretbox" (#8574, @Payback159) (See Notes 2)
• Explicit container_manager variable for Etcd hosts (#8521, @vi7)
• Improve first_kube_control_plane variable management to avoid installation failures due to variable overlapping (#8388, @unai-ttxu)
• Improve offline script generate_list.sh using ansible (#8538, @tmurakam)
• [ingress-nginx] upgrade to 1.2.1
• Ingress controllers and external provisioners (respectively deployed via ingress_controller and external_provisioner roles meta dependencies) are now upgraded in upgrade-cluster.yml (#8640, @mirwan)
• Local volume provisioner tolerations removed by default. (#8805, @spaced)
• Replace CLB with NLB for kube-apiserver domain in Terraform AWS contrib code (#8578, @sophalHong)
• Split kube_feature_gates variable for different kubernetes components (#8677, @alegrey91) (See Notes 4)
• Helm-apps role for installing helm charts (#8347, @VannTen)
• Upgrade azuredisk csi to v1.10.0 (#8432, @cyril-corbon)
• Upgrade metrics-server to v0.5.2 and remove NET_BIND_SERVICE capabilities (#8338, @cyril-corbon) (See Notes 5)
• Vagrant: new var $ansible_verbosiity was introduced for setting up ansible verbosity level (#8639, @maciejaszek)
• [CI] Move from CentOS 8 to AlmaLinux 8 for kubespray CI, therefore CentOS 8 is no longer tested (#8297, @cristicalin)
• [CI] split molecule testes to run in parallel (#8756, @cristicalin)
• [container image] use focal (ubuntu 20.04) base image for our docker builds (#8631, @cristicalin)
• [coredns] Allow overriding the default CoreDNS zone's cache plugin configuration via the coredns_default_zone_cache_block variable (#8488, @Tristan971)
• [csi-snapshotter] upgraded to 5.0.0 (#8403, @cristicalin)
• [download] add capability to specify alternative download mirrors for files (#8474, @cristicalin)
• [mitogen] update to 0.3.2 (#8470, @cristicalin)
• [reset] remove containerd storage during reset (#8469, @cristicalin)
• [sysctl] set fs.may_detach_mounts=1 to address pods stuck in Terminating state (#8635, @cristicalin)

Network

• [Calico] upgrade calico to 3.19.4, 3.20.4 and 3.21.4 (default) and add 3.22.0 experimental support (#8544, @cristicalin)
• [Calico] add 3.22.1 (#8612, @cristicalin)
• [Calico] Add calico apiserver (using calico_apiserver_enabled variable) (#8690, @liupeng0518)
• [Calico] Add support for IP6_AUTODETECTION_METHOD using new variable calico_ip6_auto_method (#8541, @kakkotetsu)
• [Calico] upgrade default calico version to v3.22.3 (#8897, @germetist)
• [Calico] Add configurable ipam strictaffinity (using calico_ipam_strictaffinity param) (#8581, @eyenx)
• [Calico] Change the calico cni name from cni0 to k8s-pod-network by default (#8813, @cyclinder)
• [Calico] Fix Wireguard support for CentOS Stream 9/RHEL 9 Beta (#8625, @ThisIsQasim)
• [Calico] fix calico-kube-controllers verbs (#8847, @irizzant)
• [calico] Some commands only need to be run once (#8833, @liupeng0518)
• [calico] call calico checks early on to prevent altering the cluster with bad settings and causing traffic outages (#8707, @cristicalin)
• [calico] make calico 3.21.x the news default and drop 3.18.x (#8426, @cristicalin)
• [calico] switch default iptables backend detection to Auto (#8429, @cristicalin)
• [calico] Use vxlan instead of ipip as the default calico encapsulation mode. This change impacts existing deployments that don't explicitly set the encapsulation mode and will need to set calico_ipip_mode: Always and calico_network_backend: bird to avoid the upgrade process breaking. (#8434, @cristicalin)
• [calico] upgrade default calico version to v3.21.5 (#8745, @mzaian)
• [calico] Use ipamconfig instead of calico ipam command (#8839, @liupeng0518)
• [calico] don't clobber calico options set by the user (#8815, @cristicalin)
• [flannel] Use install-cni-plugin to fit upstream (#8714, @zhengtianbao)
• [kube-ovn] Sync some feature with upstream (#8790, @liupeng0518)
• [kube-ovn] The network plug-in kube-ovn does not require a cluster to allocate podcidr (#8454, @chenhuazhong)

Applications

• Instance customization via cloud init for openstack VMs deployed by terraform is now available. (#8394, @moss2k13) (See Notes 6)
• [MetalLB] Configure PriorityClassName for deployment (#8362, @unai-ttxu)
• [MetalLB] Improve validation conditions for BGP Peers (#8568, @kakkotetsu)
• [MetalLB] Upgrade metallb to v0.11.0 and add liveness and readiness probe (#8420, @cyril-corbon)
• [MetalLB] Allow to put node selectors and source address for each metallb peers (#8534, @hightoxicity)
• [MetalLB] Added MetalLB BGP peer password authentication option. (#8792, @Oogy)
• [MetalLB] Add images to downloads (#8715, @sathieu)
• [MetalLB] Fix wrong port name in metallb.yml.j2 (metrics not monitoring) (#8510, @binkoni)
• [OpenStack] Allow disabling port security in terraform contrib code (#8410, @cristicalin)
• [OpenStack] Updated openstack cloud controller to version v1.22.0 (#8629, @Xartos)
• [OpenStack] Create master nodes with for_each for openstack. Makes it easier to switch out master nodes via terraform. (#8709, @robinAwallace)
• [OpenStack] Fixed cluster roles for openstack cloud controlle...

v2.18.1

Feature / Major changes

• [kubernetes] Update kubernetes hashes and make 1.22.6 the default (#8467, @cristicalin)
• Allow to choose image pull commands based on container manager or override them (#8380, @sathieu)
• Improve offline script generate_list.sh using ansible (#8606, @tmurakam)
• [CI] Move from CentOS 8 to AlmaLinux 8 for kubespray CI, therefore CentOS 8 is no longer tested (#8297, @cristicalin)
• [container image] use focal (ubuntu 20.04) base image for our docker builds (#8631, @cristicalin)
• [sysctl] set fs.may_detach_mounts=1 to address pods stuck in Terminating state (#8635, @cristicalin)

Container-Managers

• [containerd] make containerd_insecure_registries into a dict similar to containerd_registries (#8340, @mircyb) (see Notes 1)
• [containerd] nerdctl insecure registry support (#8339, @mircyb)

Bug or Regression

• Fix an issue where offline script could not output URLs of both containerd and krew. (#8379, @oomichi)
• Fix container engine still installed on dedicated etcd node even if etcd_deployment_type: host (#8404, @rtsp)

Notes

1. containerd_insecure_registries needs to be updated or won't work anymore