Changing the kubernetes endpoint

[Ulrar]

Hi,

I think I misunderstood the doc when I created my cluster without realizing, because the other day after deleting an old dns entry my cluster exploded. It looks like I used my first node's dns name as the kubernetes endpoint, so presumably now all the nodes depend on that first node's name resolving to a valid api server.

Looking in the controlplane config the only place this name seems to show up is in the apiServer certSANs section. I assume that's not actually where it's pulled from, is there a procedure to change the api endpoint configured ?

I really thought I was pointing it at the VIP but I guess not. But would that even be safe ? My understanding is the VIP needs etcd to be healthy to become available, but it seems etcd can't come up on a node if it can't reach that endpoint, meaning it'd be impossible to bring the cluster back up after more than one node going down on a three node cluster if pointing at the VIP. Or am I missing something ?

Thanks

[smira]

Kubernetes endpoint is part of the machine configuration. If you have KubePrism enabled, Talos itself doesn't even use it.

Docs: https://www.talos.dev/v1.6/reference/configuration/v1alpha1/config/#Config.cluster.controlPlane.endpoint

Make sure you update all nodes to point to the new endpoint, it's safe to change it.

[Ulrar]

Hi, that's interesting, that one is pointing to the VIP as expected.
I believe KubePrism is only in 1.6.0 and I was trying to fix this up before upgrading, but I'll just upgrade now I guess.

I wonder why everything exploded when I deleted that entry then .. I definitely got a bunch of failures resolving that hostname in the dmesg of the nodes, it must be referenced somewhere. I'll upgrade and try to reproduce it then, thanks

[smira]

KubePrism is part of the machine configuration, it's available since 1.5.0, but enabled by default in 1.6.0

But it won't be enabled on upgrade, you can enable it at any moment, just follow the documentation: https://www.talos.dev/v1.6/kubernetes-guides/configuration/kubeprism/

You can change the endpoint without KubePrism as well. For worker machines, that endpoint will be used by the kubelet to talk to the controlplane.

Controlplane machines don't depend on the endpoint themselves.

[Ulrar]

Alright so I followed that, and the diff does indeed show that my b/v1.ConfigMap/kube-system/kubeconfig-in-cluster was referring to one of the node's DNS names, somehow. Not sure how I managed that but I had clearly misconfigured my cluster in a fun way, and it certainly seems to be running better with KubePrism enabled already ! I need to test it now but that at least addresses my question, thank you so much