Talos private network Communication in Hetzner Cloud

[mrclrchtr]

Hello, I am trying to create a Terraform Module to create a cluster in the Hetzner Cloud with private network usage.
The nodes should only communicate with each other via the private network.

My current configuration is basically working. You can view it here: https://github.com/hcloud-talos/terraform-hcloud-talos

1. Question (answered)

I have set up a firewall where only my client IP has access to Talos API and Kubernetes API.

What surprises me is that talosctl only works if --endpoints and --nodes have the same (public) IP when sending the command.

talosctl dashboard --nodes x.x.x.x --endpoints x.x.x.x <- working
talosctl dashboard --nodes x.x.x.x --endpoints y.y.y.y <- not working

If I open the Talos API for all IPs in the firewall, then talosctl also works with different (public) IPs. Does this mean that the nodes try to communicate via the public IP? Can I force Talos to communicate via the private IP even though I call talosctl with the public IP?

EDIT:
I have now found that out for myself. If you call talosctl with the public IP of the node, then talos tries to establish a connection from the "endpoint host" to the node with exactly this (public) IP, which should then first go out via the public IP and then directly back in again. However, the IP of the "endpoint host" is not open in the firewall, which is why this is not working. If I call with the private IP, everything works as expected.

I am currently testing whether I can simply use extraHostEntries to redirect public to private.
Nope.. thats not working: /etc/hosts, which is used by extraHostEntries maps only names on IP addresses.

TLDR;

Yes you can force Talos to use the internal IP by calling talosctl with -n internal-ip-of-th-node.

2. Question

Unfortunately I have another question: I am trying to define the interfaces explicitly in this branch: https://github.com/hcloud-talos/terraform-hcloud-talos/blob/explicit_interfaces/patches/machine-patch.yaml.tmpl

The configuration then looks like this:
(x.x.x.x is public IP, 172.31.1.1 is the predefined Gateway by Hetzner https://docs.hetzner.com/cloud/networks/faq/#are-any-ip-addresses-reserved)

machine:
    network:
        interfaces:
            - interface: eth0
              addresses:
                -  x.x.x.x/32
              routes:
                - network: 172.31.1.1/32
                  gateway: ""
                - network: 0.0.0.0/0
                  gateway: 172.31.1.1
              dhcp: false
            - interface: eth1
              addresses:
                - 10.0.0.101/24
              routes:
                - network: 10.0.0.1/24
                  gateway: ""
                - network: 10.0.0.0/16
                  gateway: 10.0.0.1
              dhcp: false

If I use talosctl dashboard:

IP           10.0.0.102/24, <public_ipv6>/64, <public_ipv4>/32
GW           172.31.1.1, fe80::1
CONNECTIVITY OK
DNS          185.12.64.1, 185.12.64.2

shouldn't the gateway `10.0.0.1 also be in there?

Thank you very much for your help!

[smira]

shouldn't the gateway `10.0.0.1 also be in there?

it's not a gateway in your config, it just says 10.0.0.1/24 are directly attached to eth1.

[mrclrchtr]

Ah okay,that's, why it wasn't working I think.

Thank you!