Talos 1.4.6 and 1.5.0-alpha.1 won't load on bare metal EX101 in Hetzner

[eshikhov]

Hi!

I have a bare metal EX101 server at Hetzner. The configuration added memory up to 128GB and another ssd disk 980GB for talos. I'm trying to run talos on it. After the boot menu appears and the first item is selected, nothing happens, only a black screen. I tried both booting from an iso image and unpacking the image to a hard drive. The result is the same. I took version 1.4.6 and 1.5.0-alpha.1
Please tell me how you can debug talos, how to find out what the download is hanging on. Maybe there are some keys for the kernel to debug talos?

[smira]

It might be as simple as wrong console settings. I assume you're using a video console, and that is sometimes tricky.

You can try console=tty1 and other Linux kernel console-related settings.

If you have access to a serial console, that works way better.

You can check that Talos listens on port :50000 after boot, in that case it works despite the fact that you don't have console access.

[eshikhov]

I see the output in the console if you enable debug log.

this is the whole output, the download does not go further

[smira]

This is expected as EFI stub prints using UEFI, but Linux will switch to using its own methods to print to the console.

[eshikhov]

there is no option to write the boot log to a file?

[smira]

There is no filesystem during the early boot, where would it write it? Talos runs unmodified Linux kernel, so any generic Linux options work.
I would rather ask for a serial console from Hetzner, this is the easiest way.

[eshikhov]

In Hetzner they answered that they can only give KVM, are there any other options how to debug?

[smira]

Please see my first answer #7465 (comment)

[frezbo]

you shoulod try with different console options. Eg:

• console=ttyS0
• console=ttyAMA0
• console=tty0 there is not standard for this , so it's kind of a trial and error process unless the motherboard/hardware manufactures provides this info (some actually do)

[eshikhov]

The problem was solved by removing the parameters from the kernel
earlyprintk=ttyS0 console=ttyS0 console=tty0
consoleblank=0
printk.devkmsg=on now you can see the installation process and the console itself in KVM, also added the "ip=" parameter to the kernel for the network to work correctly.

thanks for the ideas

[frezbo]

yeh, swapping might work

[vladimirfx]

I don't think so. According to kernel documentation output is directed to all outputs enumerated by console arg. Last one is used for /dev/console. So if kernel can't find driver for serial console boot is hang regardless of order.

Exactly in this order arguments are now (by default).

I will check opposite variant tomorrow.

[vladimirfx]

https://www.kernel.org/doc/html/v4.12/admin-guide/serial-console.html

[vladimirfx]

I've tried to change console arguments order - didn't help.

Can I file issue for remove arg feature?

UPDATE: checked on 1.5.8-beta.1

[smira]

Can I file issue for remove arg feature?

yes, please

[vladimirfx]

hm.... but we could simply swap the args? this is my understanding from reading this...

@vladimirfx does it help if you edit the args to be console=ttyS0 console=tty0 (with ttyS0 first?)

It's current default order.

[frezbo]

@vladimirfx could you check if you remove all console= args, talos boots fine and console logs are viewable?

[vladimirfx]

It is what we done fist thanks to #7465 (comment) . Yes it woks. Later I've track down that only console=ttyS0 is problematic.

[frezbo]

ahh perfect, we're planning to drop all console args for metal platform as kernel does the automatic handling, so we don;t need to add a feature to remove args which is cumbersome