ZTS: Add archlinux #16816

Kaz205 · 2024-11-28T18:41:58Z

Motivation and Context

Arch Linux gets newer kernels faster than other distros.

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Performance enhancement (non-breaking change which improves efficiency)
Code cleanup (non-breaking change which makes code smaller or more readable)
Breaking change (fix or feature that would cause existing functionality to change)
Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
Documentation (a change to man pages or other documentation)

Checklist:

My code follows the OpenZFS code style requirements.
I have updated the documentation accordingly.
I have read the contributing document.
I have added tests to cover my changes.
I have run the ZFS Test Suite with this change applied.
All commit messages are properly formatted and contain Signed-off-by.

n0xena · 2024-11-28T21:11:06Z

you might want to have a read: https://github.com/orgs/archzfs/discussions/555
the archzfs team is on it - and for now the gist is: the main branch will be the latest stable zfs upstream release (currently 2.2.6) with the latest archlinux lts kernel (currently 6.6.x) - anything else will be at least unstable or even experimental
if you want to use zfs on arch 6.12 right now: https://github.com/n0xena/archzfs/releases

snajpa · 2024-11-28T23:09:49Z

Fedora has pretty new kernels too... what we need is to test these new kernels under load, since that is how many of the problems with them manifest - low memory conditions, a lot of churn generally

With that said, I have no objection to adding Arch, agree that the sooner problems are found the better

@Kaz205 would you be around to maintain it? I'd guess it could require regular maintenance since it's a fast moving target

Harry-Chen · 2024-12-02T09:17:44Z

You might want to add ZFS-CI-Type: full to any of your commits, so that a full CI (including newly added archlinux) will run against your PR.

sempervictus · 2024-12-02T20:09:43Z

Aside from newer kernels, arch also has newer tool chains which can help identify logic or binary problems during the build and test steps.

behlendorf

We can try enabling builds for Arch. Can you just add ZFS-CI-Type: full to the commit message as suggested and force update the CI to trigger a full run on all platforms including Arch.

behlendorf · 2024-12-02T21:49:35Z

.github/workflows/scripts/qemu-4-build.sh

@@ -56,7 +56,8 @@ function linux() {
    --prefix=/usr \
    --enable-pyzfs \
    --enable-debug \
-    --enable-debuginfo
+    --enable-debuginfo \
+    --sysconfdir=/etc


Why is this needed on Arch but not elsewhere?

Looks like the configs get installed to /usr/etc/zfs instead of /etc/zfs.

This looks good and should be applied I think.

Kaz205 · 2024-12-03T15:10:35Z

Sorry for the late response.

Fedora has pretty new kernels too... what we need is to test these new kernels under load, since that is how many of the problems with them manifest - low memory conditions, a lot of churn generally

Maybe add something like "mem=1G" to the kernel cmdline?

With that said, I have no objection to adding Arch, agree that the sooner problems are found the better

@Kaz205 would you be around to maintain it? I'd guess it could require regular maintenance since it's a fast moving target

How much maintenance would this require? I was planning for this to be a one-off thing.

We can try enabling builds for Arch. Can you just add ZFS-CI-Type: full to the commit message as suggested and force update the CI to trigger a full run on all platforms including Arch.

Done.

tonyhutter · 2024-12-03T16:53:22Z

Since we don't officially support Arch, and it's going to have a lot of churn, it would be good to name the runner in such a way so that people don't freak out if it doesn't pass, like:

 qemu-x86 (ArchLinux - doesn't need to pass) Expected — Waiting for status to be reported

I'm open to other ways to do this too.

snajpa · 2024-12-04T00:00:42Z

I guess the primary question is what are you expecting Arch will provide. If it's the ability to use OpenZFS with new kernels reliably, then I think this doesn't do anything to help on that front. If it's to make sure it compiles OK on new kernels, OK, that will work.

But if you're after actual reliability, then I'd say we need more help, a kernel param doesn't move the needle either.

Those bugs that really hinder uptime because you hit them just enough to be annoying or outright damaging to operations, but not as easy enough to be able to reproduce them easily, those are the worst and those take some effort to debug. The sooner we can uncover those, the better. Knowing on which commit things break without having to bisect for it with tests that take hours is priceless :)) And that is where we could IMO do better. This class of bugs doesn't eat data or corrupt what's on disk, but uptime is a reliability metric too :)

Oftentimes that only happens after a certain amount of (meta)data turnover, or you only notice if you run without init_on_alloc/init_on_free, or some other subtly changed kernel config.

If it's enough for you to know it compiles and runs the current test suite, OK - it looks like there isn't much to maintain overall, other than the package list, which doesn't seem difficult.

mcmilk

The first two commits look fine. But adding archlinux within the testings seems to much for me.

mcmilk · 2024-12-13T19:59:28Z

.github/workflows/scripts/qemu-4-build.sh

@@ -56,7 +56,8 @@ function linux() {
    --prefix=/usr \
    --enable-pyzfs \
    --enable-debug \
-    --enable-debuginfo
+    --enable-debuginfo \
+    --sysconfdir=/etc


This looks good and should be applied I think.

mcmilk · 2024-12-13T20:02:43Z

.github/workflows/zfs-qemu.yml

@@ -22,7 +22,7 @@ jobs:
      - name: Generate OS config and CI type
        id: os
        run: |
-          FULL_OS='["almalinux8", "almalinux9", "centos-stream9", "debian11", "debian12", "fedora40", "fedora41", "freebsd13-4r", "freebsd14-0r", "freebsd14-1s", "ubuntu20", "ubuntu22", "ubuntu24"]'
+          FULL_OS='["almalinux8", "almalinux9", "archlinux", "centos-stream9", "debian11", "debian12", "fedora40", "fedora41", "freebsd13-4r", "freebsd14-0r", "freebsd14-1s", "ubuntu20", "ubuntu22", "ubuntu24"]'


Archlinux is most of the time ahead our linux-max version in the META file.
I don't think it's useful for the OpenZFS repository. You can add archlinux within you own Actions for testings an so on.

You can add archlinux within you own Actions for testings an so on.

Sounds good to me. Thanks for the review.

mcmilk · 2024-12-21T09:12:24Z

@Kaz205 - could you reabse without the last commit? That would be fine.

Signed-off-by: Kazuki Hashimoto <[email protected]>

On Linux distros like Arch Linux, files that should be in /etc/zfs end up being installed in /usr/etc/zfs. Specifying sysconfdir path fixes it. Signed-off-by: Kazuki Hashimoto <[email protected]>

Kaz205 · 2024-12-21T11:04:12Z

@Kaz205 - could you reabse without the last commit? That would be fine.

Done.

behlendorf reviewed Dec 2, 2024

View reviewed changes

behlendorf added Status: Code Review Needed Ready for review and testing Status: Revision Needed Changes are required for the PR to be accepted labels Dec 2, 2024

Kaz205 force-pushed the master branch from ba61960 to ce99d0e Compare December 3, 2024 15:06

github-actions bot removed the Status: Revision Needed Changes are required for the PR to be accepted label Dec 3, 2024

mcmilk suggested changes Dec 13, 2024

View reviewed changes

Kaz205 added 2 commits December 21, 2024 20:03

ZTS: Install more packages on Arch Linux

12a0cb9

Signed-off-by: Kazuki Hashimoto <[email protected]>

ZTS: Specify sysconfdir

f92ffba

On Linux distros like Arch Linux, files that should be in /etc/zfs end up being installed in /usr/etc/zfs. Specifying sysconfdir path fixes it. Signed-off-by: Kazuki Hashimoto <[email protected]>

Kaz205 force-pushed the master branch from ce99d0e to f92ffba Compare December 21, 2024 11:03

mcmilk approved these changes Dec 21, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ZTS: Add archlinux #16816

ZTS: Add archlinux #16816

Kaz205 commented Nov 28, 2024 •

edited

Loading

n0xena commented Nov 28, 2024

snajpa commented Nov 28, 2024 •

edited

Loading

Harry-Chen commented Dec 2, 2024

sempervictus commented Dec 2, 2024

behlendorf left a comment •

edited

Loading

behlendorf Dec 2, 2024

Kaz205 Dec 3, 2024

mcmilk Dec 13, 2024

Kaz205 commented Dec 3, 2024

tonyhutter commented Dec 3, 2024 •

edited

Loading

snajpa commented Dec 4, 2024 •

edited

Loading

mcmilk left a comment

mcmilk Dec 13, 2024

mcmilk Dec 13, 2024

Kaz205 Dec 15, 2024

mcmilk commented Dec 21, 2024

Kaz205 commented Dec 21, 2024

ZTS: Add archlinux #16816

Are you sure you want to change the base?

ZTS: Add archlinux #16816

Conversation

Kaz205 commented Nov 28, 2024 • edited Loading

Motivation and Context

Types of changes

Checklist:

n0xena commented Nov 28, 2024

snajpa commented Nov 28, 2024 • edited Loading

Harry-Chen commented Dec 2, 2024

sempervictus commented Dec 2, 2024

behlendorf left a comment • edited Loading

Choose a reason for hiding this comment

behlendorf Dec 2, 2024

Choose a reason for hiding this comment

Kaz205 Dec 3, 2024

Choose a reason for hiding this comment

mcmilk Dec 13, 2024

Choose a reason for hiding this comment

Kaz205 commented Dec 3, 2024

tonyhutter commented Dec 3, 2024 • edited Loading

snajpa commented Dec 4, 2024 • edited Loading

mcmilk left a comment

Choose a reason for hiding this comment

mcmilk Dec 13, 2024

Choose a reason for hiding this comment

mcmilk Dec 13, 2024

Choose a reason for hiding this comment

Kaz205 Dec 15, 2024

Choose a reason for hiding this comment

mcmilk commented Dec 21, 2024

Kaz205 commented Dec 21, 2024

Kaz205 commented Nov 28, 2024 •

edited

Loading

snajpa commented Nov 28, 2024 •

edited

Loading

behlendorf left a comment •

edited

Loading

tonyhutter commented Dec 3, 2024 •

edited

Loading

snajpa commented Dec 4, 2024 •

edited

Loading