OCPBUGS-25860: Disable managed interrupts for smartpqi

[MarSik]

This driver does not obey the isolcpus=managed_irq hint and is causing interference.

This kernel argument makes sure the interrupt affinity can be managed by userspace services.

The alternative approach using /etc/modprobe.d/smartpqi.conf with option smartpqi ... does not work, because the driver is loaded early at the initrd stage and we would have to rebuild the RHCOS initrd. That is much heavier than a simple kernel arg.

[openshift-ci-robot]

@MarSik: This pull request references Jira Issue OCPBUGS-25860, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.16.0) matches configured target version for branch (4.16.0)
• bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira ([email protected]), skipping review request.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This driver does not obey the isolcpus=managed_irq hint and is causing interference.

This kernel argument makes sure the interrupt affinity can be managed by userspace services.

The alternative approach using /etc/modprobe.d/smartpqi.conf with option smartpqi ... does not work, because the driver is loaded early at the initrd stage and we would have to rebuild the RHCOS initrd. That is much heavier than a simple kernel arg.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: MarSik

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [MarSik]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[MarSik]

@jmencak @yarda Could you please check my syntax for the udev regex?

[jmencak]

@jmencak @yarda Could you please check my syntax for the udev regex?

Hmm, I'm getting the following:

2024-01-12 08:30:32,715 ERROR    tuned.units.manager: failed to initialize plugin bootloader_smartpqi
2024-01-12 08:30:32,716 ERROR    tuned.units.manager: No module named 'tuned.plugins.plugin_bootloader_smartpqi'
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/tuned/units/manager.py", line 88, in create
    plugin = self._plugins_repository.create(plugin_name)
  File "/usr/lib/python3.9/site-packages/tuned/plugins/repository.py", line 34, in create
    plugin_cls = self.load_plugin(plugin_name)
  File "/usr/lib/python3.9/site-packages/tuned/utils/plugin_loader.py", line 32, in load_plugin
    return self._get_class(module_name)
  File "/usr/lib/python3.9/site-packages/tuned/utils/plugin_loader.py", line 36, in _get_class
    module = __import__(module_name)
ModuleNotFoundError: No module named 'tuned.plugins.plugin_bootloader_smartpqi'
2024-01-12 08:30:32,718 INFO     tuned.daemon.daemon: static tuning from profile 'bz' applied

With:

[main]
summary=PoC

[bootloader_smartpqi]
devices_udev_regex==DRIVER=smartpqi
cmdline_smartpqi=+smartpqi.disable_managed_interrupts=1

Did you test this?

[MarSik]

Not yet, I should have added WIP to the name..
I know what the issue is, will fix it soon.

[jmencak]

This seems to somewhat work now. But I can also see the /etc/tuned/bootcmdline set to:

TUNED_BOOT_CMDLINE="smartpqi.disable_managed_interrupts=1"

even when the smartpqi module is not loaded. Is this intended? I suspect the devices_udev_regex==DRIVER=smartpqi doesn't take any effect. WDYT?

[MarSik]

/hold

The udev regex matching does not work for the bootloader plugin.

[yanirq]

@MarSik lets rebase and check for the right udev rule

[yanirq]

/hold
Depends on #980
to pass ci/prow/e2e-no-cluster tests

[yanirq]

/hold cancel

[MarSik]

/retest

[yanirq]

@MarSik you'll probably need to run render sync

[MarSik]

/retest-required

[openshift-ci]

@MarSik: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[MarSik]

/remove-lifecycle rotten