KMM Issue to create amd-gpu driver image #1
Comments
|
Hi Kakor,
Openshift KMM does not support OKD, so it is ok that you are seeing all
those errors :). Using DTK_AUTO requires Images and ImageStreams object to
be correctly configured and maintained by the cluster, and i think that OKD
is not there.
Is there any specific reason why you need to use OKD instead of Openshift
cluster?
Thanks,
Yevgeny
…On Tue, May 7, 2024 at 5:08 PM KakorHolloway ***@***.***> wrote:
Hello,
First of all thank you very much for this operator.
I'm trying to use this solution on my OKD cluster in version 4.14 to
install an AMD RADEON RX 7900XT card as GPU to test AI models.
Unfortunately I have a problem creating the image on the internal
registry. I have this problem on kmm operator pod :
E0507 13:44:31.120907 1 controller.go:324] kmm "msg"="Reconciler error"
"error"="failed to handle build for kernel version 6.5.5-200.fc38.x86_64:
could not synchronize the build: could not make Build template: could not
get DTK image for kernel 6.5.5-200.fc38.x86_64: could not find kernel
6.5.5-200.fc38.x86_64 in kernel --> OS mapping"
"Module"={"name":"dc-harbor","namespace":"openshift-amd-gpu"}
"controller"="BuildSignReconciler" "controllerGroup"="kmm.sigs.x-k8s.io"
"controllerKind"="Module" "name"="dc-harbor"
"namespace"="openshift-amd-gpu"
"reconcileID"="ae042701-0f38-4dfe-802b-56795b3f5691"
If I remove DeviceConfig to add the DTK image value of my cluster version
on the Module object (I remove the owner reference), and if I change
DTK_AUTO in the configuration map to DTK, I can start the build .
Even with this option, I have to add a new variable named "KVERS" and
replace it with the kernel version of the DTK image. (And apply this in the
config map to replace all KERNEL_VERSION variables with KVERS).
(6.5.5-200.fc38.x86_64 to 5.14.0-284.36.1.el9_2.x86_64)
You will find my custom files attached.
cm_and_modude.txt
<https://github.com/yevgeny-shnaidman/amd-gpu-operator/files/15236644/cm_and_modude.txt>
If all those change are mades, I can build the image and use it by
recreating the DeviceConfig file but It will not work as intended because
of those logs :
I0507 13:56:47.805280 1 main.go:100] kmm-worker "msg"="Starting worker"
"git commit"="e1625b2" "version"="2.0.1"
I0507 13:56:47.805292 1 main.go:101] kmm-worker "msg"="Reading pull
secrets" "base dir"="/var/run/kmm/pull-secrets"
I0507 13:56:47.805362 1 pullsecrets.go:39] kmm-worker "msg"="Reading file"
"path"="/var/run/kmm/pull-secrets/amd-gpu-operator-kmm-module-loader-dockercfg-wjwn5/..2024_05_07_13_31_23.2974230544/.dockercfg"
"type"="kubernetes.io/dockercfg"
I0507 13:56:47.805388 1 pullsecrets.go:35] kmm-worker "msg"="Unhandled
file name; ignoring"
"path"="/var/run/kmm/pull-secrets/amd-gpu-operator-kmm-module-loader-dockercfg-wjwn5/..data"
I0507 13:56:47.805395 1 pullsecrets.go:39] kmm-worker "msg"="Reading file"
"path"="/var/run/kmm/pull-secrets/amd-gpu-operator-kmm-module-loader-dockercfg-wjwn5/.dockercfg"
"type"="kubernetes.io/dockercfg"
I0507 13:56:47.807301 1 mirror.go:43] kmm-worker "msg"="No configuration
found for registry" "image name"="harbor.kakor.ovh/public/amddriver:v1.0"
I0507 13:56:47.807314 1 worker.go:190] kmm-worker "msg"="Pulling image"
"image name"="harbor.kakor.ovh/public/amddriver:v1.0"
time="2024-05-07T13:57:12Z" level=warning msg="ignored xattrs in archive:
underlying filesystem doesn't support them" errors="[operation not
permitted]"
time="2024-05-07T13:57:12Z" level=warning msg="ignored xattrs in archive:
underlying filesystem doesn't support them" errors="[operation not
permitted]"
I0507 13:57:16.304671 1 worker.go:198] kmm-worker "msg"="Image pulled
successfully"
"dir"="/var/run/kmm/images/harbor.kakor.ovh/public/amddriver:v1.0/fs"
"image name"="/public/amddriver:v1.0"
I0507 13:57:16.304743 1 worker.go:49] kmm-worker "msg"="Unloading in-tree
module" "name"="amdgpu"
I0507 13:57:16.304831 1 modprobe.go:33] kmm-worker "msg"="Running
modprobe" "command"="/usr/sbin/modprobe -rv amdgpu"
I0507 13:57:16.306316 1 worker.go:59] kmm-worker "msg"="preparing firmware
for loading" "host mount directory"="/var/lib/firmware" "image
directory"="/var/run/kmm/images/harbor.kakor.ovh/public/amddriver:v1.0/fs/firmwareDir/updates"
I0507 13:57:16.614772 1 modprobe.go:33] kmm-worker "msg"="Running
modprobe" "command"="/usr/sbin/modprobe -vd
/var/run/kmm/images//public/amddriver:v1.0/fs/opt amdgpu"
I0507 13:57:16.615904 1 cmdlog.go:71] kmm-worker/modprobe/stderr
"msg"="modprobe: FATAL: Module amdgpu not found in directory
/var/run/kmm/images/harbor.kakor.ovh/public/amddriver:v1.0/fs/opt/lib/modules/6.5.5-200.fc38.x86_64"
E0507 13:57:16.616009 1 cmdutils.go:11] kmm-worker "msg"="Fatal error"
"error"="error while waiting on the command: exit status 1"
Thank you in advance :)
Regards
—
Reply to this email directly, view it on GitHub
<#1>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AOPNM5LW4YDLLGCHNTFSHZTZBDN5HAVCNFSM6AAAAABHLCYKESVHI2DSMVQWIX3LMV43ASLTON2WKOZSGI4DGNBWHEZTOOI>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello,
First of all thank you very much for this operator.
I'm trying to use this solution on my OKD cluster in version 4.14 to install an AMD RADEON RX 7900XT card as GPU to test AI models.
Unfortunately I have a problem creating the image on the internal registry. I have this problem on kmm operator pod :
E0507 13:44:31.120907 1 controller.go:324] kmm "msg"="Reconciler error" "error"="failed to handle build for kernel version 6.5.5-200.fc38.x86_64: could not synchronize the build: could not make Build template: could not get DTK image for kernel 6.5.5-200.fc38.x86_64: could not find kernel 6.5.5-200.fc38.x86_64 in kernel --> OS mapping" "Module"={"name":"dc-harbor","namespace":"openshift-amd-gpu"} "controller"="BuildSignReconciler" "controllerGroup"="kmm.sigs.x-k8s.io" "controllerKind"="Module" "name"="dc-harbor" "namespace"="openshift-amd-gpu" "reconcileID"="ae042701-0f38-4dfe-802b-56795b3f5691"If I remove DeviceConfig to add the DTK image value of my cluster version on the Module object (I remove the owner reference), and if I change DTK_AUTO in the configuration map to DTK, I can start the build .
Even with this option, I have to add a new variable named "KVERS" and replace it with the kernel version of the DTK image. (And apply this in the config map to replace all KERNEL_VERSION variables with KVERS). (6.5.5-200.fc38.x86_64 to 5.14.0-284.36.1.el9_2.x86_64)
You will find my custom files attached.
cm_and_modude.txt
If all those change are mades, I can build the image and use it by recreating the DeviceConfig file but It will not work as intended because of those logs :
Thank you in advance :)
Regards
The text was updated successfully, but these errors were encountered: