Pumba Stress is not working as expected

[rantidev]

Hello Team ,

I was using pumba to inject cpu stress in docker containers and it was working perfectly fine.
curl --unix-socket /var/run/docker.sock -H "Content-Type: application/json"
-d '{"Image": "gaiaadm/pumba","HostConfig": {
"Binds": [
"/var/run/docker.sock:/var/run/docker.sock"
]
}, "Cmd": ["--log-level=info", "stress" ,"--duration=1m", "re2:^pg_"]}'
-X POST http://localhost/v1.37/containers/create

However when I use the above command now to stress the docker container getting below error :

docker logs -f 5ffef0aa421354a5c94f375c90da8fcb1fff33355277ad1dbd409429fe77bbae
time="2022-12-02T08:31:07Z" level=info msg="stress testing container" dryrun=false duration=1m0s id=e49e026eb5512be937945800216403d185bf2756c9f7e8dcc8ccab21961aaa75 image="alexeiled/stress-ng:latest-ubuntu" name=/pg_container pull=true stressors="[--cpu 4 --timeout 60s]"
time="2022-12-02T08:31:11Z" level=fatal msg="error running stress command: error running chaos command: one or more stress test failed: stress-ng failed with error: stress-ng exited with error: \x02\x00\x00\x00\x00\x00\x00/cgroups can't be listed: Cgroup is not mounted\n\x02\x00\x00\x00\x00\x00\x00)cgroup controller and pathparsing failed\n"

[alexei-led]

@rantidev please provide more details about your runtime environment: OS, docker host, version, where you run pumba and target containers, etc. - everything that will help to reproduce the issue.
Please also add commands you are using to run the target container and the pumba container (docker commands and not curl)

[toky0]

Hi,
Running into the same issue.
OS: Linux docker 5.10.0-20-amd64 #1 SMP Debian 5.10.158-2 (2022-12-13) x86_64 GNU/Linux
Docker Version: 20.10.23
Docker Compose v2.15.1
Docker compose file:

services:
   redis:
     image: redis:6-alpine
     restart: unless-stopped

Debug output:

root@docker:/home/hyperglass/prometheus# pumba --log-level=debug stress -d 1m redis
DEBU[0000] using names                                   names="[redis]"
DEBU[0000] stress testing all matching containers
DEBU[0000] listing matching containers                   duration=1m0s labels="[]" limit=0 names="[redis]" pattern= random=false stressors="[--cpu 4 --timeout 60s]"
DEBU[0000] listing containers
DEBU[0000] found container                               id=f481bc6bd9bc369c9911a9a9f3e3xxxxxxxxx name=/redacted1
DEBU[0000] found container                               id=21f8d632502e09e459e4d19dfa13cb2ayyyy name=/redacted2
DEBU[0000] found container                               id=914608923f3e8211cd2d2c7e94e4705b23z name=/redacted3
DEBU[0000] found container                               id=baf4f8d39ee1d3e82234a5153cd1067a15db name=/redis
DEBU[0000] found container                               id=d121419b5b43e7dfe8515f47a44d40b8faaa name=/redacted4
DEBU[0000] stress testing container for duration         container= baf4f8d39ee1d3e82234a5153cd1067a15db duration=1m0s pull image=true stress-ng image="alexeiled/stress-ng:latest-ubuntu" stressors="[--cpu 4 --timeout 60s]"
INFO[0000] stress testing container                      dryrun=false duration=1m0s id=baf4f8d39ee1d3e82234a5153cd1067a15db image="alexeiled/stress-ng:latest-ubuntu" name=/redis pull=true stressors="[--cpu 4 --timeout 60s]"
DEBU[0000] executing stress-ng command                   image="alexeiled/stress-ng:latest-ubuntu" pull=true stressors="[--cpu 4 --timeout 60s]" target=baf4f8d39ee1d3e82234a5153cd1067a15db
DEBU[0000] pulling stress-ng image                       image="alexeiled/stress-ng:latest-ubuntu"
DEBU[0001] &{Pulling from alexeiled/stress-ng   {0 0}}
DEBU[0001] &{Digest: sha256:6c2f6a6997aeb0dc7f7299e556f67cbb7ca1b40398ef32bbeea50b90fc020ae3   {0 0}}
DEBU[0001] &{Status: Image is up to date for alexeiled/stress-ng:latest-ubuntu   {0 0}}
DEBU[0001] creating stress-ng container                  image="alexeiled/stress-ng:latest-ubuntu"
DEBU[0001] stress-ng container created, starting it      id=103c4a9221ee57ee59dba0de6ea6aa6fd65bf2f9efdf3ab577892d459c4f7e5d
FATA[0002] error running stress command: error running chaos command: one or more stress test failed: stress-ng failed with error: stress-ng exited with error: /cgroups can't be listed: Cgroup is not mounted
)cgroup controller and pathparsing failed

[alexei-led]

@toky0 thank you. I will try to reproduce in a similar environment

[MehranJanfeshan]

Any update on this error?

[alexei-led]

Not yet. I think it's related to Linux distribution you are using and the cgroups package.

Can you try to run it on Ubuntu or Redhat LTS version?

[lukas8219]

having the same issue. The container I am trying to attack is a mongo:6.0 image with a couple tc and ping utils package installed.

The same happened when trying on all images, nginx, redis and mongo.

host uname -a -> Darwin Lucas-Polesello's-MacBook-Pro 23.1.0 Darwin Kernel Version 23.1.0: Mon Oct 9 21:28:31 PDT 2023; 10002.41.9~6/RELEASE_ARM64_T8112 arm64

container uname -a -> Linux bdab893f99b2 6.11.6-orbstack-00279-g28c6c77332e6 #57 SMP Mon Nov 4 08:11:30 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
docker -v -> Docker version 27.3.1, build ce12230
docker-compose -v -> Docker Compose version 2.26.1

pumba stress --duration 100s mongo2

DEBU[0000] using names                                   names="[mongo2]"
DEBU[0000] stress testing all matching containers
DEBU[0000] listing matching containers                   duration=1m40s labels="[]" limit=0 names="[mongo2]" pattern= random=false stressors="[--cpu 4 --timeout 60s]"
DEBU[0000] listing containers
DEBU[0000] found container                               id=f54882f0956ea3370fb6354a209958f2de24bc34ad50c5b0857f8fb9bf2dd9d1 name=/determined_matsumoto
DEBU[0000] found container                               id=8ebc9117123c826d16fd7da93a9321adbb1208b7ce61e8da6605fe96ed5cd952 name=/mongo1
DEBU[0000] found container                               id=bdab893f99b2fa035ee875507b7f29f2c388c6d89fc955d0c435b00ed3d914b1 name=/mongo2
DEBU[0000] found container                               id=245f6cfedc64ca6246f1ca2df8d3d6864a65cb8295d88f082e0af4374c506200 name=/databases-redis-1
DEBU[0000] found container                               id=dd144691c31f263d47bd16419b7423621e189f0da5cd3e87a4589623198e7322 name=/databases-rabbitmq-1
DEBU[0000] found container                               id=2089d2846ba5ca090b4797e6f386ac38262b3601841daa64dc49b439d70860b4 name=/databases-nginx-1
DEBU[0000] stress testing container for duration         container=bdab893f99b2fa035ee875507b7f29f2c388c6d89fc955d0c435b00ed3d914b1 duration=1m40s pull image=true stress-ng image="alexeiled/stress-ng:latest-ubuntu" stressors="[--cpu 4 --timeout 60s]"
INFO[0000] stress testing container                      dryrun=false duration=1m40s id=bdab893f99b2fa035ee875507b7f29f2c388c6d89fc955d0c435b00ed3d914b1 image="alexeiled/stress-ng:latest-ubuntu" name=/mongo2 pull=true stressors="[--cpu 4 --timeout 60s]"
DEBU[0000] executing stress-ng command                   image="alexeiled/stress-ng:latest-ubuntu" pull=true stressors="[--cpu 4 --timeout 60s]" target=bdab893f99b2fa035ee875507b7f29f2c388c6d89fc955d0c435b00ed3d914b1
DEBU[0000] pulling stress-ng image                       image="alexeiled/stress-ng:latest-ubuntu"
DEBU[0002] &{Pulling from alexeiled/stress-ng   {0 0}}
DEBU[0002] &{Digest: sha256:6c2f6a6997aeb0dc7f7299e556f67cbb7ca1b40398ef32bbeea50b90fc020ae3   {0 0}}
DEBU[0002] &{Status: Image is up to date for alexeiled/stress-ng:latest-ubuntu   {0 0}}
DEBU[0002] creating stress-ng container                  image="alexeiled/stress-ng:latest-ubuntu"
DEBU[0002] stress-ng container created, starting it      id=ee20b6df8d73267ad46d7bd25d1d23cdf72c5d90415e9080865185c98f656be1
FATA[0003] error running stress command: error running chaos command: one or more stress test failed: stress-ng failed with error: stress-ng exited with error: /cgroups can't be listed: Cgroup is not mounted
)cgroup controller and pathparsing failed