Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubeapps-postgresql-0 CrashLoopBackOff #8146

Open
sjxuereb opened this issue Nov 25, 2024 · 4 comments
Open

kubeapps-postgresql-0 CrashLoopBackOff #8146

sjxuereb opened this issue Nov 25, 2024 · 4 comments
Labels
kind/bug An issue that reports a defect in an existing feature

Comments

@sjxuereb
Copy link

sjxuereb commented Nov 25, 2024

Describe the bug

Postgres keeps crashing after installation. I tried with a standard installation and also with an nfs storageclass. it keeps crashing on the simplest of installations/configuration. I tried installing kubeapps several times.

After logging in kubeapps dashboard the catalog takes a long time to load and then I notice the postgres pod crashes. The pod revives and crashes on every dashboard command.

kubeapps-postgresql-0 0/1 CrashLoopBackOff

OS: Ubuntu 24.04.1 LTS
K8: Client Version: v1.31.3, Kustomize Version: v5.4.2, Server Version: v1.31.3
Helm: Version:"v3.16.3"

CHART NAME: kubeapps
CHART VERSION: 17.0.3
APP VERSION: 2.12.0**

To Reproduce

helm install kubeapps oci://registry-1.docker.io/bitnamicharts/kubeapps --namespace kapps --set useHelm3=true --set postgresql.resourcesPreset=2xlarge

Control 10g Ram /8 cores/ 32G disk
Workers 8g Ram /4cores/ 32G disk

Expected behavior

Normal behavior of the postgres and kubeapps dashboard.

Screenshots

Additional context
I also tried with a storageclass. Same results, postgres keeps crashing.

--set global.defaultStorageClass=nfs-client
--set postgresql.primary.persistence.enabled=true
--set redis.master.persistence.enabled=true
--set redis.replica.persistence.enabled=true

@sjxuereb sjxuereb added the kind/bug An issue that reports a defect in an existing feature label Nov 25, 2024
@sjxuereb
Copy link
Author

I have since re-installed the chart as:

helm install kubeapps oci://registry-1.docker.io/bitnamicharts/kubeapps -n kubeapps
--set global.defaultStorageClass=nfs-client
--set frontend.service.type=LoadBalancer
--set postgresql.primary.persistence.enabled=true
--set postgresql.auth.postgresPassword=password
--set postgresql.resourcesPreset=medium
--set redis.master.persistence.enabled=true
--set redis.auth.password=password
--set redis.master.resourcesPreset=medium

The data folder has been created on the nfs and postgres runs for a few minutes. It then dies out of memory.

journalctl -k | grep 'Killed process' ( on the worker node gives ) :
Memory cgroup out of memory: Killed process 32350 (postgres)

As an extra test I installed the postgres chart on its own and it worked with no issues. However the kubeapps embedded chart is failing with OOMKilled
https://github.com/bitnami/charts/tree/main/bitnami/postgresql#parameters

Any help please. Thanks

@singatias
Copy link

I had the same issue following the guide to test it out

@fea359
Copy link

fea359 commented Nov 29, 2024

Changing the StatefulSet of postgresql and increasing the upper limits of CPU and memory resources, for example, changing the CPU to 2 and the memory to 4096m, or directly changing to unlimited, can solve the problem.

@beltran-rubo
Copy link
Collaborator

That looks like a very specific issue for NFS storage volume. Depending on the performance of the dis, it is possible in certain scenarios require to increase the CPU & memory for postgres.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug An issue that reports a defect in an existing feature
Projects
None yet
Development

No branches or pull requests

4 participants