Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Observed panics during cluster topology change #694

Closed
jstncbllr opened this issue Dec 12, 2024 · 2 comments · Fixed by #695
Closed

Observed panics during cluster topology change #694

jstncbllr opened this issue Dec 12, 2024 · 2 comments · Fixed by #695

Comments

@jstncbllr
Copy link

jstncbllr commented Dec 12, 2024

While switching a cluster in AWS from redis to valkey, also modifying the topology, I observed a couple of panics like:

PANIC: runtime error: index out of range [423] with length 0

runtime/debug.Stack()
	/go/pkg/mod/golang.org/[email protected]/src/runtime/debug/stack.go:24 +0x5e
golang.org/x/sync/singleflight.newPanicError({0x21c0a20, 0xc1983cad50})
	/go/pkg/mod/golang.org/x/[email protected]/singleflight/singleflight.go:44 +0x25
golang.org/x/sync/singleflight.(*Group).doCall.func2.1()
	/go/pkg/mod/golang.org/x/[email protected]/singleflight/singleflight.go:193 +0x34
panic({0x21c0a20?, 0xc1983cad50?})
	/go/pkg/mod/golang.org/[email protected]/src/runtime/panic.go:770 +0x132
github.com/redis/rueidis.(*clusterClient).redirectOrNew(0xc002032000, {0xc20be37d4a, 0x51}, {0x2a415e0, 0xc0024a1e00}, 0x1a7, 0x1)
	/go/pkg/mod/github.com/redis/[email protected]/cluster.go:446 +0x4de
github.com/redis/rueidis.(*clusterClient).do(0xc002032000, {0x2a2d000, 0xc20a6deed0}, {0xc194081a40, 0x2000, 0x1a7})
	/go/pkg/mod/github.com/redis/[email protected]/cluster.go:478 +0x6b1
github.com/redis/rueidis.(*clusterClient).Do(0xc036798f78?, {0x2a2d000?, 0xc20a6deed0?}, {0xc194081a40?, 0x8f20?, 0x3679?})
	/go/pkg/mod/github.com/redis/[email protected]/cluster.go:461 +0x7b

Not sure why. Perhaps the SHARDS command didn't return any groups for a short while?

Possibly there could be a more graceful behavior in the client in this case?

@rueian
Copy link
Collaborator

rueian commented Dec 12, 2024

Hi @jstncbllr,

Thanks for reporting the issue. We are fixing it #695.

@rueian
Copy link
Collaborator

rueian commented Dec 12, 2024

If you are in a hurry, you can try the fix with v1.0.52-alpha.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants