Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LeaderElector calculates when the lease is expired incorrectly #1599

Open
gdemeter-kx opened this issue Nov 27, 2024 · 0 comments
Open

LeaderElector calculates when the lease is expired incorrectly #1599

gdemeter-kx opened this issue Nov 27, 2024 · 0 comments
Assignees

Comments

@gdemeter-kx
Copy link

gdemeter-kx commented Nov 27, 2024

Describe the bug
In case a Lease does not currently have a Leader who would renew the lease and it is already expired, the new Leader can't
immediately acquire it.

During leader election observedTime is set to observedTime = DateTimeOffset.Now which makes the calculation whether the Lease is expired incorrect. Eventually it will become the leader, but for config.LeaseDuration the Lease may not have a leader.

I created the following change on my fork, which seems to solve the issue:

  • observedTime should be the RenewTime of the observedRecord
  • Checking whether a Lease is expired should be calculated based on the observedRecord RenewTime and it's config. Not ours.

image

Kubernetes C# SDK Client Version
e.g. 9.0.1

Server Kubernetes Version
e.g. 1.22.3

Dotnet Runtime Version
e.g. net6

To Reproduce
Steps to reproduce the behavior:

  • Create a LeaderElector and dispose it. Wait for the Lease to be expired. Create another one for the same Lease. It will take config.LeaseDuration time to acquire the Lease even though it is expired.

Expected behavior
Considering the scenario explained in the To Reproduce part, I would expect the second Leader to immediately get the Lease.

KubeConfig
If applicable, add a KubeConfig file with secrets redacted.

Where do you run your app with Kubernetes SDK (please complete the following information):

  • OS: [e.g. Linux]
  • Environment [e.g. container]
  • Cloud [e.g. Azure]

Additional context
Add any other context about the problem here.

@gdemeter-kx gdemeter-kx changed the title LeaderElector calculates when the lease is expired initially LeaderElector calculates when the lease is expired wrong Nov 27, 2024
@gdemeter-kx gdemeter-kx changed the title LeaderElector calculates when the lease is expired wrong LeaderElector calculates when the lease is expired incorrectly Nov 28, 2024
@tg123 tg123 self-assigned this Nov 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants