Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: cluster-level resource scheduling suspend and resume capabilities #5937

Merged
merged 2 commits into from
Jan 2, 2025

Conversation

Monokaix
Copy link
Contributor

@Monokaix Monokaix commented Dec 11, 2024

What type of PR is this?
/kind api-change
/kind feature

What this PR does / why we need it:
Add rb suspension capability, background: #5690

Which issue(s) this PR fixes:
Part of #5690

Special notes for your reviewer:
It extends the Suspension fileds and support rb susupend and resume, mostly like the previous issue #5217 and PRs implemented in it.
This pr just implemented binding suspension in scheduler , setting status condition in controller will be implemented in another pr after #6002 merged because it may lead to status update conflict between scheduler and controller.

Does this PR introduce a user-facing change?:

`API Change`: Introduced `Scheduling` suspension in both `ResourceBinding` and `ClusterResourceBinding` which will be used for third-party systems to suspend application scheduling.

@karmada-bot karmada-bot added the kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API label Dec 11, 2024
@karmada-bot karmada-bot added the kind/feature Categorizes issue or PR as related to a new feature. label Dec 11, 2024
@karmada-bot
Copy link
Collaborator

Welcome @Monokaix! It looks like this is your first PR to karmada-io/karmada 🎉

@karmada-bot karmada-bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Dec 11, 2024
@karmada-bot karmada-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Dec 11, 2024
@karmada-bot karmada-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Dec 11, 2024
@Monokaix Monokaix force-pushed the rb-suspend branch 2 times, most recently from c4d5f12 to 08407d6 Compare December 12, 2024 06:56
@karmada-bot karmada-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Dec 12, 2024
@Monokaix
Copy link
Contributor Author

@karmada-bot karmada-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Dec 12, 2024
@codecov-commenter
Copy link

codecov-commenter commented Dec 12, 2024

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 48.39%. Comparing base (ba1e68d) to head (53dd5b6).

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5937      +/-   ##
==========================================
+ Coverage   48.37%   48.39%   +0.02%     
==========================================
  Files         665      665              
  Lines       54795    54811      +16     
==========================================
+ Hits        26507    26527      +20     
+ Misses      26569    26567       -2     
+ Partials     1719     1717       -2     
Flag Coverage Δ
unittests 48.39% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@karmada-bot karmada-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Dec 12, 2024
@Monokaix Monokaix force-pushed the rb-suspend branch 5 times, most recently from 187911e to 208246d Compare December 28, 2024 10:38
Comment on lines 325 to 327
// Suspension defines the policy for suspending of propagation, dispatching and scheduling suspension are independent
// of each other. The propagation suspension inherited from PropagationPolicy can be transparently transmitted here,
// and ResourceBinding can also set scheduling suspension separately.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Suspension defines the policy for suspending of propagation, dispatching and scheduling suspension are independent
// of each other. The propagation suspension inherited from PropagationPolicy can be transparently transmitted here,
// and ResourceBinding can also set scheduling suspension separately.
// Suspension defines the policy for suspending dispatching and scheduling.

Comment on lines 331 to 334
// Scheduling controls whether scheduling should be suspended, the scheduler will pause scheduling and not
// process resource binding after it's created when value is true and resume scheduling when it's false or nil.
// This is usually used for admission checks by third-party such as quota check, multi-tenancy prioritization
// and maybe more checks in the future before scheduler really does scheduling.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Scheduling controls whether scheduling should be suspended, the scheduler will pause scheduling and not
// process resource binding after it's created when value is true and resume scheduling when it's false or nil.
// This is usually used for admission checks by third-party such as quota check, multi-tenancy prioritization
// and maybe more checks in the future before scheduler really does scheduling.
// Scheduling controls whether scheduling should be suspended, the scheduler will pause scheduling and not
// process resource binding when the value is true and resume scheduling when it's false or nil.
// This is designed for third-party systems to temporarily pause the scheduling of applications, which enabling
// manage resource allocation, prioritize critical workloads, etc.
//
// It is expected that third-party systems use an admission webhook to suspend scheduling at the time of
// ResourceBinding creation. Once a ResourceBinding has been scheduled, it cannot be paused afterward, as it may
// lead to ineffective suspension.
// +optional

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. note that scheduling can not be paused after it has been scheduled.
  2. +optional

Copy link
Member

@RainbowMango RainbowMango left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good to me except the condition part.
I suggest moving the condition out of this PR and starting another PR to handle it.

pkg/util/binding.go Outdated Show resolved Hide resolved
pkg/util/binding.go Outdated Show resolved Hide resolved
@Monokaix
Copy link
Contributor Author

Generally looks good to me except the condition part. I suggest moving the condition out of this PR and starting another PR to handle it.

You mean seperate the webhook validation to another pr?

@RainbowMango
Copy link
Member

I mean not introduce the condition by this PR:

	// SchedulingSuspended represents the condition that the ResourceBinding or ClusterResourceBinding is suspended to schedule.
	SchedulingSuspended string = "SchedulingSuspended"

@karmada-bot karmada-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jan 2, 2025
@Monokaix
Copy link
Contributor Author

Monokaix commented Jan 2, 2025

I mean not introduce the condition by this PR:

	// SchedulingSuspended represents the condition that the ResourceBinding or ClusterResourceBinding is suspended to schedule.
	SchedulingSuspended string = "SchedulingSuspended"

Upated.

Comment on lines 360 to 361
// +listType=map
// +listMapKey=type
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why add these? What does it mean?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be useful to manage conditions as a map. but current PR doesn't require it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we can track it at #6002, which is needed by ApplyStatus method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed.

@RainbowMango RainbowMango added this to the v1.13 milestone Jan 2, 2025
@RainbowMango RainbowMango added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Jan 2, 2025
Copy link
Member

@RainbowMango RainbowMango left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@karmada-bot karmada-bot added the lgtm Indicates that a PR is ready to be merged. label Jan 2, 2025
@karmada-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: RainbowMango

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@karmada-bot karmada-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 2, 2025
@karmada-bot karmada-bot merged commit 7a85687 into karmada-io:master Jan 2, 2025
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants