Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Add support to specify dedicated host ID and Affinity #5113

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

✨ Add support to specify dedicated host ID and Affinity #5113

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions api/v1beta1/awscluster_conversion.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ func (src *AWSCluster) ConvertTo(dstRaw conversion.Hub) error {
dst.Status.Bastion.PrivateDNSName = restored.Status.Bastion.PrivateDNSName
dst.Status.Bastion.PublicIPOnLaunch = restored.Status.Bastion.PublicIPOnLaunch
dst.Status.Bastion.CapacityReservationID = restored.Status.Bastion.CapacityReservationID
dst.Status.Bastion.HostPlacement = restored.Status.Bastion.HostPlacement
}
dst.Spec.Partition = restored.Spec.Partition

Expand Down
4 changes: 4 additions & 0 deletions api/v1beta1/awsmachine_conversion.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ func (src *AWSMachine) ConvertTo(dstRaw conversion.Hub) error {
dst.Spec.PrivateDNSName = restored.Spec.PrivateDNSName
dst.Spec.SecurityGroupOverrides = restored.Spec.SecurityGroupOverrides
dst.Spec.CapacityReservationID = restored.Spec.CapacityReservationID
dst.Spec.HostPlacement = restored.Spec.HostPlacement

if restored.Spec.ElasticIPPool != nil {
if dst.Spec.ElasticIPPool == nil {
dst.Spec.ElasticIPPool = &infrav1.ElasticIPPool{}
Expand Down Expand Up @@ -104,6 +106,8 @@ func (r *AWSMachineTemplate) ConvertTo(dstRaw conversion.Hub) error {
dst.Spec.Template.Spec.PrivateDNSName = restored.Spec.Template.Spec.PrivateDNSName
dst.Spec.Template.Spec.SecurityGroupOverrides = restored.Spec.Template.Spec.SecurityGroupOverrides
dst.Spec.Template.Spec.CapacityReservationID = restored.Spec.Template.Spec.CapacityReservationID
dst.Spec.Template.Spec.HostPlacement = restored.Spec.Template.Spec.HostPlacement

if restored.Spec.Template.Spec.ElasticIPPool != nil {
if dst.Spec.Template.Spec.ElasticIPPool == nil {
dst.Spec.Template.Spec.ElasticIPPool = &infrav1.ElasticIPPool{}
Expand Down
10 changes: 6 additions & 4 deletions api/v1beta1/zz_generated.conversion.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

36 changes: 35 additions & 1 deletion api/v1beta2/awsmachine_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,18 @@ const (
IgnitionStorageTypeOptionUnencryptedUserData = IgnitionStorageTypeOption("UnencryptedUserData")
)

// Tenancy defines the different tenancy options for EC2 instance.
type Tenancy string

const (
// TenancyDefault means that the EC2 instance will be launched into a shared host.
TenancyDefault = Tenancy("default")
// TenancyDedicated means that AWS will create and allocate a physical host for the EC2 instance (1 instance per host).
TenancyDedicated = Tenancy("dedicated")
// TenancyHost means that the EC2 instance will be launched into one of the user's pre-allocated physical hosts (multiple instances per host).
TenancyHost = Tenancy("host")
)

// AWSMachineSpec defines the desired state of an Amazon EC2 instance.
type AWSMachineSpec struct {
// ProviderID is the unique identifier as specified by the cloud provider.
Expand Down Expand Up @@ -186,9 +198,18 @@ type AWSMachineSpec struct {
PlacementGroupPartition int64 `json:"placementGroupPartition,omitempty"`

// Tenancy indicates if instance should run on shared or single-tenant hardware.
// If host tenancy is used:
// - if there are no Dedicated Hosts with auto-placement enabled that match the instance type configuration, then it is required to set '.spec.hostPlacment.hostID' or no host would be picked.
// - if '.spec.hostPlacment.hostID' is set, the instance will be launched into the specified Host, disregarding any auto-placement settings.
// see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/dedicated-hosts-understanding.html#dedicated-hosts-auto-placement
//
// +optional
// +kubebuilder:validation:Enum:=default;dedicated;host
Tenancy string `json:"tenancy,omitempty"`
Tenancy Tenancy `json:"tenancy,omitempty"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By reading this API is not clear to me which input should I set for tenancy host with autoplacement, can we document that here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added more documentation, let me know if it makes sense.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! "If host tenancy is used and auto-placement is disabled for the Dedicated Host, It is required to set '.spec.hostPlacment.hostID'" though the requirement is actually not enforced. Can we document what happens if not set in that case, e.g. it's required or no host would be picked. Can we also document all matrix combinations this API enables: If both hostID and autoplacement are set, the hostID would take precedence? If the hostID doesn't exist would it fallback to autoplacement?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can't enforce it, because we don't know if auto-placement is enabled or not on the dedicated Host, since its created directly by the user out of band.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated.


// HostPlacement denotes the placement settings for the instance when tenancy=host.
// +optional
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be PlacementHost consistently with placementGroup and placementGroupPartition?

Can we please open ticket to capture putting all placements choices in a common struct for future api bump?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about just Placement then, so we can group all options inside it in the next api bump?

HostPlacement *HostPlacement `json:"hostPlacment,omitempty"`

// PrivateDNSName is the options for the instance hostname.
// +optional
Expand All @@ -199,6 +220,19 @@ type AWSMachineSpec struct {
CapacityReservationID *string `json:"capacityReservationId,omitempty"`
}

// HostPlacement denotes the placement settings for the instance when tenancy=host.
type HostPlacement struct {
// HostID pins the instance to a sepcific Dedicated Host.
// +optional
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't a hostID required if I set HostPlacement?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? you might just want to set the affinity.

HostID *string `json:"hostID,omitempty"`

// Affinity indicates the affinity setting between the instance and a Dedicated Host.
// When affinity is set to host, an instance launched onto a specific host always restarts on the same host if stopped. This applies to both targeted and untargeted launches.
// +optional
// +kubebuilder:validation:Enum:=default;host
Affinity *string `json:"affinity,omitempty"`
}

// CloudInit defines options related to the bootstrapping systems where
// CloudInit is used.
type CloudInit struct {
Expand Down
12 changes: 11 additions & 1 deletion api/v1beta2/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,8 +239,18 @@ type Instance struct {
PlacementGroupPartition int64 `json:"placementGroupPartition,omitempty"`

// Tenancy indicates if instance should run on shared or single-tenant hardware.
// If host tenancy is used:
// - if there are no Dedicated Hosts with auto-placement enabled that match the instance type configuration, then it is required to set '.spec.hostPlacment.hostID' or no host would be picked.
// - if '.spec.hostPlacment.hostID' is set, the instance will be launched into the specified Host, disregarding any auto-placement settings.
// see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/dedicated-hosts-understanding.html#dedicated-hosts-auto-placement
//
// +optional
// +kubebuilder:validation:Enum:=default;dedicated;host
Tenancy Tenancy `json:"tenancy,omitempty"`

// HostPlacement denotes the placement settings for the instance when tenancy=host.
// +optional
Tenancy string `json:"tenancy,omitempty"`
HostPlacement *HostPlacement `json:"hostPlacment,omitempty"`

// IDs of the instance's volumes
// +optional
Expand Down
35 changes: 35 additions & 0 deletions api/v1beta2/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

58 changes: 54 additions & 4 deletions config/crd/bases/controlplane.cluster.x-k8s.io_awsmanagedcontrolplanes.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1133,6 +1133,23 @@ spec:
description: Specifies whether enhanced networking with ENA is
enabled.
type: boolean
hostPlacment:
description: HostPlacement denotes the placement settings for
the instance when tenancy=host.
properties:
affinity:
description: |-
Affinity indicates the affinity setting between the instance and a Dedicated Host.
When affinity is set to host, an instance launched onto a specific host always restarts on the same host if stopped. This applies to both targeted and untargeted launches.
enum:
- default
- host
type: string
hostID:
description: HostID pins the instance to a sepcific Dedicated
Host.
type: string
type: object
iamProfile:
description: The name of the IAM instance profile associated with
the instance, if applicable.
Expand Down Expand Up @@ -1374,8 +1391,16 @@ spec:
description: The tags associated with the instance.
type: object
tenancy:
description: Tenancy indicates if instance should run on shared
or single-tenant hardware.
description: |-
Tenancy indicates if instance should run on shared or single-tenant hardware.
If host tenancy is used:
- if there are no Dedicated Hosts with auto-placement enabled that match the instance type configuration, then it is required to set '.spec.hostPlacment.hostID' or no host would be picked.
- if '.spec.hostPlacment.hostID' is set, the instance will be launched into the specified Host, disregarding any auto-placement settings.
see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/dedicated-hosts-understanding.html#dedicated-hosts-auto-placement
enum:
- default
- dedicated
- host
type: string
type:
description: The instance type.
Expand Down Expand Up @@ -3177,6 +3202,23 @@ spec:
description: Specifies whether enhanced networking with ENA is
enabled.
type: boolean
hostPlacment:
description: HostPlacement denotes the placement settings for
the instance when tenancy=host.
properties:
affinity:
description: |-
Affinity indicates the affinity setting between the instance and a Dedicated Host.
When affinity is set to host, an instance launched onto a specific host always restarts on the same host if stopped. This applies to both targeted and untargeted launches.
enum:
- default
- host
type: string
hostID:
description: HostID pins the instance to a sepcific Dedicated
Host.
type: string
type: object
iamProfile:
description: The name of the IAM instance profile associated with
the instance, if applicable.
Expand Down Expand Up @@ -3418,8 +3460,16 @@ spec:
description: The tags associated with the instance.
type: object
tenancy:
description: Tenancy indicates if instance should run on shared
or single-tenant hardware.
description: |-
Tenancy indicates if instance should run on shared or single-tenant hardware.
If host tenancy is used:
- if there are no Dedicated Hosts with auto-placement enabled that match the instance type configuration, then it is required to set '.spec.hostPlacment.hostID' or no host would be picked.
- if '.spec.hostPlacment.hostID' is set, the instance will be launched into the specified Host, disregarding any auto-placement settings.
see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/dedicated-hosts-understanding.html#dedicated-hosts-auto-placement
enum:
- default
- dedicated
- host
type: string
type:
description: The instance type.
Expand Down
29 changes: 27 additions & 2 deletions config/crd/bases/infrastructure.cluster.x-k8s.io_awsclusters.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2118,6 +2118,23 @@ spec:
description: Specifies whether enhanced networking with ENA is
enabled.
type: boolean
hostPlacment:
description: HostPlacement denotes the placement settings for
the instance when tenancy=host.
properties:
affinity:
description: |-
Affinity indicates the affinity setting between the instance and a Dedicated Host.
When affinity is set to host, an instance launched onto a specific host always restarts on the same host if stopped. This applies to both targeted and untargeted launches.
enum:
- default
- host
type: string
hostID:
description: HostID pins the instance to a sepcific Dedicated
Host.
type: string
type: object
iamProfile:
description: The name of the IAM instance profile associated with
the instance, if applicable.
Expand Down Expand Up @@ -2359,8 +2376,16 @@ spec:
description: The tags associated with the instance.
type: object
tenancy:
description: Tenancy indicates if instance should run on shared
or single-tenant hardware.
description: |-
Tenancy indicates if instance should run on shared or single-tenant hardware.
If host tenancy is used:
- if there are no Dedicated Hosts with auto-placement enabled that match the instance type configuration, then it is required to set '.spec.hostPlacment.hostID' or no host would be picked.
- if '.spec.hostPlacment.hostID' is set, the instance will be launched into the specified Host, disregarding any auto-placement settings.
see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/dedicated-hosts-understanding.html#dedicated-hosts-auto-placement
enum:
- default
- dedicated
- host
type: string
type:
description: The instance type.
Expand Down
25 changes: 23 additions & 2 deletions config/crd/bases/infrastructure.cluster.x-k8s.io_awsmachines.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -692,6 +692,23 @@ spec:
- message: allowed values are 'none' and 'amazon-pool'
rule: self in ['none','amazon-pool']
type: object
hostPlacment:
description: HostPlacement denotes the placement settings for the
instance when tenancy=host.
properties:
affinity:
description: |-
Affinity indicates the affinity setting between the instance and a Dedicated Host.
When affinity is set to host, an instance launched onto a specific host always restarts on the same host if stopped. This applies to both targeted and untargeted launches.
enum:
- default
- host
type: string
hostID:
description: HostID pins the instance to a sepcific Dedicated
Host.
type: string
type: object
iamInstanceProfile:
description: IAMInstanceProfile is a name of an IAM instance profile
to assign to the instance
Expand Down Expand Up @@ -1085,8 +1102,12 @@ spec:
type: string
type: object
tenancy:
description: Tenancy indicates if instance should run on shared or
single-tenant hardware.
description: |-
Tenancy indicates if instance should run on shared or single-tenant hardware.
If host tenancy is used:
- if there are no Dedicated Hosts with auto-placement enabled that match the instance type configuration, then it is required to set '.spec.hostPlacment.hostID' or no host would be picked.
- if '.spec.hostPlacment.hostID' is set, the instance will be launched into the specified Host, disregarding any auto-placement settings.
see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/dedicated-hosts-understanding.html#dedicated-hosts-auto-placement
enum:
- default
- dedicated
Expand Down
Loading
Loading