New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP Initial support for Bottlerocket OS #9855

Closed

rifelpet wants to merge 8 commits into kubernetes:master from rifelpet:bottlerocket

Member

rifelpet commented Sep 1, 2020 •

edited

Loading

https://github.com/bottlerocket-os/bottlerocket

ref: #8723

The userdata needs to be entirely different (in fact the userdata is just settings defined in TOML) so Kops needs to be aware of the AMI's OS at the time it creates userdata, so I added a new ImageFamily API field (following the name from eksctl but I'm happy to change it). I figured the new field would also be useful for Windows support, since that will need different userdata too.

This also requires that the Userdata model now depends on the CA keypair. I wasn't able to get that setup easily in unit testing, but the integration test covers it well.

for this PR:

make tests results consistent - I'm assuming i'll use the same CA key strategy used in the new PublicJWKS integration test
warn or fail API validation if certain IG fields are set that are incompatible with bottlerocket. gossip clusters wont be supported yet either since that would require access to the host's /etc/hosts. aws-iam-authenticator is also required to be enabled and setup to bind the nodes role with rbac permissions to join the cluster
images.md

for followup PRs:

add SSH support
add support for additionalUserdata so that users can provide additional bottlerocket settings. Or come up with a more integrated way of specifying these.

Limitations:

kubelet serving certificate will not work for metrics server

Contributor

k8s-ci-robot commented Sep 1, 2020

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

k8s-ci-robot added do-not-merge/work-in-progress cncf-cla: yes labels

k8s-ci-robot requested review from KashifSaadat and mikesplain

September 1, 2020 19:52

k8s-ci-robot added area/api area/provider/aws size/XL approved labels

Member Author

rifelpet commented Sep 1, 2020

/test pull-kops-e2e-kubernetes-aws

1 similar comment

Member

hakman commented Sep 1, 2020

/test pull-kops-e2e-kubernetes-aws

rifelpet force-pushed the bottlerocket branch from 000ec4c to a78b2a6 Compare

September 1, 2020 21:06

Member Author

rifelpet commented Sep 1, 2020

/test pull-kops-e2e-kubernetes-aws

Member Author

rifelpet commented Sep 1, 2020 •

edited

Loading

W0901 21:20:52.097277 6980 executor.go:131] error running task "LaunchTemplate/nodes-us-west-1a.e2e-42766ca32c-ff1eb.test-cncf-aws.k8s.io" (1m50s remaining to succeed): could not find Image for "bottlerocket/bottlerocket-aws-k8s-1.17-x86_64-v0.5.0-e0ddf1b"

apparently the bottlerocket AMIs are not published in every region, which doesn't play nicely with prow and boskos' randomly chosen regions.

aws/containers-roadmap#827 (comment)

/test pull-kops-e2e-kubernetes-aws

EDIT: and the bottlerocket AMIs have different owner IDs per region :(

rifelpet force-pushed the bottlerocket branch from a78b2a6 to 26b5a42 Compare

September 1, 2020 22:05

Member Author

rifelpet commented Sep 1, 2020

/test pull-kops-e2e-kubernetes-aws

rifelpet force-pushed the bottlerocket branch from 26b5a42 to 457e335 Compare

September 1, 2020 22:38

Member Author

rifelpet commented Sep 1, 2020

/test pull-kops-e2e-kubernetes-aws

rifelpet force-pushed the bottlerocket branch from 457e335 to f791381 Compare

September 1, 2020 23:11

Member Author

rifelpet commented Sep 1, 2020

/test pull-kops-e2e-kubernetes-aws

rifelpet force-pushed the bottlerocket branch from f791381 to 5023017 Compare

September 2, 2020 00:39

k8s-ci-robot added size/XXL and removed size/XL labels

Member Author

rifelpet commented Sep 2, 2020

/test pull-kops-e2e-kubernetes-aws

rifelpet force-pushed the bottlerocket branch from 5023017 to 09c3424 Compare

September 2, 2020 03:19

Member Author

rifelpet commented Sep 2, 2020

/test pull-kops-e2e-kubernetes-aws

rifelpet force-pushed the bottlerocket branch from 09c3424 to 6e92e6a Compare

September 2, 2020 03:27

Member Author

rifelpet commented Sep 2, 2020

/test pull-kops-e2e-kubernetes-aws

1 similar comment

Member Author

rifelpet commented Sep 2, 2020

/test pull-kops-e2e-kubernetes-aws

Contributor

mikesplain commented Sep 2, 2020

Awesome, glad to hear you started on this!

rifelpet force-pushed the bottlerocket branch from 5d21d61 to db53208 Compare

November 14, 2021 14:08

rifelpet added 7 commits

November 14, 2021 08:29


          Add API field for specifying the Image's OS

6af49a1


          Rename cloud-agnostic nodeup template

dac142f


          Add support for creating bottlerocket userdata

25f5fc1


          Add bottlerocket integration test

8ba590f


          Add preliminary documentation

1f91ee3


          Add IAMIdentityMappings

7d28945


          Include CSR approver RBAC resources in its own addon separate from no…

8add83a

…de-authorizer

rifelpet force-pushed the bottlerocket branch from db53208 to f121d7c Compare

November 14, 2021 14:41


          [WIP] Create bottlerocket instance group in e2e tests

c9ec417

rifelpet force-pushed the bottlerocket branch from f121d7c to c9ec417 Compare

November 14, 2021 14:58

Contributor

k8s-ci-robot commented Nov 14, 2021

@rifelpet: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot added the needs-rebase label

k8s-triage-robot commented Feb 26, 2022

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-ci-robot added the lifecycle/stale label

k8s-triage-robot commented Mar 28, 2022

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-ci-robot added lifecycle/rotten and removed lifecycle/stale labels

Member Author

rifelpet commented Mar 28, 2022

/remove-lifecycle rotten

k8s-ci-robot removed the lifecycle/rotten label

oded-dd commented May 11, 2022

Great Work! Any update on that?

k8s-triage-robot commented Aug 9, 2022

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-ci-robot added the lifecycle/stale label

Contributor

k8s-ci-robot commented Aug 31, 2022

@rifelpet: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-kops-verify-hashes	`9d0afd2`	link	`/test pull-kops-verify-hashes`
pull-kops-e2e-k8s-containerd	`fc9684d`	link	`/test pull-kops-e2e-k8s-containerd`
pull-kops-e2e-cni-canal	`fc9684d`	link	true	`/test pull-kops-e2e-cni-canal`
pull-kops-e2e-kubernetes-aws	`c9ec417`	link	true	`/test pull-kops-e2e-kubernetes-aws`
pull-kops-e2e-cni-cilium-ipv6	`c9ec417`	link	true	`/test pull-kops-e2e-cni-cilium-ipv6`
pull-kops-e2e-cni-weave	`c9ec417`	link	true	`/test pull-kops-e2e-cni-weave`
pull-kops-e2e-cni-kuberouter	`c9ec417`	link	true	`/test pull-kops-e2e-cni-kuberouter`
pull-kops-e2e-cni-calico-ipv6	`c9ec417`	link	true	`/test pull-kops-e2e-cni-calico-ipv6`
pull-kops-e2e-cni-amazonvpc	`c9ec417`	link	true	`/test pull-kops-e2e-cni-amazonvpc`
pull-kops-e2e-cni-calico	`c9ec417`	link	true	`/test pull-kops-e2e-cni-calico`
pull-kops-e2e-cni-flannel	`c9ec417`	link	true	`/test pull-kops-e2e-cni-flannel`
pull-kops-e2e-cni-cilium	`c9ec417`	link	true	`/test pull-kops-e2e-cni-cilium`
pull-kops-e2e-k8s-gce-cilium	`c9ec417`	link	true	`/test pull-kops-e2e-k8s-gce-cilium`
pull-kops-e2e-aws-karpenter	`c9ec417`	link	true	`/test pull-kops-e2e-aws-karpenter`
pull-kops-e2e-k8s-aws-calico	`c9ec417`	link	true	`/test pull-kops-e2e-k8s-aws-calico`
pull-kops-build	`c9ec417`	link	true	`/test pull-kops-build`
pull-kops-test	`c9ec417`	link	true	`/test pull-kops-test`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-triage-robot commented Sep 30, 2022

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-ci-robot added lifecycle/rotten and removed lifecycle/stale labels

k8s-triage-robot commented Oct 30, 2022

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen
Mark this PR as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Contributor

k8s-ci-robot commented Oct 30, 2022

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen

Mark this PR as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot closed this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/addons area/api area/documentation area/provider/aws cncf-cla: yes do-not-merge/work-in-progress lifecycle/rotten needs-rebase size/XXL