From 3e02dc8f67d71425d17d08f3b420f301a056a371 Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Mon, 2 Dec 2024 21:50:31 +0000 Subject: [PATCH 01/18] first draft Signed-off-by: Austin Abro --- 6-manifest-preview-capability/README.md | 370 ++++++++++++++++++++++++ 6-manifest-preview-capability/zep.yaml | 24 ++ 2 files changed, 394 insertions(+) create mode 100644 6-manifest-preview-capability/README.md create mode 100644 6-manifest-preview-capability/zep.yaml diff --git a/6-manifest-preview-capability/README.md b/6-manifest-preview-capability/README.md new file mode 100644 index 0000000..d88a282 --- /dev/null +++ b/6-manifest-preview-capability/README.md @@ -0,0 +1,370 @@ + + +# ZEP-NNNN: Your short, descriptive title + + + + + + +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) +- [Proposal](#proposal) + - [User Stories (Optional)](#user-stories-optional) + - [Story 1](#story-1) + - [Story 2](#story-2) + - [Risks and Mitigations](#risks-and-mitigations) +- [Design Details](#design-details) + - [Test Plan](#test-plan) + - [Prerequisite testing updates](#prerequisite-testing-updates) + - [Unit tests](#unit-tests) + - [e2e tests](#e2e-tests) + - [Graduation Criteria](#graduation-criteria) + - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy) + - [Version Skew Strategy](#version-skew-strategy) +- [Implementation History](#implementation-history) +- [Drawbacks](#drawbacks) +- [Alternatives](#alternatives) +- [Infrastructure Needed (Optional)](#infrastructure-needed-optional) + + +## Summary + + + +Introduce a new command called `zarf package preview` which will have the ability to display a zarf package yaml after templated, imports, and flavors are applied. Alongside manifests or values files that will be deployed. Additionally, enhance the `zarf package inspect` command to give it the ability to display manifests or values files. + +## Motivation + + + +There is no easy way to see a zarf.yaml file that will be created after templating and importing are finished outside of `zarf package create`. Furthermore, there is not a convenient way to see the manifests and values files that will be deployed by a Zarf package. + +This feature would put is in parity with other tools that have similar pre-deploy viewing capabilities. The most obvious example is `helm template`, as Zarf would call the Helm library template functions within `zarf package preview`. Other tools have similar capabilities such as opentofu with `tf plan`. + +This feature has been highly requested in recent months: +- request in Kubernetes slack - https://kubernetes.slack.com/archives/C03B6BJAUJ3/p1730229638367829 +- An issue has been created for this - https://github.com/zarf-dev/zarf/issues/2631 +- Defense Unicorns, an organization that relies heavily on Zarf for their deployments, have received requests for this feature in a feedback session with their partners. + +### Goals + + + +The goal for creators is that they have all the information they need to build a working package as quickly as possible. + +The goal for deployers is that they feel confident that they know what they're deploying and that the Zarf variable templating will work properly. + +### Non-Goals + + + +We do not want to show the manifests of a package that is already deployed. `zarf package inspect` works on packages in the cluster, if a user tries the `--show-manifests` flag or `--values` flag for a package in the cluster they will receive an error. There is already a [helm get manifest](https://helm.sh/docs/helm/helm_get_manifest/) command that the user may use when already deployed to a cluster. + +## Proposal + + + +Introduce a new command called `zarf package preview` command. This command will print a zarf.yaml after templates, imports, and flavors are applied. The help text and flags of this command would look like below. Importantly, the `--show-manifests` and `--show-values-files` flags would print out what the manifests and values files would look like after templating has occurred. + +``` +Usage: + zarf package preview [ PACKAGE ] [flags] + +Flags: + --create-set stringToString Specify package variables to set on the command line (KEY=value) (default []) + --deploy-set stringToString Specify deployment variables to set on the command line (KEY=value) (default []) + -f, --flavor string The flavor of components to include in the resulting package (i.e. have a matching or empty "only.flavor" key) + --show-manifests shows the manifests that would be deployed by this package + --show-values-files shows the values files that would be used by the charts in this package +``` + + +### User Stories (Optional) + + + +#### Story 1 + +As a creator of Zarf packages I want to see what my zarf.yaml will look like after templates, imports, and flavors are applied so I run `zarf package preview -f my-flavor --create-set=MY_TEMPLATE=my-val` + +#### Story 2 + +As a creator of Zarf packages I want to make sure the variables in my package can get templated properly for the expected values of the deployers. I want to check this for both manifests and values files. `zarf package preview --deploy-set=MY_VAR=my-val --show-manifests` or `zarf package preview --deploy-set=MY_VAR=my-val --show-values-file` + +#### Story 3 + +As a deployer of Zarf packages, I want to check that the variables I intend to deploy my package with are getting properly templated for both manifests and values files. + +### Risks and Mitigations + + + +This will not cause any security impacts. There is no sensitive data that this will print out that the user does not already have access to. This command will be able to print Kubernetes secrets to the console, but given this is expected given that it is printing manifests. + +This command could print Zarf variables with the `sensitive` value set to true. The value to these variables can be assigned with user input, config files, and the `default` key in the zarf.yaml. Given that these commands are expected to be run by a user actively managing a cluster or developing a package and not in an automated system we deem these risks acceptable. + +**QUESTION** Am I making too broad an assumption that this risk is acceptable? + +## Design Details + + + +[Internal variables](https://docs.zarf.dev/ref/values/#internal-values-zarf_) will be set using the default logic. This means the `ZARF_REGISTRY` variable will become `127.0.0.1:31999`, while secrets like the `GIT_AUTH_PUSH` variable will become a random string. This will follow the same logic as `zarf dev find-images`. + +### Test Plan + + + +[X] I/we understand the owners of the involved components may require updates to +existing tests to make this code solid enough prior to committing the changes necessary +to implement this proposal. + +##### Prerequisite testing updates + + + +N/A + +##### Unit tests + + + + + +- ``: `` - `` + +##### e2e tests + + + +Given the simplicity of this feature, unit tests will be adequate. + +### Graduation Criteria + + + +### Upgrade / Downgrade Strategy + + + +N/A + +### Version Skew Strategy + + + +N/A + +## Implementation History + + + +## Drawbacks + + + +## Alternatives + + + +One alternative was to use `zarf dev preview`, the thought being that this command will be run by creators while developing a package. However, given the similarities between this command and `zarf package inspect` we decided that it made for a more cohesive user experience to have both commands under the same parent. + +I pondered if we should accept a component flag as we do on deploy during package inspect or package preview. This would allow people to get a more accurate view of the manifests that they intend to deploy if they are using optional components. However, this flag would only make sense alongside the It does make the code more complicated, but it can give a better view of what will actually be deployed. Given the added complexity, we are not added this for now, but it could be an enhancement in the future. + +Another alternative would be to have completely separate commands to show the manifests. The `zarf package preview` command would exist solely to preview a package and the `zarf package inspect` command would be unchanged. Instead we'd introduce the commands `zarf dev show-manifests` and `zarf dev show-values-files`. These commands would have the `--deploy-set` and `--create-set` flags, while `zarf package preview` would only need `--set`. Additionally, this would make it easier to add a flag like `--components` which wouldn't make sense for the `zarf package preview` command as all components are included on create regardless of if they are used or not. The `show-manifests` and `show-values-files` commands would take either a zarf.yaml or a zarf package. This would also diff --git a/6-manifest-preview-capability/zep.yaml b/6-manifest-preview-capability/zep.yaml new file mode 100644 index 0000000..ec1617b --- /dev/null +++ b/6-manifest-preview-capability/zep.yaml @@ -0,0 +1,24 @@ +schema-version: 1.0.0 + +title: Manifest Preview Capability +zep-number: NNNN +authors: + - "@austinabro321" +status: provisional|implementable|implemented|deferred|rejected|withdrawn|replaced +creation-date: yyyy-mm-dd +reviewers: + - TBD +approvers: + - TBD + +# The target maturity stage in the current dev cycle for this ZEP. +stage: alpha|beta|stable + +# The most recent milestone for which work toward delivery of this ZEP has been +# done. This can be the current (upcoming) milestone, if it is being actively +# worked on. +latest-milestone: "v0.43.1" + +# The milestone at which this feature was, or is targeted to be, at each stage. +milestone: + stable: "TBD" From b140f1f77c58e377ffd8dc6bfe440ab72c662ff5 Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Mon, 2 Dec 2024 22:01:10 +0000 Subject: [PATCH 02/18] update alternative Signed-off-by: Austin Abro --- 6-manifest-preview-capability/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/6-manifest-preview-capability/README.md b/6-manifest-preview-capability/README.md index d88a282..eb1ca68 100644 --- a/6-manifest-preview-capability/README.md +++ b/6-manifest-preview-capability/README.md @@ -367,4 +367,4 @@ One alternative was to use `zarf dev preview`, the thought being that this comma I pondered if we should accept a component flag as we do on deploy during package inspect or package preview. This would allow people to get a more accurate view of the manifests that they intend to deploy if they are using optional components. However, this flag would only make sense alongside the It does make the code more complicated, but it can give a better view of what will actually be deployed. Given the added complexity, we are not added this for now, but it could be an enhancement in the future. -Another alternative would be to have completely separate commands to show the manifests. The `zarf package preview` command would exist solely to preview a package and the `zarf package inspect` command would be unchanged. Instead we'd introduce the commands `zarf dev show-manifests` and `zarf dev show-values-files`. These commands would have the `--deploy-set` and `--create-set` flags, while `zarf package preview` would only need `--set`. Additionally, this would make it easier to add a flag like `--components` which wouldn't make sense for the `zarf package preview` command as all components are included on create regardless of if they are used or not. The `show-manifests` and `show-values-files` commands would take either a zarf.yaml or a zarf package. This would also +Another alternative would be to have completely separate commands that show the zarf.yaml file and the commands that show the manifests and values files. The `zarf package preview` command would exist solely to view a zarf.yaml and the `zarf package inspect` command would be unchanged. Instead we'd introduce the commands `zarf dev show-manifests` and `zarf dev show-values-files`. These commands would have the `--deploy-set` and `--create-set` flags, while `zarf package preview` would only need `--set`. This would increase the surface area of the CLI, but give commands a more distinct purpose. This would help the flags be more sensible as well. Helm templating would require a `--kube-version` for charts with a certain version set. A `--components` flag would be nice for users who want to see the manifests they are deploying without optional components included, but wouldn't make sense for the `zarf package preview` command since all components are included on create regardless of if they are used or not. The `show-manifests` and `show-values-files` commands would take either a zarf.yaml or a zarf package. From 3ba26f6de7342ab38d38e7883ca2a66c3125f051 Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Tue, 3 Dec 2024 16:15:01 +0000 Subject: [PATCH 03/18] edits Signed-off-by: Austin Abro --- 6-manifest-preview-capability/README.md | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/6-manifest-preview-capability/README.md b/6-manifest-preview-capability/README.md index eb1ca68..ad18a5c 100644 --- a/6-manifest-preview-capability/README.md +++ b/6-manifest-preview-capability/README.md @@ -78,7 +78,6 @@ any additional information provided beyond the standard ZEP template. - [Implementation History](#implementation-history) - [Drawbacks](#drawbacks) - [Alternatives](#alternatives) -- [Infrastructure Needed (Optional)](#infrastructure-needed-optional) ## Summary @@ -98,7 +97,9 @@ feedback and reduce unnecessary changes. [documentation style guide]: https://docs.zarf.dev/contribute/style-guide/ --> -Introduce a new command called `zarf package preview` which will have the ability to display a zarf package yaml after templated, imports, and flavors are applied. Alongside manifests or values files that will be deployed. Additionally, enhance the `zarf package inspect` command to give it the ability to display manifests or values files. +Users need an easier way to view what their zarf.yaml will look like after templating, imports, and flavors are applied. Users need a way to view their manifests and values files after Zarf variables are applied. + +This will be accomplished through a new CLI command `zarf package preview` and enhancements to `zarf package inspect` ## Motivation @@ -113,9 +114,9 @@ or other references to show the community's interest in the ZEP. [kubernetes slack]: https://kubernetes.slack.com/archives/C03B6BJAUJ3 --> -There is no easy way to see a zarf.yaml file that will be created after templating and importing are finished outside of `zarf package create`. Furthermore, there is not a convenient way to see the manifests and values files that will be deployed by a Zarf package. +Forcing users to use the the print before the (y/n) prompt in `zarf package create` to view their "complied" zarf.yaml has been a less than ideal UX. Users not being able to see if their manifests are templated properly before create and deploy can lead to much longer cycle time. A Helm template is less than a few seconds, whereas create + deploy could take several minutes or even an hour+. -This feature would put is in parity with other tools that have similar pre-deploy viewing capabilities. The most obvious example is `helm template`, as Zarf would call the Helm library template functions within `zarf package preview`. Other tools have similar capabilities such as opentofu with `tf plan`. +This feature would put is in parity with other tools that have similar pre-deploy viewing capabilities. The most obvious example is `helm template`, as Zarf would essentially run a Helm template to get the manifests. Other tools have similar capabilities such as opentofu with `tf plan`. This feature has been highly requested in recent months: - request in Kubernetes slack - https://kubernetes.slack.com/archives/C03B6BJAUJ3/p1730229638367829 @@ -152,7 +153,7 @@ desired outcome and how success will be measured. The "Design Details" section below is for the real nitty-gritty. --> -Introduce a new command called `zarf package preview` command. This command will print a zarf.yaml after templates, imports, and flavors are applied. The help text and flags of this command would look like below. Importantly, the `--show-manifests` and `--show-values-files` flags would print out what the manifests and values files would look like after templating has occurred. +Introduce a new command called `zarf package preview`. This command will print a zarf.yaml after templates, imports, and flavors are applied. The help text and flags of this command would look like below. Importantly, the `--show-manifests` and `--show-values-files` flags would print out what the manifests and values files would look like after templating has occurred. ``` Usage: @@ -186,7 +187,7 @@ As a creator of Zarf packages I want to make sure the variables in my package ca #### Story 3 -As a deployer of Zarf packages, I want to check that the variables I intend to deploy my package with are getting properly templated for both manifests and values files. +As a deployer of Zarf packages, I want to check that the variables I intend to deploy my package with are getting properly templated for both manifests and values files so I run `zarf package preview --deploy-set=MY_VAR=my-val --show-manifests` or `zarf package preview --deploy-set=MY_VAR=my-val --show-values-file` ### Risks and Mitigations @@ -202,7 +203,7 @@ How will UX be reviewed, and by whom? This will not cause any security impacts. There is no sensitive data that this will print out that the user does not already have access to. This command will be able to print Kubernetes secrets to the console, but given this is expected given that it is printing manifests. -This command could print Zarf variables with the `sensitive` value set to true. The value to these variables can be assigned with user input, config files, and the `default` key in the zarf.yaml. Given that these commands are expected to be run by a user actively managing a cluster or developing a package and not in an automated system we deem these risks acceptable. +This command could print Zarf variables with the `sensitive` value set to true. These variables can be set using values that a user already has access to: user input, configuration files, or the default key in the zarf.yaml file. Given that these commands are expected to be run by a user actively managing a cluster or developing a package and not in an automated system we deem these risks acceptable. **QUESTION** Am I making too broad an assumption that this risk is acceptable? @@ -215,7 +216,7 @@ required) or even code snippets. If there's any ambiguity about HOW your proposal will be implemented, this is the place to discuss that. --> -[Internal variables](https://docs.zarf.dev/ref/values/#internal-values-zarf_) will be set using the default logic. This means the `ZARF_REGISTRY` variable will become `127.0.0.1:31999`, while secrets like the `GIT_AUTH_PUSH` variable will become a random string. This will follow the same logic as `zarf dev find-images`. +[Internal variables](https://docs.zarf.dev/ref/values/#internal-values-zarf_) will be set using the default logic. This means the `ZARF_REGISTRY` variable will become `127.0.0.1:31999`, while secrets like the `GIT_AUTH_PUSH` variable will become a random string. This will follow the same logic as `zarf dev find-images`. ### Test Plan @@ -265,8 +266,6 @@ This can inform certain test coverage improvements that we want to do before extending the production code to implement this enhancement. --> -- ``: `` - `` - ##### e2e tests -[Internal variables](https://docs.zarf.dev/ref/values/#internal-values-zarf_) will be set using the default logic. This means the `ZARF_REGISTRY` variable will become `127.0.0.1:31999`, while secrets like the `GIT_AUTH_PUSH` variable will become a random string. This will follow the same logic as `zarf dev find-images`. +[Internal variables](https://docs.zarf.dev/ref/values/#internal-values-zarf_) will be set using the default logic besides sensitive values which will be set to "PLACEHOLDER". This means the `ZARF_REGISTRY` variable will become `127.0.0.1:31999`, while secrets like the `GIT_AUTH_PUSH` will be set to "PLACEHOLDER". ### Test Plan From 727490cf68ce966d9f55170c7ff99e5b440a1935 Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Tue, 3 Dec 2024 16:31:34 +0000 Subject: [PATCH 05/18] grammar Signed-off-by: Austin Abro --- 6-manifest-preview-capability/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/6-manifest-preview-capability/README.md b/6-manifest-preview-capability/README.md index 314f44d..28e3ae3 100644 --- a/6-manifest-preview-capability/README.md +++ b/6-manifest-preview-capability/README.md @@ -216,7 +216,7 @@ required) or even code snippets. If there's any ambiguity about HOW your proposal will be implemented, this is the place to discuss that. --> -[Internal variables](https://docs.zarf.dev/ref/values/#internal-values-zarf_) will be set using the default logic besides sensitive values which will be set to "PLACEHOLDER". This means the `ZARF_REGISTRY` variable will become `127.0.0.1:31999`, while secrets like the `GIT_AUTH_PUSH` will be set to "PLACEHOLDER". +[Internal variables](https://docs.zarf.dev/ref/values/#internal-values-zarf_) will be set using the default logic besides sensitive values which will be set to "PLACEHOLDER". For example, the `ZARF_REGISTRY` variable will become `127.0.0.1:31999`, while `ZARF_GIT_AUTH_PUSH` will be set to "PLACEHOLDER". ### Test Plan From 41c11cfb0f015cd5a1823b0c46d0667e69611987 Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Tue, 3 Dec 2024 21:59:24 +0000 Subject: [PATCH 06/18] WIP Signed-off-by: Austin Abro --- 6-manifest-preview-capability/README.md | 46 +--- 8-show-manifests-and-values/README.md | 326 ++++++++++++++++++++++++ 8-show-manifests-and-values/zep.yaml | 34 +++ 3 files changed, 370 insertions(+), 36 deletions(-) create mode 100644 8-show-manifests-and-values/README.md create mode 100644 8-show-manifests-and-values/zep.yaml diff --git a/6-manifest-preview-capability/README.md b/6-manifest-preview-capability/README.md index 28e3ae3..c4f33a1 100644 --- a/6-manifest-preview-capability/README.md +++ b/6-manifest-preview-capability/README.md @@ -97,9 +97,9 @@ feedback and reduce unnecessary changes. [documentation style guide]: https://docs.zarf.dev/contribute/style-guide/ --> -Users need an easier way to view what their zarf.yaml will look like after templating, imports, and flavors are applied. Users need a way to view their manifests and values files after Zarf variables are applied. +Users need an easier way to view what their zarf.yaml will look like after templating, imports, and flavors are applied. -This will be accomplished through a new CLI command `zarf package preview` and enhancements to `zarf package inspect` +This will be accomplished through a new CLI command `zarf package preview` ## Motivation @@ -114,14 +114,9 @@ or other references to show the community's interest in the ZEP. [kubernetes slack]: https://kubernetes.slack.com/archives/C03B6BJAUJ3 --> -Forcing users to use the the print before the (y/n) prompt in `zarf package create` to view their "complied" zarf.yaml has been a less than ideal UX. Users not being able to see if their manifests are templated properly before create and deploy can lead to much longer cycle time. A Helm template is less than a few seconds, whereas create + deploy could take several minutes or even an hour+. +Forcing users to use the the print before the (y/n) prompt in `zarf package create` to view their zarf.yaml after templating, imports, and flavors are applied is not ideal. Having a separate command will improve the UX, and an alternate path to this information will open up the possibility of `zarf package create` proceeding without prompting the user. -This feature would put is in parity with other tools that have similar pre-deploy viewing capabilities. The most obvious example is `helm template`, as Zarf would essentially run a Helm template to get the manifests. Other tools have similar capabilities such as opentofu with `tf plan`. - -This feature has been highly requested in recent months: -- request in Kubernetes slack - https://kubernetes.slack.com/archives/C03B6BJAUJ3/p1730229638367829 -- An issue has been created for this - https://github.com/zarf-dev/zarf/issues/2631 -- Defense Unicorns, an organization that relies heavily on Zarf for their deployments, have received requests for this feature in a feedback session with their partners. +This following issues requests a feature similar to `zarf package preview` - https://github.com/zarf-dev/zarf/issues/2631 ### Goals @@ -130,9 +125,7 @@ List the specific goals of the ZEP. What is it trying to achieve? How will we know that this has succeeded? --> -The goal for creators is that they have all the information they need to build a working package as quickly as possible. - -The goal for deployers is that they feel confident that they know what they're deploying and that the Zarf variable templating will work properly. +We know we are successful if package creators have a convenient way to view their zarf.yaml after templating, imports, and flavors are applied. ### Non-Goals @@ -141,8 +134,6 @@ What is out of scope for this ZEP? Listing non-goals helps to focus discussion and make progress. --> -We do not want to show the manifests of a package that is already deployed. `zarf package inspect` works on packages in the cluster, if a user tries the `--show-manifests` flag or `--values` flag for a package in the cluster they will receive an error. There is already a [helm get manifest](https://helm.sh/docs/helm/helm_get_manifest/) command that the user may use when already deployed to a cluster. - ## Proposal -Introduce a new command called `zarf package preview`. This command will print a zarf.yaml after templates, imports, and flavors are applied. The help text and flags of this command would look like below. Importantly, the `--show-manifests` and `--show-values-files` flags would print out what the manifests and values files would look like after templating has occurred. +Introduce a new command called `zarf package preview`. This command will print a zarf.yaml after templates, imports, and flavors are applied. +Example help text: ``` Usage: zarf package preview [ PACKAGE ] [flags] Flags: - --create-set stringToString Specify package variables to set on the command line (KEY=value) (default []) - --deploy-set stringToString Specify deployment variables to set on the command line (KEY=value) (default []) + --set stringToString Specify package variables to set on the command line (KEY=value) (default []) -f, --flavor string The flavor of components to include in the resulting package (i.e. have a matching or empty "only.flavor" key) - --show-manifests shows the manifests that would be deployed by this package - --show-values-files shows the values files that would be used by the charts in this package ``` @@ -179,15 +168,7 @@ bogged down. #### Story 1 -As a creator of Zarf packages I want to see what my zarf.yaml will look like after templates, imports, and flavors are applied so I run `zarf package preview -f my-flavor --create-set=MY_TEMPLATE=my-val` - -#### Story 2 - -As a creator of Zarf packages I want to make sure the variables in my package can get templated properly for the expected values of the deployers. I want to check this for both manifests and values files. `zarf package preview --deploy-set=MY_VAR=my-val --show-manifests` or `zarf package preview --deploy-set=MY_VAR=my-val --show-values-file` - -#### Story 3 - -As a deployer of Zarf packages, I want to check that the variables I intend to deploy my package with are getting properly templated for both manifests and values files so I run `zarf package preview --deploy-set=MY_VAR=my-val --show-manifests` or `zarf package preview --deploy-set=MY_VAR=my-val --show-values-file` +As a creator of Zarf packages I want to see what my zarf.yaml will look like after templates, imports, and flavors are applied so I run `zarf package preview -f my-flavor --set=MY_TEMPLATE=my-val` ### Risks and Mitigations @@ -201,12 +182,6 @@ How will security be reviewed, and by whom? How will UX be reviewed, and by whom? --> -This will not cause any security impacts. There is no sensitive data that this will print out that the user does not already have access to. This command will be able to print Kubernetes secrets to the console, but given this is expected given that it is printing manifests. - -This command could print Zarf variables with the `sensitive` value set to true. These variables can be set using values that a user already has access to: user input, configuration files, or the default key in the zarf.yaml file. Given that these commands are expected to be run by a user actively managing a cluster or developing a package and not in an automated system we deem these risks acceptable. - -**QUESTION** Am I making too broad an assumption that this risk is acceptable? - ## Design Details -[Internal variables](https://docs.zarf.dev/ref/values/#internal-values-zarf_) will be set using the default logic besides sensitive values which will be set to "PLACEHOLDER". For example, the `ZARF_REGISTRY` variable will become `127.0.0.1:31999`, while `ZARF_GIT_AUTH_PUSH` will be set to "PLACEHOLDER". +The zarf.yaml in this command will be printed to stdout. ### Test Plan @@ -364,6 +339,5 @@ information to express the idea and why it was not acceptable. One alternative was to use `zarf dev preview`, the thought being that this command will be run by creators while developing a package. However, given the similarities between this command and `zarf package inspect` we decided that it made for a more cohesive user experience to have both commands under the same parent. -I pondered if we should accept a component flag as we do on deploy during package inspect or package preview. This would allow people to get a more accurate view of the manifests that they intend to deploy if they are using optional components. However, this flag would only make sense alongside the It does make the code more complicated, but it can give a better view of what will actually be deployed. Given the added complexity, we are not added this for now, but it could be an enhancement in the future. Another alternative is to separate commands dedicated to show the different types of files. `zarf package preview` would be introduced to show the zarf.yaml file. `zarf dev show-manifests` and `zarf dev show-values-files` would show the manifests and values files respectively. The `zarf package inspect` command would be unchanged. Separating `show-manifests` and `show-values-files` would increase the surface area of the CLI, but give the commands a more distinct purpose, and more sensible flags. `show-manifests` and `show-values-files` would have the `--deploy-set` and `--create-set` flags, while `zarf package preview` would only need `--set`. `show-manifests` and `show-values-files` would need a `--kube-version` flag so they can still template charts that have a specific version requirement outside of the default. A `--components` flag could be added for to `show-manifests` and `show-values-files` for users who want to see the manifests they are deploying without optional components included, but wouldn't make sense for the `zarf package preview` command since all components are included on create regardless of if they are used or not. The `show-manifests` and `show-values-files` commands would take either a zarf.yaml or a zarf package. diff --git a/8-show-manifests-and-values/README.md b/8-show-manifests-and-values/README.md new file mode 100644 index 0000000..71b5afa --- /dev/null +++ b/8-show-manifests-and-values/README.md @@ -0,0 +1,326 @@ + + +# ZEP-NNNN: Your short, descriptive title + + + + + + +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) +- [Proposal](#proposal) + - [User Stories (Optional)](#user-stories-optional) + - [Story 1](#story-1) + - [Story 2](#story-2) + - [Risks and Mitigations](#risks-and-mitigations) +- [Design Details](#design-details) + - [Test Plan](#test-plan) + - [Prerequisite testing updates](#prerequisite-testing-updates) + - [Unit tests](#unit-tests) + - [e2e tests](#e2e-tests) + - [Graduation Criteria](#graduation-criteria) + - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy) + - [Version Skew Strategy](#version-skew-strategy) +- [Implementation History](#implementation-history) +- [Drawbacks](#drawbacks) +- [Alternatives](#alternatives) + + +## Summary + + + +Both creators and deployers need a way to view their manifests and values files after Zarf variables are applied. + +This will be accomplished through new CLI commands `zarf dev show-manifests`, `zarf dev show-values-fils`, `zarf package show-manifests`, and `zarf package show-values-files` + +## Motivation + + + +Viewing manifests and values files after templating would be useful for both creators and deployers. Catching a mistake in templating early can reduce cycle time. A Helm template is almost instant, whereas create + deploy could take several minutes or even an hour+. + +A user could achieve a similar effect to this command by unarchiving a package and running `helm template` on their chart. This is a poor UX, and the `helm template` may fail depending on where variable templating is used within the chart. + +This feature has been highly requested in recent months: +- request in Kubernetes slack - https://kubernetes.slack.com/archives/C03B6BJAUJ3/p1730229638367829 +- An issue has been created for this - https://github.com/zarf-dev/zarf/issues/2631 +- Defense Unicorns, an organization that relies heavily on Zarf for their deployments, have received requests for this feature in a feedback session with their partners. + +### Goals + + + +This is successful for creators when they can view manifests or values files prior to building their package to gain confidence that their templating is working correctly before creating their packages. + +This is successful for deployers when they are able to view their templated manifests and values files before deploying. + +### Non-Goals + + + +We do not want to show the manifests of a package that is already deployed. Users can use the [helm get manifest](https://helm.sh/docs/helm/helm_get_manifest/) command to see what Zarf has deployed to the cluster. + +## Proposal + + + +Introduce four new commands. `zarf package show-manifests`, `zarf package show-values-files`, `zarf dev show-manifests`, and `zarf dev show-values-files`. The `package` commands will an already built package, local or OCI, while the `dev` commands will take a package directory. + +The help text for `zarf package show-manifests` will look like below. `zarf package show-values-files` will include the same flags. +``` +Usage: + zarf package show-manifests [ PACKAGE_SOURCE ] [flags] + +Flags: + --set stringToString Specify deployment variables to set on the command line (KEY=value) (default []) + --kube-version Override the default helm template KubeVersion when performing a package chart template + --components Comma-separated list of components whose manifests should be displayed. Adding this flag will skip the prompts for selected components. Globbing component names with '*' and deselecting 'default' components with a leading '-' are also supported + --confirm Confirms package deployment without prompting. Skips prompts to configure variables and select optional components +``` + +``` +Usage: + zarf dev show-manifests [ PACKAGE_SOURCE ] [flags] + +Flags: + --create-set stringToString Specify package variables to set on the command line (KEY=value) (default []) + --deploy-set stringToString Specify deployment variables to set on the command line (KEY=value) (default []) + -f, --flavor string The flavor of components to include in the resulting package (i.e. have a matching or empty "only.flavor" key) + --kube-version Override the default helm template KubeVersion when performing a package chart template + Confirms package deployment without prompting. Skips prompts to configure variables and select optional components +``` + +### User Stories (Optional) + + + +#### Story 1 + +As a creator of Zarf packages I want to make sure the variables in my package can get templated properly for the expected values of the deployers. I want to check this for both manifests and values files so I run `zarf dev show-manifests path/to/package-dir --deploy-set=MY_VAR=my-val` and `zarf dev show-values-files path/to/package-dir --deploy-set=MY_VAR=my-val` + +#### Story 2 + +As a deployer of Zarf packages, I want to check that the variables I intend to deploy my package with are getting properly templated for both manifests and values files so I run `zarf package show-manifests zarf-package-podinfo-amd64.tar.zst --deploy-set=MY_VAR=my-val` and `zarf package show-values-files zarf-package-podinfo-amd64.tar.zst --deploy-set=MY_VAR=my-val` + +### Risks and Mitigations + + + +Security risks are minimal. This command could print Zarf variables with the `sensitive` value set to true. These variables would be set using values that a user already has access to: user input, configuration files, or the default key in the zarf.yaml file. Given that these commands are expected to be run by a user developing a package or actively managing a cluster or and not in an automated system we deem these risks acceptable. + +## Design Details + + + +[Internal variables](https://docs.zarf.dev/ref/values/#internal-values-zarf_) will be set using the default logic besides sensitive values which will be set to "PLACEHOLDER". For example, the `ZARF_REGISTRY` variable become `127.0.0.1:31999`, while `ZARF_GIT_AUTH_PUSH` will be set to "PLACEHOLDER". + +Manifests and values files will be printed to standard out. + +### Test Plan + + + +[ ] I/we understand the owners of the involved components may require updates to +existing tests to make this code solid enough prior to committing the changes necessary +to implement this proposal. + +### Graduation Criteria + + + +Given the simplicity of this feature, unit tests will be adequate. + +### Upgrade / Downgrade Strategy + + + +N/A + +### Version Skew Strategy + + + +## Implementation History + + + +## Drawbacks + + + +## Alternatives + + + +Both commands could have a `--component` flag similar to `zarf package deploy`. This would allow users to get a more accurate view of the manifests that they intend to deploy if their packages include optional components. This would introduce another situation where diff --git a/8-show-manifests-and-values/zep.yaml b/8-show-manifests-and-values/zep.yaml new file mode 100644 index 0000000..d982c61 --- /dev/null +++ b/8-show-manifests-and-values/zep.yaml @@ -0,0 +1,34 @@ +schema-version: 1.0.0 + +title: ZEP Template +zep-number: NNNN +authors: + - "@jane.doe" +status: provisional|implementable|implemented|deferred|rejected|withdrawn|replaced +creation-date: yyyy-mm-dd +reviewers: + - TBD + - "@alice.doe" +approvers: + - TBD + - "@oscar.doe" + +see-also: + - "/1234-we-heard-you-like-zeps" + - "/5678-everyone-gets-a-zep" +replaces: + - "/3456-replaced-zep" + +# The target maturity stage in the current dev cycle for this ZEP. +stage: alpha|beta|stable + +# The most recent milestone for which work toward delivery of this ZEP has been +# done. This can be the current (upcoming) milestone, if it is being actively +# worked on. +latest-milestone: "v0.39" + +# The milestone at which this feature was, or is targeted to be, at each stage. +milestone: + alpha: "v0.39" + beta: "v0.40" + stable: "v1.0" From 2bd1041d08bef6ed320956cda7266e1a7efe3576 Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Wed, 4 Dec 2024 15:40:12 +0000 Subject: [PATCH 07/18] WIP Signed-off-by: Austin Abro --- 6-manifest-preview-capability/README.md | 29 ++++++++++++++++--------- 8-show-manifests-and-values/README.md | 2 ++ 2 files changed, 21 insertions(+), 10 deletions(-) diff --git a/6-manifest-preview-capability/README.md b/6-manifest-preview-capability/README.md index c4f33a1..48ece81 100644 --- a/6-manifest-preview-capability/README.md +++ b/6-manifest-preview-capability/README.md @@ -97,7 +97,7 @@ feedback and reduce unnecessary changes. [documentation style guide]: https://docs.zarf.dev/contribute/style-guide/ --> -Users need an easier way to view what their zarf.yaml will look like after templating, imports, and flavors are applied. +Users need an easier way to view what their zarf.yaml will look like after they're rendered by Zarf before create. A rendered zarf.yaml has had templating, imports, and flavors applied. This will be accomplished through a new CLI command `zarf package preview` @@ -114,9 +114,9 @@ or other references to show the community's interest in the ZEP. [kubernetes slack]: https://kubernetes.slack.com/archives/C03B6BJAUJ3 --> -Forcing users to use the the print before the (y/n) prompt in `zarf package create` to view their zarf.yaml after templating, imports, and flavors are applied is not ideal. Having a separate command will improve the UX, and an alternate path to this information will open up the possibility of `zarf package create` proceeding without prompting the user. +Forcing users to use the the print before the (y/n) prompt in `zarf package create` to view their rendered zarf.yaml is not ideal. Having a separate command will improve the UX, and an alternate path to this information will open up the possibility of `zarf package create` proceeding without prompting the user. -This following issues requests a feature similar to `zarf package preview` - https://github.com/zarf-dev/zarf/issues/2631 +The following issues requests a feature similar to `zarf package preview` - https://github.com/zarf-dev/zarf/issues/2631 ### Goals @@ -125,7 +125,7 @@ List the specific goals of the ZEP. What is it trying to achieve? How will we know that this has succeeded? --> -We know we are successful if package creators have a convenient way to view their zarf.yaml after templating, imports, and flavors are applied. +We know we are successful if package creators have a convenient way to view their rendered zarf.yaml. ### Non-Goals @@ -134,6 +134,8 @@ What is out of scope for this ZEP? Listing non-goals helps to focus discussion and make progress. --> +This command should stay extremely simple, and only print the rendered zarf.yaml. It will not output SBOMs, or manifests. + ## Proposal +The risks are minimal, the feature is simple and easy to maintain. + +The Zarf core team will review the UX + ## Design Details -The zarf.yaml in this command will be printed to stdout. +This command should print the zarf.yaml to stdout ### Test Plan @@ -278,6 +288,8 @@ If this feature will eventually be deprecated, plan for it: - Wait at least two versions before fully removing it. --> +This feature will be released immediately as GA given the low risk. + ### Upgrade / Downgrade Strategy One alternative was to use `zarf dev preview`, the thought being that this command will be run by creators while developing a package. However, given the similarities between this command and `zarf package inspect` we decided that it made for a more cohesive user experience to have both commands under the same parent. - - -Another alternative is to separate commands dedicated to show the different types of files. `zarf package preview` would be introduced to show the zarf.yaml file. `zarf dev show-manifests` and `zarf dev show-values-files` would show the manifests and values files respectively. The `zarf package inspect` command would be unchanged. Separating `show-manifests` and `show-values-files` would increase the surface area of the CLI, but give the commands a more distinct purpose, and more sensible flags. `show-manifests` and `show-values-files` would have the `--deploy-set` and `--create-set` flags, while `zarf package preview` would only need `--set`. `show-manifests` and `show-values-files` would need a `--kube-version` flag so they can still template charts that have a specific version requirement outside of the default. A `--components` flag could be added for to `show-manifests` and `show-values-files` for users who want to see the manifests they are deploying without optional components included, but wouldn't make sense for the `zarf package preview` command since all components are included on create regardless of if they are used or not. The `show-manifests` and `show-values-files` commands would take either a zarf.yaml or a zarf package. diff --git a/8-show-manifests-and-values/README.md b/8-show-manifests-and-values/README.md index 71b5afa..2f74c87 100644 --- a/8-show-manifests-and-values/README.md +++ b/8-show-manifests-and-values/README.md @@ -324,3 +324,5 @@ information to express the idea and why it was not acceptable. --> Both commands could have a `--component` flag similar to `zarf package deploy`. This would allow users to get a more accurate view of the manifests that they intend to deploy if their packages include optional components. This would introduce another situation where + +Another alternative is to separate commands dedicated to show the different types of files. `zarf package preview` would be introduced to show the zarf.yaml file. `zarf dev show-manifests` and `zarf dev show-values-files` would show the manifests and values files respectively. The `zarf package inspect` command would be unchanged. Separating `show-manifests` and `show-values-files` would increase the surface area of the CLI, but give the commands a more distinct purpose, and more sensible flags. `show-manifests` and `show-values-files` would have the `--deploy-set` and `--create-set` flags, while `zarf package preview` would only need `--set`. `show-manifests` and `show-values-files` would need a `--kube-version` flag so they can still template charts that have a specific version requirement outside of the default. A `--components` flag could be added for to `show-manifests` and `show-values-files` for users who want to see the manifests they are deploying without optional components included, but wouldn't make sense for the `zarf package preview` command since all components are included on create regardless of if they are used or not. The `show-manifests` and `show-values-files` commands would take either a zarf.yaml or a zarf package. \ No newline at end of file From d16b10a1dfd98085c406bfa2773d849e681971d6 Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Wed, 4 Dec 2024 16:12:02 +0000 Subject: [PATCH 08/18] clean up Signed-off-by: Austin Abro --- 6-manifest-preview-capability/README.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/6-manifest-preview-capability/README.md b/6-manifest-preview-capability/README.md index 48ece81..dbe711e 100644 --- a/6-manifest-preview-capability/README.md +++ b/6-manifest-preview-capability/README.md @@ -125,7 +125,7 @@ List the specific goals of the ZEP. What is it trying to achieve? How will we know that this has succeeded? --> -We know we are successful if package creators have a convenient way to view their rendered zarf.yaml. +We know this is successful if package creators have a convenient way to view their rendered zarf.yaml before creating their package. ### Non-Goals @@ -148,7 +148,7 @@ below is for the real nitty-gritty. Introduce a new command called `zarf package preview`. This command will print a zarf.yaml after templates, imports, and flavors are applied. -This command will not support the differential flag until the differential flag is out of beta. The Zarf team is undecided on whether we will keep the differential flag or build an alternative solution. The implementation of the differential flag will be simple if/when we choose to add it. +This command will not support the differential flag until the differential flag is out of beta. The Zarf team is undecided on whether we will keep the differential flag or build an alternative solution. The implementation of the differential flag will be simple if/when it is added. The registry override flag will not be supported as this flag does not have an effect on the rendered zarf.yaml. @@ -190,8 +190,6 @@ How will UX be reviewed, and by whom? The risks are minimal, the feature is simple and easy to maintain. -The Zarf core team will review the UX - ## Design Details -Users need an easier way to view what their zarf.yaml will look like after they're rendered by Zarf before create. A rendered zarf.yaml has had templating, imports, and flavors applied. +Users need an easier way to view what their zarf.yaml will look like after they're rendered by Zarf before `zarf package create`. A rendered zarf.yaml has had templating, imports, and flavors applied. -This will be accomplished through a new CLI command `zarf package preview` +This will be accomplished through a new CLI command - `zarf package preview` ## Motivation @@ -114,7 +114,7 @@ or other references to show the community's interest in the ZEP. [kubernetes slack]: https://kubernetes.slack.com/archives/C03B6BJAUJ3 --> -Forcing users to use the the print before the (y/n) prompt in `zarf package create` to view their rendered zarf.yaml is not ideal. Having a separate command will improve the UX, and an alternate path to this information will open up the possibility of `zarf package create` proceeding without prompting the user. +Forcing users to use the print before the (y/n) prompt in `zarf package create` to view their rendered zarf.yaml is not ideal. Having a separate command improves the UX by providing users with an easier way to view the rendered zarf.yaml. It also opens the possibility of allowing zarf package create to proceed without requiring user confirmation. The following issues requests a feature similar to `zarf package preview` - https://github.com/zarf-dev/zarf/issues/2631 @@ -125,7 +125,7 @@ List the specific goals of the ZEP. What is it trying to achieve? How will we know that this has succeeded? --> -We know this is successful if package creators have a convenient way to view their rendered zarf.yaml before creating their package. +This ZEP is successful when package creators have a convenient way to view their rendered zarf.yaml before creating their package. ### Non-Goals @@ -134,8 +134,6 @@ What is out of scope for this ZEP? Listing non-goals helps to focus discussion and make progress. --> -This command should stay extremely simple, and only print the rendered zarf.yaml. It will not output SBOMs, or manifests. - ## Proposal -Introduce a new command called `zarf package preview`. This command will print a zarf.yaml after templates, imports, and flavors are applied. +Introduce a new command called `zarf package preview`. This command will print the zarf.yaml after templates, imports, and flavors are applied. -This command will not support the differential flag until the differential flag is out of beta. The Zarf team is undecided on whether we will keep the differential flag or build an alternative solution. The implementation of the differential flag will be simple if/when it is added. +The differential flag will not be supported until it exits beta. The Zarf team is undecided on whether we will keep the differential flag or build an alternative solution. The implementation of the differential flag will be simple if/when it is added. The registry override flag will not be supported as this flag does not have an effect on the rendered zarf.yaml. @@ -160,7 +158,7 @@ Usage: Flags: --set stringToString Specify package variables to set on the command line (KEY=value) (default []) -f, --flavor string The flavor of components to include in the resulting package (i.e. have a matching or empty "only.flavor" key) - --confirm Confirm package creation without prompting + --confirm Confirm package preview without prompting ``` ### User Stories (Optional) @@ -199,7 +197,7 @@ required) or even code snippets. If there's any ambiguity about HOW your proposal will be implemented, this is the place to discuss that. --> -This command should print the zarf.yaml to stdout +This command should print the zarf.yaml to stdout. ### Test Plan @@ -347,4 +345,4 @@ not need to be as detailed as the proposal, but should include enough information to express the idea and why it was not acceptable. --> -One alternative was to use `zarf dev preview`, the thought being that this command will be run by creators while developing a package. However, given the similarities between this command and `zarf package inspect` we decided that it made for a more cohesive user experience to have both commands under the same parent. +One alternative was to use `zarf dev preview`, since this command will be run by creators while developing a package. However, given the similarities between this command and `zarf package inspect` we decided that it made for a more cohesive user experience to have both commands under the same parent. From 7c80334b449556af241b3f1e8141d8d8e678601a Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Wed, 4 Dec 2024 16:38:47 +0000 Subject: [PATCH 10/18] cleaner word choice Signed-off-by: Austin Abro --- 6-manifest-preview-capability/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/6-manifest-preview-capability/README.md b/6-manifest-preview-capability/README.md index 175127d..c193fa1 100644 --- a/6-manifest-preview-capability/README.md +++ b/6-manifest-preview-capability/README.md @@ -116,7 +116,7 @@ or other references to show the community's interest in the ZEP. Forcing users to use the print before the (y/n) prompt in `zarf package create` to view their rendered zarf.yaml is not ideal. Having a separate command improves the UX by providing users with an easier way to view the rendered zarf.yaml. It also opens the possibility of allowing zarf package create to proceed without requiring user confirmation. -The following issues requests a feature similar to `zarf package preview` - https://github.com/zarf-dev/zarf/issues/2631 +This issue asks for a better way to view what a zarf.yaml looks like at create time - https://github.com/zarf-dev/zarf/issues/2631 ### Goals @@ -146,9 +146,9 @@ below is for the real nitty-gritty. Introduce a new command called `zarf package preview`. This command will print the zarf.yaml after templates, imports, and flavors are applied. -The differential flag will not be supported until it exits beta. The Zarf team is undecided on whether we will keep the differential flag or build an alternative solution. The implementation of the differential flag will be simple if/when it is added. +The differential flag will not be supported until it exits beta. The Zarf team is undecided on whether we will keep the differential flag or build an alternative solution. Implementing the differential flag will be simple if/when it's out of beta. -The registry override flag will not be supported as this flag does not have an effect on the rendered zarf.yaml. +The registry override flag will not be supported as this flag does not have an effect on the rendered zarf.yaml during `zarf package create`. Example help text: ``` @@ -345,4 +345,4 @@ not need to be as detailed as the proposal, but should include enough information to express the idea and why it was not acceptable. --> -One alternative was to use `zarf dev preview`, since this command will be run by creators while developing a package. However, given the similarities between this command and `zarf package inspect` we decided that it made for a more cohesive user experience to have both commands under the same parent. +One alternative was to use `zarf dev preview`, since package creators will run this command during development. However, given the similarities between this command and `zarf package inspect` we decided that having both commands under the same parent made for a more cohesive user experience. From 12a19e4b794697097b74772687a1beb625c56821 Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Wed, 4 Dec 2024 16:47:39 +0000 Subject: [PATCH 11/18] separate Signed-off-by: Austin Abro --- 8-show-manifests-and-values/README.md | 328 -------------------------- 8-show-manifests-and-values/zep.yaml | 34 --- 2 files changed, 362 deletions(-) delete mode 100644 8-show-manifests-and-values/README.md delete mode 100644 8-show-manifests-and-values/zep.yaml diff --git a/8-show-manifests-and-values/README.md b/8-show-manifests-and-values/README.md deleted file mode 100644 index 2f74c87..0000000 --- a/8-show-manifests-and-values/README.md +++ /dev/null @@ -1,328 +0,0 @@ - - -# ZEP-NNNN: Your short, descriptive title - - - - - - -- [Summary](#summary) -- [Motivation](#motivation) - - [Goals](#goals) - - [Non-Goals](#non-goals) -- [Proposal](#proposal) - - [User Stories (Optional)](#user-stories-optional) - - [Story 1](#story-1) - - [Story 2](#story-2) - - [Risks and Mitigations](#risks-and-mitigations) -- [Design Details](#design-details) - - [Test Plan](#test-plan) - - [Prerequisite testing updates](#prerequisite-testing-updates) - - [Unit tests](#unit-tests) - - [e2e tests](#e2e-tests) - - [Graduation Criteria](#graduation-criteria) - - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy) - - [Version Skew Strategy](#version-skew-strategy) -- [Implementation History](#implementation-history) -- [Drawbacks](#drawbacks) -- [Alternatives](#alternatives) - - -## Summary - - - -Both creators and deployers need a way to view their manifests and values files after Zarf variables are applied. - -This will be accomplished through new CLI commands `zarf dev show-manifests`, `zarf dev show-values-fils`, `zarf package show-manifests`, and `zarf package show-values-files` - -## Motivation - - - -Viewing manifests and values files after templating would be useful for both creators and deployers. Catching a mistake in templating early can reduce cycle time. A Helm template is almost instant, whereas create + deploy could take several minutes or even an hour+. - -A user could achieve a similar effect to this command by unarchiving a package and running `helm template` on their chart. This is a poor UX, and the `helm template` may fail depending on where variable templating is used within the chart. - -This feature has been highly requested in recent months: -- request in Kubernetes slack - https://kubernetes.slack.com/archives/C03B6BJAUJ3/p1730229638367829 -- An issue has been created for this - https://github.com/zarf-dev/zarf/issues/2631 -- Defense Unicorns, an organization that relies heavily on Zarf for their deployments, have received requests for this feature in a feedback session with their partners. - -### Goals - - - -This is successful for creators when they can view manifests or values files prior to building their package to gain confidence that their templating is working correctly before creating their packages. - -This is successful for deployers when they are able to view their templated manifests and values files before deploying. - -### Non-Goals - - - -We do not want to show the manifests of a package that is already deployed. Users can use the [helm get manifest](https://helm.sh/docs/helm/helm_get_manifest/) command to see what Zarf has deployed to the cluster. - -## Proposal - - - -Introduce four new commands. `zarf package show-manifests`, `zarf package show-values-files`, `zarf dev show-manifests`, and `zarf dev show-values-files`. The `package` commands will an already built package, local or OCI, while the `dev` commands will take a package directory. - -The help text for `zarf package show-manifests` will look like below. `zarf package show-values-files` will include the same flags. -``` -Usage: - zarf package show-manifests [ PACKAGE_SOURCE ] [flags] - -Flags: - --set stringToString Specify deployment variables to set on the command line (KEY=value) (default []) - --kube-version Override the default helm template KubeVersion when performing a package chart template - --components Comma-separated list of components whose manifests should be displayed. Adding this flag will skip the prompts for selected components. Globbing component names with '*' and deselecting 'default' components with a leading '-' are also supported - --confirm Confirms package deployment without prompting. Skips prompts to configure variables and select optional components -``` - -``` -Usage: - zarf dev show-manifests [ PACKAGE_SOURCE ] [flags] - -Flags: - --create-set stringToString Specify package variables to set on the command line (KEY=value) (default []) - --deploy-set stringToString Specify deployment variables to set on the command line (KEY=value) (default []) - -f, --flavor string The flavor of components to include in the resulting package (i.e. have a matching or empty "only.flavor" key) - --kube-version Override the default helm template KubeVersion when performing a package chart template - Confirms package deployment without prompting. Skips prompts to configure variables and select optional components -``` - -### User Stories (Optional) - - - -#### Story 1 - -As a creator of Zarf packages I want to make sure the variables in my package can get templated properly for the expected values of the deployers. I want to check this for both manifests and values files so I run `zarf dev show-manifests path/to/package-dir --deploy-set=MY_VAR=my-val` and `zarf dev show-values-files path/to/package-dir --deploy-set=MY_VAR=my-val` - -#### Story 2 - -As a deployer of Zarf packages, I want to check that the variables I intend to deploy my package with are getting properly templated for both manifests and values files so I run `zarf package show-manifests zarf-package-podinfo-amd64.tar.zst --deploy-set=MY_VAR=my-val` and `zarf package show-values-files zarf-package-podinfo-amd64.tar.zst --deploy-set=MY_VAR=my-val` - -### Risks and Mitigations - - - -Security risks are minimal. This command could print Zarf variables with the `sensitive` value set to true. These variables would be set using values that a user already has access to: user input, configuration files, or the default key in the zarf.yaml file. Given that these commands are expected to be run by a user developing a package or actively managing a cluster or and not in an automated system we deem these risks acceptable. - -## Design Details - - - -[Internal variables](https://docs.zarf.dev/ref/values/#internal-values-zarf_) will be set using the default logic besides sensitive values which will be set to "PLACEHOLDER". For example, the `ZARF_REGISTRY` variable become `127.0.0.1:31999`, while `ZARF_GIT_AUTH_PUSH` will be set to "PLACEHOLDER". - -Manifests and values files will be printed to standard out. - -### Test Plan - - - -[ ] I/we understand the owners of the involved components may require updates to -existing tests to make this code solid enough prior to committing the changes necessary -to implement this proposal. - -### Graduation Criteria - - - -Given the simplicity of this feature, unit tests will be adequate. - -### Upgrade / Downgrade Strategy - - - -N/A - -### Version Skew Strategy - - - -## Implementation History - - - -## Drawbacks - - - -## Alternatives - - - -Both commands could have a `--component` flag similar to `zarf package deploy`. This would allow users to get a more accurate view of the manifests that they intend to deploy if their packages include optional components. This would introduce another situation where - -Another alternative is to separate commands dedicated to show the different types of files. `zarf package preview` would be introduced to show the zarf.yaml file. `zarf dev show-manifests` and `zarf dev show-values-files` would show the manifests and values files respectively. The `zarf package inspect` command would be unchanged. Separating `show-manifests` and `show-values-files` would increase the surface area of the CLI, but give the commands a more distinct purpose, and more sensible flags. `show-manifests` and `show-values-files` would have the `--deploy-set` and `--create-set` flags, while `zarf package preview` would only need `--set`. `show-manifests` and `show-values-files` would need a `--kube-version` flag so they can still template charts that have a specific version requirement outside of the default. A `--components` flag could be added for to `show-manifests` and `show-values-files` for users who want to see the manifests they are deploying without optional components included, but wouldn't make sense for the `zarf package preview` command since all components are included on create regardless of if they are used or not. The `show-manifests` and `show-values-files` commands would take either a zarf.yaml or a zarf package. \ No newline at end of file diff --git a/8-show-manifests-and-values/zep.yaml b/8-show-manifests-and-values/zep.yaml deleted file mode 100644 index d982c61..0000000 --- a/8-show-manifests-and-values/zep.yaml +++ /dev/null @@ -1,34 +0,0 @@ -schema-version: 1.0.0 - -title: ZEP Template -zep-number: NNNN -authors: - - "@jane.doe" -status: provisional|implementable|implemented|deferred|rejected|withdrawn|replaced -creation-date: yyyy-mm-dd -reviewers: - - TBD - - "@alice.doe" -approvers: - - TBD - - "@oscar.doe" - -see-also: - - "/1234-we-heard-you-like-zeps" - - "/5678-everyone-gets-a-zep" -replaces: - - "/3456-replaced-zep" - -# The target maturity stage in the current dev cycle for this ZEP. -stage: alpha|beta|stable - -# The most recent milestone for which work toward delivery of this ZEP has been -# done. This can be the current (upcoming) milestone, if it is being actively -# worked on. -latest-milestone: "v0.39" - -# The milestone at which this feature was, or is targeted to be, at each stage. -milestone: - alpha: "v0.39" - beta: "v0.40" - stable: "v1.0" From c80bcb723c848dfabd8fd07563e828eaf8618174 Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Wed, 4 Dec 2024 16:49:03 +0000 Subject: [PATCH 12/18] package preview Signed-off-by: Austin Abro --- {6-manifest-preview-capability => 6-package-preview}/README.md | 0 {6-manifest-preview-capability => 6-package-preview}/zep.yaml | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename {6-manifest-preview-capability => 6-package-preview}/README.md (100%) rename {6-manifest-preview-capability => 6-package-preview}/zep.yaml (100%) diff --git a/6-manifest-preview-capability/README.md b/6-package-preview/README.md similarity index 100% rename from 6-manifest-preview-capability/README.md rename to 6-package-preview/README.md diff --git a/6-manifest-preview-capability/zep.yaml b/6-package-preview/zep.yaml similarity index 100% rename from 6-manifest-preview-capability/zep.yaml rename to 6-package-preview/zep.yaml From 084fb47a16ecf53e4e67c9154400a0d90266f578 Mon Sep 17 00:00:00 2001 From: Austin Abro <37223396+AustinAbro321@users.noreply.github.com> Date: Tue, 10 Dec 2024 09:54:02 -0500 Subject: [PATCH 13/18] Apply suggestions from code review Co-authored-by: Maciej Szulik --- 6-package-preview/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/6-package-preview/README.md b/6-package-preview/README.md index c193fa1..c2d7840 100644 --- a/6-package-preview/README.md +++ b/6-package-preview/README.md @@ -97,7 +97,7 @@ feedback and reduce unnecessary changes. [documentation style guide]: https://docs.zarf.dev/contribute/style-guide/ --> -Users need an easier way to view what their zarf.yaml will look like after they're rendered by Zarf before `zarf package create`. A rendered zarf.yaml has had templating, imports, and flavors applied. +Users need an easier way to view what their package will look like after they're rendered by Zarf before `zarf package create`. A rendered package has had templating, imports, and flavors applied. This will be accomplished through a new CLI command - `zarf package preview` @@ -114,7 +114,7 @@ or other references to show the community's interest in the ZEP. [kubernetes slack]: https://kubernetes.slack.com/archives/C03B6BJAUJ3 --> -Forcing users to use the print before the (y/n) prompt in `zarf package create` to view their rendered zarf.yaml is not ideal. Having a separate command improves the UX by providing users with an easier way to view the rendered zarf.yaml. It also opens the possibility of allowing zarf package create to proceed without requiring user confirmation. +Forcing users to use the print before the (y/n) prompt in `zarf package create` to view their rendered package is not ideal. Having a separate command improves the UX by providing users with an easier way to view the rendered zarf.yaml. It also opens the possibility of allowing zarf package create to proceed without requiring user confirmation. This issue asks for a better way to view what a zarf.yaml looks like at create time - https://github.com/zarf-dev/zarf/issues/2631 @@ -144,9 +144,9 @@ desired outcome and how success will be measured. The "Design Details" section below is for the real nitty-gritty. --> -Introduce a new command called `zarf package preview`. This command will print the zarf.yaml after templates, imports, and flavors are applied. +Introduce a new command called `zarf package preview`. This command will print the package after templates, imports, and flavors are applied. -The differential flag will not be supported until it exits beta. The Zarf team is undecided on whether we will keep the differential flag or build an alternative solution. Implementing the differential flag will be simple if/when it's out of beta. +The `--differential` flag available in `zarf package create` will not be supported until it reaches stable level. The Zarf team is undecided on whether we will keep the differential flag or build an alternative solution. The registry override flag will not be supported as this flag does not have an effect on the rendered zarf.yaml during `zarf package create`. @@ -172,7 +172,7 @@ bogged down. #### Story 1 -As a creator of Zarf packages I want to see what my zarf.yaml will look like after templates, imports, and flavors are applied so I run `zarf package preview -f my-flavor --set=MY_TEMPLATE=my-val` +As a creator of Zarf packages I want to see what my rendered `zarf.yaml` will look like after templates, imports, and flavors are applied, with a simple invocation: `zarf package preview -f my-flavor --set=MY_TEMPLATE=my-val` ### Risks and Mitigations From 156a77ce7f9181c3b68114129613653426a739cd Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Tue, 10 Dec 2024 15:03:04 +0000 Subject: [PATCH 14/18] package to package definition Signed-off-by: Austin Abro --- 6-package-preview/README.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/6-package-preview/README.md b/6-package-preview/README.md index c2d7840..258ab49 100644 --- a/6-package-preview/README.md +++ b/6-package-preview/README.md @@ -97,7 +97,7 @@ feedback and reduce unnecessary changes. [documentation style guide]: https://docs.zarf.dev/contribute/style-guide/ --> -Users need an easier way to view what their package will look like after they're rendered by Zarf before `zarf package create`. A rendered package has had templating, imports, and flavors applied. +Users need an easier way to view what their package definition will look like after it's rendered by Zarf before `zarf package create`. A rendered package definition has had templating, imports, and flavors applied. This will be accomplished through a new CLI command - `zarf package preview` @@ -114,7 +114,7 @@ or other references to show the community's interest in the ZEP. [kubernetes slack]: https://kubernetes.slack.com/archives/C03B6BJAUJ3 --> -Forcing users to use the print before the (y/n) prompt in `zarf package create` to view their rendered package is not ideal. Having a separate command improves the UX by providing users with an easier way to view the rendered zarf.yaml. It also opens the possibility of allowing zarf package create to proceed without requiring user confirmation. +Forcing users to use the print before the (y/n) prompt in `zarf package create` to view their rendered package definition is not ideal. Having a separate command improves the UX by providing users with an easier way to view the rendered package definition. It also opens the possibility of allowing `zarf package create` to proceed without requiring user confirmation. This issue asks for a better way to view what a zarf.yaml looks like at create time - https://github.com/zarf-dev/zarf/issues/2631 @@ -144,7 +144,7 @@ desired outcome and how success will be measured. The "Design Details" section below is for the real nitty-gritty. --> -Introduce a new command called `zarf package preview`. This command will print the package after templates, imports, and flavors are applied. +Introduce a new command called `zarf package preview`. This command will print the package definition after templates, imports, and flavors are applied. The `--differential` flag available in `zarf package create` will not be supported until it reaches stable level. The Zarf team is undecided on whether we will keep the differential flag or build an alternative solution. @@ -153,7 +153,7 @@ The registry override flag will not be supported as this flag does not have an e Example help text: ``` Usage: - zarf package preview [ PACKAGE ] [flags] + zarf package preview [ DIRECTORY ] [flags] Flags: --set stringToString Specify package variables to set on the command line (KEY=value) (default []) @@ -172,7 +172,7 @@ bogged down. #### Story 1 -As a creator of Zarf packages I want to see what my rendered `zarf.yaml` will look like after templates, imports, and flavors are applied, with a simple invocation: `zarf package preview -f my-flavor --set=MY_TEMPLATE=my-val` +As a creator of Zarf packages I want to see what my package definition will look like after templates, imports, and flavors are applied, with a simple invocation: `zarf package preview -f my-flavor --set=MY_TEMPLATE=my-val` ### Risks and Mitigations @@ -186,7 +186,7 @@ How will security be reviewed, and by whom? How will UX be reviewed, and by whom? --> -The risks are minimal, the feature is simple and easy to maintain. +There are no risks, the feature is simple and easy to maintain. ## Design Details @@ -197,7 +197,7 @@ required) or even code snippets. If there's any ambiguity about HOW your proposal will be implemented, this is the place to discuss that. --> -This command should print the zarf.yaml to stdout. +This command should print the package definition to stdout. ### Test Plan @@ -284,7 +284,7 @@ If this feature will eventually be deprecated, plan for it: - Wait at least two versions before fully removing it. --> -This feature will be released immediately as GA given the low risk. +This feature will be released immediately as GA given the lack of risk. ### Upgrade / Downgrade Strategy From 68e78f848ae46a503d1109857f87208e329d39bd Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Tue, 10 Dec 2024 15:06:27 +0000 Subject: [PATCH 15/18] goals and non goals Signed-off-by: Austin Abro --- 6-package-preview/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/6-package-preview/README.md b/6-package-preview/README.md index 258ab49..e6b8991 100644 --- a/6-package-preview/README.md +++ b/6-package-preview/README.md @@ -125,7 +125,7 @@ List the specific goals of the ZEP. What is it trying to achieve? How will we know that this has succeeded? --> -This ZEP is successful when package creators have a convenient way to view their rendered zarf.yaml before creating their package. +- preview package contents without creating the package ### Non-Goals @@ -134,6 +134,8 @@ What is out of scope for this ZEP? Listing non-goals helps to focus discussion and make progress. --> +- Provide additional templating mechanism + ## Proposal -Introduce a new command called `zarf package preview`. This command will print the package definition after templates, imports, and flavors are applied. - -The `--differential` flag available in `zarf package create` will not be supported until it reaches stable level. The Zarf team is undecided on whether we will keep the differential flag or build an alternative solution. +Introduce a new command called `zarf package preview`. This command will print the package definition after templates, imports, and flavors are applied. The `--differential` flag available in `zarf package create` will not be supported until it reaches stable level. Component actions will not run during `zarf package preview`. -The registry override flag will not be supported as this flag does not have an effect on the rendered zarf.yaml during `zarf package create`. +The registry override flag will not be supported as this flag does not have an effect on the package definition during `zarf package create`. Example help text: ``` From 0c6f07fa521090a46b3fd359ae8cad9a0462c528 Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Tue, 10 Dec 2024 15:07:28 +0000 Subject: [PATCH 16/18] caps Signed-off-by: Austin Abro --- 6-package-preview/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/6-package-preview/README.md b/6-package-preview/README.md index e6b8991..d2d78aa 100644 --- a/6-package-preview/README.md +++ b/6-package-preview/README.md @@ -125,7 +125,7 @@ List the specific goals of the ZEP. What is it trying to achieve? How will we know that this has succeeded? --> -- preview package contents without creating the package +- Preview package contents without creating the package ### Non-Goals From 0f577d4e0c4cbd32c0a8da726c8e565b1f473d05 Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Wed, 11 Dec 2024 17:56:52 +0000 Subject: [PATCH 17/18] build out alternatives Signed-off-by: Austin Abro --- 6-package-preview/README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/6-package-preview/README.md b/6-package-preview/README.md index d2d78aa..c48c923 100644 --- a/6-package-preview/README.md +++ b/6-package-preview/README.md @@ -345,4 +345,10 @@ not need to be as detailed as the proposal, but should include enough information to express the idea and why it was not acceptable. --> +### DEV Parent command + One alternative was to use `zarf dev preview`, since package creators will run this command during development. However, given the similarities between this command and `zarf package inspect` we decided that having both commands under the same parent made for a more cohesive user experience. + +### Build Preview Functionality Into a Flag on Package Create + +We could instead build the functionality here into a flag called `--dry-run` on `zarf package create`. This may lead to more discoverability of this functionality. Additionally, it may have a more natural workflow. A user would run the create command with all their usual flags and the `--dry-run` flag. Once everything looks correct, they would just remove the `--dry-run` flag. \ No newline at end of file From 38e8b1f63875b42d137c7f9cd26703974a247b44 Mon Sep 17 00:00:00 2001 From: Austin Abro Date: Wed, 11 Dec 2024 19:11:07 +0000 Subject: [PATCH 18/18] package preview Signed-off-by: Austin Abro --- 6-package-preview/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/6-package-preview/README.md b/6-package-preview/README.md index c48c923..c2fac89 100644 --- a/6-package-preview/README.md +++ b/6-package-preview/README.md @@ -45,7 +45,7 @@ The latest instructions for this template can be found in [this repo](/NNNN-zep- longer appropriate, updates to the list must be approved by the remaining approvers. --> -# ZEP-NNNN: Your short, descriptive title +# ZEP-6: preview rendered package definition