ImageCustomizer - Service and Overlay recommendations for Verity-enabled images. #10608
Conversation
| @@ -550,6 +546,23 @@ os: | |||
| corruptionOption: panic | |||
| ``` | |||
|
|
|||
| When enabling the Verity feature, it is recommended that overlays are applied to | |||
There was a problem hiding this comment.
I don’t want the official recommendations for MIC+verity to include overlays. They are a lazy person’s solution to provisioning an image for verity.
Instead, the recommendations should be to create a writable persistent partition (e.g. /var) and then redirect other files to the writable partition using symlinks and the like.
For example, verity-config.yaml redirects the SSH host keys from /etc to /var.
| @@ -550,6 +546,23 @@ os: | |||
| corruptionOption: panic | |||
| ``` | |||
|
|
|||
| When enabling the Verity feature, it is recommended that overlays are applied to | |||
There was a problem hiding this comment.
I suspect that over time the recommendations for verity images could end up being quite long. There are lots of different services/programs available on Azure Linux and a number of them may need special handling under verity.
In addition, there may be multiple options for a service/program with different tradeoffs. Some users may want a service/program configuration option changeable for convenience. While other users may want it immutable for security.
As such, I recommend splitting off the verity recommendations into its own doc.
| recommended to create a writable persistent partition (e.g., /var) for any | ||
| directories that require write access. Critical files and directories, such as | ||
| SSH host keys or logs, can be redirected to the writable partition using | ||
| symlinks or similar methods. See [Verity Filesystem Layout |
There was a problem hiding this comment.
why not call out overlays here as well?
There was a problem hiding this comment.
Yeah, Chris mentioned that overlays should not be part of the official MIC+Verity recommendations, so I removed the overlays related suggestions.
There was a problem hiding this comment.
My interpretation of Chris' comment was that it shouldn't be the sole recommendation but IMO we should include it as one possible option.
There was a problem hiding this comment.
It makes sense to me. Agree.
| @@ -511,10 +511,6 @@ Example: `noatime,nodiratime` | |||
| Specifies the configuration for dm-verity root integrity verification. Please | |||
| execute `sudo modprobe nbd` before building the image with verity enablement. | |||
There was a problem hiding this comment.
While you're in the area, it might be goodness to remove the reference to nbd.
| @@ -550,6 +546,27 @@ os: | |||
| corruptionOption: panic | |||
| ``` | |||
|
|
|||
| The Verity-enabled root filesystem is always mounted as read-only. Its root hash | |||
There was a problem hiding this comment.
I would move all of this prose to the verity.md file. And then in this file, only say something like: There are multiple ways to configure a verity enabled image. For recommendations see, [Verity Image Recommendations](./verity.md).
| @@ -0,0 +1,16 @@ | |||
| # Verity Filesystem Layout Recommendations | |||
There was a problem hiding this comment.
I would probably title this document 'Verity Image Recommendations`.
| Some configurations can be made flexible to allow changes, while others may be | ||
| set as immutable for enhanced security. | ||
|
|
||
| For example, if your image includes the `systemd-growfs-root.service`, which |
There was a problem hiding this comment.
Instead of using prose, let's try to make the recommendations more structured. Also, including example configs would be goodness.
For example:
## systemd-growfs-root
This service attempts to resize the root filesystem, which fails since verity makes the root filesystem readonly and a fixed size.
### Solution 1: Do nothing
Since the root filesystem is readonly, the `systemd-growfs-root` service will fail.
However, the only impact will be an error in the boot logs.
### Solution 2: Disable service
Disabling the service removes the error from the boot logs.
`` `yaml
os:
services:
disable:
- systemd-growfs-root
`` `Do something similar for other services (e.g. cloud-init, ssh, auditd, docker, etc.).
| security. Some configurations can be made flexible to allow changes, while | ||
| others may be set as immutable for enhanced security. | ||
|
|
||
| ## systemd-growfs-root |
There was a problem hiding this comment.
It might be goodness to create a dedicated section for the var partition, since so many partitions need it to be writable. That way, you can avoid sections pointing to the cloud-init section.
| - systemd-growfs-root | ||
| ``` | ||
|
|
||
| ## cloud-init-local, cloud-config, cloud-final |
There was a problem hiding this comment.
nit: Just ## cloud-init
While there are multiple systemd services included with cloud-init, they are all part of the same application.
| - systemd-growfs-root | ||
| ``` | ||
|
|
||
| ## cloud-init-local, cloud-config, cloud-final |
There was a problem hiding this comment.
cloud-init is an odd one. cloud-init has a bunch of different features that can be used to configure the system. (e.g. user accounts, networking, etc.) But a lot of those features typically require the /etc directory to be writable.
I think the general recommendation should be to disable cloud-init and not use it.
Though it is possible that some cloud-init features can be used under verity. For those, we may end up giving them dedicated sections in this doc.
There was a problem hiding this comment.
I have re-write the cloud-init part, however, the cloud-init services I faced so far (cloud-init-local, cloud-config, cloud-final), they all need write access to /etc, so I think for the section of cloud-init features that can be used under verity, the doc can be updated when users meet them.
| The following example script can be added to the `postCustomization` scripts to | ||
| automatically move the SSH host keys and update the `sshd_config` file: | ||
|
|
||
| ```bash |
There was a problem hiding this comment.
You should provide a sample config, not just the script.
Note: Scripts can now be specified inline using content instead of path.
| The following example script can be added to the `postCustomization` scripts to | ||
| automatically move the SSH host keys and update the `sshd_config` file: | ||
|
|
||
| ```bash |
There was a problem hiding this comment.
The sshd-keygen.service file also need to be updated.
Note: Additional files now support content instead of source.
| security. Some configurations can be made flexible to allow changes, while | ||
| others may be set as immutable for enhanced security. | ||
|
|
||
| ## systemd-growfs-root |
There was a problem hiding this comment.
You should add a section for networking.
In Mariner 3.0, there are no longer any default network settings. In non-verity images, this doesn't matter too much since cloud-init will provide default networking settings. But cloud-init fails to provision the network in verity images since /etc is not writable. Hence, it is a good idea for verity images to specify network settings. The test verity config does this.
There was a problem hiding this comment.
Added, please review.
Merge Checklist
All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)
*-staticsubpackages, etc.) have had theirReleasetag incremented../cgmanifest.json,./toolkit/scripts/toolchain/cgmanifest.json,.github/workflows/cgmanifest.json)./LICENSES-AND-NOTICES/SPECS/data/licenses.json,./LICENSES-AND-NOTICES/SPECS/LICENSES-MAP.md,./LICENSES-AND-NOTICES/SPECS/LICENSE-EXCEPTIONS.PHOTON)*.signatures.jsonfilessudo make go-tidy-allandsudo make go-test-coveragepassSummary
What does the PR accomplish, why was it needed?
Service and Overlay recommendations for Verity-enabled images.
Clear services down errors when booting Verity enabled images.
Does this affect the toolchain?
NO
Test Methodology