Skip to content

Commit

Permalink
docs: fill recipe pages with content
Browse files Browse the repository at this point in the history
  • Loading branch information
the-dipsy committed Mar 15, 2024
1 parent c96853d commit 2a2719e
Show file tree
Hide file tree
Showing 5 changed files with 372 additions and 3 deletions.
56 changes: 55 additions & 1 deletion docs/recipes-libraries.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,60 @@
---
parent: Recipes
nav_order: 50
nav_order: 30
---

# Component Libraries
Outsourcing parts of your configuration into reusable libraries is a great way
to keep your projects manageable and create a good foundation for future ones.
You can even nest libraries as deeply as you like.

You won't have to learn a new build system or package manager, because
you can simply employ *Git* submodules for this.

## Creating a Library
All you need to do to create a library is initialize a separate repository
(`git init`) and populate it with your source files.

The [Project Structure][structure] page's recommendations about version control
and main components apply to libraries as well, except for the convention to
refrain from taking positional and keyword arguments in your main component.
The parameters defined in your main (and other) component's signatures should
in fact be the only way to parameterize your library.

Projects incorporating your library can, of course, access all of its files and
components. Depending on its purpose, it might make sense to design your
library to expose its entire functionality through the main component, though.

What's only a strong recommendation in the context of standalone projects is
vital when it comes to libraries: All references to files and components within
your library must be relative, while references to standard library components
must be absolute. Use `_.my_component()` but
`magic(load.toml(_/"config.toml"))`. Otherwise, your library will break when
moved within the directory structure.

The return values of your library can be whatever you need. They might be
completely custom data, functions, classes, etc. They might also be a list or
dict to populate some field of the *Butane* configuration you are constructing.
Or they might be a complete configuration, ready to be merged with the rest of
your projects' configs using the [*merge* standard library component][merge].

When your library has arrived at a stable state, consider adding a version tag
(`git tag vX.Y.Z`) to easily manage different projects using different versions
of your library.

[structure]: recipes-structure.html
[merge]: components-stdlib.html#create-merge-fields-for-inline-local-andor-remote-configs

## Adding a Library to Your Project
A good place to add all your project's libraries is the *lib* directory. Its
purpose will be obvious, you'll know where to look for your libraries, and you
avoid cluttering your repository's root directory with external code.

To add a library to your project, simply add it as a *Git* submodule and
optionally check out a version tag. Execute, e.g., `git submodule add SOURCE
lib/NAME && cd lib/NAME && git checkout vX.Y.Z` to add a library from a
specific source at a specific version to your *lib* directory with a specific
name. Don't forget to add and commit these changes in your project repository.

If your library has a *main.pyro* component, you can now simply reference and
execute it using `_.lib.NAME(...)`.
109 changes: 108 additions & 1 deletion docs/recipes-remote.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,113 @@
---
parent: Recipes
nav_order: 60
nav_order: 40
---

# Remote Configuration
Serving your configurations over *HTTP(S)* has the great advantage that you
neither have to include any potentially secret configuration details for your
final system on your unencrypted installer media, nor do you need to create new
media when your configuration changes.

This page outlines the installation of an encrypted server with mutually
secured remote configuration loading.

The machine you run *Pyromaniac* on and the one you want the system to be
installed on need to share a network, and your firewall and *NAT*
configurations need to allow the target machine to open *TCP* connections
to a port on the machine running *Pyromaniac*. The shared network can be the
internet, but a more restricted one would be preferable to reduce the attack
surface. If your IT infrastructure allows it, you may even directly connect the
two machines using some network cable and static IP addressing.

## Installer Preparation
The first thing we need to do is create our *ISO* installer. You'll need to
know the disk device you want to install *CoreOS* to and your *Pyromaniac*
machine's address.

Assuming you would like to install to */dev/sda* and your *Pyromaniac* *HTTPS*
server will run on port *4433* at *192.168.0.10*, you can create your installer
in a single command:

```sh
pyromaniac --iso --iso-disk /dev/sda \
--address 'https://192.168.0.10:4433/' \
<<< '`remote.merge()`' > installer.iso
```

A self-signed certificate and random credentials for encryption and mutual
authentication will be embedded into the installer by default.

## Setting Up Disk Encryption
To use disk encryption in *CoreOS*, you'll need to use *Clevis* pinning as
[described in the CoreOS docs][luks]. You may still want to choose and store
the encryption keys for your root partition and further data partitions
yourself.

This configuration piece will encrypt your root partition with a key of your
choosing to demonstrate the retrieval of secret keys over *HTTP(S)*. This would
be more useful for persistent data partitions in practice. Consider using
*Clevis* pinning for the root partition instead as [described in the CoreOS
docs][luks]. The configuration described here will break unattended upgrades
because you'll need to manually type in the encryption key every time the
machine is rebooted.

```yaml
storage.luks[0]:
name: root
label: luks-root
device: /dev/disk/by-partlabel/root
key_file: `contents(remote.url / "root.secret", remote.headers)`
wipe_volume: true
```
Remember that the `remote` variable will only be available in your main
component. If you configure storage in a separate component, you'll need to
pass the *URL* and headers as arguments to it.

[luks]: https://docs.fedoraproject.org/en-US/fedora-coreos/storage/#_encrypted_storage_luks

## Perform the Installation
You can now start the *Pyromaniac* *HTTPS* server with the same address used
for the *ISO* generation using the following command:

```
pyromaniac --serve --address 'https://192.168.0.10:4433/' .
```

If you boot your installer medium, it will request the */config.ign* path from
your server, which will compile your configuration and send it back to the
installer. During the storage setup, the installer will request the encryption
key from the */root.secret* path. *Pyromaniac* will prompt you for the secret
in the terminal and respond to the request with whatever line you type.

The installer might request your configuration multiple times during the
installation process, and it will be recompiled every time. You might run into
problems if your code is not deterministic and depends on randomness to compile
your configuration.

After the installation finishes, the server will be up and running as specified
in your configuration.

## Serving Static Configurations
You might already have a readily compiled *Ignition* file or don't want the
server to recompile your configuration every time if compilation takes a bit
longer.

You can easily serve an *Ignition* file named *config.ign* by writing a
one-liner using the `ignition.config.replace` field:

```
pyromaniac --serve --address 'https://192.168.0.10:4433/' \
<<< 'ignition.config.replace: `contents(Path("config.ign"))`'
```

## For Debugging
Remote configuration loading can accelerate testing of your configs in virtual
machines as well. Start the installation as usual but make a snapshot right
before the configuration is loaded. You can then restore that snapshot whenever
you'd like to test a new version of your configuration and have the
provisioning of your machine happen within seconds.

Since the *HTTP(S)* server will recompile your configuration on every request,
you can simply keep it running while tinkering with your configs.
87 changes: 86 additions & 1 deletion docs/recipes-rootless-podman.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,91 @@
---
parent: Recipes
nav_order: 40
nav_order: 60
---

# Rootless Podman
Even though you can easily embed raw executables and systemd services into your
deployments, *Fedora CoreOS* comes with *docker* and *podman* preinstalled and
is optimized for container workloads.

In regards to building secure systems, the *podman* engine has the major
advantage of being geared towards rootless containers. You can use this to add
an extra layer of separation by running different services as different *Linux*
users. When an attacker manages to compromise your service and also to break
out of the container, they will still not have control over the entire system
or other containers running on it.

## Systemd Units
The recommended way to manage *podman* containers outside of platforms like
*Kubernetes* is using [Quadlet][quadlet]. *Quadlet* allows you to specify your
container deployments inside special kinds of *systemd* unit files like the
following.

`my-service.container`
```ini
[Unit]
Description=My Service

[Container]
Image=docker.io/my/image

[Service]
Restart=always

[Install]
WantedBy=multi-user.target
```

You can use the *tree* standard library component to include an entire
directory of such units into your deployment like this:

```python
---
dirs = directories(".", ".config/containers", "myuser")
units = tree(".config/containers/systemd", _/"units", "myuser")
---

storage:
directories: `dirs + units['directories']`
files: `units['files']`
```

[quadlet]: https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html

## Templating
The great thing about *Pyromaniac* is that you can outsource code and build
abstractions very easily. You can write a component for constructing a unit
file from a Jinja template or even for creating units from scratch.

A component for creating *Quadlet* units from scratch might look something like
this:

`quadlet.pyro`
```python
(user: str, name: str, ext: str = "container", **sections: dict[str, str] = [])
---
lines = []
for section, fields in sections.items():
lines.append(f"[{section.capitalize()}]")
for key, value in fields.items():
key = "".join(w.capitalize() for w in key.split("_"))
lines.append(f"{key}={value}")

file(f".config/containers/systemd/{name}.{ext}", "\n".join(lines), user)
```

It could be used to add a file for the unit from above as follows:

```python
---
unit = quadlet(
"myuser", "my-service",
unit={"description": "My Service"},
container={"image": "docker.io/my/image"},
service={"restart": "always"},
install={"wanted_by": "multi-user.target"},
)
---

storage.files[0]: `unit`
```
112 changes: 112 additions & 0 deletions docs/recipes-structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,115 @@ nav_order: 20
---

# Project Structure
*Pyromaniac* leaves you with a lot of freedom on how and in which language to
declare what your server should look like. This page contains some
recommendations for structuring your *Pyromaniac* projects.

The directory structure for a larger project could, e.g., look something like
this:

```
.
├─ README.md
├─ main.pyro
├─ config.toml
├─ component_a.pyro
├─ component_b.pyro
├─ .gitmodules
├─📂 component_c
│ ├─ main.pyro
│ └─ component_d.pyro
└─📂 lib
├─📂 library_a
│ ├─ main.pyro
│ └─ ...
└─📂 library_b
├─ main.pyro
└─ ...
```

## Version Control
First of all, *Pyromaniac* puts great emphasis on maintainability and clarity
by enabling you to set up your servers in a declarative and modular fashion
without boilerplate or unwieldy tooling. All you need is a couple of plain text
files with whatever file structure you see fit.

This makes it easier than ever to put your server configuration under source
control. Whether you want to publish your configuration on *GitHub*, store it
on a self-hosted repository platform, or just on your local machine: Go ahead
and initialize a *Git* repository and come up with a suitable commit and
deployment strategy.

While you're at it: Document your strategy and the required commands to build
your project in the repository's *README.md*.

## Main Component
*Pyromaniac* allows you to supply your main component on standard input or
specify any file or pipe to read it from. The convention, however, is to place
it in a file named *main.pyro* in your repository's root directory.

Even though excess positional command line parameters to *Pyromaniac* will
be passed on to your main component, it is recommended to place your
customizations in a configuration file instead, as described in the next
section.

If you follow these conventions, you will be able to consistently just run
`pyromaniac . > config.ign` in your project's root directories, instead of
having to document project-specific build commands.

You have a lot of freedom in how your `main.pyro` produces its final result.
Splitting your configurations up and organizing them in trees of components and
libraries is where *Pyromaniac* shines, though.

## Configuration
*Pyromaniac* makes it very easy to build abstractions and keep your
projects configurable. Consider placing all deployment or brand-specific
information in a central configuration file or directory. Depending on your
use case, you might only want to commit a generic template for that
configuration to your main project's source control.

A solid choice would be to create a *config.toml* (and/or *config.toml.tmpl*)
file, load it from your main component using the [*load.toml*][toml] and
[*magic*][magic] standard library components, and pass the options on as
arguments to your other components.

Your main component might look something like this:

`main.pyro`
```python
---
config = magic(load.toml(_/"config.toml"))

merge(
my_server(**config.server),
my_storage(config.storage.root_size or 8000),
)
```

[toml]: components-stdlib.html#load-toml-from-disk-injecting-variables-using-jinja
[magic]: components-stdlib.html#wrap-value-in-magic-type-for-convenient-member-access-and-default-handling

## Libraries
Information technology is all about managing complexity and breaking hard
problems down into easier ones. It is very straightforward to outsource bits
of your configuration into reusable components or libraries in *Pyromaniac*.

Consider bundling all such libraries in a *lib* directory as *Git* submodules.
This allows you to maintain and version control your libraries separately and
keep your individual projects more manageable. Read more about libraries on the
[Component Libraries][libraries] recipe page.

[libraries]: recipes-libraries.html

## Building on Top of Pyromaniac
If you have built an extensive component framework for making the declaration
of new systems for your use cases a breeze, you can even package it as your own
container image based on *Pyromaniac*.

Add your components to the image's */usr/local/lib/pyromaniac* directory and
optionally configure a new entry point to make your image behave however you
like. Your *Python* scripts can simply import the `pyromaniac` module and
build on top of it. You can even go so far as to specifying your entirely own
*YAML* or *TOML* based specification format, have your *Pyromaniac* components
transform it into a corresponding *Butane* configuration, and produce bootable
disk images within no time.
Loading

0 comments on commit 2a2719e

Please sign in to comment.