Skip to content

Commit

Permalink
Add trustroot testdata generator and embedded data (#1325)
Browse files Browse the repository at this point in the history
* Add trustroot testdata generator and embedded data

Signed-off-by: Cody Soyland <[email protected]>

* Update whitespace action

Signed-off-by: Cody Soyland <[email protected]>

* Update codegen

Signed-off-by: Cody Soyland <[email protected]>

* Add Makefile target for generate-testdata

Signed-off-by: Cody Soyland <[email protected]>

* Update comments about data format

Signed-off-by: Cody Soyland <[email protected]>

---------

Signed-off-by: Cody Soyland <[email protected]>
  • Loading branch information
codysoyland authored Mar 23, 2024
1 parent 1fe635e commit 46f0020
Show file tree
Hide file tree
Showing 23 changed files with 1,193 additions and 167 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/whitespace.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ jobs:
- name: Check out code
uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633 # v4.1.2

- uses: chainguard-dev/actions/trailing-space@84c993eaf02da1c325854fb272a4df9184bd80fc # main
- uses: chainguard-dev/actions/trailing-space@7071df0659dbd4a79804731f0da2d0f1dba0b356 # main
if: ${{ always() }}

- uses: chainguard-dev/actions/eof-newline@84c993eaf02da1c325854fb272a4df9184bd80fc # main
- uses: chainguard-dev/actions/eof-newline@7071df0659dbd4a79804731f0da2d0f1dba0b356 # main
if: ${{ always() }}
3 changes: 3 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -189,3 +189,6 @@ docs/generate-api:
`find ./pkg/apis/policy/v1alpha1/ -iname '*types.go' | sort -r | tr '\n' ' '` \
> docs/api-types/index-v1alpha1.md;

.PHONY: generate-testdata
generate-testdata:
go run hack/gentestdata/gentestdata.go
16 changes: 8 additions & 8 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ require (
golang.org/x/net v0.22.0
golang.org/x/sys v0.18.0 // indirect
golang.org/x/time v0.5.0
google.golang.org/grpc v1.61.1 // indirect
google.golang.org/grpc v1.62.0 // indirect
google.golang.org/protobuf v1.33.0 // indirect
gopkg.in/yaml.v3 v3.0.1
k8s.io/api v0.29.3
Expand All @@ -61,6 +61,7 @@ require (
github.com/docker/docker v26.0.0+incompatible
github.com/docker/go-connections v0.5.0
github.com/go-jose/go-jose/v3 v3.0.3
github.com/sigstore/scaffolding v0.6.17
github.com/sigstore/sigstore/pkg/signature/kms/aws v1.8.2
github.com/sigstore/sigstore/pkg/signature/kms/azure v1.8.2
github.com/sigstore/sigstore/pkg/signature/kms/gcp v1.8.2
Expand Down Expand Up @@ -166,7 +167,7 @@ require (
github.com/go-openapi/loads v0.21.5 // indirect
github.com/go-openapi/runtime v0.27.1 // indirect
github.com/go-openapi/spec v0.20.14 // indirect
github.com/go-openapi/strfmt v0.22.0 // indirect
github.com/go-openapi/strfmt v0.22.1 // indirect
github.com/go-openapi/swag v0.22.9 // indirect
github.com/go-openapi/validate v0.22.6 // indirect
github.com/gobuffalo/flect v1.0.2 // indirect
Expand Down Expand Up @@ -199,7 +200,6 @@ require (
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 // indirect
github.com/miekg/pkcs11 v1.1.1 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
Expand All @@ -217,9 +217,9 @@ require (
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_golang v1.18.0 // indirect
github.com/prometheus/client_model v0.5.0 // indirect
github.com/prometheus/common v0.45.0 // indirect
github.com/prometheus/client_golang v1.19.0 // indirect
github.com/prometheus/client_model v0.6.0 // indirect
github.com/prometheus/common v0.48.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect
github.com/prometheus/statsd_exporter v0.22.8 // indirect
github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 // indirect
Expand All @@ -228,7 +228,7 @@ require (
github.com/sassoftware/relic v7.2.1+incompatible // indirect
github.com/secure-systems-lab/go-securesystemslib v0.8.0 // indirect
github.com/shibumi/go-pathspec v1.3.0 // indirect
github.com/sigstore/timestamp-authority v1.2.1 // indirect
github.com/sigstore/timestamp-authority v1.2.2 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/sourcegraph/conc v0.3.0 // indirect
github.com/spf13/afero v1.11.0 // indirect
Expand All @@ -245,7 +245,7 @@ require (
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/yashtewari/glob-intersection v0.2.0 // indirect
go.mongodb.org/mongo-driver v1.13.1 // indirect
go.mongodb.org/mongo-driver v1.14.0 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.48.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.48.0 // indirect
Expand Down
52 changes: 22 additions & 30 deletions go.sum

Large diffs are not rendered by default.

247 changes: 247 additions & 0 deletions hack/gentestdata/gentestdata.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,247 @@
// Copyright 2024 The Sigstore Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package main

import (
"bytes"
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/x509"
"crypto/x509/pkix"
"encoding/json"
"encoding/pem"
"flag"
"log"
"math/big"
"os"
"path"
"path/filepath"
"time"

"github.com/sigstore/cosign/v2/pkg/cosign"
"github.com/sigstore/policy-controller/pkg/apis/config"
testing "github.com/sigstore/policy-controller/pkg/reconciler/testing/v1alpha1"
"github.com/sigstore/scaffolding/pkg/repo"
"github.com/sigstore/sigstore/pkg/cryptoutils"
)

// This program generates test data for the trustroot reconciler.
//
// To run this program, you can use the following command from the root of the repo:
// $ go run hack/gentestdata/gentestdata.go
// or,
// $ make generate-testdata
//
// The output of this program can be used to update the `marshalledEntry.json`
// file in the `pkg/reconciler/trustroot/testdata` package.
//
// Do not rely on the output of this program to produce valid results. Always
// verify the output manually before committing.

var (
dir = flag.String("output-dir", "pkg/reconciler/trustroot/testdata", "Output directory")
)

func main() {
flag.Parse()
ctfePK, ctfeLogID := genPK()
rekorPK, rekorLogID := genPK()
fulcioChain := genCertChain(x509.KeyUsage(x509.ExtKeyUsageCodeSigning))
fulcioChainConcat := bytes.Join(fulcioChain, nil)
tsaChain := genCertChain(x509.KeyUsage(x509.ExtKeyUsageTimeStamping))
tsaChainConcat := bytes.Join(tsaChain, nil)

sigstoreKeysMap := map[string]string{
"ctfe": string(ctfePK),
"fulcio": string(fulcioChainConcat),
"rekor": string(rekorPK),
"tsa": string(tsaChainConcat),
}
marshalledEntry, err := genTrustRoot(sigstoreKeysMap)
if err != nil {
log.Fatal(err)
}

marshalledEntryFromMirrorFS, tufRepo, rootJSON, err := genTUFRepo(sigstoreKeysMap)
if err != nil {
log.Fatal(err)
}

mustWriteFile("ctfePublicKey.pem", ctfePK)
mustWriteFile("ctfeLogID.txt", []byte(ctfeLogID))
mustWriteFile("rekorPublicKey.pem", rekorPK)
mustWriteFile("rekorLogID.txt", []byte(rekorLogID))
mustWriteFile("fulcioCertChain.pem", fulcioChainConcat)
mustWriteFile("tsaCertChain.pem", tsaChainConcat)
mustWriteFile("marshalledEntry.json", marshalledEntry)
mustWriteFile("marshalledEntryFromMirrorFS.json", marshalledEntryFromMirrorFS)
mustWriteFile("tufRepo.tar", tufRepo)
mustWriteFile("root.json", rootJSON)
}

func mustWriteFile(path string, data []byte) {
err := os.WriteFile(filepath.Join(*dir, path), data, 0600)
if err != nil {
log.Fatalf("failed to write file %s: %v", path, err)
}
}

func genPK() ([]byte, string) {
priv, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
log.Fatalf("failed to generate ecdsa key: %v", err)
}
der, err := x509.MarshalPKIXPublicKey(priv.Public().(*ecdsa.PublicKey))
if err != nil {
log.Fatalf("failed to marshal ecdsa key: %v", err)
}
pemPK := pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: der})

// generate log id
pk, err := cryptoutils.UnmarshalPEMToPublicKey(pemPK)
if err != nil {
log.Fatalf("failed to unmarshal ecdsa key: %v", err)
}
logID, err := cosign.GetTransparencyLogID(pk)
if err != nil {
log.Fatalf("failed to get transparency log id: %v", err)
}
return pemPK, logID
}

func genCertChain(keyUsage x509.KeyUsage) [][]byte {
// Create a new CA certificate
caPriv, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
log.Fatalf("failed to generate ecdsa key: %v", err)
}
template := &x509.Certificate{
SerialNumber: new(big.Int).SetInt64(1),
Subject: pkix.Name{CommonName: "ca"},
NotBefore: time.Now(),
NotAfter: time.Now().AddDate(10, 0, 0),
KeyUsage: x509.KeyUsageCertSign,
BasicConstraintsValid: true,
IsCA: true,
}
caCertBytes, err := x509.CreateCertificate(rand.Reader, template, template, caPriv.Public(), caPriv)
if err != nil {
log.Fatalf("failed to create x509 certificate: %v", err)
}

caCert, err := x509.ParseCertificate(caCertBytes)
if err != nil {
log.Fatalf("failed to parse x509 certificate: %v", err)
}

// Create a new leaf certificate
leafPriv, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
log.Fatalf("failed to generate ecdsa key: %v", err)
}
leafCert, err := x509.CreateCertificate(rand.Reader, &x509.Certificate{
SerialNumber: new(big.Int).SetInt64(2),
Subject: pkix.Name{CommonName: "leaf"},
NotBefore: time.Now(),
NotAfter: time.Now().AddDate(10, 0, 0),
KeyUsage: keyUsage,
}, caCert, &leafPriv.PublicKey, caPriv)
if err != nil {
log.Fatalf("failed to create x509 certificate: %v", err)
}

return [][]byte{pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: leafCert}), pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: caCertBytes})}
}

func genTrustRoot(sigstoreKeysMap map[string]string) (marshalledEntry []byte, err error) {
trustRoot := testing.NewTrustRoot("test-trustroot", testing.WithSigstoreKeys(sigstoreKeysMap))
sigstoreKeys := &config.SigstoreKeys{}
sigstoreKeys.ConvertFrom(context.Background(), trustRoot.Spec.SigstoreKeys)
err = populateLogIDs(sigstoreKeys)
if err != nil {
return nil, err
}
return json.MarshalIndent(sigstoreKeys, "", " ")
}

func populateLogIDs(sigstoreKeys *config.SigstoreKeys) error {
for i := range sigstoreKeys.TLogs {
logID, err := genLogID(sigstoreKeys.TLogs[i].PublicKey)
if err != nil {
return err
}
sigstoreKeys.TLogs[i].LogID = logID
}
for i := range sigstoreKeys.CTLogs {
logID, err := genLogID(sigstoreKeys.CTLogs[i].PublicKey)
if err != nil {
return err
}
sigstoreKeys.CTLogs[i].LogID = logID
}
return nil
}

func genLogID(pkBytes []byte) (string, error) {
pk, err := cryptoutils.UnmarshalPEMToPublicKey(pkBytes)
if err != nil {
return "", err
}
return cosign.GetTransparencyLogID(pk)
}

func genTUFRepo(sigstoreKeysMap map[string]string) ([]byte, []byte, []byte, error) {
files := map[string][]byte{}
files["rekor.pem"] = []byte(sigstoreKeysMap["rekor"])
files["ctfe.pem"] = []byte(sigstoreKeysMap["ctfe"])
files["fulcio.pem"] = []byte(sigstoreKeysMap["fulcio"])

defer os.RemoveAll(path.Join(os.TempDir(), "tuf")) // TODO: Update scaffolding to use os.MkdirTemp and remove this
ctx := context.Background()
local, dir, err := repo.CreateRepo(ctx, files)
if err != nil {
return nil, nil, nil, err
}
meta, err := local.GetMeta()
if err != nil {
return nil, nil, nil, err
}
rootJSON, ok := meta["root.json"]
if !ok {
return nil, nil, nil, err
}

var compressed bytes.Buffer
if err := repo.CompressFS(os.DirFS(dir), &compressed, map[string]bool{"keys": true, "staged": true}); err != nil {
return nil, nil, nil, err
}

trustRoot := &config.SigstoreKeys{
CertificateAuthorities: []config.CertificateAuthority{{CertChain: []byte(sigstoreKeysMap["fulcio"])}},
TLogs: []config.TransparencyLogInstance{{PublicKey: []byte(sigstoreKeysMap["rekor"])}},
CTLogs: []config.TransparencyLogInstance{{PublicKey: []byte(sigstoreKeysMap["ctfe"])}},
}
err = populateLogIDs(trustRoot)
if err != nil {
return nil, nil, nil, err
}
trustRootBytes, err := json.MarshalIndent(trustRoot, "", " ")
if err != nil {
return nil, nil, nil, err
}
return trustRootBytes, compressed.Bytes(), rootJSON, nil
}
1 change: 1 addition & 0 deletions pkg/reconciler/trustroot/testdata/ctfeLogID.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
83e749763552c099b251d441566b9c12f160b24fbff28ab08d2681757d8acbde
4 changes: 4 additions & 0 deletions pkg/reconciler/trustroot/testdata/ctfePublicKey.pem
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEZ4cgFaCk7JtO/wxDw2E1S3U+97F0
2dF2fixniThvXgbxAQ+bkQ4dQUNwN46QcCzwYuJc9742Vi6LvNx7X7427A==
-----END PUBLIC KEY-----
33 changes: 33 additions & 0 deletions pkg/reconciler/trustroot/testdata/fulcioCert.pem
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
-----BEGIN CERTIFICATE-----
MIIFwzCCA6ugAwIBAgIIfUmh4cIZr8QwDQYJKoZIhvcNAQELBQAwfjEMMAoGA1UE
BhMDVVNBMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1TYW4gRnJhbmNp
c2NvMRYwFAYDVQQJEw01NDggTWFya2V0IFN0MQ4wDAYDVQQREwU1NzI3NDEZMBcG
A1UEChMQTGludXggRm91bmRhdGlvbjAeFw0yMzEyMTQxODUxMzlaFw0yNDEyMTQx
ODUxMzlaMH4xDDAKBgNVBAYTA1VTQTETMBEGA1UECBMKQ2FsaWZvcm5pYTEWMBQG
A1UEBxMNU2FuIEZyYW5jaXNjbzEWMBQGA1UECRMNNTQ4IE1hcmtldCBTdDEOMAwG
A1UEERMFNTcyNzQxGTAXBgNVBAoTEExpbnV4IEZvdW5kYXRpb24wggIiMA0GCSqG
SIb3DQEBAQUAA4ICDwAwggIKAoICAQDHVwB8bv84fUgVOqjjWtMAK4i5Zl93I9ai
zh9S/qIuJNnKx1tA87xZcAuO5riq/kXA2fZGnnP4Vsp9VaVjK9o7+1QP2rFJ4p5r
rQlZFovvrD1e6jEaoMc06v+YY4yl37b17W9sfd+5x5wZ0ArRjPAihpdVjYJwlqDR
B0AlSo6Vq/aM9QejMG4CS1jXrEEUV8MwRNjyT2xdR4vkc6wj47A1/rknjCtMsieS
eSmH/ZDamUGuUh5ej4/dmCiLw93Rou/yLlDcvAcFVzrrLMF/lRwUDUgoH1XDlpeC
C1r5HB6jp1Huap9gcLNS3UCIZVpNDO0A3pjYaLBQ3bfHe6QxKuQcEd+VKqyP9SoP
dNn31cygF28VR+k+0jU5uXxW7ilXrv7DVYMOcMNZCDA0BQdH/A3fO0ri+8t2Luo+
EilRWROBsJTuC28sesYc5NUUoszxVUoQFAhkxE6k5rGIzxO8XplgLjx0IPxU0wjj
VhcBa7AKkAMT7gDrPXijhJbv7Q3QVkChOdj6VTPagCS+JtWBkzGvCNJmaIrbLdWF
TtDMXfSSZoRyn/aXjQr/OFzBf6dDxJqEMvdD5T5Gg1sldZ00KLKqEx25i8HVZ8Xo
V4jrZOH1b9nZa3DGZOPmditlqUppvJ7c6OIGqkpE1o8mcNKko/p0dCwcHQtXgIN5
76foyNG+twIDAQABo0UwQzAOBgNVHQ8BAf8EBAMCAQYwEgYDVR0TAQH/BAgwBgEB
/wIBATAdBgNVHQ4EFgQU6A9czPqMog/PFdvjxH3V/56BBhcwDQYJKoZIhvcNAQEL
BQADggIBAAGqm7dJS+pNgCEUDE79S2r6c+BcH6DwTFvAujE0yvdTRdAVIo73CsqP
W4cDFuCw2ekOhD17JUT+9PEGJv++u16X4tLHVI5QHPleU/qzZHSEIYt0AE+y9JEL
R2RT0g11YToGzhIAto5OpOvBb1z+Q8uP5g4eK7Y8J2lVRkDk/62EtsaHTWgv9hJJ
qsdwoUMVWxn/s0oanPjyGBMSwpoFDXX/k14NDsCGp7d2e5/DxjgYAenDTtnID3VK
kvP46spBZ4yEbNIywjaubSXnNLsx2cY8Ypih23e8c1uQJ3O44FDYXVcqYZX9UOrK
HS0aE5VpU5J/j2fr4hGE3SfRXXDizcZJcVWPL+k1DHKWlCREMYw12ha3Oe0uIlwK
W7syTNnn8NgxxRgM4f83n0C/00CSqiTm8MYya3ue0m2gmCg6TguALbcIqZ3tEK3K
vvNIbgxM0ZSePI8YktvtLTQsRK8bbianOht+CwYD2NnFKo68G0l57ByKXze0wG18
i943+NTOvU/Le+8SEwJ4asRld3v3L8pCpNAM7JX12zoqisAnCCj3hu6waA5XvMeh
STj8yYtIxP1l1I1qfRJzMB9nGv9KzwmozHiw3oGJr/G3j1u1krrQfj4S6z16Bq29
nfILFnmk/MoeqYS6DBRY80b60289+R7CSCB5OQbQYvmjy/sxvcNO
-----END CERTIFICATE-----
18 changes: 18 additions & 0 deletions pkg/reconciler/trustroot/testdata/fulcioCertChain.pem
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
-----BEGIN CERTIFICATE-----
MIIBPjCB5KADAgECAgECMAoGCCqGSM49BAMCMA0xCzAJBgNVBAMTAmNhMB4XDTI0
MDMyMTIxMzcwNVoXDTM0MDMyMTIxMzcwNVowDzENMAsGA1UEAxMEbGVhZjBZMBMG
ByqGSM49AgEGCCqGSM49AwEHA0IABNnZTptnN0TWM6BRIPn/KLgo2u/W5Vt8lmOM
6xYfr1uXobdkmcUI+qMxAmXhOHDhcXgQKlgZuivcd8XwmOlpQ0SjMzAxMA4GA1Ud
DwEB/wQEAwIGwDAfBgNVHSMEGDAWgBRz6KN30XFdWO9mNjwtziSnqItmEjAKBggq
hkjOPQQDAgNJADBGAiEA9dnInoX3QVoKbqGohmvuHjcw3SLi3cYMkMCGyLI3sioC
IQDqFTNB7UGQG2HCCXoGO+hHd1uCDEz2i+56JDXYSiKnOQ==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIBSjCB8aADAgECAgEBMAoGCCqGSM49BAMCMA0xCzAJBgNVBAMTAmNhMB4XDTI0
MDMyMTIxMzcwNVoXDTM0MDMyMTIxMzcwNVowDTELMAkGA1UEAxMCY2EwWTATBgcq
hkjOPQIBBggqhkjOPQMBBwNCAAREu5I6L0ARFHjrcT+YWXuKOyo57mqOB6mCz74o
4Puipf3w8Ciuh9tnN2I1FlZ+gL3j9RKn613E399EUHkjpOoro0IwQDAOBgNVHQ8B
Af8EBAMCAgQwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUc+ijd9FxXVjvZjY8
Lc4kp6iLZhIwCgYIKoZIzj0EAwIDSAAwRQIgGpcv3B78/j4Ru+AqVA934rCGqM/X
83pUXjS4/PUsP3UCIQDlosQuYkks7zlgY7rCYMF6Nqo/1OvTOwy9V2yY3v0a4A==
-----END CERTIFICATE-----
Loading

0 comments on commit 46f0020

Please sign in to comment.