Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installer #231

Draft
wants to merge 157 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
157 commits
Select commit Hold shift + click to select a range
530e58f
updating installer docs
Nov 22, 2019
3c51e34
updating installer docs
Nov 22, 2019
885e5e5
updating installer docs
Nov 27, 2019
4aea7f9
updating installer docs
Dec 2, 2019
116fab1
updating installer docs
Dec 3, 2019
8e1a2bf
updating installer docs
Dec 18, 2019
315e584
updating installer docs
Dec 19, 2019
8d7b296
updating installer docs
Dec 19, 2019
faf94eb
updating installer docs
Dec 19, 2019
f2f5ace
updating installer docs
Jan 2, 2020
55255a0
updating installer docs
Jan 3, 2020
00dac4d
updating installer docs
Jan 6, 2020
17ac9ea
updating installer docs
Jan 13, 2020
488309a
updating installer docs
Jan 14, 2020
89bb1c1
updating installer docs
Jan 15, 2020
01047ab
updating installer docs
Jan 16, 2020
aa51349
updating installer docs
Jan 16, 2020
edea153
updating installer docs
Feb 4, 2020
5a03368
updating installer docs
Feb 4, 2020
66489c6
updating installer docs
Feb 4, 2020
bad04b0
updating installer docs
Feb 6, 2020
13a06dd
updating installer docs
Feb 7, 2020
7f25ffe
updating installer docs
Feb 7, 2020
908de23
updating installer docs
Feb 8, 2020
6462053
updating installer docs
Feb 26, 2020
0dcf8f3
updating installer docs
Feb 26, 2020
1d69ec5
updating installer docs
Mar 4, 2020
f94950d
updating installer docs
Mar 6, 2020
16db906
updating installer docs
Mar 7, 2020
a62ea46
updating installer docs
Mar 7, 2020
72b838b
updating installer docs
Mar 9, 2020
51f5039
updating installer docs
Mar 12, 2020
4a07735
updating installer docs
Mar 18, 2020
53c8051
updating installer docs
Mar 18, 2020
d2220ed
updating installer docs
Mar 25, 2020
cc6990b
updating installer docs
Mar 25, 2020
de3cc57
updating installer docs
Mar 26, 2020
cb60a26
updating installer docs
Apr 6, 2020
6abe045
updating installer docs
Apr 6, 2020
7b73071
updating installer docs
Apr 22, 2020
149e688
updating installer docs
Apr 22, 2020
58c519c
updating installer docs
Apr 28, 2020
5951d1c
updating installer docs
May 12, 2020
e3502c1
updating installer docs
May 15, 2020
b57754e
updating installer docs
May 18, 2020
b2c43f4
updating installer docs
May 18, 2020
bcd8ad2
updating installer docs
Jun 2, 2020
fdbdadc
updating installer docs
Jun 11, 2020
cb2713c
updating installer docs
Jun 11, 2020
38644ce
updating installer docs
Jun 11, 2020
90c9988
updating installer docs
Jun 13, 2020
45f0973
updating installer docs
Jun 22, 2020
ad007d3
updating installer docs
Jul 17, 2020
75f5cfb
updating installer docs
Jul 17, 2020
be1a87f
updating installer docs
Jul 30, 2020
d881854
updating installer docs
Aug 13, 2020
4b643d6
updating installer docs
Aug 21, 2020
aa8440e
updating installer docs
Oct 22, 2020
adeb7a7
updating installer docs
Oct 29, 2020
e267ac9
updating installer docs
Nov 9, 2020
0d4c5b0
updating installer docs
Nov 17, 2020
ce3623c
updating installer docs
Nov 17, 2020
03f6531
updating installer docs
Nov 17, 2020
f26ff6c
updating installer docs
Nov 20, 2020
d9b978e
updating installer docs
Nov 23, 2020
23df1b7
updating installer docs
Dec 11, 2020
75f8bf7
updating installer docs
Dec 14, 2020
da55d4a
updating installer docs
Dec 14, 2020
485a601
updating installer docs
Jan 22, 2021
4fff321
updating installer docs
Feb 3, 2021
2e0b465
updating installer docs
Feb 24, 2021
daadca3
updating installer docs
Feb 25, 2021
50666f8
updating installer docs
Mar 15, 2021
eeb25a7
updating installer docs
Mar 30, 2021
4cb41e3
updating installer docs
Mar 30, 2021
370ed3b
updating installer docs
Mar 31, 2021
5956d07
updating installer docs
Apr 5, 2021
11154be
updating installer docs
Apr 27, 2021
47bca97
updating installer docs
May 22, 2021
cb1c470
updating installer docs
Jun 24, 2021
8990707
updating installer docs
Aug 17, 2021
17dc6d0
updating installer docs
Aug 27, 2021
9f61866
updating installer docs
Aug 27, 2021
73c5bfb
updating installer docs
Sep 8, 2021
752eab0
updating installer docs
Sep 28, 2021
106e05f
updating installer docs
Oct 28, 2021
e5c857e
updating installer docs
Nov 17, 2021
526552a
updating installer docs
Dec 7, 2021
7749a2a
updating installer docs
Dec 14, 2021
ef117e1
updating installer docs
Dec 14, 2021
8f7e65d
updating installer docs
Dec 14, 2021
6f0c73d
updating installer docs
Dec 16, 2021
9b57306
updating installer docs
Dec 16, 2021
da0b0b4
updating installer docs
Dec 16, 2021
d72311a
updating installer docs
Dec 16, 2021
78de718
updating installer docs
Dec 16, 2021
98a03e5
updating installer docs
Dec 23, 2021
1c4e1f9
updating installer docs
Jan 20, 2022
d0a73c7
updating installer docs
Jan 21, 2022
9483fab
updating installer docs
Feb 7, 2022
b14136c
updating installer docs
Feb 23, 2022
52d2c32
updating installer docs
Apr 1, 2022
85b35a7
updating installer docs
Apr 19, 2022
c0fcfef
updating installer docs
Apr 25, 2022
8e2de2d
updating installer docs
Apr 26, 2022
1ab60de
updating installer docs
May 4, 2022
9c1af46
updating installer docs
May 6, 2022
8ec1b7b
updating installer docs
May 24, 2022
78ad78e
updating installer docs
May 25, 2022
4c53348
updating installer docs
Jul 7, 2022
cd0e97a
updating installer docs
Aug 18, 2022
09b137a
updating installer docs
Sep 19, 2022
b8638a8
updating installer docs
Oct 13, 2022
377f15d
updating installer docs
Oct 14, 2022
1095085
updating installer docs
Nov 3, 2022
672a48a
updating installer docs
Nov 16, 2022
57a0a1e
updating installer docs
Dec 8, 2022
a85ec0e
updating installer docs
Jan 5, 2023
f4fec54
updating installer docs
Mar 30, 2023
3b031d6
updating installer docs
Apr 7, 2023
4825a5f
updating installer docs
Apr 27, 2023
35bde42
updating installer docs
May 16, 2023
857bc6d
updating installer docs
Jun 1, 2023
31db90c
updating installer docs
Jun 5, 2023
659bc3d
Merge branch 'master' into installer
mbreitung Jun 27, 2023
15a3a24
updating installer docs
Jul 5, 2023
e709dbd
updating installer docs
Jul 18, 2023
499f99c
updating installer docs
Jul 27, 2023
9d6b3b5
updating installer docs
Aug 1, 2023
e032122
updating installer docs
Aug 28, 2023
0486513
updating installer docs
Sep 5, 2023
d9ccc5c
updating installer docs
Sep 15, 2023
3a71852
updating installer docs
Oct 30, 2023
724b33f
updating installer docs
Oct 31, 2023
14179f7
updating installer docs
Nov 7, 2023
f32fdff
updating installer docs
Dec 12, 2023
0aede39
updating installer docs
Dec 14, 2023
2c3e94a
updating installer docs
Jan 22, 2024
8343794
updating installer docs
Jan 25, 2024
4121338
updating installer docs
Jan 31, 2024
d84b6e1
updating installer docs
Feb 2, 2024
3db556f
updating installer docs
Feb 27, 2024
d69c8ec
updating installer docs
Mar 2, 2024
2164b86
updating installer docs
Mar 12, 2024
5665e61
updating installer docs
Mar 18, 2024
ced0cbe
updating installer docs
Mar 30, 2024
6a7a64a
updating installer docs
Apr 3, 2024
6b00fcc
updating installer docs
Apr 30, 2024
cb8bb47
updating installer docs
May 22, 2024
b3928d4
updating installer docs
Jun 21, 2024
7eb566f
updating installer docs
Jul 30, 2024
e6483a7
updating installer docs
Jul 30, 2024
4e730b0
updating installer docs
Jul 30, 2024
01a0c7e
updating installer docs
Aug 1, 2024
31ff9a6
updating installer docs
Sep 6, 2024
9bc4533
updating installer docs
Nov 29, 2024
76a6d48
updating installer docs
Dec 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
334 changes: 334 additions & 0 deletions installer/README.md

Large diffs are not rendered by default.

284 changes: 284 additions & 0 deletions installer/docs/01-command_line_arguments.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,284 @@
<!-- Space: IONP -->
<!-- Parent: Installer -->
<!-- Parent: Git Synced Docs -->
<!-- Title: Command Line Arguments -->
<!-- Layout: plain -->

# Command Line Arguments

<br/>

## Command: `deploy`

`--skip-namespace`

- installer does not deploy the `namespace.yaml` manifest.
It expects the Namespace to exist and to match the value in `values.yaml`
If there is a mismatch, the installer will fail as no validation is in place.

`--skip-pull-secret`

- The services require the pull secret to exist with the expected name (`sysdigcloud-pull-secret`) and to have access to the registry.

- if the pull secret is missing, the behaviour could be unpredictable:
some Pods could start if they can find the image locally and if their `imagePullPolicy`
is not `Always`
- Other Pods will fail because they can't pull the image

`--skip-serviceaccount`

- The user must provide service accounts with the exact same name expected:

```text
sysdig-serviceaccount.yaml: name: sysdig
sysdig-serviceaccount.yaml: name: node-labels-to-files
sysdig-serviceaccount.yaml: name: sysdig-with-root
sysdig-serviceaccount.yaml: name: sysdig-elasticsearch
sysdig-serviceaccount.yaml: name: sysdig-cassandra
```

- One implication of this is that unless the `node-to-labels` ServiceAccount is added,
rack awareness will not be available for any datastore.
Another implication is that if the ServiceAccount(s) are missing, the user will have to `describe`
the StatefulSet because Pods will not start at all:

```text
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 2m29s statefulset-controller create Claim data-sysdigcloud-cassandra-0 Pod sysdigcloud-cassandra-0 in StatefulSet sysdigcloud-cassandra success
Warning FailedCreate 67s (x15 over 2m29s) statefulset-controller create Pod sysdigcloud-cassandra-0 in StatefulSet sysdigcloud-cassandra failed error: pods "sysdigcloud-cassandra-0" is forbidden: error looking up service account benedetto/sysdig-cassandra: serviceaccount "sysdig-cassandra" not found
```

`--skip-storageclass`

- installer does not apply the StorageClass manifest.
It expects the storageClassName specified in values.yaml to exist.

`--disable-proxy`

- This flag allows disabling an existing configuration for proxy. Several services can be configured to use a proxy to go out to the Internet. For example `scanningv2-pkgmeta`, `certmanager`, `eventsForwarder` etc.
- If it becomes necessary to remove such configuration, this flag can be used to remove the proxy configuration.
- This flag also applies to `generate`, `diff` and `import`.

## Command: `import`

`--zookeeper-workloadname <string value>`

- This is the value that will be used for the `zookeeper` StatefulSet.
The default value is `zookeeper`, this argument must be used when the
actual name of the StatefulSet in the cluster differs

`--kafka-workloadname <value>`

- Same as above for `kafka`

`--cassandra-workloadname <value>`

- Same as above for `cassandra`

`--use-import-v2`

- This flag will use the new import logic, which will import the values from the cluster and then generate the manifests based on the imported values. Defaults to `false`, which means the old import logic will be used, unless the `--use-import-v2` flag is provided. Import V2 is supported starting from version 6.6.0, and is expected to become the default in the future.

## Command: `update-license`

** WARNING: THIS FEATURE requires `kubectl` to be at least version `1.20.0` **

This command performs the minimal changes and restarts to apply a new license.
Based on [this page](https://docs.sysdig.com/en/docs/administration/on-premises-deployments/upgrade-an-on-premises-license/)

This command performs the following:

- Gets a new license from either `--license` or from `--license-file name.ext`

- applies the license to `common-config` and to the relevant Secret of the following backend services:

- `api`
- `collector`
- `worker`

- If `secure` and `anchore` are enabled, it also applies and restarts all Anchore services.

## Command: `image-list`

This command prints to `stdout` (and optionally to a file) a list of all images in a generated stack.

It requires a `values.yaml` and it produces a list of images based on that `values.yaml`.

It does not require a live cluster, and it does not fetches any value from a live cluster, if one is accessible.

### Flags

`-f <filename>` - write the list to a file. If the file already exists, it will be overwritten.

### Example

```log
./installer/out/installer-darwin-amd64 image-list
I1118 18:48:44.643520 97065 main.go:64] Installer version
I1118 18:48:44.646391 97065 values.go:122] using namespace sysdig from values.yaml
I1118 18:48:44.660236 97065 imagelist.go:44] installerVersion: darwin amd64 gc
I1118 18:48:44.660263 97065 imagelist.go:13] generating manifests
I1118 18:48:44.722172 97065 validate.go:1255] skipping Kubernetes version validation for PostgreSQL because HA is not enabled
I1118 18:48:44.723158 97065 generate.go:171] validation stage:generate passed
I1118 18:49:00.625921 97065 generate.go:234] Generating kubernetes manifests
I1118 18:49:00.642116 97065 generate.go:253] Generating kubernetes manifests for dependencies
I1118 18:49:00.987615 97065 imagelist.go:20] extracting images from generated manifests
I1118 18:49:01.147089 97065 imagelist.go:23] writing images list to file image_list.txt
I1118 18:49:01.147276 97065 imagelist.go:30] found 72 images in the generated manifests
quay.io/sysdig/activity-audit-api:6.0.0.12431
quay.io/sysdig/certman-janitor:6.0.0.12431
quay.io/sysdig/nginx:6.0.0.12431
quay.io/sysdig/anchore:0.8.1-49
quay.io/sysdig/postgres:12.10.0.0
quay.io/sysdig/cp-kafka-6:0.2.1
quay.io/sysdig/kube-rbac-proxy:v0.8.0
quay.io/sysdig/secure-onboarding-api:6.0.0.12431
quay.io/sysdig/ui-monitor-nginx:6.0.0.12431
quay.io/sysdig/sysdig-worker:6.0.0.12431
quay.io/sysdig/profiling-api:6.0.0.12431
quay.io/sysdig/scanning-retention-mgr:6.0.0.12431
quay.io/sysdig/sysdig-api:6.0.0.12431
quay.io/sysdig/helm-renderer:1.0.677
quay.io/sysdig/cp-zookeeper-6:0.4.0
quay.io/sysdig/redis-sentinel-6:1.0.1
quay.io/sysdig/activity-audit-janitor:6.0.0.12431
quay.io/sysdig/secure-todo-worker:6.0.0.12431
quay.io/sysdig/reporting-init:6.0.0.12431
quay.io/sysdig/certman:6.0.0.12431
quay.io/sysdig/sysdig-meerkat-collector:6.0.0.12431
quay.io/sysdig/policies:6.0.0.12431
quay.io/sysdig/profiling-worker:6.0.0.12431
quay.io/sysdig/cloudsec-api:6.0.0.12431
quay.io/sysdig/compliance-api:6.0.0.12431
quay.io/sysdig/elasticsearch-tools:0.0.35
quay.io/sysdig/events-forwarder:6.0.0.12431
quay.io/sysdig/ingress-default-backend:1.5
docker.io/sysdig/falco_rules_installer:latest
quay.io/sysdig/events-api:6.0.0.12431
quay.io/sysdig/events-forwarder-api:6.0.0.12431
quay.io/sysdig/promqlator:0.99.0-master.2022-10-03T12-41-14Z.2f800e101b
quay.io/sysdig/ui-secure-nginx:6.0.0.12431
quay.io/sysdig/reporting-worker:6.0.0.12431
quay.io/sysdig/scanning-ve-janitor:6.0.0.12431
quay.io/sysdig/rapid-response-janitor:6.0.0.12431
quay.io/sysdig/compliance-worker:6.0.0.12431
quay.io/sysdig/events-janitor:6.0.0.12431
quay.io/sysdig/events-dispatcher:6.0.0.12431
quay.io/sysdig/haproxy-ingress:1.1.5-v0.10
quay.io/sysdig/sysdig-meerkat-api:6.0.0.12431
quay.io/sysdig/metadata-service-operator:1.0.1.23
quay.io/sysdig/netsec:6.0.0.12431
quay.io/sysdig/nats-exporter:0.9.0.2
quay.io/sysdig/secure-prometheus:2.17.2
quay.io/sysdig/opensearch-1:0.0.16
quay.io/sysdig/events-gatherer:6.0.0.12431
quay.io/sysdig/reporting-api:6.0.0.12431
quay.io/sysdig/promchap:0.99.0-master.2022-11-18T13-46-40Z.d6b3d10f83
quay.io/sysdig/redis-6:1.0.1
quay.io/sysdig/ui-admin-nginx:6.0.0.12431
quay.io/sysdig/admission-controller-api:6.0.0.12431
quay.io/sysdig/scanning:6.0.0.12431
quay.io/sysdig/sysdig-alert-notifier:6.0.0.12431
quay.io/sysdig/cassandra:0.0.36
quay.io/sysdig/metadata-service-server:1.10.63
quay.io/sysdig/rapid-response-connector:6.0.0.12431
quay.io/sysdig/secure-todo-api:6.0.0.12431
quay.io/sysdig/api-docs:6.0.0.12431
quay.io/sysdig/cloudsec-worker:6.0.0.12431
quay.io/sysdig/sysdig-collector:6.0.0.12431
quay.io/sysdig/events-ingestion:6.0.0.12431
quay.io/sysdig/rsyslog:8.2102.0.4
quay.io/sysdig/sysdig-meerkat-aggregator:6.0.0.12431
quay.io/sysdig/secure-todo-janitor:6.0.0.12431
quay.io/sysdig/sysdig-alert-manager:6.0.0.12431
quay.io/sysdig/redis-exporter-1:1.0.9
quay.io/sysdig/ui-inspect-nginx:6.0.0.12431
```

## Command: `diff`

Will perform a diff between the platform objects in a running k8s cluster, and the generated manifests based on some values.

`--write-diff`

- Will write the diff on the filesystem organized in subfolders, rather than printing it to the stdout.

`--out-diff-dir`

- Allows you to specify a custom path for the diff files being written on the filesystem. Will be used only if also `--write-diff` is provided. If not set will use a temporary directory.

`--cleanup`

- If set, will attempt to automatically delete any generated diff files on the filesystem if the directory used to store the diff files already exists. Requires both `--write-diff` and `--out-diff-dir` to be set.

`--secure`

- applies some filters to the produced diff in order to avoid printing sensitive informations. This is useful if you need to share diffs to user who shouldn't have access to credentials.

`--summary`

- Only prints a summary of the diff errors.

Diff command also has options inherited from the generate command options. See **generate** command section.

### Sub-Command: secure-diff [DEPRECATED]

Performs a diff not showing sensitive information.
This subcommand is DEPRECATED and will be removed starting from version 6.7.0, you can have the same effect with the diff command and the flag `--secure`.

## Command: `generate`

`--manifest-directory`

- Set the location where the installer will write the genearted manifests.

`--skip-generate`

- Skips generating Kubernetes manifests and attempts to diff whatever is in the manifests directory. Manifest directory can be specified using `--manifest-directory <dir>` flag.

`--skip-import`

- Skips the import phase, which would try to import values from a running cluster.

`--skip-validation`

- Skips validation checks.

`--ignore-kubeconfig-errors`

- This will ignore all errors from trying to parse kubeconfig file.

`--preserve-templates`

- Preserve directory installer templates are extracted to, this should only be used for debugging purposes

`--k8s-server-version`

- Sets the `kubernetesServerVersion` within values.

`--helm-install`

- The installer will extract the necessary files for an installation using the `helm` command only. By default it will create a directory `helm-install` in the directory where the installer is being executed. Content of the directory:

- `values.hi.yaml`: the complete values generated by the `installer`
- `values.hi.nats.yaml` and `values.hi.nats.global.yaml`: values for the rendering of NATSJS
- `charts`: the Helm charts that make up the Sysdig onprem stack

`--helm-install-out-dir`

- To use a custom directory to output the files generated by `--helm-install` instead of the default.

## Command: `list-resources`

Will list all the required resources and limits for a planned deployment, based on the the defaults, provided values, and overlays.
This command expects to have a `generated` folder. If one doesn't exist, it can be created within the scope of this command, using the `--generate-manifests` flag.

`--generate-manifests`

- Generate Kubernetes manifests before generating the list of resources. Defaults to `false`.

`--node-count`

- Number of nodes in the target cluster. This impacts the resource calculation, because DaemonSets get deployed on every (tolerated) node in the cluster. Defaults to `1`.
Loading