Merge branch 'master' into krr-and-action-pubsub

.gitignore

@@ -20,3 +20,4 @@ generated_values*.yaml
 skaffold.dev.yaml
 tests/last_used_tag.txt
 pytest.ini
+gen-config-test

docs/configuration/configuring-sinks.rst

@@ -18,7 +18,8 @@ Sinks are defined in Robusta's Helm chart, using the ``sinksConfig`` value:
         name: my_teams_sink           # arbitrary name
         webhook_url: <placeholder>    # a sink-specific parameter
         stop: false                   # optional (see `Routing Alerts to only one Sink`)
-        match: {}                     # optional routing rules (see below)
+        scope: {}                     # optional routing rules
+        match: {}                     # optional routing rules (deprecated; see below)
         default: true                 # optional (see below)
 
 To add a sink, update ``sinksConfig`` according to the instructions in :ref:`Sinks Reference`. Then do a :ref:`Helm Upgrade <Simple Upgrade>`.
@@ -28,7 +29,7 @@ Integrate as many sinks as you like.
 .. _sink-matchers:
 
 
-Routing Alerts to only one Sink
+Routing Alerts to Only One Sink
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 By default, alerts are sent to all sinks that matches the alerts.
@@ -44,17 +45,129 @@ The sinks evaluation order, is the order defined in ``generated_values.yaml``.
         name: production_sink
         slack_channel: production-notifications
         api_key: secret-key
-        match:
-          namespace: production
+        scope:
+          include:
+            - namespace: production
         stop: true
 
 
-Routing Alerts to Specific Sinks
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Routing Alerts To Specific Sinks
+***************************************
+
+Define which messages a sink accepts using ``scope``.
+
+For example, **Slack**  can be integrated to receive high-severity messages in a specific namespace. Other messages will not be sent to this **Slack** sink.
+
+.. code-block:: yaml
+
+    sinksConfig:
+    - slack_sink:
+        name: test_sink
+        slack_channel: test-notifications
+        api_key: secret-key
+        scope:
+          include: # more options available - see below
+            - namespace: [prod]
+              severity: HIGH
+
+Each attribute expression used in the ``scope`` specification can be 1 item, or a list, where each is either a regex or an exact match
+
+``Scope`` allows specifying a set of ``include`` and ``exclude`` sections:
+
+.. code-block:: yaml
+
+    sinksConfig:
+    - slack_sink:
+        name: prod_slack_sink
+        slack_channel: prod-notifications
+        api_key: secret-key
+        scope:
+        # AND between namespace and labels, but OR within each selector
+          include:
+            - namespace: default
+              labels: "instance=1,foo!=x.*"
+            - namespace: bla
+              name:
+              - foo
+              - qux
+          exclude:
+            - type: ISSUE
+              title: .*crash.*
+            - name: bar[a-z]*
+
+
+In order for a message to be sent to a ``Sink``, it must match **one of** the ``include`` sections, and **must not** match **all** the ``exclude`` sections.
+
+When multiple attributes conditions are present, all must be satisfied.
+
+The following attributes can be included in an ``include``/``excluded`` block:
+
+- ``title``: e.g. ``Crashing pod foo in namespace default``
+- ``name`` : the Kubernetes object name
+- ``namespace``: the Kubernetes object namespace
+- ``node`` : the Kubernetes node name
+- ``severity``: one of ``INFO``, ``LOW``, ``MEDIUM``, ``HIGH``
+- ``type``: one of ``ISSUE``, ``CONF_CHANGE``, ``HEALTH_CHECK``, ``REPORT``
+- ``kind``: one of ``deployment``, ``node``, ``pod``, ``job``, ``daemonset``
+- ``source``: one of ``NONE``, ``KUBERNETES_API_SERVER``, ``PROMETHEUS``, ``MANUAL``, ``CALLBACK``
+- ``identifier``: e.g. ``report_crash_loop``
+- ``labels``: A comma separated list of ``key=val`` e.g. ``foo=bar,instance=123``
+- ``annotations``: A comma separated list of ``key=val`` e.g. ``app.kubernetes.io/name=prometheus``
+
+.. note::
+
+    ``labels`` and ``annotations`` are both the Kubernetes resource labels and annotations
+    (e.g. pod labels) and the Prometheus alert labels and annotations. If both contains the
+    same label/annotation, the value from the Prometheus alert is preferred.
+
+
+.. details:: How do I find the ``identifier`` value to use in a match block? (deprecated)
+
+    For Prometheus alerts, it's always the alert name.
+
+    .. TODO: update after we finish our improvements here:
+    .. For builtin APIServer alerts, it can vary, but common values are ``report_crash_loop``, ``image_pull_backoff_reporter``, ``ConfigurationChange/KubernetesResource/Change``, and ``job_failure``.
+
+    For custom playbooks, it's the value you set in :ref:`create_finding<create_finding>` under ``aggregation_key``.
+
+    Ask us in Slack if you need help.
+
+By default, every message is sent to every matching sink. To change this behaviour, you can mark a sink as :ref:`non-default <Non-default sinks>`.
+
+The top-level mechanism works as follows:
+
+#. If the notification is **excluded** by any of the sink ``scope`` excludes - drop it
+#. If the notification is **included** by any of the sink ``scope`` includes - accept it
+#. If the notification is **included** by any of the sink ``matchers`` - accept it (Deprecated)
+
+Any of (but not both) of the ``include`` and ``exclude`` may be left undefined or empty.
+An undefined/empty ``include`` section will effectively allow all alerts, and an
+undefined/empty ``exclude`` section will not exclude anything.
+
+Inside the ``include`` and ``exclude`` section, at the topmost level, the consecutive
+items act with the OR logic, meaning that it's enough to match a single item in the
+list in order to allow/reject a message. The same applies to the items listed under
+each attribute name.
+
+Within a specific ``labels`` or ``annotations`` expression, the logic is ``AND``
+
+.. code-block:: yaml
+
+    ....
+        scope:
+          include:
+            - labels: "instance=1,foo=x.*"
+    .....
+
+The above requires that the ``instance`` will have a value of ``1`` **AND** the ``foo`` label values starts with ``x``
+
+Match Section (Deprecated)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Define which messages a sink accepts using *matchers*.
 
-For example, Slack can be integrated to receive high-severity messages in a specific namespace. Other messages will not be sent to Slack.
+For example, Slack can be integrated to receive high-severity messages in a specific
+namespace. Other messages will not be sent to Slack.
 
 .. code-block:: yaml
 
@@ -86,11 +199,12 @@ The following attributes can be included in a *match* block:
 
 .. note::
 
-    ``labels`` and ``annotations`` are both the Kubernetes resource labels and annotations (e.g. pod labels) and the Prometheus alert labels and annotations.
-    If both contains the same label/annotation, the value from the Prometheus alert is preferred.
+    ``labels`` and ``annotations`` are both the Kubernetes resource labels and annotations
+    (e.g. pod labels) and the Prometheus alert labels and annotations. If both contains the
+    same label/annotation, the value from the Prometheus alert is preferred.
 
 
-.. details:: How do I find the ``identifier`` value to use in a match block?
+.. details:: How do I find the ``identifier`` value to use in a match block? (deprecated)
 
     For Prometheus alerts, it's always the alert name.
 
@@ -103,8 +217,8 @@ The following attributes can be included in a *match* block:
 
 By default, every message is sent to every matching sink. To change this behaviour, you can mark a sink as :ref:`non-default <Non-default sinks>`.
 
-Matches Can Be Lists Or Regexes
-********************************************
+Match Section (Deprecated): Matches Can Be Lists or Regexes
+***********************************************************
 
 *match* rules support both regular expressions and lists of exact values:
 
@@ -122,8 +236,8 @@ Matches Can Be Lists Or Regexes
 
 Regular expressions must be in `Python re module format <https://docs.python.org/3/library/re.html#regular-expression-syntax>`_, as passed to `re.match <https://docs.python.org/3/library/re.html#re.match>`_.
 
-Matching Labels and Annotations
-********************************************
+Match Section (Deprecated): Matching Labels and Annotations
+***********************************************************
 
 Special syntax is used for matching labels and annotations:
 
@@ -139,8 +253,8 @@ Special syntax is used for matching labels and annotations:
 
 The syntax is similar to Kubernetes selectors, but only `=` conditions are allowed, not `!=`
 
-Or Between Matches
-********************************************
+Match Section (Deprecated): Or Between Matches
+**********************************************
 
 You can use `Or` between *match* rules:
 

docs/playbook-reference/defining-playbooks/external-playbook-repositories.rst

@@ -14,15 +14,58 @@ use in :ref:`customPlaybooks<customPlaybooks>`.
 
 External actions are loaded using the ``playbookRepos`` Helm value, with either HTTPs or SSH.
 
-If your repository is not in ``github.com`` or ``bitbucket.org`` (default verified domains), please add your repository domains:
+If you are going to be using an external repository via HTTPS, you just need to configure
+correct read access credentials (see below). When connecting via SSH, however, there is an
+additional requirement to verify the remote host's identity on the client side, as SSH
+generally does not provide any method of doing that automatically (in contrast with HTTPS,
+which relies on the well established cryptographic infrastructure of certificates).
+
+In order to streamline the process of SSH host key verification, Robusta ships with verified
+host keys for the following popular Git providers:
+
+* github.com
+* gitlab.com
+* bitbucket.org
+* ssh.dev.azure.com
+
+If you are using a Git service outside of that list, you should add its SSH host keys in Robusta
+configuration. This is done via the `CUSTOM_SSH_HOST_KEYS` environment variable with the list
+of keys separated by newlines:
 
 .. code-block:: yaml
 
     runner:
       additional_env_vars:
-        - name: GIT_REPOS_VERIFIED_HOSTS
-          value: "ssh.dev.azure.com gitlab.com"
+        - name: CUSTOM_SSH_HOST_KEYS
+        # codeberg.org host keys
+        - value: |
+            |1|TVOSCWl9+tXzKniecqFzaidE+yA=|XgOrtH2kjzERBPrbC9aGbaisnDE= ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8hZi7K1/2E2uBX8gwPRJAHvRAob+3Sn+y2hxiEhN0buv1igjYFTgFO2qQD8vLfU/HT/P/rqvEeTvaDfY1y/vcvQ8+YuUYyTwE2UaVU5aJv89y6PEZBYycaJCPdGIfZlLMmjilh/Sk8IWSEK6dQr+g686lu5cSWrFW60ixWpHpEVB26eRWin3lKYWSQGMwwKv4LwmW3ouqqs4Z4vsqRFqXJ/eCi3yhpT+nOjljXvZKiYTpYajqUC48IHAxTWugrKe1vXWOPxVXXMQEPsaIRc2hpK+v1LmfB7GnEGvF1UAKnEZbUuiD9PBEeD5a1MZQIzcoPWCrTxipEpuXQ5Tni4mN
+            |1|Zht5NJQx7c6F9fzemGK15ewk4lE=|D8ZMvKG5X9HEAUqWZaGJOwpBb7s= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL2pDxWr18SoiDJCGZ5LmxPygTlPu+cCKSkpqkvCyQzl5xmIMeKNdfdBpfbCGDPoZQghePzFZkKJNR/v9Win3Sc=
+            |1|KMrl/f5rYsb8KkF7rHCwBuo49Do=|wkmvtUU1nQTyj+ZNyVqZlO0oP5o= ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIVIC02vnjFyL+I4RHfvIGNtOgJMe769VTF1VR4EB3ZB
+
+Another option to automate host key verification is the `GIT_REPOS_VERIFIED_HOSTS` environment
+variable.
+
+.. warning::
+
+    **DANGER ZONE**
 
+    Using the `GIT_REPOS_VERIFIED_HOSTS` variable is generally not recommended due to
+    security issues. Each host added this way will be automatically trusted *without*
+    an actual host key verification, potentially allowing man-in-the-middle attacks with
+    catastrophic implications. For more information, see
+    `here <https://www.ssh.com/academy/attack/man-in-the-middle>`_.
+
+    Please make sure you know what you are doing before using this functionality.
+
+An example of using that configuration option:
+
+.. code-block:: yaml
+
+    runner:
+      additional_env_vars:
+        - name: GIT_REPOS_VERIFIED_HOSTS
+          value: "ssh.yourhost.com ssh.anotherhost.com"
 
 Loading Actions from Public Git Repo
 ------------------------------------------

docs/setup-robusta/configuration-secrets.rst

@@ -16,7 +16,6 @@ Robusta can pull values from Kubernetes secrets for:
 * Sink Configuration
 * Global Config
 * Action Parameters
-* Robusta RSA keys
 
 To do so, first define an environment variable based on a Kubernetes secret. Add to Robusta's Helm values:
 
@@ -41,5 +40,3 @@ Then reference that environment variable in other Helm values using the special
 
 Finally, make sure the Kubernetes secret actually exists. In this example, create a Secret named ``my-robusta-secrets``
 with a ``secret_grafana_key`` value inside.
-
-For Robusta RSA keys, you can define the ``existingSecret`` parameter. The secret must have the `pub` and `prv` keys.

docs/setup-robusta/gitops/argocd.rst

@@ -39,9 +39,6 @@ Example ``generated_values.yaml``:
     enablePlatformPlaybooks: true
     runner:
       sendAdditionalTelemetry: true
-    rsa:
-      prv: xxxxxx
-      pub: xxxxxx
 
 .. Options
 .. ^^^^^^^^^^^^^

docs/setup-robusta/gitops/flux.rst

@@ -40,9 +40,6 @@ Example ``generated_values.yaml``:
     enablePlatformPlaybooks: true
     runner:
       sendAdditionalTelemetry: true
-    rsa:
-      prv: xxxxxx
-      pub: xxxxxx
 
 .. admonition:: Secrets handling
     :class: note

helm/robusta/templates/NOTES.txt

@@ -5,12 +5,16 @@ This data is extremely limited and contains only general metadata to help us und
 If you are willing to share additional data, please do so! It really help us improve Robusta.
 
 You can set sendAdditionalTelemetry: true as a Helm value to send exception reports and additional data.
-This is disabled by default. 
+This is disabled by default.
 
 To opt-out of telemetry entirely, set a ENABLE_TELEMETRY=false environment variable on the robusta-runner deployment.
 
 {{- range .Values.sinksConfig }}
 {{if .robusta_sink}}
 Visit the web UI at: https://platform.robusta.dev/
 {{- end }}
-{{- end }}
+{{- end }}
+
+{{- if .Values.rsa }}
+NOTICE: RSA is no longer used by default and can be removed from the values file
+{{- end }}

helm/robusta/values.yaml

@@ -459,7 +459,7 @@ platformPlaybooks:
   sinks:
     - "robusta_ui_sink"
 
-- name: "WeeklyPopeyScan"
+- name: "WeeklyPopeyeScan"
   triggers:
   - on_schedule:
       fixed_delay_repeat:
@@ -692,11 +692,6 @@ kube-prometheus-stack:
       limits:
         memory: 256Mi
 
-rsa: ~
-  # @param existingSecret Name of existing secret containing the rsa keys
-  # NOTE: Must contain the keys `pub` and `prv`
-  # existingSecret: my-robusta-rsa-keys
-
 # custom parameters for OpenShift clusters
 openshift:
   enabled: false

playbooks/robusta_playbooks/event_enrichments.py

@@ -245,8 +245,10 @@ def deployment_events_enricher(event: DeploymentEvent, params: ExtendedEventEnri
                     event.add_enrichment([events_table_block], {SlackAnnotations.ATTACHMENT: True},
                                          enrichment_type=EnrichmentType.k8s_events, title="Deployment Events")
     else:
-        pods = list_pods_using_selector(dep.metadata.namespace, dep.spec.selector, "status.phase=Running")
-        event.add_enrichment([MarkdownBlock(f"*Replicas: Desired ({dep.spec.replicas}) --> Running ({len(pods)})*")])
+        available_replicas = dep.status.availableReplicas if dep.status.availableReplicas else 0
+        event.add_enrichment(
+            [MarkdownBlock(f"*Replicas: Desired ({dep.spec.replicas}) --> Running ({available_replicas})*")])
+
         events_table_block = get_resource_events_table(
             "*Deployment events:*",
             dep.kind,