Skip to content
This repository has been archived by the owner on Aug 22, 2024. It is now read-only.

Configuration settings reference

Jump to bottom
Alexey Lesovsky edited this page Jul 3, 2021 · 14 revisions

Configuration settings reference.

IMPORTANT NOTES:

  • Support of environment variables merged into master, but not released yet. It is planned for 0.6.0 release.

pgSCV configuration settings can be defined using YAML configuration file or environment variables.

  • Location of the configuration file could be specified at startup using --config-file option. It is recommended to store configuration in /etc/pgscv.yaml. When configuration file is specified, all environment variables are ignored, even if corresponding settings are not specified in config file.
  • When configuration file is not specified, pgSCV is looking for configuration settings in environment variables.
  • If no environment variables found, pgSCV runs with default settings.

YAML settings:

  • listen_address: network address and port where the application should listen on. Environment variable: PGSCV_LISTEN_ADDRESS. Default value: 127.0.0.1:9890.

  • authentication: enable basic authentication and TLS encryption. Disabled by default.

    • username: authentication username. Environment variable: PGSCV_AUTH_USERNAME. Default value: "" (disabled).
    • password: authentication password. Environment variable: PGSCV_AUTH_PASSWORD. Default value: "" (disabled).
    • keyfile: path to file with the secret key in the PEM format. Environment variable: PGSCV_AUTH_KEYFILE. Default value: "" (disabled).
    • certfile: path to file with the certificate in the PEM format. Environment variable: PGSCV_AUTH_CERTFILE. Default value: "" (disabled).

  • autoupdate: controls tracking new versions and auto-update procedure. Environment variable: PGSCV_AUTOUPDATE. Default value: "off" (disabled). Valid values are:

    • off - auto-update is disabled
    • devel - auto-update allowed for release candidates (not recommended)
    • stable - auto-update allowed only for stable releases (recommended)

  • no_track_mode: controls tracking of sensitive information, such as query texts. Environment variable: PGSCV_NO_TRACK_MODE. Default value: false (disabled).

  • send_metrics_url: URL of the remote service where collected metrics should be sent. Environment variable: PGSCV_SEND_METRICS_URL. Default value: "" (disabled).

  • api_key: API key for accessing to Weaponry service. Environment variable: PGSCV_API_KEY. Default value: "". only for Weaponry users.

  • services: dictionary of services to which pgSCV should connect and monitor. Environment variable: see here for details. Defining services automatically disables auto-discovery. Empty by default, looking for services using auto-discovery. See example for details.

    • service_type: type of the service, must be one of postgres, pgbouncer.
    • conninfo: connection string or DSN for connecting to service.

  • defaults: default requisites for connecting to auto-discovered services (no environment variables supported):

    • postgres_dbname: database name for connecting to services which are identified as Postgres. Default value: "postgres".
    • postgres_username: username for connecting to services which are identified as Postgres. Default value: "pgscv".
    • postgres_password: password for connecting to services which are identified as Postgres. Default value: "".
    • pgbouncer_dbname: database name for connecting to services which are identified as Pgbouncer. Default value: "pgbouncer".
    • pgbouncer_username: username for connecting to services which are identified as Pgbouncer. Default value: "pgscv".
    • pgbouncer_password: password for connecting to services which are identified as Pgbouncer. Default value: "".

  • databases: regular expression which defines databases from which builtin metrics should be collected. Environment variable: PGSCV_DATABASES. This option is not propagated on user-defined metrics specified in collector settings.

  • filters: per-collector filtering rules for including or excluding specific collector objects. Exclude rules has higher priority over include rules. No environment variables supported.

    • collector_name/label_name: exact name of collector.
      • include: regexp string for including objects.
      • exclude: regexp string for excluding objects. Has higher priority over include.

    Supported filters. Currently, only the following list of filters are available:

    • diskstats/device: exclude: ^(ram|loop|fd|sr|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$
    • netdev/device: exclude: docker|virbr
    • filesystem/fstype: include: ^(ext3|ext4|xfs|btrfs)$

  • disable_collectors: list of collectors which should be disabled. Environment variable: PGSCV_DISABLE_COLLECTORS - comma separated list of collectors. Default value: [] (all collectors are enabled).

  • collectors: per-collectors settings. Each entry is a name of collector. Main purpose of collectors is configuring user-defined metrics. No environment variables supported.

    • subsystems: per-subsystems settings. Each entry defines exact subsystem.
      • databases: defines regular expression for databases where user-defined metrics of this subsystem should be collected
      • query: defines query used for getting metrics data
      • metrics: list of user metrics should be collected from queried data
        • name: metric name
        • usage: type of metric: GAUGE or COUNTER
        • value: name of column with metric value
        • labeled_values: name of label and names of columns with metric values
        • labels: list of columns names with label values
        • description: metric description

YAML Configuration file example

Complete YAML configuration file example:

listen_address: 127.0.0.1:9890
authentication:
  username: monitoring
  password: supersecretpassword
  keyfile: /etc/ssl/private/ssl-cert-snakeoil.key
  certfile: /etc/ssl/certs/ssl-cert-snakeoil.pem
autoupdate: off
no_track_mode: false
send_metrics_url: https://push.weaponry.io
api_key: 12345678-abcd-1234-abcd-123456789012
services:
  "postgres:5432":
    service_type: "postgres"
    conninfo: "postgres://[email protected]:5432/postgres"
  "pgbouncer:6432": 
    service_type: "pgbouncer"
    conninfo: "postgres://[email protected]:6432/pgbouncer"
defaults:
    postgres_username: "monitoring"
    postgres_password: "supersecret"
    pgbouncer_username: "monitoring"
    pgbouncer_password: "supersecret"
databases: "^(a-z_)+_prod$"
collectors:
  postgres/custom:
    filters:
      database:
        exclude: "^(test|staging)_.+$"
    subsystems:
      activity:
        query: "select datname as database,xact_commit,xact_rollback,blks_read as read,blks_write as write from pg_stat_database"
        metrics:
          - name: xact_commit_total
            usage: COUNTER
            labels:
              - database
            value: xact_commit
            description: "Total number of transactions had been committed by each database."
          - name: "blocks_total"
            usage: COUNTER
            labels:
              - database
            labeled_values:
              access: [ "read", "write" ]
            description: "Total number of blocks had been processed by each database."
      bgwriter:
        query: "select maxwritten_clean from pg_stat_bgwriter"
        metrics:
          - name: "maxwritten_clean_total"
            usage: COUNTER
            value: maxwritten_clean
            description: "Total number of times bgwriter have to be paused."

Configuring connection settings using environment variables.

Connecting to network services (Postgres or Pgbouncer) is possible without YAML configuration, using environment variables only:

  • POSTGRES_DSN is used for defining connection string used for connecting to Postgres services.
  • DATABASE_DSN is used for defining connection string used for connecting to Postgres services. This is an alias for POSTGRES_DSN, which allows to use an environment variable shared across several applications.
  • PGBOUNCER_DSN is used for defining connection string used for connecting to Pgbouncer services.
  • PATRONI_URL is used for defining base connection URI used for connecting to Patroni services.

It is possible to use expanded syntax and specify more than one connection strings:

  • POSTGRES_DSN<SERVICE_ID>
  • POSTGRES_DSN_<SERVICE_ID>
  • DATABASE_DSN<SERVICE_ID>
  • DATABASE_DSN_<SERVICE_ID>
  • PGBOUNCER_DSN<SERVICE_ID>
  • PGBOUNCER_DSN_<SERVICE_ID>

Where <SERVICE_ID> is alpha-numeric value used as value for service_id label of exposed metrics.

It is recommended to use common env. variables syntax: uppercase letters, digits, and the '_' (underscore). For more fine-grained service_id naming use YAML configuration.

Important note: Make sure values used as a SERVICE_ID are unique across all used environment variables.

Connection string could be specified in libpq supported formats: as a keyword/value, or a connection URI string.

Examples:

$ DATABASE_DSN="host=127.0.0.1 port=5432 user=pgscv dbname=postgres" pgscv

$ POSTGRES_DSN="postgresql://pgscv:[email protected]:5432/postgres" pgscv

$ PGBOUNCER_DSN="postgresql://pgscv:[email protected]:6432/pgbouncer" pgscv

$ POSTGRES_DSN_SRV1="postgresql://pgscv:[email protected]:5432/postgres" POSTGRES_DSN_SRV2="postgresql://pgscv:[email protected]:5432/postgres" pgscv
Clone this wiki locally