adding support for env variable expansion in log config. see #337

Signed-off-by: Syed Nihal <[email protected]>

README.md

@@ -93,3 +93,32 @@ If the filename ends with `.url` suffix, the content will be processed as a URL
 | `LOG_FORMAT`               | Set a log format. (JSON or LOGFMT)                                                                                                                                                                                                                                                                                                  | false    | `JSON`                                    | string  |
 | `LOG_TZ`                   | Set the log timezone. (LOCAL or UTC)                                                                                                                                                                                                                                                                                                | false    | `LOCAL`                                   | string  |
 | `LOG_CONFIG`               | Log configuration file path. If not configured, uses the default log config for backward compatibility support. When not configured `LOG_LEVEL, LOG_FORMAT and LOG_TZ` would be used. Refer to [Python logging](https://docs.python.org/3/library/logging.config.html) for log configuration. For sample configuration file  refer to file examples/example_logconfig.yaml | false    | -                                         | string  |
+
+# Environment variable expansion in LOG_CONFIG
+Kiwigrid k8s-sidecar supports expansion fo environment variables expansion in the log config.
+This can be done by wrapping the name of environment variable in the regex placeholder `$(<env_var_name>)` in the log config.
+At the startup, the k8s container will look for the regex wrapper and replace all the matched occurrences with the content of the environment variables.
+
+For instance the below snippet from the log config,
+
+```commandline
+version: 1
+disable_existing_loggers: false
+
+root:
+    level: $(LV_DBG)
+    handlers: [console]
+...
+```
+
+would be read as below, replacing the content of `$(LV_DBG)` with the value of environment variable `LV_DBG`.
+
+```commandline
+version: 1
+disable_existing_loggers: false
+
+root:
+    level: DEBUG
+    handlers: [console]
+...
+```

src/logger.py

@@ -1,6 +1,7 @@
 import logging
 import os
 import sys
+import re
 import yaml
 from datetime import datetime
 from typing import Optional
@@ -107,11 +108,27 @@ def add_fields(self, log_record, record, message_dict):
     }
 }
 
+def expand_env(data):
+    placeholder_pattern = r'\$\((.*?)\)'
+
+    def replace_placeholder(s):
+        env = s.group(1)
+        env_value = os.getenv(env)
+        if env_value is None:
+            print(f'unable to expand environment variable {env} in LOG_CONFIG. reason: env variable not set')
+            sys.exit(1)
+        else:
+            return env_value
+
+    processed_data = re.sub(placeholder_pattern, replace_placeholder, data)
+    return yaml.load(processed_data, Loader=yaml.FullLoader)
+
 def get_log_config():
     if log_conf_file != "" :
         try:
             with open(log_conf_file, 'r') as stream:
-                config = yaml.load(stream, Loader=yaml.FullLoader)
+                data = stream.read()
+                config = expand_env(data)
             return config
         except FileNotFoundError:
             msg = "Config file: "+ log_conf_file + " Not Found"