Docker Secrets For Images That Don't Support Them In Production (Docker Swarm)

[bbkysf]

Hello,

first of all everything mentioned here is for docker swarm production environment therefore I would really appreciate to point out my mistakes if you see any bad practices and also I'm seeking for production grade solutions for the problem mentioned in the title.

As we all know using docker secrets for official images that support them is pretty straight forward in your stack yaml file define your secrets and use them in environment variables with a "_FILE" at the end of it. almost every famous database is using this convention. So as far as I know It won't be visible to anyone except PID1

version: '3.8'

services:
  db:
    image: dbimage
    environment:
      - DB_PASSWORD_FILE=/run/secrets/db_pass
      
#  ... other settings ...

secrets:
  db_pass:
    external: true

Now there are lots of images out there that don't support this convention. two of them that I encountered recently is Docker official registry and Drone CI from Harness.

How official images support docker secrets

I studied couple of official database Dockerfiles (MySQL and PostgreSQL) to see how they support secrets and generally its a script like this running in their entrypoint script file :

#!/usr/bin/env bash

# ... some other code ...

# usage: file_env VAR [DEFAULT]
#    ie: file_env 'XYZ_DB_PASSWORD' 'example'
# (will allow for "$XYZ_DB_PASSWORD_FILE" to fill in the value of
#  "$XYZ_DB_PASSWORD" from a file, especially for Docker's secrets feature)
file_env() {
	local var="$1"
	local fileVar="${var}_FILE"
	local def="${2:-}"
	if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
		echo >&2 "error: both $var and $fileVar are set (but are exclusive)"
		exit 1
	fi
	local val="$def"
	if [ "${!var:-}" ]; then
		val="${!var}"
	elif [ "${!fileVar:-}" ]; then
		val="$(< "${!fileVar}")"
	fi
	export "$var"="$val"
	unset "$fileVar"
}

# ... some other code ...

file_env "DB_PASSWORD" "a_default_value"

# ... some other code ...

Docker Secrets for official registry image

for Docker private registry image I created another image using registry:2 as the base image and inserted this script before it's default entrypoint script and then replaced it to support docker secrets for aws s3 credentials to setup s3 storage for the registry.

entrypoint.sh

#!/usr/bin/env bash

# usage: file_env VAR [DEFAULT]
#    ie: file_env 'XYZ_DB_PASSWORD' 'example'
# (will allow for "$XYZ_DB_PASSWORD_FILE" to fill in the value of
#  "$XYZ_DB_PASSWORD" from a file, especially for Docker's secrets feature)
file_env() {
	local var="$1"
	local fileVar="${var}_FILE"
	local def="${2:-}"
	if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
		echo >&2 "error: both $var and $fileVar are set (but are exclusive)"
		exit 1
	fi
	local val="$def"
	if [ "${!var:-}" ]; then
		val="${!var}"
	elif [ "${!fileVar:-}" ]; then
		val="$(< "${!fileVar}")"
	fi
	export "$var"="$val"
	unset "$fileVar"
}

file_env 'REGISTRY_STORAGE_S3_ACCESSKEY' 

file_env 'REGISTRY_STORAGE_S3_SECRETKEY' 


# default registry:2 entrypoint script file:

set -e

case "$1" in
    *.yaml|*.yml) set -- registry serve "$@" ;;
    serve|garbage-collect|help|-*) set -- registry "$@" ;;
esac

exec "$@"

Dockerfile

FROM registry:2

RUN apk add --no-cache bash

COPY entrypoint.sh /entrypoint.sh

also it is worth mentioning that I installed bash to be able to run the bash script I extracted from database images.

Drone CI

I couldn't figure out how this image works and where is it's entrypoint script to replace it. I tried to build an image with drone image as its base and run the script from a separate file before the original entrypoint script but it just doesn't work.

non-working Dockerfile

FROM drone/drone:2

RUN apk add --no-cache bash

COPY file_env.sh /bin/file_env.sh

RUN chmod +x /bin/file_env.sh

ENTRYPOINT [ "sh", "-c","/bin/file_env.sh /bin/drone-server"]

# It doesn't work

so I come up with another solution that works but it is even more hacky and spits out credentials to container logs on start-up which I believe is a bad thing. The solution was to define another entrypoint for it in the stack yaml file.

version: '3.8'

services:
  drone:
    image: drone/drone:2
    entrypoint:
      [
        'sh',
        '-xc',
        'DRONE_SECRET=`cat /run/secrets/drone_secret`  /bin/drone-server',
      ]
    secrets:
      - drone_secret

# ... other settings ...

secrets:
  drone_secret:
    external: true

P.S. Drone CI developers are not interested in supporting docker swarm so that's not an option unfortunately.

So what is the correct way

All of this feels really hacky to me. I was wondering if there is a better way to handle this situation.
if not, I would like to hear any suggestions for the second one where using cat in entrypoint script is printing credentials into container logs.

Thanks a lot!

[BretFisher]

I would need to see your full Dockerfile and entrypoint script for Drone to see why it's not working. I looked up the drone dockerfile itself but it didn't clue me into anything useful.

Make sure in your entrypoint script you pass execution, which is how a "script plus command" works in entrypoint+cmd. exec "$@" should be the last line in the shell script.

See my example of how that works here: https://youtu.be/C1GE07UEFDo

Also, I've used Drone in the past, and much prefer my CI to be the one built into my code storage, which is way less work to maintain, with many more features. See my recent video on my opinions of CI/CD solutions and why I prefer GitHub Actions.

[bbkysf]

Thanks for your reply,
here is both Dockerfile and script file for Drone

Dockerfile

FROM drone/drone:2

RUN apk add --no-cache bash

COPY file_env.sh /bin/file_env.sh

RUN chmod +x /bin/file_env.sh

ENTRYPOINT [ "sh", "-c","/bin/file_env.sh /bin/drone-server"]

Script file

#!/usr/bin/env bash

# usage: file_env VAR [DEFAULT]
#    ie: file_env 'XYZ_DB_PASSWORD' 'example'
# (will allow for "$XYZ_DB_PASSWORD_FILE" to fill in the value of
#  "$XYZ_DB_PASSWORD" from a file, especially for Docker's secrets feature)
file_env() {
	local var="$1"
	local fileVar="${var}_FILE"
	local def="${2:-}"
	if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
		echo >&2 "error: both $var and $fileVar are set (but are exclusive)"
		exit 1
	fi
	local val="$def"
	if [ "${!var:-}" ]; then
		val="${!var}"
	elif [ "${!fileVar:-}" ]; then
		val="$(< "${!fileVar}")"
	fi
	export "$var"="$val"
	unset "$fileVar"
}

file_env 'DRONE_GITHUB_CLIENT_ID' 
file_env 'DRONE_GITHUB_CLIENT_SECRET' 
file_env 'DRONE_RPC_SECRET' 

By the way what's your opinion about building these kind of custom images? Because it's like forking a project and you would miss any upates to it unless you check and rebuild your own image manually.

I also tried something else. I've put the above script in a swarm config and made it available to the container using configs setting in stack yaml file, and then defined a custom entrypoint so you don't need to build a custom image. but unfortunately I ran into permission issues, the error says you don't have permission to run the script file. I was interesting to see if you can make this approach work as it will be way more better solution (no custom images)

docker-compose.yml

version: '3.8'

services:
  server:
    image: drone/drone:2
    entrypoint:
      [
        'sh',
        '-xc',
        'chmod +x /bin/file_env.sh /bin/file_env.sh /bin/drone-server',
      ]
    configs:
      - source: file_env
      - target: /bin/file_env.sh

# ...rest of the code...

By the way thanks for your suggestion about Github Actions I will look into it but at the moment I need to stay with open source projects.
and Also I will watch the video as soon as I can.

Regards.