This blog series is split into four parts:
- General information of secrets management
- Example how to store secrets into a version control
- Example how to use encrypted secrets in CI-pipelines
- Security issue and threat analysis based on previous examples
After the secrets are stored securely on a version control, it is time to look how a continuous integration pipeline should handle the secrets. In this guide, I’ll use Gitlab’s pipelines.
Configuring GCP
First of all, you have to create a new service account and a new KMS key which is used only by the service account.
- Head to https://console.cloud.google.com/iam-admin/serviceaccounts, select the project used and create a service account
- Create the private key in JSON format and temporarily store it on your computer. It will be used later when configuring Gitlab CI-pipeline.
- Then open https://console.cloud.google.com/security/kms, select your preferred keyring (or create a new one) and create a new key
- Purpose: Symmetric encrypt/decrypt
- Rotation period: 7 days (I’ll tell about this on the last blog post)
- Select the key created and add the service account with decryption access (’Cloud KMS CryptoKey Decrypter’) to it
Modifying access to the cryptographic key
Encrypting with a new key
If you have already encrypted secrets, you must add the key to the secrets after setting up the new cryptographic key.
# Adding GCP keysops -r -i --add-gcp-kms projects/<gcp project>/locations/global/keyRings/sops/cryptoKeys/gitlab-test api-key.json# Removing GCP key# sops -r -i --rm-gcp-kms projects/<gcp project>/locations/global/keyRings/sops/cryptoKeys/gitlab-test api-key.json# -r option: 'generate a new data encryption key and reencrypt all values with the new key' |
For .sops.yaml configuration, you have to add the new cryptographic key and re-encrypt the secrets. The previous blog post contained the Git pre-commit script which will re-encrypt all secrets if the configuration file is changed.
creation_rules: # Global enc-files (typically for testing and dev-environment) - path_regex: .*.enc(.yaml|.json)?$ gcp_kms: projects/<gcp dev project>/locations/global/keyRings/sops/cryptoKeys/dev-sops-key,projects/<gcp dev project>/locations/global/keyRings/sops/cryptoKeys/gitlab-test |
Gitlab configuration
Variables
You can add variables for the CI-pipeline in the project or group settings. I’ll add the following variables to the project settings (Gitlab > Project > Settings > CI/CD > Variables).
|
Key
|
Value
|
|---|---|
| GCLOUD_PROJECT_NAME | <project name> |
| GCLOUD_SERVICE_KEY | <service account key file content in JSON-format) |
| SOPS_VERSION | 3.4.0 |
CI Pipeline
Build contains following steps:
- Authenticate and setup gcloud CLI
- Install SOPS
- Decrypt secrets and cache them for the next build steps
- decrypt_secrets.sh -script is the same that was used in the previous blog post.
decrypt secrets: image: google/cloud-sdk:alpine cache: # Caches decrypted secrets to next build step (among with other untracked files) untracked: true script: # Authentication for google cloud - echo "$GCLOUD_SERVICE_KEY" > ${HOME}/gcloud-service-key.json - export GOOGLE_APPLICATION_CREDENTIALS=${HOME}/gcloud-service-key.json - gcloud auth activate-service-account --key-file ${HOME}/gcloud-service-key.json - gcloud config set project $GCLOUD_PROJECT_NAME # Install SOPS - apk update && apk add --no-cache ca-certificates - wget https://github.com/mozilla/sops/releases/download/${SOPS_VERSION}/sops-${SOPS_VERSION}.linux -O /usr/local/bin/sops - chmod 0755 /usr/local/bin/sops # Install GNU findutils for proper regex support - apk add findutils - sh decrypt_secrets.sh |
Gitlab missing features and bugs
- https://gitlab.com/gitlab-org/gitlab-ce/issues/43980 – If you make parallel jobs which stores to cache at same time, it will be incomplete.
- https://gitlab.com/gitlab-org/gitlab-ce/issues/42056 – You can use only one cache, no multiple. Maybe in future, you can cache decrypted secrets to one specific cache.
- https://gitlab.com/gitlab-org/gitlab-ce/issues/59937 – cache: untracked will always store all untracked files to cache. In this case we would like to cache only decrypted secrets, nothing else.
- https://gitlab.com/gitlab-org/gitlab-ce/issues/17850 – There is no regex support for caching specific files.
In the last part of blog series: what vulnerabilities my examples could contain and how to manage them.
