ops: add HashiCorp Vault for secrets management
Replaces .env files across all KiteStacks apps. Vault runs as a Docker container bound to 127.0.0.1:8200 with file storage backend. - apps/vault/: compose file + vault.hcl config (TLS disabled, localhost only) - scripts/vault-env.sh: fetches secret from Vault KV and injects as env vars before running docker compose (drops the .env pattern entirely) - scripts/vault-init.sh: one-time init — GPG-encrypts unseal keys to ~/.vault-keys.gpg, creates kitestacks policy + limited app token - scripts/vault-unseal.sh: post-restart unseal via GPG-decrypted key - docs/vault-setup.md: full setup guide including secret migration steps Usage: vault-env.sh kitestacks/authentik -- docker compose up -d Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
5b3698191e
commit
dbcf51993d
6 changed files with 321 additions and 0 deletions
15
apps/vault/config/vault.hcl
Normal file
15
apps/vault/config/vault.hcl
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
storage "file" {
|
||||
path = "/vault/data"
|
||||
}
|
||||
|
||||
listener "tcp" {
|
||||
address = "0.0.0.0:8200"
|
||||
tls_disable = true
|
||||
}
|
||||
|
||||
# Only accept connections from localhost and internal Docker network.
|
||||
# Vault is NOT exposed on a public port — access via SSH tunnel or from monk only.
|
||||
api_addr = "http://127.0.0.1:8200"
|
||||
cluster_addr = "http://127.0.0.1:8201"
|
||||
|
||||
ui = true
|
||||
26
apps/vault/docker-compose.yml
Normal file
26
apps/vault/docker-compose.yml
Normal file
|
|
@ -0,0 +1,26 @@
|
|||
services:
|
||||
vault:
|
||||
image: hashicorp/vault:1.17
|
||||
container_name: vault
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
# Bound to localhost only — never expose Vault to the internet
|
||||
- "127.0.0.1:8200:8200"
|
||||
environment:
|
||||
VAULT_ADDR: "http://127.0.0.1:8200"
|
||||
cap_add:
|
||||
- IPC_LOCK
|
||||
volumes:
|
||||
- vault_data:/vault/data
|
||||
- ./config/vault.hcl:/vault/config/vault.hcl:ro
|
||||
command: server -config=/vault/config/vault.hcl
|
||||
healthcheck:
|
||||
test: ["CMD", "vault", "status", "-address=http://127.0.0.1:8200"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
|
||||
volumes:
|
||||
vault_data:
|
||||
name: vault_data
|
||||
113
docs/vault-setup.md
Normal file
113
docs/vault-setup.md
Normal file
|
|
@ -0,0 +1,113 @@
|
|||
# HashiCorp Vault: Secrets Management
|
||||
|
||||
Vault replaces `.env` files across all KiteStacks apps. Secrets live in Vault's encrypted storage; nothing sensitive is ever written to disk in plaintext or committed to git.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
monk (T14s)
|
||||
└── Docker: hashicorp/vault:1.17
|
||||
├── Bound to 127.0.0.1:8200 only (never public)
|
||||
├── Storage: vault_data Docker volume (encrypted at rest)
|
||||
└── secret/kitestacks/<app>
|
||||
├── kitestacks/authentik → AUTHENTIK_SECRET_KEY, PG_PASS
|
||||
├── kitestacks/cloudflared → TUNNEL_TOKEN
|
||||
├── kitestacks/kite-ai → WEBUI_SECRET_KEY, ...
|
||||
└── kitestacks/openproject → OPENPROJECT_OIDC_SECRET
|
||||
|
||||
scripts/vault-env.sh pulls secrets at deploy time and injects them as env vars.
|
||||
No .env files. No secrets in git.
|
||||
```
|
||||
|
||||
## One-time setup
|
||||
|
||||
### 1. Install Vault CLI on monk
|
||||
|
||||
```bash
|
||||
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
|
||||
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
|
||||
sudo apt update && sudo apt install -y vault
|
||||
```
|
||||
|
||||
### 2. Start Vault
|
||||
|
||||
```bash
|
||||
cd ~/kitestacks-homelab/apps/vault
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### 3. Initialize and unseal
|
||||
|
||||
```bash
|
||||
export VAULT_ADDR=http://127.0.0.1:8200
|
||||
# Needs GPG key set up: gpg --gen-key if you don't have one
|
||||
GPG_RECIPIENT=kenpat7177@gmail.com bash ~/kitestacks-homelab/scripts/vault-init.sh
|
||||
```
|
||||
|
||||
This saves encrypted keys to `~/.vault-keys.gpg` and creates a `kitestacks` policy token.
|
||||
|
||||
### 4. Write your first secrets
|
||||
|
||||
```bash
|
||||
export VAULT_ADDR=http://127.0.0.1:8200
|
||||
# Use the app token created by vault-init.sh
|
||||
# (or root token from: gpg --decrypt ~/.vault-keys.gpg | python3 -c "import json,sys; print(json.load(sys.stdin)['root_token'])")
|
||||
|
||||
vault kv put secret/kitestacks/authentik \
|
||||
AUTHENTIK_SECRET_KEY="$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | head -c 64)" \
|
||||
PG_PASS="your-postgres-password"
|
||||
|
||||
vault kv put secret/kitestacks/cloudflared \
|
||||
TUNNEL_TOKEN="your-cf-tunnel-token"
|
||||
|
||||
vault kv put secret/kitestacks/openproject \
|
||||
OPENPROJECT_OIDC_SECRET="your-oidc-secret"
|
||||
```
|
||||
|
||||
### 5. Write app token to ~/.vault-token
|
||||
|
||||
```bash
|
||||
# The app token was printed by vault-init.sh — paste it here:
|
||||
echo "hvs.YOURTOKEN" > ~/.vault-token
|
||||
chmod 600 ~/.vault-token
|
||||
```
|
||||
|
||||
## Deploying apps with Vault
|
||||
|
||||
Instead of `docker compose up -d`, use vault-env.sh:
|
||||
|
||||
```bash
|
||||
# Add scripts/ to PATH or use full path
|
||||
PATH="$HOME/kitestacks-homelab/scripts:$PATH"
|
||||
|
||||
vault-env.sh kitestacks/authentik -- \
|
||||
docker compose -f apps/authentik/docker-compose.yml up -d
|
||||
|
||||
vault-env.sh kitestacks/cloudflared -- \
|
||||
docker compose -f apps/cloudflared/docker-compose.yml up -d
|
||||
```
|
||||
|
||||
## After a reboot (unseal)
|
||||
|
||||
Vault is sealed after every restart. Unseal it before any deployments:
|
||||
|
||||
```bash
|
||||
export VAULT_ADDR=http://127.0.0.1:8200
|
||||
bash ~/kitestacks-homelab/scripts/vault-unseal.sh
|
||||
```
|
||||
|
||||
Or add to a post-boot script / systemd unit that runs after docker.
|
||||
|
||||
## Verifying secrets
|
||||
|
||||
```bash
|
||||
vault kv get secret/kitestacks/authentik
|
||||
vault kv list secret/kitestacks/
|
||||
```
|
||||
|
||||
## Why not just use Docker secrets?
|
||||
|
||||
Docker Swarm secrets are great but require Swarm mode. For a single-node compose setup, Vault gives you:
|
||||
- Central secret versioning and audit log
|
||||
- Rotation without touching compose files
|
||||
- Same pattern you'd use in a real Kubernetes/Nomad environment
|
||||
64
scripts/vault-env.sh
Executable file
64
scripts/vault-env.sh
Executable file
|
|
@ -0,0 +1,64 @@
|
|||
#!/usr/bin/env bash
|
||||
# vault-env.sh — fetch secrets from Vault and run a command with them as env vars
|
||||
#
|
||||
# Usage:
|
||||
# vault-env.sh <vault-path> -- docker compose up -d
|
||||
# vault-env.sh kitestacks/authentik -- docker compose -f apps/authentik/docker-compose.yml up -d
|
||||
# vault-env.sh kitestacks/cloudflared -- docker compose -f apps/cloudflared/docker-compose.yml up -d
|
||||
#
|
||||
# Vault secret paths follow the pattern: kitestacks/<app>
|
||||
# Each key in the secret maps directly to an env var name.
|
||||
#
|
||||
# Example: vault kv put kitestacks/authentik \
|
||||
# AUTHENTIK_SECRET_KEY="abc123" \
|
||||
# PG_PASS="postgres-pw"
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
VAULT_ADDR="${VAULT_ADDR:-http://127.0.0.1:8200}"
|
||||
VAULT_TOKEN_FILE="${HOME}/.vault-token"
|
||||
|
||||
if [[ $# -lt 3 ]] || [[ "$2" != "--" ]]; then
|
||||
echo "Usage: vault-env.sh <secret-path> -- <command...>"
|
||||
echo "Example: vault-env.sh kitestacks/authentik -- docker compose up -d"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
SECRET_PATH="$1"
|
||||
shift 2 # remove path and --
|
||||
|
||||
# Check Vault is reachable
|
||||
if ! vault status -address="${VAULT_ADDR}" &>/dev/null; then
|
||||
echo "ERROR: Vault is not reachable at ${VAULT_ADDR}"
|
||||
echo " Start it: cd apps/vault && docker compose up -d"
|
||||
echo " Unseal: VAULT_ADDR=${VAULT_ADDR} vault operator unseal"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check we have a token
|
||||
if [[ -z "${VAULT_TOKEN:-}" ]] && [[ ! -f "${VAULT_TOKEN_FILE}" ]]; then
|
||||
echo "ERROR: No VAULT_TOKEN set and no ~/.vault-token file found"
|
||||
echo " Login: vault login -address=${VAULT_ADDR}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Fetch the secret as JSON and convert to KEY=VALUE exports
|
||||
SECRET_JSON=$(vault kv get -address="${VAULT_ADDR}" -format=json "secret/${SECRET_PATH}" 2>/dev/null) || {
|
||||
echo "ERROR: Secret not found at secret/${SECRET_PATH}"
|
||||
echo " Create it: vault kv put secret/${SECRET_PATH} KEY=value ..."
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Build env from the secret's data map
|
||||
declare -a ENV_ARGS=()
|
||||
while IFS='=' read -r key value; do
|
||||
ENV_ARGS+=("${key}=${value}")
|
||||
done < <(echo "${SECRET_JSON}" | python3 -c "
|
||||
import json, sys
|
||||
data = json.load(sys.stdin)
|
||||
for k, v in data['data']['data'].items():
|
||||
print(f'{k}={v}')
|
||||
")
|
||||
|
||||
# Execute the command with secrets injected as env vars
|
||||
exec env "${ENV_ARGS[@]}" "$@"
|
||||
74
scripts/vault-init.sh
Executable file
74
scripts/vault-init.sh
Executable file
|
|
@ -0,0 +1,74 @@
|
|||
#!/usr/bin/env bash
|
||||
# vault-init.sh — initialize Vault and save keys securely
|
||||
# Run ONCE after first `docker compose up` in apps/vault/
|
||||
# Keys are GPG-encrypted and stored in ~/.vault-keys.gpg
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
VAULT_ADDR="${VAULT_ADDR:-http://127.0.0.1:8200}"
|
||||
KEYS_FILE="${HOME}/.vault-keys.gpg"
|
||||
GPG_RECIPIENT="${GPG_RECIPIENT:-${USER}}"
|
||||
|
||||
export VAULT_ADDR
|
||||
|
||||
echo "=== KiteStacks Vault Initialization ==="
|
||||
echo "Vault address: ${VAULT_ADDR}"
|
||||
echo ""
|
||||
|
||||
# Check not already initialized
|
||||
if vault status 2>/dev/null | grep -q "Initialized.*true"; then
|
||||
echo "Vault is already initialized."
|
||||
read -rp "Re-initialize? This DESTROYS all secrets. [y/N] " ans
|
||||
[[ "${ans}" == "y" ]] || exit 0
|
||||
fi
|
||||
|
||||
# Initialize
|
||||
echo "Initializing Vault (1 key share, 1 key threshold for homelab simplicity)..."
|
||||
INIT_OUTPUT=$(vault operator init -key-shares=1 -key-threshold=1 -format=json)
|
||||
|
||||
UNSEAL_KEY=$(echo "${INIT_OUTPUT}" | python3 -c "import json,sys; d=json.load(sys.stdin); print(d['unseal_keys_b64'][0])")
|
||||
ROOT_TOKEN=$(echo "${INIT_OUTPUT}" | python3 -c "import json,sys; d=json.load(sys.stdin); print(d['root_token'])")
|
||||
|
||||
echo ""
|
||||
echo "Encrypting keys with GPG (recipient: ${GPG_RECIPIENT}) ..."
|
||||
echo "${INIT_OUTPUT}" | gpg --encrypt --recipient "${GPG_RECIPIENT}" --output "${KEYS_FILE}"
|
||||
echo "Keys saved to: ${KEYS_FILE}"
|
||||
echo ""
|
||||
|
||||
# Unseal immediately
|
||||
echo "Unsealing Vault ..."
|
||||
vault operator unseal "${UNSEAL_KEY}"
|
||||
echo ""
|
||||
|
||||
# Log in with root token
|
||||
vault login "${ROOT_TOKEN}"
|
||||
echo ""
|
||||
|
||||
# Enable KV v2 secret engine at secret/
|
||||
vault secrets enable -path=secret kv-v2 2>/dev/null || echo "(secret/ already enabled)"
|
||||
echo ""
|
||||
|
||||
# Create a policy for kitestacks apps
|
||||
vault policy write kitestacks - <<'POLICY'
|
||||
path "secret/data/kitestacks/*" {
|
||||
capabilities = ["read"]
|
||||
}
|
||||
POLICY
|
||||
|
||||
# Create an app token with limited policy
|
||||
APP_TOKEN=$(vault token create -policy=kitestacks -display-name="kitestacks-apps" -format=json | python3 -c "import json,sys; print(json.load(sys.stdin)['auth']['client_token'])")
|
||||
|
||||
echo ""
|
||||
echo "=== SETUP COMPLETE ==="
|
||||
echo ""
|
||||
echo "Root token: stored in ${KEYS_FILE} (gpg-encrypted)"
|
||||
echo "App token: add to ~/.vault-token OR set VAULT_TOKEN env var"
|
||||
echo ""
|
||||
echo " echo '${APP_TOKEN}' > ~/.vault-token && chmod 600 ~/.vault-token"
|
||||
echo ""
|
||||
echo "Add your first secret:"
|
||||
echo " vault kv put secret/kitestacks/authentik AUTHENTIK_SECRET_KEY=... PG_PASS=..."
|
||||
echo ""
|
||||
echo "Then deploy with:"
|
||||
echo " vault-env.sh kitestacks/authentik -- docker compose -f apps/authentik/docker-compose.yml up -d"
|
||||
echo ""
|
||||
29
scripts/vault-unseal.sh
Executable file
29
scripts/vault-unseal.sh
Executable file
|
|
@ -0,0 +1,29 @@
|
|||
#!/usr/bin/env bash
|
||||
# vault-unseal.sh — unseal Vault after a container restart
|
||||
# Decrypts the GPG-encrypted keys file and unseals automatically.
|
||||
# Add to startup: after `docker compose up`, call this script.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
VAULT_ADDR="${VAULT_ADDR:-http://127.0.0.1:8200}"
|
||||
KEYS_FILE="${HOME}/.vault-keys.gpg"
|
||||
|
||||
export VAULT_ADDR
|
||||
|
||||
if vault status 2>/dev/null | grep -q "Sealed.*false"; then
|
||||
echo "Vault is already unsealed."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [[ ! -f "${KEYS_FILE}" ]]; then
|
||||
echo "ERROR: ${KEYS_FILE} not found — run vault-init.sh first"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Decrypting unseal key ..."
|
||||
UNSEAL_KEY=$(gpg --decrypt "${KEYS_FILE}" 2>/dev/null \
|
||||
| python3 -c "import json,sys; d=json.load(sys.stdin); print(d['unseal_keys_b64'][0])")
|
||||
|
||||
echo "Unsealing Vault at ${VAULT_ADDR} ..."
|
||||
vault operator unseal "${UNSEAL_KEY}"
|
||||
echo "Vault unsealed."
|
||||
Loading…
Add table
Add a link
Reference in a new issue