kenpat 5b3698191e ops: add nightly Docker volume backup to SAMURAI

- scripts/backup-volumes.sh: tar each named volume via alpine, rsync to
  SAMURAI (Tailscale 100.74.x.x) at 02:00; 7-day retention; preflight
  checks Tailscale + SSH before starting
- scripts/setup-samurai-ssh.sh: one-time SSH key install to SAMURAI
- scripts/monk-backup.{service,timer}: systemd units for nightly schedule
- docs/backup-setup.md: full setup instructions incl. Windows OpenSSH
  config and admin authorized_keys fix

Phase 2 (MinIO S3 on SAMURAI) tracked as TODO in backup-volumes.sh.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-06-19 02:59:14 -05:00

2.8 KiB

Raw Blame History

Docker Volume Backup: monk → SAMURAI

Nightly rsync of named Docker volumes to SAMURAI (Windows 11, Tailscale).

Architecture

monk (T14s)
└── Docker named volumes
    ├── kite-ai_open-webui
    ├── osticket_osticket_db / osticket_uploads
    ├── portainer_data
    ├── prometheus_prometheus-data
    └── uptime-kuma_uptime-kuma
         │
         │  tar.gz via alpine container
         │  rsync over SSH (Tailscale)
         ▼
SAMURAI (Windows 11, 100.74.x.x)
└── C:\KiteBackups\monk\<TIMESTAMP>\
    ├── kite-ai_open-webui.tar.gz
    ├── osticket_osticket_db.tar.gz
    └── ...

7-day retention (older dirs pruned automatically)

Phase 2 (TODO)

Deploy MinIO on SAMURAI and push archives as S3 objects using mc put.

One-time setup

1. Enable OpenSSH Server on SAMURAI

In PowerShell (admin):

Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0
Start-Service sshd
Set-Service -Name sshd -StartupType Automatic
# Allow Tailscale traffic (adjust rule name if needed)
New-NetFirewallRule -Name "sshd-tailscale" -DisplayName "OpenSSH via Tailscale" -Enabled True -Direction Inbound -Protocol TCP -Action Allow -LocalPort 22 -RemoteAddress 100.64.0.0/10

2. Install SSH key from monk

cd ~/kitestacks-homelab
SAMURAI_USER=kenpat bash scripts/setup-samurai-ssh.sh

If your SAMURAI account is in the Administrators group, Windows ignores ~\.ssh\authorized_keys. Run this in PowerShell admin instead:

$key = Get-Content "$env:USERPROFILE\.ssh\authorized_keys" -ErrorAction SilentlyContinue
if (-not $key) { $key = Get-Content "$env:ProgramData\ssh\authorized_keys" }
Add-Content -Force "$env:ProgramData\ssh\administrators_authorized_keys" $key
icacls "$env:ProgramData\ssh\administrators_authorized_keys" /inheritance:r /grant "SYSTEM:(F)" /grant "BUILTIN\Administrators:(F)"

3. Create backup directory on SAMURAI

New-Item -ItemType Directory -Path "C:\KiteBackups\monk" -Force

4. Install systemd units on monk

sudo cp ~/kitestacks-homelab/scripts/monk-backup.service /etc/systemd/system/
sudo cp ~/kitestacks-homelab/scripts/monk-backup.timer /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now monk-backup.timer

Verify:

systemctl list-timers monk-backup.timer
# Run immediately to test:
sudo systemctl start monk-backup.service
journalctl -u monk-backup.service -f

Logs

tail -f /var/log/kitestacks/backup-volumes.log

Restore a volume

# Copy archive back from SAMURAI
scp -i ~/.ssh/id_ed25519_samurai kenpat@100.74.x.x:/cygdrive/c/KiteBackups/monk/<TIMESTAMP>/osticket_osticket_db.tar.gz /tmp/

# Restore into a volume
docker run --rm -v osticket_osticket_db:/target alpine sh -c \
  "cd /target && tar xzf -" < /tmp/osticket_osticket_db.tar.gz

2.8 KiB Raw Blame History