diff --git a/build-guide/without-ai/README.md b/build-guide/without-ai/README.md new file mode 100644 index 0000000..4048b03 --- /dev/null +++ b/build-guide/without-ai/README.md @@ -0,0 +1,620 @@ +# Build KiteStacks Manually + +This is the step-by-step, command-by-command build guide. Every file, every command, every setting. No AI. Build this from scratch on a blank Ubuntu 24.04 machine. + +**Convention:** Replace `` with your actual values. Replace `kitestacks.com` with your own domain. + +--- + +## Step 1 — Docker Engine + +```bash +# Remove any old Docker installs +sudo apt remove docker docker-engine docker.io containerd runc + +# Install dependencies +sudo apt update +sudo apt install -y ca-certificates curl gnupg lsb-release + +# Add Docker's official GPG key +sudo install -m 0755 -d /etc/apt/keyrings +curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \ + sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg +sudo chmod a+r /etc/apt/keyrings/docker.gpg + +# Add Docker repository +echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \ + https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | \ + sudo tee /etc/apt/sources.list.d/docker.list > /dev/null + +# Install Docker Engine + Compose v2 +sudo apt update +sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin + +# Add your user to docker group (re-login after this) +sudo usermod -aG docker $USER + +# Create the shared network for all homelab containers +docker network create kitestacks + +# Verify +docker run --rm hello-world +docker network ls | grep kitestacks +``` + +--- + +## Step 2 — Tailscale + +```bash +curl -fsSL https://tailscale.com/install.sh | sh +sudo tailscale up +# Follow the auth link printed in the terminal +tailscale status +``` + +Do the same on the cloud VPS when you get to Step 10. + +--- + +## Step 3 — Project Structure + +```bash +mkdir -p ~/kitestacks-live/docker/{authentik,bookstack,cloudflared,forgejo,grafana,karakeep,kavita,kitestacks-portal,osticket,portainer,prometheus} +``` + +--- + +## Step 4 — Cloudflare Tunnel + +```bash +mkdir -p ~/kitestacks-live/docker/cloudflared +``` + +Create `~/kitestacks-live/docker/cloudflared/.env`: +``` +TUNNEL_TOKEN= +``` + +Create `~/kitestacks-live/docker/cloudflared/docker-compose.yml`: +```yaml +services: + cloudflared: + image: cloudflare/cloudflared:latest + container_name: cloudflared + restart: unless-stopped + command: tunnel --no-autoupdate run + environment: + - TUNNEL_TOKEN=${TUNNEL_TOKEN} + networks: + - kitestacks + +networks: + kitestacks: + external: true +``` + +```bash +cd ~/kitestacks-live/docker/cloudflared +docker compose up -d +docker logs cloudflared --tail 20 +``` + +In Cloudflare Zero Trust → Networks → Tunnels → your tunnel → Public Hostnames, add: +- `www.kitestacks.com` → `http://kitestacks-portal:80` +- `auth.kitestacks.com` → `http://authentik:9000` +(Add more subdomains as you deploy each service) + +--- + +## Step 5 — Authentik (SSO) + +Authentik needs PostgreSQL and Redis. In this setup, both live on kscloud1 (Step 10). For a single-machine setup, run them locally. + +Create `~/kitestacks-live/docker/authentik/docker-compose.yml`: +```yaml +services: + postgresql: + image: docker.io/library/postgres:16-alpine + restart: unless-stopped + container_name: authentik-postgresql + healthcheck: + test: ["CMD-SHELL", "pg_isready -d ${POSTGRES_DB} -U ${POSTGRES_USER}"] + start_period: 20s + interval: 30s + retries: 5 + timeout: 5s + volumes: + - database:/var/lib/postgresql/data + environment: + POSTGRES_PASSWORD: ${PG_PASS} + POSTGRES_USER: ${PG_USER} + POSTGRES_DB: ${PG_DB} + networks: + - kitestacks + + redis: + image: docker.io/library/redis:alpine + restart: unless-stopped + container_name: authentik-redis + healthcheck: + test: ["CMD-SHELL", "redis-cli ping | grep PONG"] + start_period: 20s + interval: 30s + retries: 5 + timeout: 3s + networks: + - kitestacks + + server: + image: ghcr.io/goauthentik/server:2024.12.3 + restart: unless-stopped + container_name: authentik + command: server + environment: + AUTHENTIK_REDIS__HOST: redis + AUTHENTIK_POSTGRESQL__HOST: postgresql + AUTHENTIK_POSTGRESQL__USER: ${PG_USER} + AUTHENTIK_POSTGRESQL__NAME: ${PG_DB} + AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS} + AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY} + volumes: + - ./media:/media + - ./custom-templates:/templates + depends_on: + - postgresql + - redis + networks: + - kitestacks + + worker: + image: ghcr.io/goauthentik/server:2024.12.3 + restart: unless-stopped + container_name: authentik-worker + command: worker + environment: + AUTHENTIK_REDIS__HOST: redis + AUTHENTIK_POSTGRESQL__HOST: postgresql + AUTHENTIK_POSTGRESQL__USER: ${PG_USER} + AUTHENTIK_POSTGRESQL__NAME: ${PG_DB} + AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS} + AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY} + volumes: + - /var/run/docker.sock:/var/run/docker.sock + - ./media:/media + - ./custom-templates:/templates + depends_on: + - postgresql + - redis + networks: + - kitestacks + +volumes: + database: + +networks: + kitestacks: + external: true +``` + +Create `~/kitestacks-live/docker/authentik/.env`: +``` +PG_PASS= +PG_USER=authentik +PG_DB=authentik +AUTHENTIK_SECRET_KEY= +``` + +Generate the secret key: +```bash +openssl rand -hex 32 +``` + +```bash +cd ~/kitestacks-live/docker/authentik +docker compose up -d +``` + +Browse to `https://auth.kitestacks.com/if/flow/initial-setup/` to complete setup. + +--- + +## Step 6 — Portainer + +Create `~/kitestacks-live/docker/portainer/docker-compose.yml`: +```yaml +services: + portainer: + image: portainer/portainer-ce:latest + container_name: portainer + restart: unless-stopped + volumes: + - /var/run/docker.sock:/var/run/docker.sock + - portainer_data:/data + networks: + - kitestacks + +volumes: + portainer_data: + +networks: + kitestacks: + external: true +``` + +```bash +cd ~/kitestacks-live/docker/portainer +docker compose up -d +``` + +Add Cloudflare Tunnel route: `portainer.kitestacks.com` → `https://portainer:9443` + +In Portainer UI → Settings → Authentication → OAuth: +- Client ID: `portainer` (from Authentik provider) +- Auth URL: `https://auth.kitestacks.com/application/o/authorize/` +- Token URL: `https://auth.kitestacks.com/application/o/token/` +- Resource URL: `https://auth.kitestacks.com/application/o/userinfo/` +- Redirect URL: `https://portainer.kitestacks.com` + +Pre-create your Authentik user as admin before their first login: +```bash +TOKEN=$(curl -sk -X POST https://portainer.kitestacks.com/api/auth \ + -H "Content-Type: application/json" \ + -d '{"username":"admin","password":""}' | python3 -c \ + "import sys,json; print(json.load(sys.stdin)['jwt'])") + +curl -sk -X POST "https://portainer.kitestacks.com/api/users" \ + -H "Authorization: Bearer $TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"username":"user@example.com","role":1}' +``` + +--- + +## Step 7 — Forgejo + +Create `~/kitestacks-live/docker/forgejo/docker-compose.yml`: +```yaml +services: + forgejo: + image: codeberg.org/forgejo/forgejo:11 + container_name: forgejo + restart: unless-stopped + volumes: + - ./data:/data + - /etc/timezone:/etc/timezone:ro + - /etc/localtime:/etc/localtime:ro + ports: + - "2222:22" + - "3006:3000" + networks: + - kitestacks + +networks: + kitestacks: + external: true +``` + +```bash +cd ~/kitestacks-live/docker/forgejo +docker compose up -d +``` + +Browse to `http://localhost:3006` to complete initial setup. + +Add CF Tunnel route: `gitforge.kitestacks.com` → `http://forgejo:3000` + +Set up SSH config (`~/.ssh/config`): +``` +Host gitforge.kitestacks.com + HostName gitforge.kitestacks.com + Port 2222 + User git + IdentityFile ~/.ssh/id_ed25519 +``` + +Generate API token: +```bash +docker exec -u git forgejo forgejo admin user generate-access-token \ + --username --token-name "cli-token" --raw \ + --scopes "read:user,write:user,read:repository,write:repository" +``` + +--- + +## Step 8 — BookStack + +Create `~/kitestacks-live/docker/bookstack/docker-compose.yml`: +```yaml +services: + bookstack: + image: lscr.io/linuxserver/bookstack:latest + container_name: bookstack + restart: unless-stopped + environment: + - PUID=1000 + - PGID=1000 + - APP_URL=https://wiki.kitestacks.com + - DB_HOST=bookstack-db + - DB_PORT=3306 + - DB_USER=bookstack + - DB_PASS= + - DB_DATABASE=bookstackapp + - AUTH_METHOD=oidc + - OIDC_ISSUER=https://auth.kitestacks.com/application/o/bookstack/ + - OIDC_ISSUER_DISCOVER=true + - OIDC_CLIENT_ID=bookstack + - OIDC_CLIENT_SECRET= + - OIDC_USER_ATTRIBUTE=email + - APP_KEY= + volumes: + - ./config:/config + ports: + - "6875:80" + depends_on: + - bookstack-db + networks: + - kitestacks + + bookstack-db: + image: lscr.io/linuxserver/mariadb:latest + container_name: bookstack-db + restart: unless-stopped + environment: + - PUID=1000 + - PGID=1000 + - MYSQL_ROOT_PASSWORD= + - MYSQL_DATABASE=bookstackapp + - MYSQL_USER=bookstack + - MYSQL_PASSWORD= + volumes: + - ./db:/config + networks: + - kitestacks + +networks: + kitestacks: + external: true +``` + +Generate APP_KEY: +```bash +docker run --rm --entrypoint /bin/bash lscr.io/linuxserver/bookstack:latest appkey +``` + +In Authentik, create an OAuth2 Provider for BookStack: +- Name: `bookstack` +- Redirect URIs: `https://wiki.kitestacks.com/oidc/callback` +- `issuer_mode`: `per_provider` + +```bash +cd ~/kitestacks-live/docker/bookstack +docker compose up -d + +# Fix cache permissions (prevents "unknown error" on OIDC login) +docker exec bookstack chown -R abc:users /config/www/framework/cache/ +``` + +Add CF Tunnel route: `wiki.kitestacks.com` → `http://bookstack:80` + +--- + +## Step 9 — Monitoring + +### Prometheus + Node Exporter + +Create `~/kitestacks-live/docker/prometheus/docker-compose.yml`: +```yaml +services: + prometheus: + image: prom/prometheus:latest + container_name: prometheus + restart: unless-stopped + volumes: + - ./prometheus.yml:/etc/prometheus/prometheus.yml + - prometheus_data:/prometheus + networks: + - kitestacks + + node-exporter: + image: prom/node-exporter:latest + container_name: node-exporter + restart: unless-stopped + network_mode: host + pid: host + volumes: + - /:/host:ro,rslave + command: + - '--path.rootfs=/host' + +volumes: + prometheus_data: + +networks: + kitestacks: + external: true +``` + +Create `~/kitestacks-live/docker/prometheus/prometheus.yml`: +```yaml +global: + scrape_interval: 15s + +scrape_configs: + - job_name: 'node-monk' + static_configs: + - targets: ['localhost:9100'] + + - job_name: 'node-kscloud1' + static_configs: + - targets: [':9100'] +``` + +### Grafana + +Create `~/kitestacks-live/docker/grafana/docker-compose.yml`: +```yaml +services: + grafana: + image: grafana/grafana:latest + container_name: grafana + restart: unless-stopped + environment: + - GF_SERVER_ROOT_URL=https://grafana.kitestacks.com + - GF_AUTH_GENERIC_OAUTH_ENABLED=true + - GF_AUTH_GENERIC_OAUTH_NAME=Authentik + - GF_AUTH_GENERIC_OAUTH_CLIENT_ID=grafana + - GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET= + - GF_AUTH_GENERIC_OAUTH_SCOPES=openid email profile + - GF_AUTH_GENERIC_OAUTH_AUTH_URL=https://auth.kitestacks.com/application/o/authorize/ + - GF_AUTH_GENERIC_OAUTH_TOKEN_URL=https://auth.kitestacks.com/application/o/token/ + - GF_AUTH_GENERIC_OAUTH_API_URL=https://auth.kitestacks.com/application/o/userinfo/ + volumes: + - grafana_data:/var/lib/grafana + networks: + - kitestacks + +volumes: + grafana_data: + +networks: + kitestacks: + external: true +``` + +```bash +cd ~/kitestacks-live/docker/grafana +docker compose up -d +``` + +After first login: Configuration → Data Sources → Add Prometheus → URL `http://prometheus:9090` + +--- + +## Step 10 — Cloud Replica (kscloud1) + +### On the VPS + +```bash +# Install Docker (same commands as Step 1) +# Install Tailscale (same as Step 2) +# Create project directories +mkdir -p /opt/kitestacks/docker/{authentik,bookstack,cloudflared,grafana,portainer} + +# Copy docker-compose files from monk via SCP or git +# Example for cloudflared: +scp ~/kitestacks-live/docker/cloudflared/docker-compose.yml root@:/opt/kitestacks/docker/cloudflared/ +``` + +### Move Authentik DB to kscloud1 + +```bash +# On kscloud1: bring up just postgres and redis from authentik compose +cd /opt/kitestacks/docker/authentik +docker compose up -d postgresql redis + +# On monk: update Authentik env to point to kscloud1 Tailscale IP +# AUTHENTIK_REDIS__HOST: +# AUTHENTIK_POSTGRESQL__HOST: +docker compose up -d +``` + +### Add second cloudflared connector on kscloud1 + +```bash +# Same TUNNEL_TOKEN as monk +cd /opt/kitestacks/docker/cloudflared +docker compose up -d + +# Verify 2 connectors in Cloudflare Zero Trust +docker exec cloudflared cloudflared tunnel info +``` + +### Test failover + +```bash +# Stop all services on monk +docker stop $(docker ps -q) + +# Browse to https://wiki.kitestacks.com from a browser — kscloud1 should serve it +# Check Cloudflare analytics for which connector is active +``` + +--- + +## Step 11 — OSTicket (Help Desk) + +Create `~/kitestacks-live/docker/osticket/docker-compose.yml`: +```yaml +services: + osticket-db: + image: mariadb:11 + container_name: osticket-db + restart: unless-stopped + environment: + - MYSQL_ROOT_PASSWORD= + - MYSQL_DATABASE=osticket + - MYSQL_USER=osticket + - MYSQL_PASSWORD= + volumes: + - ./db:/var/lib/mysql + networks: + - kitestacks + + osticket: + image: ghcr.io/tiredofit/docker-osticket:latest + container_name: osticket-app + restart: unless-stopped + environment: + - DB_HOST=osticket-db + - DB_NAME=osticket + - DB_USER=osticket + - DB_PASS= + - SMTP_HOST=smtp.gmail.com + - SMTP_PORT=587 + - SMTP_FROM=kitestacks.helpdesk@gmail.com + - SMTP_USER=kitestacks.helpdesk@gmail.com + - SMTP_PASS= + - SMTP_TLS=true + depends_on: + - osticket-db + networks: + - kitestacks + +networks: + kitestacks: + external: true +``` + +--- + +## Reference: Useful Commands + +```bash +# Check all running containers +docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}" + +# Follow logs for a service +docker logs --tail 50 -f + +# Restart a service +cd ~/kitestacks-live/docker/ && docker compose restart + +# Pull latest image and redeploy +docker compose pull && docker compose up -d + +# BookStack: clear config cache +docker exec bookstack php /app/www/artisan config:clear +docker exec bookstack php /app/www/artisan cache:clear + +# Portainer: reset admin password +docker stop portainer +docker run --rm -v portainer_data:/data portainer/helper-reset-password +docker start portainer + +# Check Tailscale connectivity +tailscale status + +# SSH to kscloud1 +ssh -i ~/.ssh/id_ed25519_kscloud1 root@ +```