kenpat
1e8319ee75
Complete documentation suite for KiteStacks covering all 11 services across 2-host active-active architecture. Includes beginner track (with AI, 8 files) and advanced track (without AI, 7 files) with time estimates, real troubleshooting cases, and command-by-command explanations. Updates certifications roadmap to reflect July 7 2026 A+ Core 2 exam goal. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
347 lines
9 KiB
Markdown
347 lines
9 KiB
Markdown
# Without AI — Part 3: Python Basics
|
||
|
||
**Track:** Advanced (No AI)
|
||
**Time for this section:** 1–2 weeks
|
||
|
||
Python is used in this homelab for:
|
||
1. **Database operations** — copying SQLite databases safely between machines
|
||
2. **HTTP requests** — hitting APIs to configure services
|
||
3. **The metrics API** — the Python FastAPI service that feeds live stats to the portal
|
||
4. **One-off automation** — scripts that are too complex for Bash
|
||
|
||
You do not need to be a Python developer. You need to read Python code, understand
|
||
what it does, modify it for your situation, and write simple scripts.
|
||
|
||
---
|
||
|
||
## Installing Python
|
||
|
||
Ubuntu 24.04 comes with Python 3 already installed:
|
||
```bash
|
||
python3 --version # Should show 3.12.x or similar
|
||
pip3 --version # Package manager for Python
|
||
```
|
||
|
||
Install the packages used in this homelab:
|
||
```bash
|
||
pip3 install requests fastapi uvicorn psutil
|
||
```
|
||
|
||
---
|
||
|
||
## Python Syntax Basics
|
||
|
||
Python uses indentation (spaces) to define blocks of code instead of `{}` like many
|
||
other languages. This is critical — wrong indentation causes errors.
|
||
|
||
```python
|
||
# This is a comment
|
||
|
||
name = "kenpat" # string
|
||
port = 3000 # integer
|
||
price = 4.99 # float
|
||
is_running = True # boolean
|
||
|
||
print(name) # prints: kenpat
|
||
print(f"Port is {port}") # f-string: prints: Port is 3000
|
||
print(f"{name!r}") # repr: prints: 'kenpat' (with quotes)
|
||
```
|
||
|
||
---
|
||
|
||
## Data Structures
|
||
|
||
```python
|
||
# List (ordered, mutable)
|
||
services = ["forgejo", "grafana", "authentik"]
|
||
services.append("portainer") # add to end
|
||
services[0] # "forgejo" (zero-indexed)
|
||
services[-1] # "portainer" (last item)
|
||
len(services) # 4
|
||
|
||
for service in services:
|
||
print(service)
|
||
|
||
# Dictionary (key-value pairs, like JSON)
|
||
monitor = {
|
||
"name": "Forgejo",
|
||
"url": "https://gitforge.kitestacks.com",
|
||
"id": 16,
|
||
"active": True
|
||
}
|
||
|
||
monitor["name"] # "Forgejo"
|
||
monitor.get("missing", "default") # "default" (safe get with fallback)
|
||
monitor.keys() # dict_keys(["name", "url", "id", "active"])
|
||
|
||
for key, value in monitor.items():
|
||
print(f"{key}: {value}")
|
||
|
||
# List of dicts (very common in API responses)
|
||
monitors = [
|
||
{"id": 16, "name": "Forgejo"},
|
||
{"id": 17, "name": "Grafana"},
|
||
]
|
||
for m in monitors:
|
||
print(m["id"], m["name"])
|
||
```
|
||
|
||
---
|
||
|
||
## Functions and Conditionals
|
||
|
||
```python
|
||
def check_service(name, url):
|
||
"""Check if a service URL is reachable."""
|
||
if not url.startswith("https://"):
|
||
return False
|
||
print(f"Checking {name} at {url}")
|
||
return True
|
||
|
||
result = check_service("Grafana", "https://grafana.kitestacks.com")
|
||
print(result) # True
|
||
```
|
||
|
||
**Conditionals:**
|
||
```python
|
||
status = 200
|
||
|
||
if status == 200:
|
||
print("OK")
|
||
elif status in (301, 302):
|
||
print("Redirect")
|
||
elif status >= 500:
|
||
print("Server error")
|
||
else:
|
||
print(f"Unexpected status: {status}")
|
||
```
|
||
|
||
---
|
||
|
||
## Working with JSON
|
||
|
||
Almost every API in this homelab sends and receives JSON (JavaScript Object Notation).
|
||
Python's `json` module converts between JSON strings and Python dicts/lists:
|
||
|
||
```python
|
||
import json
|
||
|
||
# JSON string to Python dict
|
||
data = json.loads('{"name": "Forgejo", "id": 16}')
|
||
print(data["name"]) # Forgejo
|
||
|
||
# Python dict to JSON string
|
||
obj = {"monitors": [1, 2, 3]}
|
||
json_str = json.dumps(obj, indent=2)
|
||
print(json_str)
|
||
# {
|
||
# "monitors": [1, 2, 3]
|
||
# }
|
||
|
||
# Read JSON from a file
|
||
with open("/tmp/kuma.meta.json") as f:
|
||
kuma_data = json.load(f)
|
||
|
||
# Parse Uptime Kuma heartbeat data
|
||
for monitor_id, heartbeats in kuma_data.get("heartbeatList", {}).items():
|
||
if heartbeats:
|
||
last = heartbeats[-1]
|
||
status = "UP" if last["status"] == 1 else "DOWN"
|
||
print(f"Monitor {monitor_id}: {status}")
|
||
```
|
||
|
||
---
|
||
|
||
## HTTP Requests with `requests`
|
||
|
||
The `requests` library makes HTTP calls easy:
|
||
|
||
```python
|
||
import requests
|
||
|
||
# GET request
|
||
response = requests.get("https://gitforge.kitestacks.com/api/v1/repos/search",
|
||
headers={"Authorization": "token your-api-token"},
|
||
timeout=5)
|
||
|
||
print(response.status_code) # 200
|
||
data = response.json() # Parse JSON response body
|
||
print(data["data"][0]["name"]) # First repo name
|
||
|
||
# POST request with JSON body
|
||
response = requests.post(
|
||
"https://auth.kitestacks.com/api/v3/core/tokens/",
|
||
headers={"Authorization": "Bearer your-admin-token"},
|
||
json={"identifier": "my-token", "user": "kenpat"},
|
||
timeout=5
|
||
)
|
||
|
||
if response.ok: # True for 2xx status codes
|
||
print("Token created:", response.json()["key"])
|
||
else:
|
||
print(f"Failed: {response.status_code} {response.text}")
|
||
```
|
||
|
||
---
|
||
|
||
## SQLite — The Key Database Skill in This Homelab
|
||
|
||
SQLite is a database that lives in a single file. Uptime Kuma, Kavita, and other services
|
||
use SQLite. You used Python's `sqlite3` module to copy databases safely between machines.
|
||
|
||
```python
|
||
import sqlite3
|
||
|
||
# Connect to a database file
|
||
conn = sqlite3.connect("/path/to/kuma.db")
|
||
|
||
# Run a query
|
||
cursor = conn.execute("SELECT id, name, url FROM monitor ORDER BY id")
|
||
rows = cursor.fetchall() # Get all results
|
||
for row in rows:
|
||
print(row[0], row[1], row[2])
|
||
|
||
# Insert data
|
||
conn.execute(
|
||
"INSERT INTO monitor (name, type, url, active) VALUES (?, ?, ?, ?)",
|
||
("BookStack", "http", "https://wiki.kitestacks.com", 1)
|
||
)
|
||
conn.commit() # Save changes (without commit, nothing is written)
|
||
|
||
# Use a transaction explicitly (safer for multiple changes)
|
||
conn.execute("BEGIN")
|
||
conn.execute("UPDATE monitor SET active=1 WHERE id=26")
|
||
conn.execute("UPDATE monitor SET active=1 WHERE id=27")
|
||
conn.execute("COMMIT")
|
||
|
||
conn.close()
|
||
```
|
||
|
||
### The `backup()` Method — Copying Databases Safely
|
||
|
||
SQLite databases in WAL mode (write-ahead log) cannot be copied with a plain file copy
|
||
while they are in use. The `Connection.backup()` method creates a consistent snapshot:
|
||
|
||
```python
|
||
import sqlite3
|
||
|
||
def safe_backup(source_path, dest_path):
|
||
"""Copy a SQLite database safely, even if it's in use."""
|
||
src = sqlite3.connect(source_path)
|
||
dst = sqlite3.connect(dest_path)
|
||
src.backup(dst) # Creates a consistent copy
|
||
dst.close()
|
||
src.close()
|
||
print(f"Backed up {source_path} to {dest_path}")
|
||
|
||
safe_backup("/src/kuma.db", "/out/kuma.db.backup")
|
||
```
|
||
|
||
**Why a plain `cp` would fail:** SQLite in WAL mode has two extra files:
|
||
`kuma.db-wal` (uncommitted changes) and `kuma.db-shm` (shared memory). If you copy
|
||
the main file without those, or in the wrong order, you get a corrupted database.
|
||
`Connection.backup()` handles all of this correctly.
|
||
|
||
---
|
||
|
||
## Writing a Simple FastAPI Service
|
||
|
||
The kitestacks-metrics-api is a Python FastAPI service. Understanding it helps you
|
||
modify or extend it:
|
||
|
||
```python
|
||
from fastapi import FastAPI
|
||
import psutil
|
||
|
||
app = FastAPI()
|
||
|
||
@app.get("/api/health")
|
||
def health():
|
||
return {"ok": True}
|
||
|
||
@app.get("/api/metrics")
|
||
def metrics():
|
||
return {
|
||
"cpu_percent": psutil.cpu_percent(interval=1),
|
||
"ram_percent": psutil.virtual_memory().percent,
|
||
"ram_used_gb": psutil.virtual_memory().used / 1e9,
|
||
"disk_percent": psutil.disk_usage("/").percent,
|
||
}
|
||
```
|
||
|
||
Run it:
|
||
```bash
|
||
uvicorn myapi:app --host 0.0.0.0 --port 8000
|
||
```
|
||
|
||
`psutil` reads these values from the host's `/proc` filesystem. When running inside
|
||
a Docker container with `pid: host`, it reads the HOST's stats.
|
||
|
||
---
|
||
|
||
## Environment Variables in Python
|
||
|
||
```python
|
||
import os
|
||
|
||
token = os.environ.get("FORGEJO_TOKEN") # None if not set
|
||
token = os.environ.get("FORGEJO_TOKEN", "") # Empty string if not set
|
||
token = os.environ["FORGEJO_TOKEN"] # KeyError if not set (explicit)
|
||
|
||
# Check and fail clearly
|
||
token = os.environ.get("FORGEJO_TOKEN")
|
||
if not token:
|
||
raise ValueError("FORGEJO_TOKEN environment variable is required")
|
||
```
|
||
|
||
---
|
||
|
||
## File Operations
|
||
|
||
```python
|
||
import os
|
||
|
||
# Read a file
|
||
with open("/tmp/kuma.json") as f:
|
||
content = f.read()
|
||
|
||
# Write a file
|
||
with open("/tmp/output.sql", "w") as f:
|
||
f.write("UPDATE ServerSetting SET Value='test' WHERE \"Key\"=40;\n")
|
||
|
||
# Check if a file exists
|
||
if os.path.exists("/data/kuma.db"):
|
||
print("Database found")
|
||
|
||
# Delete a file safely
|
||
for fname in ["/data/kuma.db-shm", "/data/kuma.db-wal"]:
|
||
if os.path.exists(fname):
|
||
os.remove(fname)
|
||
print(f"Removed {fname}")
|
||
|
||
# List files in a directory
|
||
for filename in os.listdir("/app/data"):
|
||
print(filename)
|
||
```
|
||
|
||
---
|
||
|
||
## Practice Exercises
|
||
|
||
1. Write a Python script that reads `monitors.json` from Uptime Kuma's API response
|
||
and prints each monitor's name and status
|
||
|
||
2. Write a script that connects to a SQLite database, lists all tables, and prints
|
||
the first 5 rows of the `monitor` table
|
||
|
||
3. Write a script that uses `requests` to check if all 11 KiteStacks URLs return
|
||
a status code between 200 and 399, and prints a summary
|
||
|
||
4. Read the kitestacks-metrics-api source code and understand what each endpoint does
|
||
|
||
5. Modify the `safe_backup()` function to also delete `-shm` and `-wal` files from
|
||
the destination before writing (prevents WAL conflicts after restore)
|
||
|
||
---
|
||
|
||
**Next:** [Part 4 — Docker Deep Dive](04-docker-deep-dive.md)
|