kitestacks-homelab/homelab-mastery/build-guide/without-ai/05-networking.md

# Without AI — Part 5: Networking

**Track:** Advanced (No AI)  
**Time for this section:** 1–2 weeks

Networking is the hardest part to learn and the most important. Every problem in this
homelab ultimately involves a packet trying to get somewhere. If you understand how
packets travel, you can debug anything.

---

## IP Addresses

Every device on a network has an IP address — a number that identifies it.

**IPv4:** Four octets (0–255) separated by dots: `192.168.1.205`

**Classes of addresses:**

| Range | Who Owns It | Used For |
|-------|------------|---------|
| `10.0.0.0/8` | Private | Corporate networks, VPNs |
| `172.16.0.0/12` | Private | Docker bridge networks |
| `192.168.0.0/16` | Private | Home networks (your router) |
| `100.64.0.0/10` | Shared | Tailscale uses this range |
| Everything else | Public | Routable on the internet |

Private addresses are not routable on the internet. Your home router uses NAT
(Network Address Translation) to let private-addressed devices reach the internet.

---

## Subnetting and CIDR Notation

CIDR (Classless Inter-Domain Routing) notation describes a range of IP addresses:
```
192.168.1.0/24
              │
              └── prefix length: how many bits are fixed
```

An IPv4 address is 32 bits. A `/24` means the first 24 bits are fixed (the network),
leaving 8 bits for hosts. `2^8 = 256` addresses, minus network (`.0`) and broadcast (`.255`)
= 254 usable host addresses.

| CIDR | Addresses | Usable | Example |
|------|-----------|--------|---------|
| `/32` | 1 | 1 | A single IP |
| `/31` | 2 | 2 | Point-to-point link |
| `/30` | 4 | 2 | Small link |
| `/29` | 8 | 6 | Small subnet |
| `/28` | 16 | 14 | |
| `/27` | 32 | 30 | |
| `/26` | 64 | 62 | |
| `/25` | 128 | 126 | |
| `/24` | 256 | 254 | Typical home/office LAN |
| `/16` | 65,536 | 65,534 | Large network |
| `/12` | 1,048,576 | — | Docker range: 172.16.0.0/12 |
| `/8` | 16,777,216 | — | 10.x.x.x range |

**Subnetting practice:** Calculating the host range of `172.17.0.0/16`:
- Fixed: `172.17` (first 16 bits)
- Variable: last 16 bits
- Host range: `172.17.0.1` to `172.17.255.254`
- This covers all of `172.17.x.x`

**Why `/12` covers all Docker networks:**
`172.16.0.0/12` covers `172.16.0.0` through `172.31.255.255`.
Docker creates bridge networks in the `172.17.x.x`, `172.18.x.x`, etc. ranges.
All of those are inside `172.16.0.0/12` — so one ufw rule covers all Docker bridges.

---

## Ports

A port is a 16-bit number (0–65535) that identifies which service on a host should
handle a connection.

```
IP address = the building
Port       = the apartment number
```

**Well-known ports (0–1023):**
| Port | Protocol | Service |
|------|----------|---------|
| 22 | TCP | SSH |
| 25 | TCP | SMTP (email sending) |
| 53 | UDP/TCP | DNS |
| 80 | TCP | HTTP |
| 443 | TCP | HTTPS |
| 5432 | TCP | PostgreSQL |
| 6379 | TCP | Redis |

**Ephemeral ports (49152–65535):** OS assigns these randomly for outbound connections.

**In Docker:**
```yaml
ports:
  - "9100:9100"   # host:container — both the same number
```
Container port 9100 is mapped to host port 9100.
External systems connect to the host IP on port 9100.
Internally, containers on the Docker network use the container port directly.

---

## DNS (Domain Name System)

DNS is a distributed database that maps names to IP addresses.

**The hierarchy:**
```
. (root)
└── com
    └── kitestacks
        ├── www      →  Cloudflare anycast IP
        ├── auth     →  Cloudflare anycast IP
        └── grafana  →  Cloudflare anycast IP
```

**Resolution process for `grafana.kitestacks.com`:**
1. Browser checks local cache — not found
2. Browser asks OS resolver (usually `127.0.0.53`)
3. OS asks the configured DNS server (your home router, or 8.8.8.8)
4. Resolver asks root nameservers: "who handles `.com`?"
5. Root says: "Ask Verisign's servers"
6. Resolver asks Verisign: "who handles `kitestacks.com`?"
7. Verisign says: "Ask Cloudflare's nameservers (`vera.ns.cloudflare.com`)"
8. Resolver asks Cloudflare: "what is `grafana.kitestacks.com`?"
9. Cloudflare returns: "Cloudflare's anycast IP: 104.x.x.x"
10. Browser connects to 104.x.x.x on port 443

**Internal Docker DNS:**
Inside the `kitestacks` Docker network, Docker runs a DNS server at `127.0.0.11`.
When cloudflared resolves `grafana`, Docker DNS returns the container's bridge IP.

```bash
# Check what an external name resolves to
dig grafana.kitestacks.com

# Check DNS from inside a container
docker exec cloudflared nslookup grafana
docker exec cloudflared cat /etc/resolv.conf   # Shows the DNS server: 127.0.0.11
```

---

## HTTP and HTTPS

**HTTP:** Plain text request/response protocol. Anyone who can see the traffic can read it.

```
GET /api/health HTTP/1.1
Host: grafana.kitestacks.com
Accept: application/json

HTTP/1.1 200 OK
Content-Type: application/json

{"ok": true}
```

**HTTPS:** HTTP inside a TLS-encrypted tunnel. The connection is encrypted from client to
Cloudflare's edge. Between Cloudflare and your containers (inside Docker network), it is
plain HTTP — this is fine because that traffic never leaves the host.

**TLS handshake (simplified):**
1. Client says "hello, I support these cipher suites"
2. Server sends its certificate (proves it is `kitestacks.com`)
3. Client verifies certificate against trusted Certificate Authorities
4. Both sides agree on encryption keys (Diffie-Hellman key exchange)
5. Encrypted connection established
6. HTTP requests flow inside this encrypted tunnel

In this homelab, Cloudflare handles TLS entirely. Your containers never see TLS.

---

## Cloudflare Tunnel — Technical Details

**What cloudflared actually does:**

```bash
# Watch cloudflared connect
docker logs cloudflared -f
# You see: "Connection established" connIndex=0 location=ORD
# ORD = Chicago data center (or nearest Cloudflare POP to you)
```

cloudflared establishes persistent multiplexed HTTP/2 connections to Cloudflare's
edge network. When a request comes in:

```
Internet user → Cloudflare edge → tunnel (HTTP/2 multiplexed) → cloudflared
                                                                       ↓
cloudflared reads Ingress rules from Cloudflare API:
  grafana.kitestacks.com → http://grafana:3000

cloudflared → Docker DNS → grafana container IP → sends request
```

The tunnel connection uses QUIC (UDP-based) when possible, falls back to HTTPS/TCP.

**Active-Active with two connectors:**
Each connector registers separately. Cloudflare maintains a list of active connectors.
Incoming requests are distributed across connectors by Cloudflare — no configuration
needed on your end. If one connector drops, the others take all traffic within seconds.

---

## Tailscale — WireGuard Under the Hood

Tailscale is a managed WireGuard VPN. Understanding WireGuard explains Tailscale.

**WireGuard:**
- Modern VPN protocol, designed in 2016
- Uses UDP (faster than TCP-based VPNs like OpenVPN)
- Cryptography: Curve25519 key exchange, ChaCha20-Poly1305 encryption
- Each peer has a public/private key pair (like SSH keys)
- Configured via static peer lists with IP allowances

**The NAT problem:** Home machines are behind NAT. Their public IP is the router's IP,
not their own. Two NAT-ed machines cannot easily make direct connections.

**Tailscale's solution — UDP hole punching:**
1. Both machines connect to Tailscale's coordination server (DERP)
2. Tailscale orchestrates a "hole punch": both machines send packets to each other
   simultaneously, which opens NAT mappings on both routers
3. Direct WireGuard connection established peer-to-peer
4. Tailscale coordination servers are no longer involved in the data path

```bash
# Check Tailscale status
tailscale status

# See your device's Tailscale IP
tailscale ip -4

# Check connectivity to kscloud1
tailscale ping 100.123.x.x

# See if connection is direct or via relay
tailscale status --json | python3 -m json.tool | grep -A5 "kscloud1"
```

**Why Tailscale IPs are stable:** Each device's `100.x.x.x` IP is tied to its machine
identity in Tailscale's database. It does not change when you move networks or reconnect.

---

## Firewalls (ufw)

ufw (Uncomplicated Firewall) is a frontend for iptables/nftables.

**kscloud1's firewall configuration:**
```bash
# View current rules
sudo ufw status verbose

# Default policies
sudo ufw default deny incoming    # Block all inbound by default
sudo ufw default allow outgoing   # Allow all outbound

# Allow specific services
sudo ufw allow ssh                 # Allow SSH (port 22)
sudo ufw allow from 172.16.0.0/12 to any port 8000 proto tcp  # Docker → metrics API

# Why 172.16.0.0/12 and not just the specific Docker subnet?
# Docker creates a new bridge network with a random 172.x subnet for each network.
# /12 covers ALL possible Docker subnets so this rule always works.
```

**The ufw/Docker conflict:** Docker modifies iptables rules directly, bypassing ufw.
This means Docker's port mappings (`-p 9100:9100`) are accessible regardless of ufw rules.
Only services running in `network_mode: host` are controlled by ufw.

kscloud1's metrics API uses `network_mode: host`, so it needs an explicit ufw allow rule
for Docker containers to reach it.

---

## Reverse Proxies

A reverse proxy receives requests on behalf of backend services:

```
Client → Reverse Proxy → Backend A
                      → Backend B
                      → Backend C
```

In this homelab:
- **Cloudflare + cloudflared** — the primary reverse proxy routing by hostname
- **nginx (homepage container)** — secondary proxy forwarding `/api/*` to metrics API

nginx config that proxies API calls:
```nginx
location /api/ {
    proxy_pass http://host.docker.internal:8000/;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
}
```

`host.docker.internal` resolves to the host machine's IP from inside a Docker container.
This lets the nginx container reach the metrics API running in `network_mode: host`.

---

## Diagnosing Network Problems

**"I can't reach the service from outside"**
```bash
# Is cloudflared running and connected?
docker logs cloudflared | tail -20

# Is the target container running and on the right network?
docker inspect homepage --format '{{range .NetworkSettings.Networks}}{{println .}}{{end}}'

# Can cloudflared reach the container?
docker exec cloudflared curl -s http://homepage:3000
```

**"Two containers can't talk to each other"**
```bash
# Are they on the same network?
docker network inspect kitestacks | grep -A5 "Containers"

# DNS resolution working?
docker exec service-a nslookup service-b

# Is the target port open inside the container?
docker exec service-b ss -tlnp
```

**"The database won't accept connections"**
```bash
# Is Postgres listening?
docker exec authentik-postgres ss -tlnp | grep 5432

# From another container, can we reach it?
docker exec authentik nc -zv authentik-postgres 5432

# Is it bound to the right interface on kscloud1?
docker exec authentik-postgres ss -tlnp | grep 5432
# Should show: *:5432 or 100.123.x.x:5432, not 127.0.0.1:5432
```

---

**Next:** [Part 6 — Full Build](06-full-build.md)