kitestacks-homelab/homelab-mastery/concepts/oauth2-oidc.md

OAuth2 and OIDC — How SSO Actually Works

This is the concept that most people get wrong. Understanding it cold will impress any interviewer.

---

The Problem SSO Solves

Without SSO: 11 services = 11 separate user databases. To add a friend:

• Create account in Forgejo
• Create account in Grafana
• Create account in Open WebUI
• Create account in Kavita
• ... 11 times

To remove their access: 11 places to deactivate.

With SSO: 1 account in Authentik. Access to all 11 services. Deactivate once.

---

OAuth2 — Authorization, Not Authentication

OAuth2 is commonly misunderstood. It was designed for authorization (what can you access?) not authentication (who are you?).

The core flow (Authorization Code):

1. You click "Sign in with Authentik" in Grafana

2. Grafana redirects your browser to Authentik:
   GET https://auth.kitestacks.com/application/o/authorize/
   ?client_id=grafana
   &redirect_uri=https://grafana.kitestacks.com/login/generic_oauth
   &response_type=code
   &scope=openid email profile
   &state=random_string_to_prevent_csrf

3. Authentik presents login page
   You enter username + password
   Authentik validates credentials against its database

4. Authentik redirects your browser BACK to Grafana:
   GET https://grafana.kitestacks.com/login/generic_oauth
   ?code=abc123xyz  ← authorization code (short-lived, one-time use)
   &state=random_string  ← must match what Grafana sent in step 2

5. Grafana's backend (not browser) calls Authentik directly:
   POST https://auth.kitestacks.com/application/o/token/
   client_id=grafana
   client_secret=<secret>
   code=abc123xyz
   grant_type=authorization_code

6. Authentik validates: code exists? client_secret correct? 
   Returns: access_token + id_token (JWTs)

7. Grafana reads the user's info from the token
   Logs the user in

Why the code exchange in steps 5-6? The authorization code goes through the browser (URL redirect) — visible in browser history, logs, etc. The actual tokens go server-to-server (Grafana backend → Authentik). This keeps tokens out of the browser.

---

The invalid_grant Bug — Explained

This is the exact bug you hit and fixed. Now understand why:

In step 4, Authentik stores the code=abc123xyz in its database. In step 5, Grafana sends that code to Authentik to exchange for tokens.

With active-active failover:

• Step 4 might hit monk's cloudflared connector → monk's Authentik → code stored in monk's Postgres
• Step 5 might hit kscloud1's cloudflared connector → kscloud1's Authentik → looks for code in kscloud1's Postgres → NOT FOUND → invalid_grant

The fix: Both Authentik instances share ONE Postgres database (on kscloud1, via Tailscale). The code is always found regardless of which connector handles each request.

This is a real distributed systems problem — stateful operations across load-balanced nodes — and you diagnosed and fixed it.

---

OIDC — Adding Identity on Top of OAuth2

OAuth2 tells you what a user can access. It doesn't tell you who they are.

OpenID Connect (OIDC) is a layer on top of OAuth2 that adds identity. It introduces the ID Token — a JWT (JSON Web Token) that contains claims about the user:

{
  "sub": "user-uuid-from-authentik",
  "email": "kenpat7177@gmail.com",
  "name": "kenpat",
  "preferred_username": "kenpat7177",
  "iat": 1234567890,
  "exp": 1234571490,
  "iss": "https://auth.kitestacks.com/application/o/kavita/"
}

The app reads these claims to create or update the user's local account. This is why when you log into Grafana with Authentik, Grafana knows your username and email without you creating a separate Grafana account.

---

JWT — What It Is

A JWT (JSON Web Token) is a signed, base64-encoded string with three parts:

eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJ1c2VyaWQiLCJlbWFpbCI6ImtlbnBhdEBraXRlc3RhY2tzLmNvbSJ9.SIGNATURE
     HEADER                    PAYLOAD (claims)                          SIGNATURE

The signature is created by Authentik using a private RSA key. Any app can verify it using Authentik's public key (available at the JWKS endpoint). This means apps can validate tokens without calling Authentik again.

What this means practically: Authentik signs the JWT. Grafana doesn't need to call Authentik to validate it — it just checks the signature with Authentik's public key. This is stateless authentication.

---

The Discovery Document

Every OIDC provider exposes a discovery document:

https://auth.kitestacks.com/application/o/grafana/.well-known/openid-configuration

This JSON file tells any OIDC client where to find every endpoint:

• authorization_endpoint — where to send the user to log in
• token_endpoint — where to exchange codes for tokens
• userinfo_endpoint — where to get user details
• jwks_uri — where to find the public keys for JWT validation

Apps use this URL to auto-configure themselves. That's why you set OPENID_PROVIDER_URL in Open WebUI and it figured out the rest.

---

Redirect URI — Why It Must Match Exactly

In step 2 of the flow, Grafana sends:

&redirect_uri=https://grafana.kitestacks.com/login/generic_oauth

Authentik checks: is this redirect URI registered for this client? If not, it refuses — this prevents attackers from creating a malicious app that tricks users into sending their authorization codes to an attacker's server.

This is why Karakeep broke until you fixed the redirect URI. Karakeep uses NextAuth.js with provider ID custom, so the actual callback path was /api/auth/callback/custom — not /api/auth/callback/authentik. The URI had to match exactly.

---

Provider Patterns in This Homelab

Native OIDC (app handles SSO itself):

App	How it works
Grafana	Generic OAuth2 env vars (GF_AUTH_GENERIC_OAUTH_*)
Open WebUI	OAUTH_CLIENT_ID + OPENID_PROVIDER_URL env vars
Kavita	Settings UI → OIDC section (must use UI, not database)
Karakeep	NextAuth.js with custom provider ID in .env
Forgejo	OAuth2 authentication source in Forgejo admin UI

The Authentik proxy pattern (for apps with no native SSO support): Authentik acts as a reverse proxy in front of the app. The user authenticates with Authentik, and only authenticated requests are forwarded to the app. Uptime Kuma and Prometheus use this pattern (not yet fully deployed — requires Cloudflare route update).

---

What to Say About SSO

"I implemented single sign-on across all eleven services using Authentik as the OIDC identity provider. Each service is registered as an OAuth2 client with a unique client ID and redirect URI. The OAuth2 authorization code flow means user credentials only ever go to Authentik — other services receive a signed JWT and never see the password. I hit a distributed systems issue in production where authorization codes were being invalidated by active-active load balancing across two hosts — I diagnosed it by tracing the OAuth2 flow and fixed it by sharing a single Postgres database between both Authentik instances over a private Tailscale network."