kitestacks-homelab/homelab-mastery/concepts/linux.md

Linux — Commands and Concepts to Own

You've been running Linux commands without fully owning them. This fixes that.

---

The Filesystem

Everything in Linux is a file. The filesystem tree starts at / (root):

/
├── etc/         Configuration files (system-wide)
├── home/        User home directories (/home/kenpat)
├── opt/         Optional/third-party software (kscloud1 services live here)
├── proc/        Virtual filesystem — running processes, kernel info
│   ├── uptime       System uptime in seconds
│   └── net/route    Routing table (used by metrics API to find active interface)
├── sys/         Virtual filesystem — hardware/kernel info
├── var/         Variable data — logs, databases, cache
└── usr/         User programs and libraries

---

File Permissions

Every file has three permission sets: owner, group, others.

ls -la ~/kitestacks-live/docker/kavita/config/kavita.db
-rw-r--r-- 1 kenpat kenpat 2.4M Jun 11 kavita.db

Breaking it down:

• - — it's a file (not a directory d or symlink l)
• rw- — owner (kenpat): read + write
• r-- — group (kenpat): read only
• r-- — others: read only

chmod changes permissions. chown changes owner.

Why this mattered: When syncing kavita.db to kscloud1, you ran chown 1000:1000 because the Kavita container runs as user ID 1000. If the file is owned by the wrong user ID, the container can't write to it.

chmod 644 kavita.db      # rw-r--r--
chmod 755 script.sh      # rwxr-xr-x (executable)
chown 1000:1000 kavita.db  # set owner to UID 1000, GID 1000
chown -R kenpat:kenpat ./  # recursive (-R) on a directory

---

Processes

ps aux                   # all running processes
ps aux | grep forgejo    # find forgejo processes
kill 1234               # send SIGTERM to PID 1234 (polite stop)
kill -9 1234            # send SIGKILL (force kill, no cleanup)

systemctl manages systemd services (services that start on boot):

systemctl status tailscaled     # is tailscale running?
systemctl restart tailscaled    # restart it
systemctl enable tailscaled     # start on boot
journalctl -u tailscaled        # logs for tailscale service

Your containers don't use systemd — Docker manages them with restart: unless-stopped.

---

Networking Commands

# What ports is this machine listening on?
ss -tlnp                 # TCP listening, numeric, with process
ss -tlnp | grep :3006   # is Forgejo's port bound?

# Test connectivity
ping 8.8.8.8             # can I reach Google DNS?
curl -I https://auth.kitestacks.com  # HTTP headers from Authentik
curl -s http://localhost:8000/api/health  # test metrics API

# DNS lookup
dig www.kitestacks.com   # full DNS query details
nslookup gitforge.kitestacks.com  # simpler DNS lookup

# Firewall
sudo ufw status          # what rules are active?
sudo ufw allow 22/tcp    # allow SSH
sudo ufw deny 3306/tcp   # block MySQL from outside

---

Piping and Redirection

The | (pipe) sends output of one command as input to another:

docker ps | grep forgejo           # filter docker ps output
cat prometheus.yml | grep job      # find job lines in config
docker logs authentik 2>&1 | grep ERROR  # show only errors

2>&1 redirects stderr (error output, stream 2) to stdout (stream 1) — so errors appear in the same stream as normal output and can be piped.

> redirects output to a file (overwrites):

pg_dump authentik > authentik-backup.sql

>> appends to a file:

echo "new line" >> ~/.ssh/config

---

SSH

SSH (Secure Shell) gives you a terminal session on a remote machine.

ssh kenpat@5.78.x.x                    # basic SSH
ssh -i ~/.ssh/id_ed25519_kscloud1 kenpat@5.78.x.x  # specify key
ssh -L 5099:localhost:5000 kenpat@5.78.x.x          # local port forward

Local port forward (-L local:remote_host:remote_port): ssh -L 5099:localhost:5000 kenpat@kscloud1 means:

• Traffic to YOUR localhost:5099
• Gets tunneled through the SSH connection
• And hits kscloud1's localhost:5000

You used this to access kscloud1's Kavita instance (running on port 5000) from your browser at http://localhost:5099 — without opening that port to the internet.

---

sudo and Non-Interactive Usage

sudo runs a command as root. It normally prompts for your password.

The kscloud1 problem: In automated scripts, there's no terminal to enter a password. Solution:

echo YOUR_PASSWORD | sudo -S command
# -S reads password from stdin instead of terminal

You used this to run ufw commands non-interactively. In real production environments, this is handled differently (sudoers file with NOPASSWD for specific commands, or SSH key-based service accounts).

---

Grep, Sed, Awk

grep finds lines matching a pattern:

grep "error" /var/log/syslog          # lines containing "error"
grep -i "error" logfile               # case-insensitive
grep -n "AUTHENTIK" docker-compose.yml # show line numbers
grep -r "search-term" /opt/kitestacks/  # recursive search in directory

sed (stream editor) modifies text:

sed 's/old_text/new_text/g' file.txt  # replace all occurrences
sed -i 's/old/new/g' file.txt         # -i edits file in place

awk processes structured text (columns):

grep PG_PASS .env | cut -d= -f2-      # get value after = (including trailing =)
# cut -d= splits on =, -f2- means "field 2 and everything after"
# This is why: PG_PASS=abc= → if you use -f2, you get "abc" (loses trailing =)
# With -f2-, you get "abc=" (correct)

---

What to Say About Linux

"All services run on Linux hosts. I'm comfortable with file permissions, process management, SSH configuration (including local port forwarding for secure access to non-exposed services), firewall rules with ufw, and command-line tools like grep, curl, and docker CLI. I diagnosed and fixed a network configuration issue on the cloud VPS where ufw's default-deny policy was blocking Docker container traffic to a host-network-mode service."