Threat Detection — Logs, Alerts, Signals & Indicators

How defenders identify malicious activity before it becomes a breach.

Indicators of Compromise (IoCs)

IoCs are artifacts left behind by malicious activity — the digital equivalent of footprints. They only help if you're actively looking. Most successful breaches go undetected for weeks or months because nobody checked the logs.

Common IoCs to watch for:
  Auth failures         → brute force in progress
  Auth success from new IP → possible credential compromise
  New listening ports   → backdoor or unexpected service
  Unexpected cron jobs  → persistence mechanism
  Unknown processes     → malware running in background
  High outbound traffic → data exfiltration
  Failed sudo attempts  → privilege escalation attempts
  Modified system files → rootkit or tampering
  New user accounts     → attacker creating persistence

Key Log Files to Know

Linux logs are your primary detection tool. Know where to look before you need to.

/var/log/auth.log         → SSH logins, sudo, su, PAM events
/var/log/syslog           → general system messages
/var/log/kern.log         → kernel messages
/var/log/dpkg.log         → package installs/removals
/var/log/nginx/access.log → all HTTP requests
/var/log/nginx/error.log  → nginx errors and anomalies
/var/log/fail2ban.log     → bans and unbans

# systemd (use journalctl for these)
journalctl -u sshd        → SSH daemon logs
journalctl -u nginx       → nginx logs
journalctl _COMM=sudo     → all sudo usage

Grep Patterns for Common Attacks

# SSH brute force — top attacking IPs
grep "Failed password" /var/log/auth.log \
  | awk '{print $11}' | sort | uniq -c | sort -rn | head -20

# Successful logins — who got in?
grep "Accepted" /var/log/auth.log

# Sudo usage — what commands were run as root?
grep "sudo:" /var/log/auth.log | grep "COMMAND"

# Web scanning (404 floods = directory brute force)
awk '$9 == 404' /var/log/nginx/access.log \
  | awk '{print $1}' | sort | uniq -c | sort -rn | head -10

# Look for common web attack patterns
grep -E "\.\.\/|etc/passwd|cmd=|exec\(" /var/log/nginx/access.log

# New user accounts created (persistence)
grep "new user" /var/log/auth.log

Live Process & Network Inspection

When something seems wrong, these commands give you a snapshot of what's actually running right now — processes, connections, and open files.

# Who is logged in right now?
who
w
last | head -20              → recent logins

# What processes are running?
ps auxf                      → all processes, tree view
top / htop                   → live resource usage
pstree -p                    → process tree with PIDs

# What ports are open? What owns them?
ss -tlnp                     → TCP listeners + process names
ss -tnp state established    → active TCP connections

# Suspicious: process listening on unknown port?
lsof -i :4444                → what's on port 4444?
lsof -p <PID>                → all files/sockets for a process

# Check scheduled tasks (persistence mechanism)
crontab -l                   → current user's cron
cat /etc/crontab
ls -la /etc/cron.d/
ls -la /etc/cron.daily/

Building a Detection Habit

Detection only works if it's routine. You can't spot anomalies without knowing what normal looks like. Spend 5 minutes weekly on these checks — before you need them.

Weekly homelab security check:
  systemctl list-units --failed           → any crashed services?
  journalctl -p 3 --since "7 days ago"   → errors this week
  grep "Accepted" /var/log/auth.log       → who logged in?
  ss -tlnp | grep -v LISTEN              → unexpected open ports?
  df -h                                   → disk not filling up?
  last | head -20                         → recent login history

Automated option: send weekly digest via cron
  0 9 * * 1 /usr/local/bin/security-digest.sh | mail -s "Weekly Check" [email protected]

≈·*•—[ A|S ]—•*·≈

- Crafted by Axiom|Spectre