Skip to main content

Linux Deep Dive for Platform Engineers

A comprehensive guide to Linux internals and system programming essential for platform engineering roles.

📚 Essential Resources​

📖 Must-Read Books​

🎥 Video Resources​

🎓 Courses & Training​

📰 Blogs & Articles​

🔧 Essential Tools & References​

💬 Communities & Forums​

🎯 Interview Preparation​

Process Management​

Process Lifecycle​

Key Concepts:

  • Process creation (fork/exec)
  • Process states (Running, Sleeping, Stopped, Zombie)
  • Process scheduling
  • Inter-process communication (IPC)

Common Interview Questions:

  1. Explain the difference between a process and a thread
  2. What happens when you run a command in Linux?
  3. How do you debug a zombie process?
  4. Explain different IPC mechanisms

Practical Commands:

# Process monitoring
ps aux | grep <process>
top -H -p <pid>  # Show threads
strace -p <pid>  # Trace system calls
lsof -p <pid>    # List open files

# Process control
nice -n 10 command     # Run with lower priority
renice -n 5 -p <pid>   # Change priority
kill -SIGTERM <pid>    # Graceful termination

Resources:

Memory Management​

Virtual Memory Concepts:

  • Address space layout
  • Page tables and TLB
  • Memory allocation (malloc/free)
  • Copy-on-write (COW)
  • Memory overcommit

Memory Debugging:

# Memory analysis
free -h              # System memory overview
vmstat 1            # Virtual memory statistics
pmap -x <pid>       # Process memory map
cat /proc/<pid>/smaps  # Detailed memory usage

# Memory profiling
valgrind --leak-check=full ./program
perf record -g ./program
perf report

Common Issues:

  • Memory leaks
  • Buffer overflows
  • Out of Memory (OOM) killer
  • Page faults and swapping

Resources:

File Systems​

File System Architecture​

Key Components:

  • Inodes and dentries
  • VFS (Virtual File System) layer
  • Block devices and I/O scheduling
  • Journaling mechanisms

File System Types:

# Common file systems
ext4     # Default for many distributions
xfs      # High performance, large files
btrfs    # Copy-on-write, snapshots
zfs      # Advanced features, not in mainline kernel

# Check file system
df -Th              # File system types and usage
mount | column -t   # Mounted file systems
lsblk -f           # Block devices and file systems

Performance Tuning:

# I/O monitoring
iostat -x 1         # I/O statistics
iotop               # I/O by process
blktrace /dev/sda   # Block layer tracing

# File system tuning
tune2fs -l /dev/sda1  # ext4 parameters
xfs_info /mount/point # XFS information

Resources:

Networking Stack​

TCP/IP Implementation​

Network Layers in Linux:

Application Layer
    ↓
Socket API (sys_socket, sys_connect, etc.)
    ↓
Transport Layer (TCP/UDP)
    ↓
Network Layer (IP)
    ↓
Link Layer (Ethernet)
    ↓
Network Device Driver

Socket Programming:

// Basic TCP server
int server_fd = socket(AF_INET, SOCK_STREAM, 0);
bind(server_fd, (struct sockaddr *)&address, sizeof(address));
listen(server_fd, 3);
int new_socket = accept(server_fd, (struct sockaddr *)&address, &addrlen);

Network Debugging:

# Packet analysis
tcpdump -i eth0 -nn port 80
tshark -i eth0 -f "tcp port 443"

# Connection monitoring
ss -tulpn          # All listening ports
netstat -an        # All connections
lsof -i :80        # Process on port 80

# Network performance
iperf3 -s          # Server mode
iperf3 -c server   # Client mode

Resources:

Network Namespaces and Virtual Networking​

Container Networking:

# Network namespace operations
ip netns add myns
ip netns exec myns ip link list
ip link add veth0 type veth peer name veth1
ip link set veth1 netns myns

# Bridge networking
brctl addbr br0
brctl addif br0 eth0
ip link set br0 up

iptables and Netfilter:

# Common iptables rules
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A FORWARD -i eth0 -o eth1 -j ACCEPT
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

# Connection tracking
conntrack -L
cat /proc/net/nf_conntrack

Resources:

System Calls and Kernel Interface​

Essential System Calls​

Process Management:

fork()      // Create new process
exec()      // Execute new program
wait()      // Wait for child process
exit()      // Terminate process
clone()     // Create thread/process with shared resources

File Operations:

open()      // Open file
read()      // Read from file descriptor
write()     // Write to file descriptor
close()     // Close file descriptor
mmap()      // Memory-map file

Tracing System Calls:

# strace examples
strace -e trace=open,read,write ls
strace -c command    # Summary of system calls
strace -f -p <pid>   # Trace with children

# Advanced tracing with perf
perf trace -p <pid>
perf stat command

Resources:

Performance Analysis​

CPU Performance​

CPU Profiling Tools:

# CPU usage analysis
mpstat -P ALL 1     # Per-CPU statistics
pidstat 1           # Process CPU usage
perf top            # Real-time CPU profiling

# Flame graphs
perf record -F 99 -ag -- sleep 30
perf script | flamegraph.pl > flame.svg

Resources:

Memory Performance​

Memory Analysis:

# Page cache and buffers
vmstat 1
sar -r 1

# Memory pressure
cat /proc/pressure/memory
dmesg | grep -i memory

# NUMA statistics
numactl --hardware
numastat

Storage Performance​

I/O Analysis:

# Disk I/O patterns
iostat -xz 1
iotop -o
blktrace -d /dev/sda -o trace
blkparse trace

# File system cache
echo 3 > /proc/sys/vm/drop_caches  # Clear cache
vmtouch file.txt  # Check if file is cached

Security Features​

Linux Security Modules (LSM)​

SELinux:

# SELinux management
getenforce
setenforce 0/1
sestatus
ausearch -m avc

# Context management
ls -Z file.txt
chcon -t httpd_sys_content_t /var/www/html
restorecon -R /var/www

AppArmor:

# AppArmor status
aa-status
aa-enforce /etc/apparmor.d/profile
aa-complain /etc/apparmor.d/profile

Resources:

Capabilities and Namespaces​

Linux Capabilities:

# View capabilities
getcap /usr/bin/ping
capsh --print

# Set capabilities
setcap cap_net_raw+ep /usr/bin/ping

Namespaces:

# List namespaces
lsns
ls -la /proc/*/ns/

# Enter namespace
nsenter -t <pid> -n -m
unshare --net --pid --fork bash

Troubleshooting Toolkit​

Essential Commands Reference​

# System information
uname -a            # Kernel version
lsb_release -a      # Distribution info
dmidecode           # Hardware information
lscpu               # CPU information
lspci               # PCI devices
lsusb               # USB devices

# Process debugging
gdb -p <pid>        # Attach debugger
pstack <pid>        # Print stack trace
ltrace command      # Library call tracing

# Kernel debugging
dmesg -T            # Kernel messages with timestamps
journalctl -xe      # System logs
sysctl -a           # Kernel parameters

Interview Preparation​

Common Linux Interview Topics​

  1. Boot Process

    • BIOS/UEFI → Bootloader → Kernel → Init system
    • Systemd vs SysV init
    • Runlevels and targets

  2. Package Management

    • RPM vs DEB
    • Dependency resolution
    • Repository management

  3. System Administration

    • User and group management
    • Permission models
    • Cron and scheduling
    • Log management

  4. Kernel Concepts

    • Kernel modules
    • Device drivers
    • Kernel parameters (sysctl)

Practice Scenarios​

  1. Debug high CPU usage
  2. Investigate memory leak
  3. Troubleshoot network connectivity
  4. Analyze slow disk I/O
  5. Secure a Linux system

Additional Resources​

Books​

Online Resources​

Certifications​