Skip to content

OS Fundamentals

Module Overview

This module covers essential operating system concepts crucial for understanding security, system administration, and penetration testing across different platforms.

Learning Objectives

By the end of this module, you will be able to:

  • Understand operating system architecture and components
  • Navigate and manage Linux and Windows file systems
  • Comprehend process management and inter-process communication
  • Configure and analyze network services and protocols
  • Implement security hardening techniques

Operating System Architecture

graph TB
    A[Applications] --> B[System Calls]
    B --> C[Kernel]
    C --> D[Hardware Abstraction Layer]
    D --> E[Hardware]

    C --> F[Process Management]
    C --> G[Memory Management]
    C --> H[File System]
    C --> I[Network Stack]
    C --> J[Device Drivers]

Core OS Components

Component Function Security Relevance
Kernel Core OS functions, hardware abstraction Kernel exploits, privilege escalation
Shell User interface, command interpreter Command injection, shell escapes
File System Data storage and organization File permissions, ACLs
Network Stack Network communication protocols Network attacks, packet analysis
Process Manager Process creation and scheduling Process injection, privilege escalation

File Systems Deep Dive

Linux File System Hierarchy

# Essential system directories
/bin          # Essential user binaries
/sbin         # System administration binaries
/etc          # Configuration files
/var          # Variable data (logs, spool)
/home         # User home directories
/tmp          # Temporary files
/proc         # Process and kernel information
/sys          # Hardware and kernel interface
# Permission types
r (4) = Read
w (2) = Write  
x (1) = Execute

# Examples
-rwxr-xr-x  # 755: Owner full, group/others read+execute
-rw-r--r--  # 644: Owner read+write, others read-only
-rwsr-xr-x  # 4755: SUID bit set
-rwxr-sr-x  # 2755: SGID bit set
-rwxr-xr-t  # 1755: Sticky bit set
# Extended attributes
lsattr file.txt           # List attributes
chattr +i file.txt        # Make immutable
chattr +a file.txt        # Append-only

# Access Control Lists (ACLs)
getfacl file.txt          # View ACL
setfacl -m u:user:rw file.txt  # Modify ACL

# SELinux contexts
ls -Z file.txt            # View SELinux context
chcon -t httpd_exec_t file.txt  # Change context

Windows File System

# File system information
Get-WmiObject -Class Win32_LogicalDisk

# NTFS permissions
Get-Acl C:\Windows\System32\config\SAM | Format-List

# Alternate Data Streams
dir /r file.txt           # Show ADS
echo "hidden" > file.txt:hidden  # Create ADS
more < file.txt:hidden    # Read ADS
# Registry hives
HKEY_LOCAL_MACHINE (HKLM)    # Machine-wide settings
HKEY_CURRENT_USER (HKCU)     # Current user settings
HKEY_CLASSES_ROOT (HKCR)     # File associations
HKEY_USERS (HKU)             # All user profiles
HKEY_CURRENT_CONFIG (HKCC)   # Current hardware profile

# Registry operations
Get-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion"
Set-ItemProperty -Path "HKCU:\Software\Test" -Name "Value" -Value "Data"

Process Management

Linux Process Control

# Process listing and analysis
ps aux                    # All processes
ps -elf                   # Extended format
pstree                    # Process tree
top                       # Real-time process monitor
htop                      # Enhanced process monitor

# Process details
cat /proc/PID/status      # Process status
cat /proc/PID/cmdline     # Command line
cat /proc/PID/environ     # Environment variables
lsof -p PID               # Open files
# Common signals
kill -TERM PID            # Terminate gracefully (15)
kill -KILL PID            # Force kill (9)
kill -STOP PID            # Suspend process (19)
kill -CONT PID            # Resume process (18)
kill -USR1 PID            # User-defined signal 1 (10)

# Signal handling in scripts
trap 'echo "Interrupted"' INT
trap 'cleanup; exit' EXIT
# Background processes
command &                 # Run in background
nohup command &           # Run immune to hangups
disown %1                 # Remove from job table

# Job management
jobs                      # List active jobs
fg %1                     # Bring job to foreground
bg %1                     # Send job to background

Windows Process Management

# Process information
Get-Process               # List all processes
Get-Process -Name "notepad" | Format-List *

# Detailed process info
Get-WmiObject Win32_Process | Select Name, ProcessId, CommandLine

# Process tree
Get-CimInstance Win32_Process | Select Name, ProcessId, ParentProcessId
# Service operations
Get-Service               # List all services
Start-Service "ServiceName"
Stop-Service "ServiceName"
Restart-Service "ServiceName"

# Service configuration
Get-WmiObject Win32_Service | Where {$_.Name -eq "ServiceName"}
sc.exe config ServiceName start= disabled

Memory Management

Virtual Memory Concepts

The relationship between virtual and physical memory can be expressed as:

\[Virtual\ Address = Page\ Number \times Page\ Size + Offset\]

Where the Memory Management Unit (MMU) translates virtual addresses to physical addresses using page tables.

Linux Memory Analysis

# System memory
free -h                   # Memory usage summary
cat /proc/meminfo         # Detailed memory info
vmstat 1                  # Virtual memory statistics

# Process memory
cat /proc/PID/maps        # Memory mapping
cat /proc/PID/smaps       # Detailed mapping info
pmap PID                  # Process memory map
# Memory pressure
echo 3 > /proc/sys/vm/drop_caches  # Clear caches
sysctl vm.swappiness=10            # Reduce swap usage

# OOM killer
echo -1000 > /proc/PID/oom_score_adj  # Protect from OOM
dmesg | grep -i "killed process"       # OOM events

Windows Memory Management

# Memory information
Get-WmiObject Win32_PhysicalMemory | Select Capacity, Speed
Get-Counter "\Memory\Available MBytes"

# Process memory usage
Get-Process | Sort-Object WorkingSet -Descending | Select -First 10

# Virtual memory
Get-WmiObject Win32_PageFileUsage

Network Fundamentals

TCP/IP Stack Analysis

graph TB
    A[Application Layer] --> B[Transport Layer]
    B --> C[Network Layer]
    C --> D[Data Link Layer]
    D --> E[Physical Layer]

    A --> F[HTTP, HTTPS, SSH, FTP]
    B --> G[TCP, UDP]
    C --> H[IP, ICMP, ARP]
    D --> I[Ethernet, WiFi]
    E --> J[Physical Hardware]

Network Configuration

# Interface configuration
ip addr show              # Show interfaces
ip route show             # Show routing table
ip neigh show             # Show ARP table

# Network statistics
ss -tuln                  # Socket statistics
netstat -rn               # Routing table
iptables -L -n            # Firewall rules

# DNS configuration
cat /etc/resolv.conf      # DNS servers
systemd-resolve --status  # Systemd DNS info
# Network configuration
Get-NetAdapter            # Network adapters
Get-NetIPConfiguration    # IP configuration
Get-NetRoute              # Routing table

# Network connections
Get-NetTCPConnection      # TCP connections
Get-NetUDPEndpoint        # UDP endpoints

# Firewall
Get-NetFirewallRule       # Firewall rules
Get-NetFirewallProfile    # Firewall profiles

Protocol Analysis

# Using tcpdump
tcpdump -i eth0 -w capture.pcap
tcpdump -r capture.pcap host 192.168.1.1

# Using Wireshark/tshark
tshark -i eth0 -f "tcp port 80"
tshark -r capture.pcap -Y "http.request.method==GET"
# Port scanning
nmap -sS target           # SYN scan
nmap -sU target           # UDP scan
nmap -A target            # Aggressive scan

# Service enumeration
nmap -sV -p- target       # Version detection
nc -zv target 1-1000      # Simple port scan

Security Hardening

Linux Hardening Checklist

Category Hardening Measure Implementation
Updates Keep system updated apt update && apt upgrade
Users Disable root login PermitRootLogin no in sshd_config
Firewall Enable iptables/ufw ufw enable
Services Disable unnecessary services systemctl disable service
Permissions Fix file permissions chmod, chown appropriately
Logging Enable comprehensive logging Configure rsyslog/journald

Windows Hardening

# Security policies
secedit /export /cfg security.inf  # Export security config

# User rights assignments
Get-WmiObject Win32_UserAccount | Where {$_.Disabled -eq $false}

# Password policies
net accounts                        # Account policies
# Security-related registry keys
Get-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Control\Lsa"

# Disable SMBv1
Set-SmbServerConfiguration -EnableSMB1Protocol $false

# Enable Windows Defender
Set-MpPreference -DisableRealtimeMonitoring $false

Practical Exercises

Exercise 1: File System Analysis

Challenge

Analyze a compromised system's file system for evidence of intrusion.

Investigation Steps
# 1. Check recently modified files
find / -mtime -1 -type f 2>/dev/null

# 2. Look for unusual SUID binaries
find / -perm -4000 -type f 2>/dev/null | sort

# 3. Examine log files
tail -f /var/log/auth.log
grep -i "failed\|error" /var/log/syslog

# 4. Check for hidden files
find / -name ".*" -type f 2>/dev/null

Exercise 2: Process Forensics

Challenge

Identify and analyze suspicious processes on a Windows system.

Analysis Approach
flowchart TD
    A[List All Processes] --> B[Check Process Trees]
    B --> C[Analyze Network Connections]
    C --> D[Examine Process Details]
    D --> E[Check File Signatures]
    E --> F[Correlate with Logs]

Performance Monitoring

System Resource Monitoring

# CPU usage monitoring
top -p PID                # Monitor specific process
sar -u 1 10               # CPU utilization
iostat -x 1               # I/O statistics

# Load average interpretation
uptime                    # Current load
cat /proc/loadavg         # Detailed load info
# Memory monitoring
watch -n1 'free -h'       # Real-time memory usage
smem -pk                  # Memory usage by process

# Swap analysis
swapon --show             # Swap information
cat /proc/swaps           # Swap usage details

Performance Tuning

The performance optimization formula:

\[Performance = \frac{Throughput}{Latency} \times Efficiency\]

Where efficiency factors include CPU scheduling, memory allocation, and I/O optimization.

Key Takeaways

Core Concepts

  • Understanding OS internals is crucial for effective security assessment
  • File systems store critical security information and potential attack vectors
  • Process management knowledge enables detection of malicious activities
  • Network stack comprehension aids in traffic analysis and incident response
  • Security hardening requires systematic approach across all OS components

Additional Resources


This module establishes the foundational operating system knowledge necessary for advanced security topics and practical system administration tasks.