Infrastructure

Introduction Infrastructure as Code (IaC) is how modern teams build reliable systems. Instead of manually clicking through cloud consoles or SSHing into servers, you define infrastructure in code—testable, version-controlled, repeatable. This guide shows you practical patterns for Terraform, Ansible, and Kubernetes with real examples, not just theory. Why Infrastructure as Code? Consider a production outage scenario: Without IaC: Database server dies You manually recreate it through AWS console (30 minutes) Forgot to enable backups? Another 15 minutes Need to reconfigure custom security groups? More time Total recovery: 2-4 hours Risk of missing steps = still broken With IaC: …

Executive Summary Layer 4 (Transport Layer) Load Balancing distributes traffic at the TCP/UDP level, before any application-level processing. Unlike Layer 7 (HTTP), L4 LBs don’t inspect request content—they simply route packets based on IP protocol data. When to use L4: Raw throughput requirements (millions of requests/sec) Non-HTTP protocols (gRPC, databases, MQTT, game servers) TLS passthrough (encrypted SNI unavailable) Extreme latency sensitivity When NOT to use L4: HTTP/HTTPS (use Layer 7 instead) Request-based routing (path-based, host-based) Simple workloads with <1M req/sec Fundamentals L4 vs L7: Quick Comparison Aspect Layer 4 (TCP/UDP) Layer 7 (HTTP/HTTPS) What it sees IP/port/protocol HTTP headers, body, cookies Routing based on Destination IP, port, protocol Host, path, query string, cookies Throughput Very high (millions pps) Lower (thousands rps) Latency <1ms typical 5-50ms typical Protocols TCP, UDP, QUIC, SCTP HTTP/1.1, HTTP/2, HTTPS, WebSocket Encryption Can passthrough TLS Can terminate/re-encrypt Best for Databases, non-HTTP, TLS passthrough Web apps, microservices, APIs Core Concepts Listeners: Defined by (protocol, port). Example: TCP:443, UDP:5353 …

Executive Summary Layer 7 (Application Layer) Load Balancing routes traffic based on HTTP/HTTPS semantics: hostnames, paths, headers, cookies, and body content. Unlike Layer 4, L7 LBs inspect and understand application protocols. When to use L7: HTTP/HTTPS workloads (99% of web apps) Host-based or path-based routing (SaaS multi-tenant) Advanced features: canary deployments, content-based routing API gateways with authentication/authorization WebSockets, gRPC, Server-Sent Events (SSE) When NOT to use L7: Non-HTTP protocols (use L4) Ultra-low latency (<5ms) with extreme throughput (use L4) Binary protocols (databases, Kafka) Fundamentals L7 vs L4: What L7 Adds Feature L4 L7 Visibility IP/port/protocol Full HTTP request/response Routing based on Destination IP, port Host, path, headers, cookies, body Request modification None Rewrite, redirect, compress TLS Passthrough only Terminate + re-encrypt Session affinity IP hash (crude) Sticky cookies, affinity headers Compression No Gzip/Brotli inline WebSockets Requires passthrough Native support gRPC Via TLS passthrough Native with trailers, keep-alives Rate limiting App-level only LB-level per path/host Auth App-level only OIDC, JWT, basic @ edge Throughput Millions RPS Thousands-millions RPS Latency <1ms 1-10ms Core L7 Concepts Listeners: HTTP port 80, HTTPS port 443 (often combined as single listener with TLS upgrade) …

Executive Summary Linux is a layered system: from firmware through kernel subsystems to containerized applications. Understanding these layers—and their interdependencies—is critical for reliable, secure, performant infrastructure. This guide covers: Layered architecture (firmware → kernel → userspace → containers) Core subsystems: process scheduling, memory, filesystems, networking systemd: unit management and service lifecycle Production best practices: security, reliability, performance, observability Note: For detailed boot flow and debugging, see the Linux Boot Flow & Debugging guide. …

Executive Summary Neo4j is a native graph database that stores data as nodes (entities) connected by relationships (edges). Unlike relational databases that normalize data into tables, Neo4j excels at traversing relationships. Quick decision: Use Neo4j for: Knowledge graphs, authorization/identity, recommendations, fraud detection, network topology, impact analysis Don’t use for: Heavy OLAP analytics, simple key-value workloads, document storage Production deployment: Kubernetes + Helm (managed) or Docker Compose + Causal Cluster (self-managed) …

Introduction Terraform state is the source of truth for your infrastructure. Proper state management is critical for team collaboration, preventing conflicts, and maintaining infrastructure integrity. This guide covers remote backends, locking mechanisms, and workspace strategies. Understanding Terraform State What is State? State is Terraform’s way of tracking which real-world resources correspond to your configuration. It’s stored in terraform.tfstate file. State file contains: Resource mappings Metadata Resource dependencies Attribute values Why State Matters Without proper state management: …

Incident Summary Date: 2025-07-22 Time: 11:20 UTC Duration: 3 hours 45 minutes Severity: SEV-2 (High) Impact: Progressive service degradation with intermittent failures Quick Facts Users Affected: ~40% experiencing intermittent errors Services Affected: Multiple microservices across 3 Kubernetes nodes Nodes Failed: 3 out of 8 worker nodes Pods Evicted: 47 pods due to disk pressure SLO Impact: 35% of monthly error budget consumed Timeline 11:20:00 - Prometheus alert: Node disk usage >85% on node-worker-3 11:22:00 - On-call engineer (Tom) acknowledged alert 11:25:00 - Checked node: 92% disk usage, mostly logs 11:28:00 - Second alert: node-worker-5 also >85% 11:30:00 - Third alert: node-worker-7 >85% 11:32:00 - Senior SRE (Rachel) joined investigation 11:35:00 - Pattern identified: All nodes running logging-agent pod 11:38:00 - First node reached 98% disk usage 11:40:00 - Kubelet started evicting pods due to disk pressure 11:42:00 - 12 pods evicted from node-worker-3 11:45:00 - User reports: Intermittent 503 errors 11:47:00 - Incident escalated to SEV-2 11:50:00 - Identified root cause: Log rotation not working for logging-agent 11:52:00 - Emergency: Manual log cleanup on affected nodes 11:58:00 - First node cleaned: 92% → 45% disk usage 12:05:00 - Second node cleaned: 88% → 40% disk usage 12:10:00 - Third node cleaned: 95% → 42% disk usage 12:15:00 - All evicted pods rescheduled and running 12:30:00 - Deployed fix for log rotation issue 12:45:00 - Monitoring shows disk usage stabilizing 13:00:00 - Implemented automated log cleanup job 13:30:00 - Added improved monitoring and alerts 14:15:00 - Verified all nodes healthy, services normal 15:05:00 - Incident marked as resolved Root Cause Analysis What Happened A logging agent (Fluentd) was deployed on all Kubernetes nodes to collect and forward logs to Elasticsearch. Due to a configuration error, log rotation was not working properly, causing log files to grow indefinitely. …