DevOps & SRE Practices

Introduction Disaster Recovery (DR) is the process, policies, and procedures for recovering and continuing technology infrastructure after a disaster. A disaster can be natural (earthquake, flood), technical (data center failure, ransomware), or human-caused (accidental deletion, security breach). Core Principle: “Hope is not a strategy. Plan for failure before it happens.” Key Concepts RTO vs RPO Time ─────────────────────────────────────────────────────────────> │ │ │ │ Disaster Detection Recovery Normal Occurs Time Begins Operations │◄──────────────────────────────────►│ │ Recovery Time │ │ Objective (RTO) │ │ │ │◄───────────►│ │ Data Loss │ (Recovery Point │ Objective - RPO) │ Recovery Time Objective (RTO) Definition: Maximum acceptable time that a system can be down after a disaster. …

Introduction Observability is the ability to understand the internal state of a system based on its external outputs. Unlike traditional monitoring, which tells you what is broken, observability helps you understand why it’s broken, even for issues you’ve never encountered before. Core Principle: “You can’t fix what you can’t see. You can’t see what you don’t measure.” The Three Pillars Overview ┌─────────────────────────────────────────┐ │ OBSERVABILITY │ ├─────────────┬──────────────┬────────────┤ │ METRICS │ LOGS │ TRACES │ ├─────────────┼──────────────┼────────────┤ │ What/When │ Why/Details │ Where │ │ Aggregated │ Individual │ Causal │ │ Time-series │ Events │ Flows │ │ Dashboards │ Search │ Waterfall │ └─────────────┴──────────────┴────────────┘ When to Use Each: …

Introduction Chaos Engineering is the discipline of experimenting on a system to build confidence in the system’s capability to withstand turbulent conditions in production. Rather than waiting for failures to happen, chaos engineering proactively injects failures to identify weaknesses before they impact users. Why does this matter? In modern distributed systems (microservices, cloud infrastructure, containers), failures are inevitable. A network can partition, a server can crash, a database can slow down. Traditional testing can’t predict all the ways these components interact when things go wrong. Chaos engineering fills this gap by deliberately causing failures in a controlled way. …

Introduction Infrastructure as Code (IaC) is how modern teams build reliable systems. Instead of manually clicking through cloud consoles or SSHing into servers, you define infrastructure in code—testable, version-controlled, repeatable. This guide shows you practical patterns for Terraform, Ansible, and Kubernetes with real examples, not just theory. Why Infrastructure as Code? Consider a production outage scenario: Without IaC: Database server dies You manually recreate it through AWS console (30 minutes) Forgot to enable backups? Another 15 minutes Need to reconfigure custom security groups? More time Total recovery: 2-4 hours Risk of missing steps = still broken With IaC: …

Executive Summary Layer 4 (Transport Layer) Load Balancing distributes traffic at the TCP/UDP level, before any application-level processing. Unlike Layer 7 (HTTP), L4 LBs don’t inspect request content—they simply route packets based on IP protocol data. When to use L4: Raw throughput requirements (millions of requests/sec) Non-HTTP protocols (gRPC, databases, MQTT, game servers) TLS passthrough (encrypted SNI unavailable) Extreme latency sensitivity When NOT to use L4: HTTP/HTTPS (use Layer 7 instead) Request-based routing (path-based, host-based) Simple workloads with <1M req/sec Fundamentals L4 vs L7: Quick Comparison Aspect Layer 4 (TCP/UDP) Layer 7 (HTTP/HTTPS) What it sees IP/port/protocol HTTP headers, body, cookies Routing based on Destination IP, port, protocol Host, path, query string, cookies Throughput Very high (millions pps) Lower (thousands rps) Latency <1ms typical 5-50ms typical Protocols TCP, UDP, QUIC, SCTP HTTP/1.1, HTTP/2, HTTPS, WebSocket Encryption Can passthrough TLS Can terminate/re-encrypt Best for Databases, non-HTTP, TLS passthrough Web apps, microservices, APIs Core Concepts Listeners: Defined by (protocol, port). Example: TCP:443, UDP:5353 …

Executive Summary Layer 7 (Application Layer) Load Balancing routes traffic based on HTTP/HTTPS semantics: hostnames, paths, headers, cookies, and body content. Unlike Layer 4, L7 LBs inspect and understand application protocols. When to use L7: HTTP/HTTPS workloads (99% of web apps) Host-based or path-based routing (SaaS multi-tenant) Advanced features: canary deployments, content-based routing API gateways with authentication/authorization WebSockets, gRPC, Server-Sent Events (SSE) When NOT to use L7: Non-HTTP protocols (use L4) Ultra-low latency (<5ms) with extreme throughput (use L4) Binary protocols (databases, Kafka) Fundamentals L7 vs L4: What L7 Adds Feature L4 L7 Visibility IP/port/protocol Full HTTP request/response Routing based on Destination IP, port Host, path, headers, cookies, body Request modification None Rewrite, redirect, compress TLS Passthrough only Terminate + re-encrypt Session affinity IP hash (crude) Sticky cookies, affinity headers Compression No Gzip/Brotli inline WebSockets Requires passthrough Native support gRPC Via TLS passthrough Native with trailers, keep-alives Rate limiting App-level only LB-level per path/host Auth App-level only OIDC, JWT, basic @ edge Throughput Millions RPS Thousands-millions RPS Latency <1ms 1-10ms Core L7 Concepts Listeners: HTTP port 80, HTTPS port 443 (often combined as single listener with TLS upgrade) …

Executive Summary Neo4j is a native graph database that stores data as nodes (entities) connected by relationships (edges). Unlike relational databases that normalize data into tables, Neo4j excels at traversing relationships. Quick decision: Use Neo4j for: Knowledge graphs, authorization/identity, recommendations, fraud detection, network topology, impact analysis Don’t use for: Heavy OLAP analytics, simple key-value workloads, document storage Production deployment: Kubernetes + Helm (managed) or Docker Compose + Causal Cluster (self-managed) …

Introduction Slow CI/CD pipelines waste developer time and delay releases. This guide covers proven techniques to optimize pipeline performance including build caching, parallel job execution, and efficient deployment strategies across popular CI/CD platforms. Build Caching Why Caching Matters Without caching: Build 1: npm install (5 min) → tests (2 min) = 7 min Build 2: npm install (5 min) → tests (2 min) = 7 min Build 3: npm install (5 min) → tests (2 min) = 7 min Total: 21 minutes With caching: …

Introduction The ELK stack (Elasticsearch, Logstash, Kibana) is powerful for log aggregation and analysis, but requires proper tuning for production workloads. This guide covers Elasticsearch index lifecycle management, Logstash pipeline optimization, and performance best practices. Elasticsearch Index Lifecycle Management (ILM) Understanding ILM ILM automates index management through lifecycle phases: Phases: Hot - Actively writing and querying Warm - No longer writing, still querying Cold - Rarely queried, compressed Frozen - Very rarely queried, minimal resources Delete - Removed from cluster Basic ILM Policy Create policy: …

Introduction GitOps is a paradigm that uses Git as the single source of truth for declarative infrastructure and applications. ArgoCD and Flux are the leading tools for implementing GitOps on Kubernetes. This guide covers deployment patterns, rollback strategies, and choosing between the two. GitOps Principles Core Concepts 1. Declarative - Everything defined in Git 2. Versioned - Git history = deployment history 3. Automated - Tools sync Git to cluster 4. Auditable - All changes tracked in Git …