Executive Summary
Layer 7 (Application Layer) Load Balancing routes traffic based on HTTP/HTTPS semantics: hostnames, paths, headers, cookies, and body content. Unlike Layer 4, L7 LBs inspect and understand application protocols.
When to use L7:
- HTTP/HTTPS workloads (99% of web apps)
- Host-based or path-based routing (SaaS multi-tenant)
- Advanced features: canary deployments, content-based routing
- API gateways with authentication/authorization
- WebSockets, gRPC, Server-Sent Events (SSE)
When NOT to use L7:
- Non-HTTP protocols (use L4)
- Ultra-low latency (<5ms) with extreme throughput (use L4)
- Binary protocols (databases, Kafka)
Fundamentals
L7 vs L4: What L7 Adds
Feature | L4 | L7 |
---|---|---|
Visibility | IP/port/protocol | Full HTTP request/response |
Routing based on | Destination IP, port | Host, path, headers, cookies, body |
Request modification | None | Rewrite, redirect, compress |
TLS | Passthrough only | Terminate + re-encrypt |
Session affinity | IP hash (crude) | Sticky cookies, affinity headers |
Compression | No | Gzip/Brotli inline |
WebSockets | Requires passthrough | Native support |
gRPC | Via TLS passthrough | Native with trailers, keep-alives |
Rate limiting | App-level only | LB-level per path/host |
Auth | App-level only | OIDC, JWT, basic @ edge |
Throughput | Millions RPS | Thousands-millions RPS |
Latency | <1ms | 1-10ms |
Core L7 Concepts
Listeners: HTTP port 80, HTTPS port 443 (often combined as single listener with TLS upgrade)
Host-based routing:
Host: api.example.com β API backend
Host: web.example.com β Web frontend
Host: admin.example.com β Admin panel
Path-based routing:
/api/* β API service
/images/* β CDN or image backend
/static/* β Static content
Header-based routing:
X-User-Type: premium β Premium tier backend
X-Client-Version: 2.x β V2 API service
Weighted routing (Canary):
New version: 5% of traffic
Old version: 95% of traffic
β Monitor error rate, latency
β Gradually shift (5% β 10% β 50% β 100%)
TLS Termination:
Client TLS β[decrypt]β LB β[HTTP]β Backend
LB holds private key; backend sees plaintext HTTP
Pros: centralized cert mgmt, easy rotation
Cons: LB sees plaintext (inspect logs carefully!)
TLS Re-encryption:
Client TLS β[decrypt]β LB β[re-encrypt]β Backend TLS
End-to-end encryption; LB doesn't see plaintext
Pros: maximum security
Cons: higher CPU, backend must handle TLS
Session Affinity (Sticky Sessions):
Cookie-based:
Set-Cookie: SERVERID=backend-a; Path=/; HttpOnly
β All requests with this cookie β backend-a
Pros: works across zones
Cons: if backend-a dies, session lost
IP hash:
Hash(client IP) % num_backends = backend_a
Pros: stateless LB
Cons: doesn't work with mobile (IP changes), CDN
SNI (Server Name Indication):
TLS ClientHello includes hostname
β LB can route based on SNI before full TLS handshake
β Single port, multiple certs (multi-tenant SaaS)
Cloud Implementation Mapping
Feature | AWS ALB | Azure App GW | GCP GLB |
---|---|---|---|
Scope | Regional | Regional | Global (anycast) |
Protocols | HTTP/1.1, HTTP/2, HTTPS, gRPC | HTTP/1.1, HTTP/2, HTTPS | HTTP/1.1, HTTP/2, HTTPS, gRPC |
Host routing | β Listener rules | β Listener rules | β URL maps |
Path routing | β Listener rules | β Listener rules | β URL maps + backends |
Header routing | β HTTP header | β Custom headers | Limited (query string via NEGs) |
Weighted routing | β Target group weights | β Backend pool weights | β Backend service weights |
Session affinity | β Sticky cookies, duration | β Affinity (cookie/IP) | β Client IP, generated cookie |
WebSockets | β Native | β Native | β Native |
gRPC | β Native | β (use Traffic Manager) | β Native |
TLS termination | β | β | β |
TLS re-encryption | β | β Backend HTTPS | β Backend HTTPS |
WAF | AWS WAF (integrated) | Azure WAF (integrated) | Cloud Armor (separate) |
Auth offload | OIDC, JWT, basic | Basic only (API mgmt for OIDC) | None (Cloud Armor layer) |
Request rewrite | β Headers, path | β Headers, path, URL | Limited (URL rewrite via NEGs) |
Access logs | S3, CloudWatch | Application Insights | Cloud Logging |
Metrics | CloudWatch | Application Insights | Cloud Monitoring |
Quotas | 1000 rules/listener | 100 rules/listener | 15k backends |
Pricing | Hourly + LCU | Hourly + processed bytes | Hourly + forwarding rule + backend |
AWS Application Load Balancer (ALB)
Best for:
- HTTP(S) routing in AWS
- Microservices with path/host routing
- gRPC services
- Tightly integrated with ECS/EKS
Key strengths:
- Listener rules (up to 1000 per listener)
- Native HTTP/2, gRPC support
- Deep Lambda integration
- OIDC/JWT auth actions
- WAF native integration
Azure Application Gateway
Best for:
- HTTP(S) routing in Azure
- Backend HTTPS with certificates
- Advanced routing with path-based rules
- URL rewriting for legacy app compatibility
Key strengths:
- Backend TLS re-encryption (mutual TLS)
- URL rewrite engine (powerful routing)
- Path maps with prefix matching
- Integrated WAF
- Multi-site routing
GCP Global HTTP(S) Load Balancer
Best for:
- Global, anycast load balancing
- Edge-based traffic management
- Multi-region failover
- Integration with Cloud CDN
Key strengths:
- Global routing with local preference
- URL maps with highly flexible backend routing
- NEGs (Network Endpoint Groups) for extreme flexibility
- Cloud CDN for content caching
- Cloud Armor for WAF/DDoS
Kubernetes: Ingress vs Gateway API
Ingress (Legacy, Still Widely Used)
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
namespace: default
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/rate-limit: "10"
spec:
ingressClassName: nginx # or "alb", "gce"
tls:
- hosts:
- api.example.com
- web.example.com
secretName: app-tls # cert-manager will populate
rules:
- host: api.example.com
http:
paths:
- path: /api/v1
pathType: Prefix
backend:
service:
name: api-v1
port:
number: 8080
- path: /api/v2
pathType: Prefix
backend:
service:
name: api-v2
port:
number: 8080
- host: web.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web-frontend
port:
number: 3000
Limitations:
- Limited routing options (no header-based, no weighted)
- Not standardized across clouds
- Each cloud provider interprets differently
Gateway API (New Standard, Recommended)
---
apiVersion: v1
kind: Namespace
metadata:
name: app
---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: aws-alb
spec:
controllerName: elbv2.k8s.aws/elbv2
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: app-gateway
namespace: app
spec:
gatewayClassName: aws-alb
listeners:
- name: https
port: 443
protocol: HTTPS
tls:
mode: Terminate
certificateRefs:
- name: app-tls
allowedRoutes:
namespaces:
from: All
- name: http
port: 80
protocol: HTTP
allowedRoutes:
namespaces:
from: All
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: api-route
namespace: app
spec:
parentRefs:
- name: app-gateway
hostnames:
- api.example.com
rules:
- matches:
- path:
type: PathPrefix
value: /api/v1
backendRefs:
- name: api-v1
port: 8080
weight: 100
- matches:
- path:
type: PathPrefix
value: /api/v2
backendRefs:
- name: api-v2
port: 8080
weight: 100
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: canary-route
namespace: app
spec:
parentRefs:
- name: app-gateway
hostnames:
- canary.example.com
rules:
- matches:
- headers:
- name: X-Canary
value: "true"
backendRefs:
- name: app-new # New version
port: 8080
weight: 100
- backendRefs:
- name: app-stable # Old version
port: 8080
weight: 95
- name: app-new
port: 8080
weight: 5
---
apiVersion: v1
kind: Service
metadata:
name: api-v1
namespace: app
spec:
type: ClusterIP
selector:
app: api
version: v1
ports:
- port: 8080
targetPort: 8080
Advantages over Ingress:
- Standardized across clouds
- Rich routing (header-based, weighted, methods)
- Better for canary/blue-green
- Roles + RBAC for multi-tenant
- Gateway owned by infra team, HTTPRoute by app team
Security
TLS Policy
# AWS ALB: Minimum TLS 1.2
resource "aws_lb_listener" "https" {
load_balancer_arn = aws_lb.app.arn
port = 443
protocol = "HTTPS"
ssl_policy = "ELBSecurityPolicy-TLS-1-2-2017-01" # TLS 1.2+
certificate_arn = aws_acm_certificate.app.arn
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.backend.arn
}
}
# Azure Application Gateway: TLS 1.2+
resource "azurerm_application_gateway" "app" {
ssl_policy {
policy_type = "Predefined"
policy_name = "AppGwSslPolicy20170401S" # TLS 1.2+, strong ciphers
min_protocol_version = "TLSv1_2"
}
}
HSTS (HTTP Strict Transport Security)
# AWS ALB: Add HSTS header
resource "aws_lb_listener_rule" "hsts" {
listener_arn = aws_lb_listener.https.arn
priority = 1
action {
type = "forward"
target_group_arn = aws_lb_target_group.backend.arn
}
# Add HSTS header (Terraform resource doesn't expose this directly)
# Use response headers action (AWS feature)
action {
type = "fixed-response"
fixed_response_config {
content_type = "text/plain"
message_body = ""
status_code = "200"
headers = {
"Strict-Transport-Security" = "max-age=31536000; includeSubDomains; preload"
}
}
}
}
WAF (Web Application Firewall)
# AWS: WAF attached to ALB
resource "aws_wafv2_web_acl" "app" {
name = "app-waf"
description = "WAF for app ALB"
scope = "REGIONAL" # For ALB/API Gateway
default_action {
allow {}
}
# Block SQL injection
rule {
name = "AWSManagedRulesSQLiRuleSet"
priority = 1
action {
block {}
}
statement {
managed_rule_group_statement {
vendor_name = "AWS"
name = "AWSManagedRulesSQLiRuleSet"
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "SQLiRuleSetMetric"
sampled_requests_enabled = true
}
}
# Block XSS
rule {
name = "AWSManagedRulesKnownBadInputsRuleSet"
priority = 2
action {
block {}
}
statement {
managed_rule_group_statement {
vendor_name = "AWS"
name = "AWSManagedRulesKnownBadInputsRuleSet"
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "BadInputsMetric"
sampled_requests_enabled = true
}
}
# Rate limit: >2000 req/5min from single IP
rule {
name = "RateLimitRule"
priority = 3
action {
block {}
}
statement {
rate_based_statement {
limit = 2000
aggregate_key_type = "IP"
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "RateLimitMetric"
sampled_requests_enabled = true
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "AppWAFMetric"
sampled_requests_enabled = true
}
}
resource "aws_wafv2_web_acl_association" "alb" {
resource_arn = aws_lb.app.arn
web_acl_arn = aws_wafv2_web_acl.app.arn
}
mTLS to Backends
# AWS ALB β Backend with client cert (TLS re-encryption)
# Note: ALB doesn't support backend client certs directly
# Workaround: Use TargetType=IP with custom security group rules
# Or: Use Envoy/Nginx sidecar with mTLS
# Azure: Native backend HTTPS with cert
resource "azurerm_application_gateway" "app" {
backend_http_settings {
name = "https-settings"
cookie_based_affinity = "Disabled"
port = 8443
protocol = "Https"
request_timeout = 20
# Path to backend cert (client certificate for mTLS)
authentication_certificate {
name = "backend-cert"
}
}
}
Reliability & Performance
Multi-AZ Balancing
# AWS ALB: Enabled by default, costs inter-AZ data transfer
resource "aws_lb" "app" {
load_balancer_type = "application"
enable_cross_zone_load_balancing = true # Cost lever
subnets = [
aws_subnet.az_a.id,
aws_subnet.az_b.id,
aws_subnet.az_c.id
]
}
Connection Pooling & Keep-Alives
# Kubernetes: Service with keep-alive timeout
apiVersion: v1
kind: Service
metadata:
name: app-backend
annotations:
# Depends on ingress controller
# NGINX:
nginx.ingress.kubernetes.io/upstream-keepalive-connections: "100"
nginx.ingress.kubernetes.io/upstream-keepalive-timeout: "60"
nginx.ingress.kubernetes.io/upstream-keepalive-requests: "1000"
Queue Management (Surge Queue)
# Azure Application Gateway: Connection draining
resource "azurerm_application_gateway" "app" {
backend_http_settings {
connection_draining {
enabled = true
drain_timeout_sec = 30
}
}
}
Canary Deployment
# AWS ALB: Weighted target groups
resource "aws_lb_target_group" "app_stable" {
name = "app-stable"
port = 8080
protocol = "HTTP"
vpc_id = aws_vpc.main.id
}
resource "aws_lb_target_group" "app_canary" {
name = "app-canary"
port = 8080
protocol = "HTTP"
vpc_id = aws_vpc.main.id
}
resource "aws_lb_listener_rule" "canary_5_percent" {
listener_arn = aws_lb_listener.https.arn
priority = 10
action {
type = "forward"
forward {
target_group {
arn = aws_lb_target_group.app_stable.arn
weight = 95
}
target_group {
arn = aws_lb_target_group.app_canary.arn
weight = 5
}
}
}
condition {
path_pattern {
values = ["/api/*"]
}
}
}
Observability
Key Metrics
# AWS ALB CloudWatch Metrics
AWS/ApplicationELB:
RequestCount: # Total requests
TargetResponseTime: # Backend latency (p50/p95/p99)
HTTPCode_Target_2XX_Count: # Success responses
HTTPCode_Target_4XX_Count: # Client errors (app bug)
HTTPCode_Target_5XX_Count: # Server errors (app crash)
HTTPCode_ELB_5XX_Count: # LB errors (capacity?)
UnHealthyHostCount: # Unhealthy targets
ProcessedBytes: # Total bytes
NewConnectionCount: # New connections/period
ActiveConnectionCount: # Concurrent connections
# SLO example:
# - P99 latency: <500ms
# - Error rate: <0.1%
# - Availability: 99.95%
Access Logs
# AWS ALB: S3 access logs
resource "aws_lb" "app" {
access_logs {
bucket = aws_s3_bucket.logs.id
prefix = "alb-logs"
enabled = true
}
}
# Log format:
# http 2025-01-15T10:30:45.123456Z app-alb 10.0.1.100:12345 10.0.2.50:8080
# 0.123 0.456 0.000 200 200 156 342 "GET http://api.example.com:80/api/v1?token=*** HTTP/1.1"
# "Mozilla/5.0..." arn:aws:elasticloadbalancing:...
Tracing Headers
# AWS ALB: Add X-Amzn-Trace-Id for correlation
resource "aws_lb_listener_rule" "add_trace_header" {
listener_arn = aws_lb_listener.https.arn
priority = 1
action {
type = "forward"
target_group_arn = aws_lb_target_group.backend.arn
}
# ALB adds these by default:
# X-Amzn-Trace-Id: Root=1-abcd1234-5678efgh
# X-Forwarded-For: client-ip
# X-Forwarded-Proto: https
}
Cost Optimization
Pricing Models
AWS ALB:
- Hourly: ~$0.0225/hour ($16-17/month)
- LCU (Load Balancer Capacity Unit): processes bytes, connections, rules
- Data transfer: inter-AZ $0.01/GB, inter-region $0.02/GB
Azure Application Gateway:
- Hourly: per tier (Standard ~$0.20/hour, WAF ~$0.30/hour)
- Throughput: metered in GB
- WAF: additional $0.50/hour
GCP Global HTTP(S) LB:
- Per forwarding rule: $0.10-0.20/month
- Data: $0.12-0.30/GB (inter-region)
- Cloud CDN caching: reduces egress
Cost Reduction
# 1. Consolidate listeners/rules
# Instead of 10 ALBs (10 Γ $16 = $160/month)
# Use 1 ALB with 20 rules (~$17 + LCU for rules)
# 2. Disable cross-zone (if targets balanced per AZ)
enable_cross_zone_load_balancing = false # Saves ~50% inter-AZ costs
# 3. Use CDN for static content
resource "aws_cloudfront_distribution" "cdn" {
enabled = true
origin {
domain_name = aws_lb.app.dns_name
origin_id = "alb"
}
# CloudFront caches, reduces ALB requests/bytes
}
# 4. Right-size idle timeout
idle_timeout = 60 # Default 350s can accumulate stale connections
# 5. Delete unused target groups
# Each TG costs LCU
IaC: Terraform Examples
AWS ALB (HTTPβHTTPS Redirect + TLS + Path Routing)
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
}
provider "aws" {
region = "us-east-1"
}
# VPC & Subnets
resource "aws_vpc" "main" {
cidr_block = "10.0.0.0/16"
enable_dns_hostnames = true
}
resource "aws_subnet" "az_a" {
vpc_id = aws_vpc.main.id
cidr_block = "10.0.1.0/24"
availability_zone = "us-east-1a"
}
resource "aws_subnet" "az_b" {
vpc_id = aws_vpc.main.id
cidr_block = "10.0.2.0/24"
availability_zone = "us-east-1b"
}
# Internet Gateway
resource "aws_internet_gateway" "main" {
vpc_id = aws_vpc.main.id
}
resource "aws_route_table" "main" {
vpc_id = aws_vpc.main.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.main.id
}
}
resource "aws_route_table_association" "az_a" {
subnet_id = aws_subnet.az_a.id
route_table_id = aws_route_table.main.id
}
resource "aws_route_table_association" "az_b" {
subnet_id = aws_subnet.az_b.id
route_table_id = aws_route_table.main.id
}
# Security Groups
resource "aws_security_group" "alb" {
name = "alb-sg"
vpc_id = aws_vpc.main.id
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
ingress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_security_group" "backend" {
name = "backend-sg"
vpc_id = aws_vpc.main.id
ingress {
from_port = 8080
to_port = 8080
protocol = "tcp"
security_groups = [aws_security_group.alb.id]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
# ACM Certificate (self-signed for demo)
resource "tls_private_key" "app" {
algorithm = "RSA"
rsa_bits = 2048
}
resource "tls_self_signed_cert" "app" {
private_key_pem = tls_private_key.app.private_key_pem
subject {
common_name = "api.example.com"
organization = "Example Inc"
}
validity_period_hours = 8760 # 1 year
}
resource "aws_acm_certificate" "app" {
private_key = tls_private_key.app.private_key_pem
certificate_body = tls_self_signed_cert.app.cert_pem
}
# ALB
resource "aws_lb" "app" {
name = "app-alb"
load_balancer_type = "application"
security_groups = [aws_security_group.alb.id]
subnets = [aws_subnet.az_a.id, aws_subnet.az_b.id]
enable_cross_zone_load_balancing = true
access_logs {
bucket = aws_s3_bucket.logs.id
prefix = "alb"
enabled = true
}
}
# S3 bucket for logs
resource "aws_s3_bucket" "logs" {
bucket = "app-alb-logs-${data.aws_caller_identity.current.account_id}"
}
data "aws_caller_identity" "current" {}
# Target Groups
resource "aws_lb_target_group" "api_v1" {
name = "api-v1"
port = 8080
protocol = "HTTP"
vpc_id = aws_vpc.main.id
health_check {
healthy_threshold = 2
unhealthy_threshold = 2
timeout = 3
interval = 30
path = "/health"
}
}
resource "aws_lb_target_group" "api_v2" {
name = "api-v2"
port = 8080
protocol = "HTTP"
vpc_id = aws_vpc.main.id
health_check {
healthy_threshold = 2
unhealthy_threshold = 2
timeout = 3
interval = 30
path = "/health"
}
}
# HTTP Listener (redirect to HTTPS)
resource "aws_lb_listener" "http" {
load_balancer_arn = aws_lb.app.arn
port = 80
protocol = "HTTP"
default_action {
type = "redirect"
redirect {
port = "443"
protocol = "HTTPS"
status_code = "HTTP_301"
}
}
}
# HTTPS Listener
resource "aws_lb_listener" "https" {
load_balancer_arn = aws_lb.app.arn
port = 443
protocol = "HTTPS"
ssl_policy = "ELBSecurityPolicy-TLS-1-2-2017-01"
certificate_arn = aws_acm_certificate.app.arn
default_action {
type = "fixed-response"
fixed_response {
content_type = "text/plain"
message_body = "Not Found"
status_code = "404"
}
}
}
# Listener Rules (path-based routing)
resource "aws_lb_listener_rule" "api_v1" {
listener_arn = aws_lb_listener.https.arn
priority = 10
action {
type = "forward"
target_group_arn = aws_lb_target_group.api_v1.arn
}
condition {
path_pattern {
values = ["/api/v1/*"]
}
}
}
resource "aws_lb_listener_rule" "api_v2" {
listener_arn = aws_lb_listener.https.arn
priority = 20
action {
type = "forward"
target_group_arn = aws_lb_target_group.api_v2.arn
}
condition {
path_pattern {
values = ["/api/v2/*"]
}
}
}
# WAF
resource "aws_wafv2_web_acl" "app" {
name = "app-waf"
scope = "REGIONAL"
default_action {
allow {}
}
rule {
name = "RateLimitRule"
priority = 1
action {
block {}
}
statement {
rate_based_statement {
limit = 2000
aggregate_key_type = "IP"
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "RateLimitMetric"
sampled_requests_enabled = true
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "AppWAFMetric"
sampled_requests_enabled = true
}
}
resource "aws_wafv2_web_acl_association" "alb" {
resource_arn = aws_lb.app.arn
web_acl_arn = aws_wafv2_web_acl.app.arn
}
output "alb_dns_name" {
value = aws_lb.app.dns_name
}
Azure Application Gateway (With WAF)
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 3.0"
}
}
}
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "main" {
name = "app-rg"
location = "East US"
}
resource "azurerm_virtual_network" "main" {
name = "app-vnet"
address_space = ["10.0.0.0/16"]
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
}
resource "azurerm_subnet" "gateway" {
name = "gateway-subnet"
resource_group_name = azurerm_resource_group.main.name
virtual_network_name = azurerm_virtual_network.main.name
address_prefixes = ["10.0.1.0/24"]
}
resource "azurerm_subnet" "backend" {
name = "backend-subnet"
resource_group_name = azurerm_resource_group.main.name
virtual_network_name = azurerm_virtual_network.main.name
address_prefixes = ["10.0.2.0/24"]
}
resource "azurerm_public_ip" "main" {
name = "app-pip"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
allocation_method = "Static"
sku = "Standard"
}
# WAF Policy
resource "azurerm_web_application_firewall_policy" "main" {
name = "app-waf"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
policy_settings {
enabled = true
mode = "Prevention"
request_body_check = true
max_request_body_size_kb = 128
file_upload_limit_mb = 100
request_body_inspect_limit_kb = 128
}
managed_rules {
managed_rule_set {
type = "OWASP"
version = "3.1"
rule_group_override {
rule_group_name = "REQUEST-930-APPLICATION-ATTACK-LFI"
disabled_rules = []
}
}
}
}
# Application Gateway
resource "azurerm_application_gateway" "main" {
name = "app-appgw"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
sku {
name = "WAF_v2"
tier = "WAF_v2"
capacity = 2
}
gateway_ip_configuration {
name = "gateway-ip-config"
subnet_id = azurerm_subnet.gateway.id
}
frontend_port {
name = "http"
port = 80
}
frontend_port {
name = "https"
port = 443
}
frontend_ip_configuration {
name = "frontend-ip"
public_ip_address_id = azurerm_public_ip.main.id
}
http_listener {
name = "http-listener"
frontend_ip_configuration_name = "frontend-ip"
frontend_port_name = "http"
protocol = "Http"
}
http_listener {
name = "https-listener"
frontend_ip_configuration_name = "frontend-ip"
frontend_port_name = "https"
protocol = "Https"
ssl_certificate_name = "app-cert"
}
backend_address_pool {
name = "backend-pool"
}
backend_http_settings {
name = "http-settings"
cookie_based_affinity = "Enabled" # Sticky sessions
port = 8080
protocol = "Http"
request_timeout = 20
health_probes = [azurerm_application_gateway_probe.health.id]
}
probe {
name = "health-probe"
host = "127.0.0.1"
path = "/health"
interval = 30
timeout = 3
unhealthy_threshold = 3
}
request_routing_rule {
name = "http-to-https"
rule_type = "Basic"
http_listener_name = "http-listener"
redirect_configuration_name = "http-to-https"
}
request_routing_rule {
name = "https-rule"
rule_type = "Basic"
http_listener_name = "https-listener"
backend_address_pool_name = "backend-pool"
backend_http_settings_name = "http-settings"
}
redirect_configuration {
name = "http-to-https"
redirect_type = "Permanent"
target_listener_name = "https-listener"
include_path = true
include_query_string = true
}
ssl_certificate {
name = "app-cert"
data = filebase64("path/to/cert.pfx")
password = "cert-password"
}
waf_configuration {
enabled = true
firewall_mode = "Prevention"
rule_set_type = "OWASP"
rule_set_version = "3.1"
}
}
GCP Global HTTP(S) Load Balancer
terraform {
required_providers {
google = {
source = "hashicorp/google"
version = "~> 5.0"
}
}
}
provider "google" {
project = "my-project"
region = "us-central1"
}
resource "google_compute_network" "main" {
name = "app-network"
auto_create_subnetworks = false
}
resource "google_compute_subnetwork" "main" {
name = "app-subnet"
ip_cidr_range = "10.0.0.0/24"
region = "us-central1"
network = google_compute_network.main.id
}
# SSL Certificate
resource "google_compute_ssl_certificate" "main" {
name = "app-cert"
private_key = file("path/to/private.key")
certificate = file("path/to/cert.crt")
}
# Health Check
resource "google_compute_health_check" "main" {
name = "app-health"
description = "Health check for app backends"
http_health_check {
port = 8080
request_path = "/health"
}
}
# Backend Service
resource "google_compute_backend_service" "main" {
name = "app-backend"
protocol = "HTTP"
port_name = "http"
health_checks = [google_compute_health_check.main.id]
session_affinity = "CLIENT_IP"
backend {
group = google_compute_instance_group.main.id
balancing_mode = "RATE"
max_rate_per_endpoint = 100
}
log_config {
enable = true
}
}
# Instance Group
resource "google_compute_instance_group" "main" {
name = "app-ig"
zone = "us-central1-a"
instances = [] # Add instances later
named_port {
name = "http"
port = 8080
}
}
# URL Map
resource "google_compute_url_map" "main" {
name = "app-url-map"
default_service = google_compute_backend_service.main.id
host_rule {
hosts = ["api.example.com"]
path_matcher = "api-paths"
}
path_matcher {
name = "api-paths"
default_service = google_compute_backend_service.main.id
path_rule {
paths = ["/api/v1/*"]
service = google_compute_backend_service.main.id
}
path_rule {
paths = ["/api/v2/*"]
service = google_compute_backend_service.main.id
}
}
}
# HTTPS Target Proxy
resource "google_compute_target_https_proxy" "main" {
name = "app-https-proxy"
url_map = google_compute_url_map.main.id
ssl_certificates = [google_compute_ssl_certificate.main.id]
}
# Global Forwarding Rule (HTTPS)
resource "google_compute_global_forwarding_rule" "https" {
name = "app-https-forwarding-rule"
target = google_compute_target_https_proxy.main.id
port_range = "443"
ip_protocol = "TCP"
}
# Forwarding Rule (HTTP redirect)
resource "google_compute_target_http_proxy" "main" {
name = "app-http-proxy"
url_map = google_compute_url_map_redirect.main.id
}
resource "google_compute_url_map_redirect" "main" {
name = "app-redirect"
default_url_redirect {
https_redirect = true
redirect_response_code = "301"
strip_query = false
}
}
resource "google_compute_global_forwarding_rule" "http" {
name = "app-http-forwarding-rule"
target = google_compute_target_http_proxy.main.id
port_range = "80"
ip_protocol = "TCP"
}
output "ip_address" {
value = google_compute_global_forwarding_rule.https.ip_address
}
Kubernetes Gateway API (AWS ALB Controller)
---
apiVersion: v1
kind: Namespace
metadata:
name: app-prod
---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: aws-alb
spec:
controllerName: elbv2.k8s.aws/elbv2
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: app-gateway
namespace: app-prod
spec:
gatewayClassName: aws-alb
listeners:
- name: http
port: 80
protocol: HTTP
- name: https
port: 443
protocol: HTTPS
tls:
mode: Terminate
certificateRefs:
- name: app-tls
allowedRoutes:
namespaces:
from: All
---
apiVersion: v1
kind: Secret
metadata:
name: app-tls
namespace: app-prod
type: kubernetes.io/tls
data:
tls.crt: <base64-encoded-cert>
tls.key: <base64-encoded-key>
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: api-route
namespace: app-prod
spec:
parentRefs:
- name: app-gateway
sectionName: https
hostnames:
- api.example.com
rules:
- matches:
- path:
type: PathPrefix
value: /api/v1
backendRefs:
- name: api-v1-svc
port: 8080
weight: 100
timeouts:
request: 30s
backendRequest: 20s
- matches:
- path:
type: PathPrefix
value: /api/v2
backendRefs:
- name: api-v2-svc
port: 8080
weight: 100
---
# Canary: 5% to new version
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: canary-route
namespace: app-prod
spec:
parentRefs:
- name: app-gateway
sectionName: https
hostnames:
- api.example.com
rules:
- matches:
- headers:
- name: X-Canary
value: "true"
backendRefs:
- name: api-v3-svc-new
port: 8080
weight: 100
- backendRefs:
- name: api-v3-svc-stable
port: 8080
weight: 95
- name: api-v3-svc-new
port: 8080
weight: 5
---
apiVersion: v1
kind: Service
metadata:
name: api-v1-svc
namespace: app-prod
spec:
type: ClusterIP
selector:
app: api
version: v1
ports:
- port: 8080
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: api-v2-svc
namespace: app-prod
spec:
type: ClusterIP
selector:
app: api
version: v2
ports:
- port: 8080
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: api-v3-svc-stable
namespace: app-prod
spec:
type: ClusterIP
selector:
app: api
version: v3
state: stable
ports:
- port: 8080
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: api-v3-svc-new
namespace: app-prod
spec:
type: ClusterIP
selector:
app: api
version: v3
state: new
ports:
- port: 8080
targetPort: 8080
Architecture Diagrams
Edge β L7 β App
ββββββββββββββββββββββββββββ
β Client / CDN Cache β
β (CloudFront/Front Door) β
ββββββββββββββ¬βββββββββββββββ
β HTTPS (TLS)
ββββββββββββββΌβββββββββββββββ
β Layer 7 Load Balancer β
β (ALB, App GW, GLB) β
β Listener: 443 HTTPS β
β Rules: host/path/header β
ββββββ¬βββββββββββββ¬βββββββββ¬β
β β β
ββββββΌββ ββββββΌββ ββββΌβββββ
β API β β Web β βAdmin β
βv1 β βFront β βPanel β
β:8080 β β:3000 β β:3001 β
ββββββββ ββββββββ βββββββββ
Client request for: GET https://api.example.com/api/v1/users
β
LB inspects: Host=api.example.com, Path=/api/v1*
β
LB route rule: /api/v1* β Target Group "api-v1"
β
Forward to: api-v1 backend :8080
Canary & Blue/Green Deployment
βββββββββββββββββββββββ
β Client Requests β
ββββββββββββ¬βββββββββββ
β
ββββββββββββΌββββββββββββ
β L7 Load Balancer β
β Weighted LB Rules β
ββββββ¬βββββββββββββ¬βββββ
β β
Stable β 5% β β 95%
β β
ββββββΌβ βββββΌββββ
βNew β βStable β
βVer β βVer β
β3.x β β2.x β
β β β β
βCanaryβ βProd β
β:8080 β β:8080 β
ββββββββ βββββββββ
Gradual Canary Shift:
Hour 1: New=5%, Stable=95% (monitor errors, latency)
Hour 2: New=10%, Stable=90% (if healthy, increase)
Hour 3: New=25%, Stable=75%
Hour 4: New=50%, Stable=50% (50/50 split)
Hour 5: New=75%, Stable=25%
Hour 6: New=100%, Stable=0% (fully rolled out)
If error rate spikes at New=25%, rollback to New=0%
Best Practices Checklist
Before Deployment
- DNS CNAME points to LB (www.example.com β alb.example.com)
- TLS certificate validated (not expired, correct domains)
- WAF rules tested (false positive rate acceptable?)
- Routing rules documented (every host/path covered)
- Header/body limits set (prevent request bombs)
- Session affinity policy decided (stateless or sticky?)
- Access logs destination ready (S3 bucket, Log Analytics, Cloud Logging)
- Health check path verified (backend responds 200 OK)
- Timeout values reasonable (p99 latency < threshold)
- Security groups allow inbound 80, 443
After Deployment
- Canary test: Send 1% traffic, monitor errors
- Zero-downtime cutover: DNS CNAME switched (app traffic still works)
- Rollback plan documented (if issues, fall back to old LB in <5min)
- Metrics baseline: RPS, p50/p95/p99, error rate by code
- SLO alerts configured (p99 > 500ms? alert on-call)
- Access logs flowing (can troubleshoot incidents)
- Request tracing: X-Forwarded-For, X-Forwarded-Proto set correctly
- Canary weights monitored (errors/latency healthy before full rollout?)
Top Pitfalls to Avoid
Pitfall | Impact | Fix |
---|---|---|
Sticky session lock-in | User stuck on crashed backend | Use stateless sessions; cache in Redis |
Small header limit | Legitimate requests rejected (large cookies) | Increase to 128KB+ |
Small body limit | Large POST/PUT rejected (file uploads) | Set to 100MB+ if needed |
TLS misconfiguration | Browser security warning, clients reject | Use TLS 1.2+, strong ciphers |
Accidental public exposure | Security group allows 0.0.0.0/0 from internet | Restrict to known IPs / WAF |
gRPC/WebSocket timeout | Connection unexpectedly closed after 60s | Set timeout to match app needs (often 5-10min) |
No connection draining | In-flight requests killed during deploy | Set drain timeout to match p99 latency |
Wrong health check path | Backend always unhealthy, all traffic dropped | Match path to app’s health endpoint |
Cross-zone disabled | Targets in AZ-2 get no traffic if AZ-1 down | Enable cross-zone (understand cost) |
No canary before rollout | Buggy version hits 100% traffic instantly | Always test with 1-5% first |
Forgotten WAF bypass | Legitimate traffic blocked (API clients) | Whitelist internal IPs, use geo-policies |
Session affinity on stateless app | Uneven load distribution (one backend hot) | Disable affinity; scale with load |
FAQ
Q: When should I switch from L4 to L7? A: If you’re routing based on HTTP paths, hosts, or headers, or if you need sticky sessions or canary deployments, move to L7. L4 is for raw throughput / non-HTTP protocols.
Q: How do I handle TLS certificates? A: Use AWS ACM (free), Azure Key Vault, or GCP Managed Certs. For Kubernetes, use cert-manager with Let’s Encrypt. Always set up auto-renewal 30 days before expiry.
Q: Should I use sticky sessions? A: Only if your app requires it (e.g., sessions stored locally). Better: move sessions to external store (Redis, DynamoDB) so app stays stateless and scales.
Q: How do I do a blue/green deployment? A: Create two complete stacks (blue=old, green=new). LB weighted 100% β blue. Test green. Shift 1% β green (canary), then 100% β green once healthy.
Q: What if my backend needs mutual TLS? A: Use TLS re-encryption (Azure natively supports). AWS ALB doesn’t support backend client certsβuse sidecar proxy (Envoy, Nginx) with mTLS.
Q: How do I reduce LB costs?
A: Consolidate into one ALB with 20 rules ($17/mo) instead of 20 ALBs ($340/mo). Disable cross-zone if targets balanced per AZ. Use CDN for static content.
Q: What’s the difference between Ingress and Gateway API? A: Ingress is legacy, provider-specific. Gateway API is standardized, more features (weighted routing, header matching). Use Gateway API for new clusters.
Q: How do I preserve client IP in Kubernetes?
A: Use externalTrafficPolicy: Local
in Service. LB will not SNAT, so backend sees real client IP. Trade-off: requests might be unbalanced across nodes.
Conclusion
Layer 7 load balancing is essential for modern HTTP workloads. Key takeaways:
- Choose L7 for HTTP(S): You get routing, TLS termination, canary, WAF
- Plan your routing: Host/path/header rules should be documented upfront
- TLS matters: Use TLS 1.2+, cert auto-renewal, HSTS headers
- Test canary first: Never roll out 100% without testing 1-5% first
- Monitor observability: RPS, latency percentiles, error rates per endpoint
- Cost optimization: Consolidate listeners/rules, leverage CDN, disable cross-zone if safe
Further Reading: