Cloud Monitoring
You can't fix what you can't see. Cloud monitoring gives you visibility into your infrastructure, applications, and costs. Without it, you're flying blind — discovering problems only when users complain.
The Three Pillars of Observability
┌────────────────────────────────────────────────────────┐
│ OBSERVABILITY PILLARS │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ LOGS │ │ METRICS │ │ TRACES │ │
│ │ │ │ │ │ │ │
│ │ Event records│ │ Numeric data │ │ Request flow │ │
│ │ with context │ │ over time │ │ across │ │
│ │ │ │ │ │ services │ │
│ │ "What │ │ "How is │ │ "Where is │ │
│ │ happened" │ │ the system │ │ latency" │ │
│ │ │ │ performing" │ │ │ │
│ │ CloudWatch │ │ CloudWatch │ │ X-Ray │ │
│ │ Logs, │ │ Metrics, │ │ │ │
│ │ CloudTrail │ │ Prometheus │ │ Jaeger, │ │
│ │ │ │ │ │ Zipkin │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ Together they answer: What happened, why, and where? │
└────────────────────────────────────────────────────────┘
AWS CloudWatch
CloudWatch is AWS's native monitoring service. It collects metrics from every AWS resource, stores logs, and lets you set alarms. Think of it as the nervous system of your cloud environment — it detects and reports on everything.
┌──────────────────────────────────────────────────────┐
│ CLOUDWATCH COMPONENTS │
│ │
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ │
│ │ Metrics │ │ Logs │ │ Alarms │ │
│ │ │ │ │ │ │ │
│ │ CPU Util% │ │ Application│ │ Threshold │ │
│ │ Memory │ │ logs │ │ breached? │ │
│ │ Network I/O│ │ VPC Flow │ │ ┌────┐ │ │
│ │ Custom │ │ logs │ │──▶│SNS │ │ │
│ │ metrics │ │ CloudTrail │ │ └────┘ │ │
│ └────────────┘ └────────────┘ │ Send alert│ │
│ └────────────┘ │
│ │
│ ┌────────────┐ ┌────────────┐ │
│ │ Dashboards │ │ Events │ │
│ │ │ │ │ │
│ │ Visualize │ │ React to │ │
│ │ everything │ │ changes │ │
│ │ on one │ │ (auto- │ │
│ │ screen │ │ remediate)│ │
│ └────────────┘ └────────────┘ │
└──────────────────────────────────────────────────────┘
AWS CloudTrail
CloudTrail records every API call made in your AWS account. Who did what, when, and from where. It's your audit trail — essential for security investigations, compliance, and debugging access issues.
API Call Flow with CloudTrail:
User/Service ──▶ AWS API ──▶ CloudTrail ──▶ S3 Bucket
│ │
│ ▼
│ ┌──────────┐
│ │ Log │
│ │ File │
│ │ │
│ │ Event: │
│ │ RunInst- │
│ │ ances │
│ │ User: │
│ │ admin@co │
│ │ m │
│ │ Time: │
│ │ 2024-... │
│ │ SourceIP:│
│ │ 1.2.3.4 │
│ └──────────┘
▼
AWS performs action
Logging Strategy
A good logging strategy captures the right data at the right level. Too little logging and you can't debug issues. Too much and you're paying for noise. Aim for structured logs with consistent formats.
┌──────────────────────────────────────────────────┐
│ LOG LEVELS GUIDE │
│ │
│ Level When to Use Retention │
│ ───── ────────── ────────── │
│ ERROR Failures that need Long-term │
│ immediate attention │
│ │
│ WARN Unexpected but not Medium-term │
│ critical │
│ │
│ INFO Key business events Short-term │
│ and milestones │
│ │
│ DEBUG Detailed diagnostic Very short │
│ info (dev/staging) (hours/days) │
│ │
│ TRACE Everything Debug only │
│ │
│ Best Practice: │
│ Production: ERROR + WARN + INFO │
│ Staging: + DEBUG │
│ Dev: + TRACE │
└──────────────────────────────────────────────────┘
Alerting and Dashboards
Alerts notify you when something goes wrong. Dashboards show you the big picture. Together they keep you informed and in control. The key is setting alerts that matter — too many false alarms and you'll start ignoring them.
┌────────────────────────────────────────────────────┐
│ DASHBOARD EXAMPLE │
│ │
│ CPU Utilization Memory Usage Request Count │
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ │
│ │ ╱╲ │ │ ▂▃▅▇▅▃▂ │ │ ▁▁▁▁▁▁▁ │ │
│ │ ╱ ╲ │ │ │ │ ▁▁▁▁▁▁▁ │ │
│ │ ╱ ╲╱╲ │ │ │ │ ▁▁▁▁▁▁▁ │ │
│ │ ╱ ╲ │ │ │ │ ▁▁▁▁▁▁▁ │ │
│ └────────────┘ └────────────┘ └────────────┘ │
│ Current: 62% Current: 4.1GB Current: 1.2k│
│ Status: OK Status: OK Status: OK │
│ │
│ Error Rate Latency (p99) Cost Today │
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ │
│ │▁▁▁▁▁▁▁▁▁▁│ │ ╱───╲ │ │ $142.50 │ │
│ │▁▁▁▁▁▁▁▁▁▁│ │ ╱ ╲ │ │ Budget:$200│ │
│ │▁▁▁▁▁▁▁▁▁▁│ │ ╱ ╲│ │ ████████░░ │ │
│ └────────────┘ └────────────┘ └────────────┘ │
│ Current: 0.1% Current: 234ms Current: 71% │
│ Status: OK Status: OK Status: OK │
└────────────────────────────────────────────────────┘
Monitoring Best Practices
[x] Monitor the four golden signals: latency, traffic,
errors, and saturation
[x] Set up alerts for symptoms (high latency) not just
causes (high CPU)
[x] Use composite alarms to reduce noise (alert only when
multiple conditions are true)
[x] Store logs centrally (CloudWatch Logs, OpenSearch)
[x] Retain metrics for at least 13 months for trending
[x] Use synthetic monitoring to catch issues before users
[x] Tag everything so you can filter dashboards by
environment, service, or team
[x] Run chaos experiments to validate your monitoring
actually detects failures