Google Cloud
Observability — Cloud Operations Suite
Collect metrics, logs, and traces with Cloud Monitoring, Logging, and Trace to run reliable systems.
Three signals plus two extras, and a strong SRE opinion baked in.
An air-traffic control centre run by the people who wrote the safety manual (SRE): radar (metrics) flags anything off-target, the flight recorder (logs) holds every detail, the route tracker (traces) shows each request's path, and the whole operation is organised around explicit reliability targets rather than guesswork.
Key Concepts
1
Cloud Monitoring metrics, dashboards, alerting policies
Cloud Logging logs, queries, sinks, log-based metrics
Cloud Trace distributed latency tracing
Cloud Profiler continuous CPU and memory profiling in production
Error Reporting deduplicated exceptions2
Logging is structured by default on GCP services, so you query fields rather than matching strings:
3
resource.type="cloud_run_revision"
severity>=ERROR
jsonPayload.orderId="A-91"4
Log-based metrics turn a log pattern into a time series you can alert on — the bridge between the two signals.
5
Sinks route logs where they belong.
-> BigQuery for analysis
-> Cloud Storage for cheap long retention
-> Pub/Sub for downstream processing6
An aggregated sink at the organisation level writes audit logs to a separate project, so a project administrator cannot delete the evidence. That is a governance control, not just plumbing.
7
The SRE framing is the distinctive part. Monitoring has first-class SLOs: define an SLI, set a target, and alert on burn rate — how fast the error budget is being consumed — rather than on a raw threshold.
8
Burn-rate alerting is the idea worth being able to explain. A fast burn pages immediately; a slow burn raises a ticket. It stops the pager firing for a blip that does not threaten the month's budget, which is the usual cause of alert fatigue.
9
Alert on symptoms, not causes. High CPU may be fine; a rising error rate or p99 latency is what users feel. Percentiles over averages, always.
10
Cloud Profiler is genuinely cheap to run continuously and answers "which function is burning the CPU" in production rather than in a synthetic benchmark.
11
Set retention deliberately. Logs kept forever are a quiet permanent cost; the _Default bucket holds 30 days, and high-volume logs can be excluded or routed to cheaper storage.
_Default
12
What the interviewer is probing.1. "What is burn-rate alerting?" Probing: the SRE approach. Stalls: "Alerting on errors."
Moves up: alerting on how fast the error budget is being consumed — a fast burn pages, a slow burn
raises a ticket — so a blip that does not threaten the month does not wake anyone.
13
2. "Why route audit logs to a separate project?" Probing: the governance reason. Stalls:
"For organisation." Moves up: an aggregated sink at organisation level means a project
administrator cannot delete the evidence.
14
3. "What is a log-based metric for?" Probing: bridging the signals. Stalls: "Counting logs."
Moves up: turning a log pattern into a time series you can alert on, when the signal only exists
in log text.
15
4. "What does Cloud Profiler give you?" Probing: the extra pillar. Stalls: "Profiling in
testing." Moves up: continuous low-overhead CPU and memory profiling in production, answering
which function burns the CPU under real traffic.