Datadog
APM & Distributed Tracing
Trace a request across services to find where latency and errors come from.
Datadog APM instruments your application to emit traces — a trace is the full journey of one request, made of spans where each span is one unit of work (an HTTP handler, a DB query, a cache call). Spans carry timing, the service and resource name, tags, and parent/child links, so a flame graph reconstructs exactly where time went.
A parcel tracking history that timestamps every depot the package passed through. When delivery is late, the timeline shows exactly which depot held it up rather than just "it was slow".
Key Concepts
1
Instrumentation is usually automatic: a per-language tracing library hooks common frameworks and clients with little or no code change. Datadog ties traces to the service catalog and to metrics generated from spans (request rate, error rate, latency percentiles — the "RED" signals), so you can move from a p99 latency alert straight into the slow traces behind it, then to the logs of that exact request via a shared trace ID.
2
What the interviewer is probing.
What the interviewer is probing.
3
1. "How does a trace get assembled across services?" *Probing:* the mechanism. *Stalls:* "The
agent." *Moves up:* a trace id propagated in headers, with each service emitting spans the backend
stitches by parent id.
1. "How does a trace get assembled across services?"
4
2. "What is a service map for?" *Probing:* the value. *Stalls:* "A picture." *Moves up:* showing
real dependencies and their error and latency rates, which is usually different from the
architecture diagram.
2. "What is a service map for?"
5
3. "What is trace sampling and why does it matter?" *Probing:* the cost angle. *Stalls:*
"Nothing." *Moves up:* keeping a fraction of traces to control cost, ideally retaining errors and
slow requests rather than sampling uniformly.
3. "What is trace sampling and why does it matter?"
6
4. "What breaks a trace most often?" *Probing:* experience. *Stalls:* "Bugs." *Moves up:* a hop
that does not propagate headers — a message queue, a thread pool, or a service not instrumented.
4. "What breaks a trace most often?"