Datadog
Log Management & Pipelines
Ingest, parse, and control the cost of logs while keeping them searchable and correlated.
Datadog Log Management ingests logs from the Agent, cloud services, and libraries, then runs them through processing pipelines that parse raw text into structured attributes (status, duration, user, trace_id). Structured logs are what make full-text and faceted search fast and what enables one-click pivots between a log line and its trace.
A mailroom that opens every letter and files the important ones in fast-access drawers while boxing the rest in cheap off-site storage — retrievable later if an audit needs them.
Key Concepts
1
A key concept is the split between ingestion and indexing. Everything you send is ingested and can be processed, but you choose — via index filters and exclusion rules — which logs are actually indexed (made searchable and retained) versus archived cheaply to object storage. Patterns and Log Rehydration let you keep costs down while still being able to bring archived logs back when an investigation needs them.