Explain Architecture — Forwarders, Indexers & Search Heads in Observability

1 / 4

Splunk

Architecture — Forwarders, Indexers & Search Heads

Understand how Splunk ingests, indexes, and searches machine data at scale.

Splunk is a platform for collecting and analysing machine data — logs, events, metrics — built around three tiers. Forwarders run on source machines and ship data to the indexing tier. Indexers receive that data, parse it into events, extract a timestamp, and write it to time-partitioned indexes on disk, where each event is also tokenised for fast search. Search heads provide the UI and coordinate searches, fanning a query out to all indexers (a map-reduce pattern) and merging the results.

A national postal sorting system: local offices (forwarders) send mail to regional sorting centres (indexers) that file everything by date, and an inquiry desk (search head) can ask every centre at once and combine the answers.

Key Concepts

At scale these tiers cluster: indexer clusters replicate data for availability and search-time performance, and search head clusters share configuration and load. Splunk's strength is schema-on-read: you index raw data with minimal upfront structure and apply field extraction at search time, so you can ask new questions of old data without re-ingesting. That flexibility is also why Splunk is associated with security (SIEM) and operational analytics, not just observability.

What the interviewer is probing.

What the interviewer is probing.

1. "What does each component do?" *Probing:* the structure. *Stalls:* "It searches." *Moves up:* forwarders collect and ship, indexers parse and store, search heads run searches and distribute them across indexers.

1. "What does each component do?"

2. "What is the difference between a universal and heavy forwarder?" *Probing:* the distinction. *Stalls:* "Size." *Moves up:* universal just ships raw data with minimal overhead; heavy can parse, filter and route before sending.

2. "What is the difference between a universal and heavy forwarder?"

3. "What is an index and why does it matter?" *Probing:* the detail. *Stalls:* "Storage." *Moves up:* the storage unit with its own retention and access control, which is how data separation and cost control are implemented.

3. "What is an index and why does it matter?"

4. "What drives Splunk licensing?" *Probing:* the commercial reality. *Stalls:* "Users." *Moves up:* daily ingest volume traditionally, so filtering at the forwarder directly reduces cost.

4. "What drives Splunk licensing?"