Apache Kafka
Schema Registry & Evolution
Let producers and consumers evolve message formats independently without breaking each other.
In a long-lived event stream, producers and consumers are deployed at different times and evolve separately. If a producer changes a message format, old consumers can break. A Schema Registry solves this by storing versioned schemas (Avro, Protobuf, or JSON Schema) centrally and enforcing compatibility rules.
A shared API contract kept in version control with a linter: anyone can propose a new version, but the linter blocks a change that would break existing callers before it is merged.
Key Concepts
1
Messages carry a small schema id instead of the full schema, so payloads stay compact; consumers fetch and cache the schema by id to deserialize. The registry enforces a compatibility mode — backward (new consumers read old data), forward (old consumers read new data), or full — rejecting a producer that tries to register an incompatible schema.
2
This shifts format errors from runtime failures in production to deploy-time validation, which is the whole point.
3
What the interviewer is probing.
What the interviewer is probing.
4
1. "What problem does a schema registry solve?" *Probing:* the purpose. *Stalls:* "Validation."
*Moves up:* producers and consumers deploy independently, so a shared, versioned contract is what
stops a producer change breaking every consumer.
1. "What problem does a schema registry solve?"
5
2. "What is the difference between backward and forward compatibility?" *Probing:* the
precision. *Stalls:* "They are the same." *Moves up:* backward means new consumers read old data;
forward means old consumers read new data — which you need depends on deploy order.
2. "What is the difference between backward and forward compatibility?"
6
3. "Which change is safe in Avro?" *Probing:* the practical detail. *Stalls:* "Any." *Moves up:*
adding a field with a default, or removing one that had a default — adding a required field is not.
3. "Which change is safe in Avro?"
7
4. "What is stored in the message itself?" *Probing:* the mechanism. *Stalls:* "The schema."
*Moves up:* a schema id, not the schema — which is what keeps the payload small.
4. "What is stored in the message itself?"