Snowflake
Snowflake Architecture: Storage, Compute & Cloud Services
Understand Snowflake's unique multi-cluster shared data architecture and its three layers.
Snowflake's whole design comes from one choice: the data does not live on the machines that query it.
Think of a library (storage) that many independent reading rooms (compute/warehouses) can access simultaneously — the librarian (cloud services) manages catalog lookups and access rules without any reading room ever locking another out of a book.
Key Concepts
1
In a traditional warehouse, each node owns a slice of the data on its own disks. Query capacity and storage
capacity are the same purchase, and two heavy queries fight over the same machines.
2
The three layers.
cloud services authentication, metadata, planning, access control
|
compute many independent clusters, sized separately
|
storage one copy of the data, in S3 / Blob / GCS3
Storage is one shared copy. Data sits in cloud object storage as compressed columnar files. It is
stored once, and every compute cluster reads the same files.
4
Compute is several separate clusters over that one copy. Each is a virtual warehouse. They do not share
anything with each other, so they cannot slow each other down:
5
ETL job -> warehouse LOAD_WH (large, runs at night)
BI dashboards -> warehouse BI_WH (small, business hours)
data science -> warehouse DS_WH (huge, on demand)6
All three read the same tables at the same moment. The overnight load cannot make the dashboards slow,
because they are not running on the same machines. In a coupled architecture that isolation needs a second
copy of the data.
7
Cloud services is the part you never see and always rely on. It authenticates you, holds all the
metadata, parses and plans the query, enforces access control, and manages the infrastructure.
8
The metadata is why Snowflake needs no indexes, no vacuum, and no manual statistics. This layer knows the
minimum and maximum of every column in every file, so it can eliminate most of the data before a warehouse
reads anything. There is nothing for you to tune because the tuning is metadata, not structures you build.
9
It is also what makes the result cache work across warehouses: the result belongs to the account, not to
the cluster that produced it.
10
What the separation actually buys. Scale compute without touching storage, and the reverse. Give each
team its own cluster without copying data. Suspend all compute overnight and pay only for storage. Resize a
warehouse for one heavy job and put it back.
11
And what it costs. Compute reads over the network, not from a local disk, so a cold query fetches from
object storage — mitigated by caching on the warehouse's local SSD, but real. And because compute is
billed by the second while running, an idle warehouse nobody suspended is a bill for nothing.
12
What the interviewer is probing.1. "What does separating storage from compute actually buy you?" Probing: the core idea.
Stalls: "Scalability." Moves up: they scale and are billed independently, so you can run several
warehouses on one copy of the data and suspend compute without losing anything.
13
2. "What does the cloud services layer do?" Probing: the third layer. Stalls: "Manages
things." Moves up: metadata, query planning, transactions, security and the result cache — it is
what makes zero-copy cloning and time travel metadata operations.
14
3. "Where is the data actually stored?" Probing: the detail. Stalls: "In Snowflake." *Moves
up:* in the cloud provider's object storage, in compressed columnar micro-partitions Snowflake owns
the format of.
15
4. "How do two teams avoid competing for resources?" Probing: the practical consequence.
Stalls: "Queues." Moves up: separate virtual warehouses over the same data — one team's heavy
query cannot slow the other's.