Snowflake
Snowflake Architecture: Storage, Compute & Cloud Services
Understand Snowflake's unique multi-cluster shared data architecture and its three layers.
Snowflake uses a multi-cluster shared data architecture that physically separates three layers: storage, compute, and cloud services. This is fundamentally different from traditional MPP databases where storage and compute are tightly coupled on the same nodes.
Think of a library (storage) that many independent reading rooms (compute/warehouses) can access simultaneously — the librarian (cloud services) manages catalog lookups and access rules without any reading room ever locking another out of a book.
Key Concepts
1
The storage layer holds all data in compressed, columnar micro-partitions on cloud object storage (S3, Azure Blob, or GCS). This layer is completely decoupled from compute, meaning data is stored once but can be accessed by many independent compute clusters simultaneously without contention.
storage layermicro-partitions
2
The compute layer consists of virtual warehouses — independent MPP compute clusters that read from the shared storage layer. Each warehouse can be sized independently, scaled up or down, and multiple warehouses can query the same data concurrently without competing for resources.
compute layervirtual warehouses
3
The cloud services layer is the "brain" that ties everything together: it handles authentication, metadata management, query parsing and optimization, access control, and infrastructure management. This layer is fully managed by Snowflake and is why users never worry about indexes, vacuuming, or manual statistics.
cloud services layer
4
This separation is the reason Snowflake can offer near-infinite concurrency — a BI team, a data science team, and an ETL pipeline can all hit the same tables at the same time using different warehouses without ever blocking each other.