NetCDF/HDF5 treat data as a single file with complex internal pointers. Updating one time-step often requires rewriting the whole file or managing distributed locks. Zarr inverts this: it treats data as a directory of independent chunks on S3.
NetCDF is monolithic. You have a single file containing temperature measurements for every grid point, every altitude, and every time-step. Modify time-step 100? Traditional workflows either rewrite the entire file or use complex journaling. Parallel writes collide. Locking becomes the bottleneck.
Zarr explodes the array. The same dataset becomes a directory structure: temperature/0/0/0.bin, temperature/0/0/1.bin… each chunk is a separate object on S3. Metadata (dimensions, data types, compression) lives in lightweight JSON files. Data lives in independent binary chunks.
This changes parallelism fundamentally. A hundred machines can write a hundred different time-steps simultaneously. No coordination. No locking. Each chunk is a separate object—S3 handles the atomicity.
The query pattern determines chunking. Climate models chunk by time (fetch all space, one time-step). Satellite data cubes chunk by space (fetch all bands, one region). The same data, organized for different access patterns.
Zarr handles N-dimensional arrays natively: Time × Elevation × Latitude × Longitude. NetCDF forces you to manage those relationships implicitly. Zarr’s chunking strategy is explicit and optimizable.
The rule: If it has a time axis, it probably belongs in Zarr. Use COG for 2D imagery. Use Zarr for everything else—weather forecasts, climate models, hyperspectral time-series. NetCDF still works, but Zarr scales on the cloud.
Whenever you’re ready, here are 4 ways I can help you grow in GIS & spatial data:
Spatial Lab – My private community where GIS professionals, data engineers, and analysts connect, swap workflows, and build repeatable systems together.
Modern GIS Accelerator – A guided program to help you break out of legacy GIS habits and learn modern, scalable workflows.
Career Compass – A career-focused program designed to help GIS pros navigate the job market, sharpen their pitch, and find roles beyond traditional GIS paths.
Sponsorship: Interested in sponsoring this newsletter (or other content)? Learn more here and fill out the form to get in touch!
