Most spatial data lakes are just S3 buckets full of files with no coordination layer. Multiple analysts writing simultaneously? Corrupted data. Schema evolution? Manual rewrites. Time travel? Impossible. Iceberg fixes this by treating object storage as a database.
Iceberg is a table format—the metadata layer between object storage and compute. It’s not a database or storage engine. It’s a contract: “Here’s how to safely read and write data to this folder on S3.” Every read and write is tracked in metadata. Multiple writers never collide because Iceberg guarantees ACID transactions directly on cheap object storage.
Think of it this way: raw Parquet files are books scattered on a shelf. Iceberg is the librarian’s catalog that knows which books exist, their state, and who’s reading them right now.
Hidden partitioning prevents disasters. Traditional Hive-style partitioning forces users to know physical folder structure (year=2024/month=05/...). Users write bad queries that scatter reads across the entire bucket. Iceberg hides partitioning—the metadata layer handles it. A query “find features with owner=’Smith'” works without the analyst knowing how data is physically organized.
Zero-copy sharing across engines. Spark reads, Trino queries, Snowflake analyzes—all reading the same Iceberg table simultaneously without copying data. The format is standardized and platform-agnostic.
The rule: Stop managing folders; start managing tables. If multiple engines or analysts touch your spatial data, you need a table format. Iceberg brings database reliability to data lake economics.
Whenever you’re ready, here are 4 ways I can help you grow in GIS & spatial data:
Spatial Lab – My private community where GIS professionals, data engineers, and analysts connect, swap workflows, and build repeatable systems together.
Modern GIS Accelerator – A guided program to help you break out of legacy GIS habits and learn modern, scalable workflows.
Career Compass – A career-focused program designed to help GIS pros navigate the job market, sharpen their pitch, and find roles beyond traditional GIS paths.
Sponsorship: Interested in sponsoring this newsletter (or other content)? Learn more here and fill out the form to get in touch!
