Spatial data scattered across nodes by creation date or object ID looks reasonable on paper. In practice, a simple intersection query triggers a full cluster shuffle. Your distributed compute grinds to a halt.
Spatial partitioning is different. Traditional partitioning divides rows. Spatial partitioning groups geometries that actually intersect. If you partition by ID, nearby features scatter across machines. A spatial join between overlapping zones requires every machine talking to every other machine. Massive data shuffle. Expensive and slow.
Two approaches: Grid-based (H3, Geohash, A5 or fixed bounding boxes) divide space into cells. Simple. Predictable. Tree-based (Quadtree, R-Tree) dynamically partition based on data density. More complex. Better for skewed data (dense cities, sparse rural areas).
The boundary problem is real: a large polygon crossing multiple partition boundaries creates ambiguity. Duplicate it in all partitions it touches (cheap storage, simplifies logic). Or expand the partition envelope to contain it (cleaner, but uneven distribution).
Data skew is your hidden killer. A city covers 10% of your area but 70% of your features. A naive grid wastes compute on empty cells while the city partition stalls. Adaptive partitioning (Quadtree) handles this, but requires maintenance.
The rule: Partition for the spatial join, not just file size. If your intersection query shuffles the entire cluster, your partitions are wrong. Colocate geometries that frequently intersect. A spatial partition strategy is the difference between queries that work and queries that explode.
Whenever you’re ready, here are 4 ways I can help you grow in GIS & spatial data:
Spatial Lab – My private community where GIS professionals, data engineers, and analysts connect, swap workflows, and build repeatable systems together.
Modern GIS Accelerator – A guided program to help you break out of legacy GIS habits and learn modern, scalable workflows.
Career Compass – A career-focused program designed to help GIS pros navigate the job market, sharpen their pitch, and find roles beyond traditional GIS paths.
Sponsorship: Interested in sponsoring this newsletter (or other content)? Learn more here and fill out the form to get in touch!
