One GeoParquet file, with its map inside
A genuine GeoParquet 2.0 file for analytics that also carries the map overviews a web map needs. The full-resolution geometry is stored once, in the Parquet rows, and read from there when you zoom in.
Open the mapCompare with two filesWhy
Cloud-native vector data usually ships as two files: a GeoParquet for queries and a PMTiles archive for the map. The tiles repeat every feature's geometry at every zoom, so the same parcels are stored twice, and the two files can drift apart.
The goal here is a single file that
- stays a standard GeoParquet: DuckDB, GDAL, QGIS, Arrow and any other reader see the same rows, columns, counts and sums;
- carries real map overviews, generalized the way a cartographer would: simplified, with small features kept visible so dense areas still read as dense;
- does not duplicate the full-resolution geometry: at close zooms, the map reads it from the Parquet rows.
The layout
PAR1 · row groups and page index · PMTiles archive · footer · PAR1
The PMTiles archive sits between the last data byte and the Parquet footer. Nothing before it moves, so every offset in the footer stays valid. The footer gets one more key-value entry:
pmtiles = {"version": 3, "offset": 616679114, "length": 182926855}
Parquet readers only follow the footer, so they never see the archive. A reader that knows the convention reads the end of the file, finds the entry and opens the archive at its offset. PMTiles offsets are relative to the start of the archive, so the archive is a byte-for-byte copy of a normal .pmtiles file.
How the map reads it
| Zoom | Where the map gets it |
|---|---|
| 6 to 13 | Baked PMTiles: parcels simplified for each zoom, tiny ones kept as small squares so cities stay dense. |
| 14 and closer | The Parquet rows: original geometry, turned into vector tiles in the browser. |
| Click | The Parquet rows: every attribute of the parcel, from the pages that contain the point. |
- One 64 KB read of the end of the file finds the
pmtilesentry. It is the last key-value of the footer, so the rest of the footer (about 430 KB here) is not needed to draw the map. - Overview zooms go through pmtiles.js with a source that adds the archive's offset to every range request. MapLibre works unchanged.
- Detail zooms come from a MapLibre custom protocol. A Web Worker reads the footer once, picks the pages whose bounding box overlaps the tile (from the page index of the
bboxcolumns), decodes them with hyparquet, reprojects, then clips and encodes the tile with geojson-vt and vt-pbf. Decoded pages are cached, so neighbouring tiles share them. - Attributes on click use the same page selection on a point: about 90 KB for a parcel.
Why zoom 14 is the dividing line
Below zoom 14, a view holds far more parcels than it can show. A dense zoom 13 view of Boston covers hundreds of thousands of full-resolution parcels, which the tiles show simplified and much lighter. From zoom 14, tiles are close to full resolution anyway, and they are the biggest zoom of a normal archive: 155 MB of 337 MB for these parcels. That zoom is what this file stops storing twice.
The data file
MassGIS Level 3 property tax parcels, statewide: 2,197,531 parcels in NAD83 / Massachusetts Mainland (EPSG:26986). To make the close-zoom reads tight, the rows were laid out for it:
- sorted along a Hilbert curve, with the parcels wider than 500 m (2% of rows) moved to the end, so they do not stretch the bounding box of every page around them;
- pages of 512 rows with a page index, so a reader can fetch a page, not a whole row group;
- no dictionary on near-unique identifiers, so a page read does not pull a dictionary as well;
- native Parquet GEOMETRY with its CRS (GeoParquet 2.0), zstd.
Readers tested
| Reader | On the file with its map inside |
|---|---|
| DuckDB 2.0 | Same count, sums and bbox-filtered queries as without the archive. |
| pyarrow | Same rows and sums; geo metadata intact. |
| GDAL (Parquet driver) | Same SQL results and geometries. |
| hyparquet | Same rows; the extra key-value is visible, nothing else. |
| Polars | Does not read GeoParquet 2.0 native geometry yet, with or without the archive. |
Removing the archive (a byte copy up to its offset, plus the original footer) gives back the plain GeoParquet byte for byte.
Numbers
| One file | Two files | |
|---|---|---|
| Storage | 800 MB | 840 MB (GeoParquet 503 + PMTiles z6-14 337) |
| First map (whole state) | 5 requests, 772 KB, plus one 64 KB read | 6 requests, 782 KB |
| Zooms 6 to 13 | Identical tiles and requests. | |
| Parcel attributes on click | about 90 KB | Only the columns kept in the tiles |
Measured with headless Chrome on a local server. Open the map and watch the network panel on the right for your own connection.
Open questions
- The page index costs space: 512-row pages make the GeoParquet 617 MB instead of 503 MB, which takes most of the 155 MB saved on zoom 14 tiles.
- Any tool that rewrites the file drops the archive. That is by design: the map is derived data, like an index.
- Where the line sits depends on the data. For points or small buildings, full resolution may pay off from a lower zoom.
Related work
The GeoParquet community is discussing overviews, and Cloud-Optimized GeoParquet orders a file's rows by level of detail so that the first row groups make a coarse map. This demo takes another path: the data keeps a purely spatial order for queries, and the overviews are real, generalized map tiles kept next to it in the same file.
Built with
tylertoo for the overview tiles, pyarrow and DuckDB for the layout, MapLibre GL JS, pmtiles.js, hyparquet, geojson-vt, vt-pbf and proj4js in the browser. Parcels: MassGIS (Bureau of Geographic Information), Commonwealth of Massachusetts. Basemap: Esri.