What's the Deal with Vector Tiles?

Over the last decade, Vector Tiles have become a hot trend in web mapping and GIS. CartoLab has been at the forefront of integrating them into custom web applications for our customers. Vector Tiles are an awesome way to display geographic data to the end-user. 

Vector Tiles, unlike older systems that send a series of static jpegs or png image files, send a series of binary files that allow for rendering maps with better detail at all zoom levels and include attribute information for the geometries. These older systems, such as Geoserver, use standards like the Web Mapping Service (WMS), where the data is returned as a bitmap image of some form.

Above: A wms image when zoomed in to greater than its default zoom level

These images, while often smaller in size than the binary pbf (Protocol Binary Format, what we’re actually referring when we discuss Vector Tiles) file, would only really look good at the zoom level that they were rendered for on the server; showing pixelation and image degradation if you attempted to zoom in on the image without retrieving more images. However, with the pbf’s in vector tiles, a single tile may replace any number of images at multiple zoom layers as long as the information in the pbf is sufficient for those zoom levels. Further, with older systems like WMS, you would have to query Web Feature Services (WFS) to get information about the geometry you’re interested in while with Vector Tiles attribute information is sent in the pbf and can be directly accessed

At CartoLab, we’re often confronted with very large (by GIS standards) data sets in a variety of formats, and it’s our job to display this massive data on a web map. When dealing with the web, speed and size are extremely important, not only to mitigate bandwidth costs but more importantly to enhance the end-user experience. We use a few different tools to produce Vector Tiles at CartoLab: Tippecanoe and The Mapbox API.

Tippecanoe

At CartoLab, we use Mapbox’s Tippecanoe extensively to generate vector tiles. Tippecanoe allows you to output custom Vector Tiles in the mbtiles format from JSON, geoJSON, geobuf, or csv files. It also allows you to name layers, decide on the minimum and maximum zoom levels for the tiles, and how the data is simplified or cleaned up for display on a map. 

The Mapbox API

Note: At the time of writing, June of 2020, the Mapbox uploads API is currently in beta, however, we’ve been able to use it to generate tilesets without issue using Python. 

The Mapbox API is feature-rich and exposes some powerful tools to developers. For the purpose of this article, I’m going to focus on the Uploads API Endpoint. The Uploads API Endpoint allows you to upload data to an AWS s3 bucket where the data can be further processed by Mapbox into a tileset. The Upload API can also be used to upload pre-generated Vector Tiles that you create in Mapbox.

Putting it All Together

Using Tippecanoe in conjunction with the Mapbox API allows us to automate and gain control over constantly changing data using vector tile creation pipelines written in Python. In the below example, we’ll use PostGIS, Python, Tippecanoe, and the Mapbox API to describe a fully automated workflow to update vector tiles served from Mapbox. 

Scenario and Solution

We have a statewide roads dataset for South Carolina that is updated by the state monthly, and we consume that into PostGIS as part of our ETL (Extract, Transform, Load) process. Each road segment is given a unique ID within our system, and any derived attributes or views are updated. We also want to make sure that our vector tiles are up to date with those unique ID’s so our web app clients can query information from them in the database.

We can use python with pandas or geopandas to export the updated roads table to a data frame using a standard SQL query. From there, we can do any further processing that needs to be done on the data, like converting date fields, and then export the data as geoJSON. 

Next, using the python subprocess module we can call Tippecanoe to format and produce our vector tiles. We can use a saved config file that the python script reads to always produce the vector tiles in the same way, every time. By using a static configuration file and passing the configuration information into the program, we achieve consistency in our final product.

Finally, we can directly upload the new vector tiles to Mapbox, replacing the road vector tiles from the previous month. In order to upload the new vector tiles, we again use Python with the Amazon Web Services SDK for Python (boto3) to upload the new tiles. If we were uninterested in the exact output of the vector tiles, we could have skipped the above Tippecanoe step and uploaded the geoJSON file instead. We upload the file by first retrieving authorization information from Mapbox for their s3 bucket. Once we have this temporary access information, we pass that information to our AWS s3 client to upload the vector tiles. After the file is sent to s3, we return the information from s3 back to Mapbox, where they will add the road vector tiles to our account.

Interested in finding out if vector tiles can help power your next web mapping application? Give us a shout and find out!