-
elevation_lookups
Takes an input file of paths described as series of points, outputs a file of data about the elevation changes along those paths.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
abstreet.org has an offline import process that combines data from OpenStreetMap, city-specific GIS datasets, and elevation into a single file. The process has to be deterministic, given the same input and code, and calling out to an external API is a non-starter. We use https://github.com/eldang/elevation_lookups, a Python library that downloads missing LIDAR or SRTM data and uses GDAL to handle batch queries. Two issues with it are having too many dependencies (so we run it in Docker) and not being able to parallelize lookups without blowing up memory, due to some GDAL caching internals.
Cool, I learned about abstreet the other day from HN! Thanks for the feedback.
These days I run everything geospatial in docker containers, the dependencies in geo are tricky.
Interesting about the memory/caching issues. I was going to suggest rasterio which I use for batch queries in https://github.com/ajnisbet/opentopodata and comes bundled with its own gdal binary, but looks like you're already using that.
I've also used zarr+tifffile for geotiffs in particular, it's faster and avoids a lot of gdal's warts, but you still need something like rasterio to read the geospatial metadata and handle projections.