Python Zarr

Open-source Python projects categorized as Zarr

Top 4 Python Zarr Projects

  1. zarr-python

    An implementation of chunked, compressed, N-dimensional arrays for Python.

  2. CodeRabbit

    CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

    CodeRabbit logo
  3. arco-era5

    Recipes for reproducing Analysis-Ready & Cloud Optimized (ARCO) ERA5 datasets.

    Project mention: Loading a trillion rows of weather data into TimescaleDB | news.ycombinator.com | 2024-04-16

    Why?

    Most weather and climate datasets - including ERA5 - are highly structured on regular latitude-longitude grids. Even if you were solely doing timeseries analyses for specific locations plucked from this grid, the strength of this sort of dataset is its intrinsic spatiotemporal structure and context, and it makes very little sense to completely destroy the dataset's structure unless you were solely and exclusively to extract point timeseries. And even then, you'd probably want to decimate the data pretty dramatically, since there is very little use case for, say, a point timeseries of surface temperature in the middle of the ocean!

    The vast majority of research and operational applications of datasets like ERA5 are probably better suited by leveraging cloud-optimized replicas of the original dataset, such as ARCO-ERA5 published on the Google Public Datasets program [1]. These versions of the dataset preserve the original structure, and chunk it in ways that are amenable to massively parallel access via cloud storage. In almost any case I've encountered in my career, a generically chunked Zarr-based archive of a dataset like this will be more than performant enough for the majority of use cases that one might care about.

    [1]: https://cloud.google.com/storage/docs/public-datasets/era5

  4. ome-zarr-py

    Implementation of next-generation file format (NGFF) specifications for storing bioimaging data in the cloud.

  5. zen3geo

    The 🌏 data science library you've been waiting for~

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Python Zarr discussion

Log in or Post with

Python Zarr related posts

  • GraphCast: AI model for faster and more accurate global weather forecasting

    2 projects | news.ycombinator.com | 14 Nov 2023
  • Open-source could finally get the world’s microscopes speaking the same language

    4 projects | news.ycombinator.com | 2 Oct 2023
  • Analysis-Ready, Cloud Optimized ERA5

    1 project | news.ycombinator.com | 21 Oct 2022

Index

What are some of the best open-source Zarr projects in Python? This list will help you:

# Project Stars
1 zarr-python 1,597
2 arco-era5 359
3 ome-zarr-py 172
4 zen3geo 81

Sponsored
CodeRabbit: AI Code Reviews for Developers
Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
coderabbit.ai

Did you know that Python is
the 2nd most popular programming language
based on number of references?