The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →
Top 5 Python open-science Projects
-
pyvista
3D plotting and mesh analysis through a streamlined interface for the Visualization Toolkit (VTK)
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
pycbc
Core package to analyze gravitational-wave data, find signals, and study their parameters. This package was used in the first direct detection of gravitational waves (GW150914), and is used in the ongoing analysis of LIGO/Virgo data.
-
itk-wasm
High performance spatial analysis in a web browser, Node.js, and across programming languages and hardware architectures
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
Project mention: Observable 2.0, a static site generator for data apps | news.ycombinator.com | 2024-02-15I think the idea of Framework is really good, but static data limits the applications, excluding monitoring and other cases in which the data is constantly changing, but the dashboard can stay as it is. For example, I'd love to see a revamped Framework version of the LHC beam monitor and related pages (see https://op-webtools.web.cern.ch/vistar/, but check again in 2 months or so, when the accelerator will be running).
In high-energy physics, ROOT is /the/ toolkit for data analysis, and I guess jsROOT (https://root.cern.ch/js/) could also be used to load data to be shown in Framework dashboards. I thought the idea of Framework as a blogging engine with powerful data visualization built-in could be very interesting. Think, for example, about physicists pulling open data (https://opendata.cern.ch) and writing about their analysis or someone pulling data from https://ourworldindata.org/ in their own visualizations to support their case while writing about a particular subject, etc.
Project mention: Loading a trillion rows of weather data into TimescaleDB | news.ycombinator.com | 2024-04-16Why?
Most weather and climate datasets - including ERA5 - are highly structured on regular latitude-longitude grids. Even if you were solely doing timeseries analyses for specific locations plucked from this grid, the strength of this sort of dataset is its intrinsic spatiotemporal structure and context, and it makes very little sense to completely destroy the dataset's structure unless you were solely and exclusively to extract point timeseries. And even then, you'd probably want to decimate the data pretty dramatically, since there is very little use case for, say, a point timeseries of surface temperature in the middle of the ocean!
The vast majority of research and operational applications of datasets like ERA5 are probably better suited by leveraging cloud-optimized replicas of the original dataset, such as ARCO-ERA5 published on the Google Public Datasets program [1]. These versions of the dataset preserve the original structure, and chunk it in ways that are amenable to massively parallel access via cloud storage. In almost any case I've encountered in my career, a generically chunked Zarr-based archive of a dataset like this will be more than performant enough for the majority of use cases that one might care about.
[1]: https://cloud.google.com/storage/docs/public-datasets/era5
Python open-science related posts
- GraphCast: AI model for faster and more accurate global weather forecasting
- Are modern physicists dancing with the devil?
- Good Series, Tutorial, or Book on Particle Physics Analysis using Python or Root for Undergraduates
- Analysis-Ready, Cloud Optimized ERA5
- Why atheists behave so unscientific?
- Es ce que les données récoltées sont disponible au public ?
- [P] Official Imagen Website by Google Brain
-
A note from our sponsor - WorkOS
workos.com | 26 Apr 2024
Index
What are some of the best open-source open-science projects in Python? This list will help you:
Project | Stars | |
---|---|---|
1 | pyvista | 2,353 |
2 | opendata.cern.ch | 635 |
3 | pycbc | 294 |
4 | itk-wasm | 177 |
5 | arco-era5 | 175 |
Sponsored