Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge. Learn more →
Top 18 Python open-data Projects
-
CKAN
CKAN is an open-source DMS (data management system) for powering data hubs and data portals. CKAN makes it easy to publish, share and use data. It powers catalog.data.gov, open.canada.ca/data, data.humdata.org among many other sites.
CKAN The Open Source Data Portal Software
-
Project mention: NFS > FUSE: Why We Built Our Own NFS Server in Rust | news.ycombinator.com | 2023-09-19
> XetHub has the world’s first natively cross-platform, user-mode filesystem implementation, allowing you to mount arbitrarily large datasets on your machine.
Not really world's first. CERN has developed EOS (https://eos-web.web.cern.ch/) for many years, and even though it's not available natively on Windows, it is available on Linux and macOS. EOS uses FUSE, though, not NFS.
> This enables you to, in just a few seconds, locally mount ~660 GB of Llama 2 models or write DuckDB queries to analyze large parquet files and scan just the data you need.
If you mount all instances of EOS at CERN on your machine with the FUSE client, that in principle mounts hundreds of PB of data from LHC experiments, although much of this data requires special permissions to be accessed. However, there's also a lot of open data. See https://opendata.cern.ch/.
-
InfluxDB
Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
-
Probaj s: https://github.com/meteostat/meteostat-python
-
-
-
upgini
Data search & enrichment library for Machine Learning → Easily find and add relevant features to your ML pipeline from hundreds of public and premium external data sources optimized for ML models with LLMs and other NNs
Project mention: The fastest way to improve quality of ML model on tabular data | /r/learnmachinelearning | 2023-06-18web: https://upgini.com
-
-
Mergify
Tired of breaking your main and manually rebasing outdated pull requests?. Managing outdated pull requests is time-consuming. Mergify's Merge Queue automates your pull request management & merging. It's fully integrated to GitHub & coordinated with any CI. Start focusing on code. Try Mergify for free.
-
Project mention: Data? Where can I find the percentage of NYC Housing stock over 100 years old? | /r/AskNYC | 2023-03-08
-
-
PatZilla
PatZilla is a modular patent information research platform and data integration toolkit with a modern user interface and access to multiple data sources.
-
For anyone interested, here is how you import custom maps (obf files) to iOS using the Files app without iTunes: Download an obf file, for instance, from here or use OSMAnd Map Creator to make one: https://github.com/pnoll1/osmand_map_creation / https://wiki.openstreetmap.org/wiki/OsmAndMapCreator
-
open-grid-emissions
Tools for producing high-quality hourly generation and emissions data for U.S. electric grids
-
-
wikdict-gen
Generation of bilingual dictionaries from Wiktionary/dbnary data for the WikDict project
Project mention: Does anyone know of an API that lets you translate a word into multiple alternative translations? | /r/LanguageTechnology | 2022-12-08Also WikDict, though I like the results from Azure better: https://www.wikdict.com/
-
Bus-Departure-Board
A selection of Python programs which will retrieve live bus and rail UK open data and output it to a ER-OLEDM032 (256X64) display screen.
I think if you have a look at this; https://github.com/jfoot/Bus-Departure-Board, this should help you out!
-
-
tamato
The Tariff Management Tool (TaMaTo) stores and manages the tariffs and controls that are applied on imports and exports at the UK border. 🍅
Project mention: It is becoming difficult for me to be productive in Python | news.ycombinator.com | 2023-02-09https://github.com/uktrade/tamato/
This is a Django based tool to manage the tax rates you pay on any thing you might trade with the UK.
It can handle all the changes to the tax from the founding of the EU, through past Brexit when the UK has its own tariff as well.
Running it locally, it may not be straightforward to get hold of the right data (you can download it, but I don't think it's a turn key thing).
-
-
Sonar
Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
The latest post mention was on 2023-09-19.
Python open-data related posts
- How to import custom maps:obf files to iOS
- How to put my "custom" layer into OsmAnd while navigating
- Are modern physicists dancing with the devil?
- Addresses - New Brunswick Canada
- You can try Google Bard now
- What would you change about London?
- Good Series, Tutorial, or Book on Particle Physics Analysis using Python or Root for Undergraduates
-
A note from our sponsor - InfluxDB
www.influxdata.com | 22 Sep 2023
Index
What are some of the best open-source open-data projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | CKAN | 3,942 |
| 2 | opendata.cern.ch | 604 |
| 3 | meteostat-python | 292 |
| 4 | wetterdienst | 287 |
| 5 | UCF-SST-CitySim-Dataset | 278 |
| 6 | upgini | 249 |
| 7 | images | 174 |
| 8 | nycdb | 162 |
| 9 | Kotori | 94 |
| 10 | PatZilla | 83 |
| 11 | osmand_map_creation | 60 |
| 12 | open-grid-emissions | 49 |
| 13 | dashmap.io | 40 |
| 14 | wikdict-gen | 32 |
| 15 | Bus-Departure-Board | 32 |
| 16 | WarsawGTFS | 27 |
| 17 | tamato | 17 |
| 18 | wikdict-web | 11 |
Write Clean Python Code. Always.
Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
www.sonarsource.com