Python CSV

Open-source Python projects categorized as CSV
Edit details
Topics: #Python #JSON #Python3 #Pandas #Excel
Collect and Analyze Billions of Data Points in Real Time
Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
www.influxdata.com
Sponsored

Top 23 Python CSV Projects

  • q

    q - Run SQL directly on delimited files and multi-file sqlite databases (by harelba)

    Project mention: I wrote this iCalendar (.ics) command-line utility to turn common calendar exports into more broadly compatible CSV files. | /r/commandline | 2023-03-24

    CSV utilities (still haven't pick a favorite one...): https://github.com/harelba/q https://github.com/BurntSushi/xsv https://github.com/wireservice/csvkit https://github.com/johnkerl/miller

  • datasette

    An open source multi-tool for exploring and publishing data

    Project mention: Seeking Help to Preserve Rare WWII Database on Windows 98 | news.ycombinator.com | 2023-09-18

    Looks like it's an Access database. Perhaps convert it to SQLite and publish with something like https://datasette.io/?

    I think the problem is, thread author doesn't know how to rip an ISO of the CD or move the database out; looks like they are getting help already though.

  • InfluxDB

    Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.

  • visidata

    A terminal spreadsheet multitool for discovering and arranging data

    Project mention: Fx – Terminal JSON Viewer | news.ycombinator.com | 2023-09-19

    [4] "Is it possible to "flatten" structured data (like JSON?)": https://github.com/saulpw/visidata/discussions/1605

  • csvkit

    A suite of utilities for converting to and working with CSV, the king of tabular file formats.

    Project mention: I wrote this iCalendar (.ics) command-line utility to turn common calendar exports into more broadly compatible CSV files. | /r/commandline | 2023-03-24

    CSV utilities (still haven't pick a favorite one...): https://github.com/harelba/q https://github.com/BurntSushi/xsv https://github.com/wireservice/csvkit https://github.com/johnkerl/miller

  • django-import-export

    Django application and library for importing and exporting data with admin integration.

    Project mention: Import or load a json into a database | /r/django | 2023-04-19

    django-import-export provides a sophisticated framework for importing data. Good if you need to do this on a regular basis and need to do some work on the data before writing to the database.

  • ethereum-etl

    Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ

  • pygraphistry

    PyGraphistry is a Python library to quickly load, shape, embed, and explore big graphs with the GPU-accelerated Graphistry visual graph analyzer

    Project mention: How to pass any first-round interview (even in a terrible talent market) | news.ycombinator.com | 2023-07-05

    I appreciate the good faith attempt:

    https://github.com/graphistry/pygraphistry

    And yes, we currently get used by data scientists and devs on problems like supply chain analysis, misinformation, cybersecurity, human trafficking. Seeing 100x+ more data than d3 and having a full env there makes their investigations easier. Our original tech helped lead to what is now Apache Arrow (we wrote the JS tier) and Nvidia RAPIDS (we wrote the precursor in js/opencl, and worked with Nvidia to restart for pydata), and are now focusing on the Nvidia Morpheus & graph AI sides for end-to-end GPU pipelines with our bigger customers (cyber, ...). To make this kind of tech easier for analysts, who are traditionally stuck with Splunk/Kibana/etc style UIs for investigations, we have been launching louie.ai with various customers. L

    Hopefully now it makes sense why we don't go far with candidates who can't have conversations on these things.

  • Mergify

    Tired of breaking your main and manually rebasing outdated pull requests?. Managing outdated pull requests is time-consuming. Mergify's Merge Queue automates your pull request management & merging. It's fully integrated to GitHub & coordinated with any CI. Start focusing on code. Try Mergify for free.

  • datamodel-code-generator

    Pydantic model and dataclasses.dataclass generator for easy conversion of JSON, OpenAPI, JSON Schema, and YAML data sources.

    Project mention: tRPC – Move Fast and Break Nothing. End-to-end typesafe APIs made easy | news.ycombinator.com | 2023-08-12

    Like generating pydantic models or dataclasses for an OpenAPI schema? I haven't needed to go in that direction myself, but this[0] looks promising!

    Apologies if I've misunderstood your comment

    https://koxudaxi.github.io/datamodel-code-generator/

  • JobFunnel

    Scrape job websites into a single spreadsheet with no duplicates.

  • DataProfiler

    What's in your data? Extract schema, statistics and entities from datasets

    Project mention: Data Profiler – What's in your data? | news.ycombinator.com | 2023-06-08

  • python-benedict

    :blue_book: dict subclass with keylist/keypath support, built-in I/O operations (base64, csv, ini, json, pickle, plist, query-string, toml, xls, xml, yaml), s3 support and many utilities.

    Project mention: What are the best Python libraries to learn for beginners? | /r/learnpython | 2023-01-30

    python-benedict: Dictionary manipulation library

  • pyexcel

    Single API for reading, manipulating and writing data in csv, ods, xls, xlsx and xlsm files

  • CleverCSV

    CleverCSV is a Python package for handling messy CSV files. It provides a drop-in replacement for the builtin CSV module with improved dialect detection, and comes with a handy command line application for working with CSV files.

    Project mention: Parquet: more than just "Turbo CSV" | /r/programming | 2023-04-03

    There’s things like this, but I consider the existence of messy, non standard CSV files (backed by a decade of experience dealing with the problem) a strong reason to not use the format ever.

  • Municipios-Brasileiros

    :house_with_garden: Dados relacionados aos municípios brasileiros

    Project mention: Dividindo o Brasil com base no paralelo 22°s por população | /r/brasil | 2022-11-27

    acochambrei rapidamente a tabela da estimativa 2021 (https://www.ibge.gov.br/estatisticas/sociais/populacao/9103-estimativas-de-populacao.html?=&t=resultados) usando as latitutes dessa base aqui: https://github.com/kelvins/Municipios-Brasileiros

  • finviz

    Unofficial API for finviz.com

    Project mention: Scraping Realtime Data from finviz | /r/algotrading | 2023-03-23

    https://github.com/mariostoev/finviz may be helpful to you

  • rows

    A common, beautiful interface to tabular data, no matter the format

  • csvs-to-sqlite

    Convert CSV files into a SQLite database

  • extract_otp_secrets

    Extract one time password (OTP) secrets from QR codes exported by two-factor authentication (2FA) apps such as "Google Authenticator". The exported QR codes from authentication apps can be captured by camera, read from images, or read from text files. The secrets can be exported to JSON or CSV, or printed as QR codes to console.

    Project mention: Google Online Security Blog: Google Authenticator now supports Google Account synchronization | /r/Android | 2023-04-24

    I used https://github.com/scito/extract_otp_secrets to export the keys from the Google Authenticator and imported them manually into Bitwarden.

  • pytablewriter

    pytablewriter is a Python library to write a table in various formats: AsciiDoc / CSV / Elasticsearch / HTML / JavaScript / JSON / LaTeX / LDJSON / LTSV / Markdown / MediaWiki / NumPy / Excel / Pandas / Python / reStructuredText / SQLite / TOML / TSV.

  • rainbow_csv

    🌈Rainbow CSV - Vim plugin: Highlight columns in CSV and TSV files and run queries in SQL-like language

    Project mention: Looking for two plugins for Log Analysis | /r/neovim | 2023-04-26

    Probably not an exact fit, but this plugin came to mind: rainbow_csv

  • sterraxcyl

    Instagram OSINT tool to export and analyse followers | following with their details

  • test-lists

    URL testing lists intended for discovering website censorship

  • pysonDB

    A Simple , ☁️ Lightweight , 💪 Efficent JSON based database for 🐍 Python. PysonDB-V2 has been released ⬇️

    Project mention: LiliDB (JSON-based database) | /r/Python | 2023-04-15

    Have a tl:dr how it differs / improves on pysonDB?

  • Sonar

    Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-09-19.

Python CSV related posts

Index

What are some of the best open-source CSV projects in Python? This list will help you:

Project Stars
1 q 9,937
2 datasette 8,272
3 visidata 6,804
4 csvkit 5,575
5 django-import-export 2,750
6 ethereum-etl 2,658
7 pygraphistry 1,906
8 datamodel-code-generator 1,834
9 JobFunnel 1,659
10 DataProfiler 1,253
11 python-benedict 1,152
12 pyexcel 1,137
13 CleverCSV 1,119
14 Municipios-Brasileiros 1,008
15 finviz 931
16 rows 848
17 csvs-to-sqlite 779
18 extract_otp_secrets 768
19 pytablewriter 568
20 rainbow_csv 526
21 sterraxcyl 407
22 test-lists 376
23 pysonDB 344

Sponsored
Write Clean Python Code. Always.
Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
www.sonarsource.com