Practical SQL for Data Analysis(what you can do without Pandas)

• vinum

  5 65 0.0 C++

  Vinum is a SQL processor for Python, designed for data analysis workflows and in-memory analytics.

Following similar observations I was wondering if one can actually execute SQL queries inside of Python process with the access to native Python functions and Numpy as UDFs. Thanks to Apache Arrow one can mix C++ and Python operators without need to copy the data and essentially combine DataFrame API with SQL, all while within the confines of the same Python process.

https://github.com/dmitrykoval/vinum

Vinum allows users to write queries which may invoke any Numpy or Python functions as UDFs available to the interpreter.

• q

  46 10,122 2.1 Python

  q - Run SQL directly on delimited files and multi-file sqlite databases (by harelba)

http://harelba.github.io/q/

q "SELECT COUNT(*) FROM ./clicks_file.csv WHERE c3 > 32.3"

It uses sqlite under the hood.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• siuba

  25 1,100 7.5 Python

  Python library for using dplyr like syntax with pandas and SQL

For what it's worth, I maintain a library called siuba that lets you generate SQL code from pandas methods.

It's crazy to me how people use SELECT * -> pandas, but also how people in SQL type a ton of code over and over.

https://github.com/machow/siuba

Suggest a related project