Show HN: Query any kind of data with SQL powered by Python

• spyql

  23 902 0.0 Jupyter Notebook

  Query data on the command line with SQL-like SELECTs powered by Python expressions

I took a look to see how this works, expecting a super-complex parser/compiler - and it's mainly some straight-forward regular expression replacements! https://github.com/dcmoura/spyql/blob/4b5ea3ff52875d2c1a544f...

I'm very impressed - this is some very neat pragmatic software design.

• Lunar

  192 4,380 9.0 Swift

  Intelligent adaptive brightness for your external monitors

I love this tool!

I use it every day to insert monitor data from Lunar (https://lunar.fyi) into The Monitor Database (https://db.lunar.fyi)

I also use it occasionaly to insert data from my Meilisearch instances into Postgres/SQLite for more accurate filtering when needed.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• dsq

  20 3,638 4.3 Go

  Commandline tool for running SQL queries against JSON, CSV, Excel, Parquet, and more.

  

Nice work! I have a similar tool built on SQLite [0] and there are a number of similar tools (compared in my readme) like octosql, q, textql, datasette, etc.

Implementation-wise, is there a reason you wrote your own SQL parser rather than using an existing library? I ask not because I'd discourage handwritten SQL parsers but that I think the approach you took is going to suffer when it comes to error messages and lexical edge cases like `SELECT(SELECT 1)`.

But I can appreciate you wanting your own syntax especially for nested objects. In dsq I solved that by giving nested fields a literal column name with a dot in it. But that means you have to quote the column name (e.g. `SELECT "city.address" FROM {}`).

The additional penalty you'd pay not backing this by SQLite or DuckDB or some other in-memory database is in query performance as the input grows. Among other existing tools like this octosql has its own query engine but cube2222 has put a lot of time into optimizing it and it's also written in Go.

Whatever the case, competition is good! Happy to chat more if you want. I love database projects!

[0] https://github.com/multiprocessio/dsq

• PonyORM

  2 3,528 5.8 Python

  Pony Object Relational Mapper

  

See also:

https://github.com/ponyorm/pony/

https://macropy3.readthedocs.io/en/latest/pinq.html#pinq

• prosto

  9 89 3.6 Python

  Prosto is a data processing toolkit radically changing how data is processed by heavily relying on functions and operations with functions - an alternative to map-reduce and join-groupby

Having Python expressions within a declarative language is a really good idea because we can combine low level logic of computations of values with high level logic of set processing.

A similar approach is implemented in the Prosto data processing toolkit:

https://github.com/asavinov/prosto

Although Prosto is viewed as an alternative to Map-Reduce by relying on functions, it also supports Python User-Defined Functions in its Column-SQL:

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Suggest a related project