Our great sponsors
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ideas
-
How Many Lines of C It Takes to Execute a and B in Python?
Recent CPython development has been towards optimizations and addressing use cases that benefit from optimizations, some coming from the faster CPython initiative. You might just get your JIT[1].
[1] https://github.com/faster-cpython/ideas/wiki/Workflow-for-3....
-
GIL removal and the Faster CPython project
The faster-cpython folks seem to be working towards a JIT (https://github.com/faster-cpython/ideas/tree/main/3.13) and both pyston and cinder have JITs. So I don't think anyone has ruled one out.
You should look into the copy & patch efforts underway for Python[0]; an actual JIT will probably never exist but I think c&p has a shot of being mainlined in the next few years, such that Python could dynamically choose to either run the interpreter or a c&p option.
-
Our Plan for Python 3.13
faster-cpython team has done a lot of work to experiment on it: https://github.com/faster-cpython/ideas/issues/485#issuecomm...
It kind of sounds like migration to register based is a foregone conclusion, but it's not very clear to me.
-
Faster CPython at PyCon, part two
lots of big ideas are still remaining to be done. One example is the register based interpreter, see https://github.com/faster-cpython/ideas/issues/485
A previous plan called for the beginning of a JIT in 3.12, seen as "Trace optimized interpreter" here: https://github.com/faster-cpython/ideas/wiki/Workflow-for-3....
-
I started a repo to gather a collection of scripts that leverage programing language quirks that cause unexpected behavior. It's just so much fun to see the wheels turning in someone's head when you show them a script like this. Please send in a PR if you feel like you have a great example!
Bignums are heap-allocated and not deduplicated, so they cease having the same identity. One day CPython might do the same, but previous attempts have always stalled.
-
Python 3.11 Delivers
Guido himself is involved in the faster-cpython project though (which is responsible for these performance improvements).
-
Codon: A high-performance Python compiler
I got a massive jump in performance when moving from Python 3.8 to 3.10 (over some function call optimizations I think, based on the project). And 3.11 got even better (up to 50% faster on special cases, and 10~15% on average) with respect to 3.10. Python 3.12 is already getting even more speedups and a there's a lot more down the road[0].
But Python core developers value keeping "not breaking anyones code" (Python 3 itself was a huge trip on that aspect and they're not making that mistake again), that's why things may seem slow on their end. But work is being done, and the results are there if you benchmark things.
[0] See https://github.com/faster-cpython/ideas/blob/main/FasterCPyt... however that's over a year old already and I'm sure I've read/heard more specifics
-
A Register-Based Python Interpreter for Beter Performance
For what it's worth, the CPython core/Faster CPython developers are actively investigating implementations of this idea: https://github.com/faster-cpython/ideas/issues/485 .
Tile38
- Show HN: TG – Fast geometry library in C
-
PostgreSQL: No More Vacuum, No More Bloat
Experimental format to help readability of a long rant:
1.
According to the OP, there's a "terrifying tale of VACUUM in PostgreSQL," dating back to "a historical artifact that traces its roots back to the Berkeley Postgres project." (1986?)
2.
Maybe the whole idea of "use X, it has been battle-tested for [TIME], is robust, all the bugs have been and keep being fixed," etc., should not really be that attractive or realistic for at least a large subset of projects.
3.
In the case of Postgres, on top of piles of "historic code" and cruft, there's the fact that each user of Postgres installs and runs a huge software artifact with hundreds or even thousands of features and dependencies, of which every particular user may only use a tiny subset.
4.
In Kleppmann's DDOA [1], after explaining why the declarative SQL language is "better," he writes: "in databases, declarative query languages like SQL turned out to be much better than imperative query APIs." I find this footnote to the paragraph a bit ironic: "IMS and CODASYL both used imperative query APIs. Applications typically used COBOL code to iterate over records in the database, one record at a time." So, SQL was better than CODASYL and COBOL in a number of ways... big surprise?
Postgres' own PL/pgSQL [2] is a language that (I imagine) most people would rather NOT use: hence a bunch of alternatives, including PL/v8, on its own a huge mass of additional complexity. SQL is definitely "COBOLESQUE" itself.
5.
Could we come up with something more minimal than SQL and looking less like COBOL? (Hopefully also getting rid of ORMs in the process). Also, I have found inspiring to see some people creating databases for themselves. Perhaps not a bad idea for small applications? For instance, I found BuntDB [3], which the developer seems to be using to run his own business [4]. Also, HYTRADBOI? :-) [5].
6.
A usual objection to use anything other than a stablished relational DB is "creating a database is too difficult for the average programmer." How about debugging PostgreSQL issues, developing new storage engines for it, or even building expertise on how to set up the instances properly and keep it alive and performant? Is that easier?
I personally feel more capable of implementing a small, well-tested, problem-specific, small implementation of a B-Tree than learning how to develop Postgres extensions, become an expert in its configuration and internals, or debug its many issues.
Another common opinion is "SQL is easy to use for non-programmers." But every person that knows SQL had to learn it somehow. I'm 100% confident that anyone able to learn SQL should be able to learn a simple, domain-specific, programming language designed for querying DBs. And how many of these people that are not able to program imperatively would be able to read a SQL EXPLAIN output and fix deficient queries? If they can, that supports even more the idea that they should be able to learn something different than SQL.
----
2: https://www.postgresql.org/docs/7.3/plpgsql-examples.html
-
Your Data Fits in RAM
I actually worked on a project that did this. We used a database called "Tile38" [1] which used an R-Tree to make geospatial queries speedy. It was pretty good.
Our dataset was ~150 GiB, I think? All in RAM. Took a while to start the server, as it all came off disk. Could have been faster. (It borrowed Redis's query language, and its storage was just "store the commands the recreate the DB, literally", IIRC. Dead simple, but a lot of slack/wasted space there.)
Overall not a bad database. Latency serving out of RAM was, as one should/would expect, very speedy!
[1]: https://tile38.com/
-
Redcon - Redis compatible server framework for Rust
I ported it from Go and use it for my Tile38 project.
- Path hints for B-trees can bring a performance increase of 150% – 300%
What are some alternatives?
vitess - Vitess is a database clustering system for horizontal scaling of MySQL.
go-mysql-elasticsearch - Sync MySQL data into elasticsearch
ledisdb - A high performance NoSQL Database Server powered by Go
goleveldb - LevelDB key/value database in Go.
groupcache - groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.
InfluxDB - Scalable datastore for metrics, events, and real-time analytics
Nuitka - Nuitka is a Python compiler written in Python. It's fully compatible with Python 2.6, 2.7, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10, and 3.11. You feed it your Python app, it does a lot of clever things, and spits out an executable or extension module.
kingshard - A high-performance MySQL proxy
goqu - SQL builder and query library for golang
prometheus - The Prometheus monitoring system and time series database.
dgraph - The high-performance database for modern applications
sqlhooks - Attach hooks to any database/sql driver