murmurhash
Pyjion
Our great sponsors
murmurhash | Pyjion | |
---|---|---|
2 | 23 | |
42 | 1,407 | |
- | - | |
5.0 | 5.0 | |
6 months ago | 29 days ago | |
C++ | C++ | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
murmurhash
-
Is anyone using PyPy for real work?
If you have very large dicts, you might find this hash table I wrote for spaCy helpful: https://github.com/explosion/preshed . You need to key the data with 64-bit keys. We use this wrapper around murmurhash for it: https://github.com/explosion/murmurhash
There's no docs so obviously this might not be for you. But the software does work, and is efficient. It's been executed many many millions of times now.
-
Data Ingestion - Build Your Own "Map Reduce"?
Some notes: We don't need Sha256 and not evey base64; nothing will happen if keys will not distribute very equally. we could take MMH3; googling "python murmurhash" gives 2 interesting results; and since both use the same cpp code, let's take the one with most stars Other options would be to simply do (% NUM_SHARDS) or even shift right (however must have shards count == power of 2).
Pyjion
-
Python 3.13 Gets a JIT
It exists, was created by microsoft employees, and is referenced in the article: https://www.trypyjion.com/
-
Is anyone using PyPy for real work?
I've actually come across and started using Pyjion recently (https://github.com/tonybaloney/pyjion); how does Pypy compare, both in terms of performance and purpose? There seems to be a lot of overlap...
-
funAndEasyToUse
Python is capable of doing things at runtime that are really hard to statically compile around, such as monkeypatching methods onto existing objects. You can compile it, but it's complicated. One strategy is to use a JIT that can observe application state at runtime and then invalidate code as it becomes obsoleted by changes, but it's complicated. See pyjion for an example.
-
Javascript has Typescript. WHY WE DONT HAVE TYPY !
When I say "Python" I am referring to the standard CPython interpreter which most people use. But there is also PyPy, which includes a Just In Time compile that compiles selected code into machine language on the fly, as needed. pyjion is another JIT compiler that generates machine language on the fly, and you can install it with pip. Or you could work for Facebook and use Cinder. Cython, Nuitka and Pyston are other alternatives.
-
How is Golang websocket better than FastAPI websocket?
and if you need more speed you can try https://www.pypy.org/ or https://github.com/tonybaloney/Pyjion or https://www.pyston.org/
-
CPython vs PyPy
Finally, there is also Pyjion which based on its website is “A drop-in JIT Compiler for Python 3.10” (https://www.trypyjion.com/). We will be covering it on a separate writeup. See you next time ;-).
- Accelerate Python code 100x by import taichi as ti
- Create CPython extensions in .NET?
-
Instant upvotes
Though some exciting stuff happening over the next few years, Python is getting faster, has been for awhile, and stuff like Pyjion https://www.trypyjion.com/, a drop in C# powered JIT compiler is starting to approach usable. Rust and Python seem to be best buds right now, so more extension libraries in rust, a newer more approachable language than say C/C++ but with a similar speed. Sign me up!
-
You think python is slow ?
Pyjion Easy to use, small compiler. Increase performance of our 🐌 CPython.
What are some alternatives?
mmh3 - Python extension for MurmurHash (MurmurHash3), a set of fast and robust hash functions.
Numba - NumPy aware dynamic Python compiler using LLVM
mrjob - Run MapReduce jobs on Hadoop or Amazon Web Services
Nuitka - Nuitka is a Python compiler written in Python. It's fully compatible with Python 2.6, 2.7, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10, and 3.11. You feed it your Python app, it does a lot of clever things, and spits out an executable or extension module.
preshed - 💥 Cython hash tables that assume keys are pre-hashed
cinder - Cinder is Meta's internal performance-oriented production version of CPython.
python-mysql-replication - Pure Python Implementation of MySQL replication protocol build on top of PyMYSQL
graalpython - A Python 3 implementation built on GraalVM
sparc-curation - code and files for SPARC curation workflows
Cython - The most widely used Python to C compiler
psycopg2cffi - Port to cffi with some speed improvements
hpy - HPy: a better API for Python