pymssql VS murmurhash

Install pymssql, connect to the database, read data from files and do a lot of inserts.

Support of vendors. For example, the MS SQL driver for Python is developed from community a nd for some time it was in a maintenance state because developer stopped the project (see this). Instead, the JDBC driver is developed directly from Microsoft

pymssql-utils is a small library that wraps pymssql with the aim to make your life easier, and your database admins slightly happier.

If you have very large dicts, you might find this hash table I wrote for spaCy helpful: https://github.com/explosion/preshed . You need to key the data with 64-bit keys. We use this wrapper around murmurhash for it: https://github.com/explosion/murmurhash

There's no docs so obviously this might not be for you. But the software does work, and is efficient. It's been executed many many millions of times now.

Some notes: We don't need Sha256 and not evey base64; nothing will happen if keys will not distribute very equally. we could take MMH3; googling "python murmurhash" gives 2 interesting results; and since both use the same cpp code, let's take the one with most stars Other options would be to simply do (% NUM_SHARDS) or even shift right (however must have shards count == power of 2).