PyInstaller VS dh-virtualenv

Not sure if fixed in this patch, but pyinstaller had an issue in 3.12.0 https://github.com/pyinstaller/pyinstaller/issues/7992

You are right. I think I've misremembered the module name - it was uwsgi, not uvicorn.

This is a github issue where I discussed my original issue with PyInstaller devs - the dev explained the situation very well: https://github.com/pyinstaller/pyinstaller/issues/6362

PyInstaller seemed like the most maintained and developed tool to freeze python script into an executable, so I went with it. As expected, the freezed interface with the model was gigabytes large, so I had to figure out how to squeeze this. Fortunately, Onnx worked wonders and packaged the model into an inference only state, so I could throw away the Pytorch and Torchtext dependencies when freezing with Pyinstaller.Now the size of the executable with the model was 43MB instead of 4GB.

You might want to see Pyinstaller and auto_py_to_exe if you want a GUI interface.

It uses the pyinstaller command behind and please read their docs if you want to know more details.

First, instead of creating a VM for windows, you may need to use a software called Wine mentioned in the pyinstaller FAQs

are you running windows 7? here's a relevant issue i found on the pyinstaller github https://github.com/pyinstaller/pyinstaller/issues/4908

pyinstaller

I'm in a devops role where we actually reroll the Tensorflow whl in-house (to get a few tweaks like specific AVX flags turned on), but because the rest of our deployment is apt/debs, we then turn around and wrap that whl in a deb using Spotify's excellent dh-virtualenv:

https://github.com/spotify/dh-virtualenv

There's no expertise for Bazel in-house; when we run the build, it seems to fail all its cache hits and then spend 12-13h in total compiling, much of which appears to be recompiling a specific version of LLVM.

Every dependency is either vendored or pinned, including some critical things that have no ABI guarantees like Eigen, which is literally pinned to some a random commit, so that causes chaos when other binaries try to link up with the underlying Tensorflow shared objects:

https://github.com/tensorflow/tensorflow/blob/master/third_p...

And when you go down a layer into CUDA, there are even more support matrixes listing exact known sets of versions of things that work together:

https://docs.nvidia.com/deeplearning/tensorrt/support-matrix...

Anyway, I'm mostly just venting here. But the whole thing is an absurd nightmare. I have no idea how a normal distro would even begin to approach the task of unvendoring this stuff and shipping a set of normal packages for it all.