mediapipe VS Pytorch

If you're looking for the differences in terms of how inference is done, I recommend you take a look at MediaPipe's source code. MediaPipe doesn't use raw code, but uses a "graph" instead (eg. pose_landmark_cpu.pbtxt), which can be visualised using MediaPipe Viz. I also used axinc-ai/ailia-models as the base (preprocessing, inference, postprocessing, etc...) which I further built upon (keypoint refinement, roi from keypoints, filtering / smoothing, etc...)

I thought of doing that but unfortunately medipipe requires a RGB input and performs better with it more on that here

You might want to check out https://github.com/open-mmlab/mmhuman3d and https://github.com/google/mediapipe

This architecture was also used in the link referenced when bringing up alternative implementations:

https://github.com/google/mediapipe/issues/877#issuecomment-...

mediapipe: runs in realtime > 20fps confirmed, training code is NOT provided

If you only care about data science and machine learning, then I would learn scikit-learn and PyTorch. Most companies and research groups have switched from Tensorflow to PyTorch, and Tensorflow itself replaced a number of frameworks before it (e.g., Caffe, Theano). I would also recommend reading An Introduction to Statistical Learning to get a basic understanding of different methods.

You might look into https://github.com/pytorch/pytorch/pull/95793.

When you said "build" pytorch I thought you meant(simplified): git clone https://github.com/pytorch/pytorch # get the source code

This is the master tracking list for MPS operator support: https://github.com/pytorch/pytorch/issues/77764

There is also some hope of things using the GPU on the M1/M2 as well. I did some testing and actually got it hooked up with some caveats. Not all PyTorch functions are mapped to work properly in the new MPS functionality Apple has provided so far. It looks like both PyTorch and Apple are working on things so this will improve. It also seems that the memory requirements of loading the models with GPU functionality are crazy high. That could be a side effect of the prototyping I did, but not sure. If you're interested, more detail can be found here.

Pytorch supports other kinds of accelerators (e.g. FPGA, and https://github.com/pytorch/glow), but unless you want to become a ML systems engineer and have money and time to throw away, or a business case to fund it, it is not worth it. In general, both pytorch and tensorflow have hardware abstractions that will compile down to device code. (XLA, https://github.com/pytorch/xla, https://github.com/pytorch/glow). TPUs and GPUs have very different strengths; so getting top performance requires a lot of manual optimizations. Considering the the cost of training LLM, it is time well spent.

While your millage (clearly) varies from mine, Anaconda is a de facto standard way to go in deep learning (and, generally, in most of the Python data science ecosystem).

For example, when you go to the front page of PyTorch (https://pytorch.org/), the default way to go is with Anaconda. It precisely makes it easy to install things regardless of the system and with matching versions. For example, out of box, it gives GPU support for Apple Silicon - not extra installation instructions.

Pip installers don't work with non-Python dependencies. Of course, you can manually install things any way you like (including inside Docker), but it is up to you to make sure that all dependencies are compatible. And it is a non-trivial task, given frequent updates of all things involved (including CUDA kernels, Python versions, PyTorch/TF versions, and all libraries related to them one way or the other).