🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
Could this work well with distributed solutions like petals?
I don't understand how petals can work though. I thought LLMs were typically quite monolithic.
An Open Source Machine Learning Framework for Everyone
Right but that's not an inherent GPU determinism issue. It's a software issue.
https://github.com/tensorflow/tensorflow/issues/3103#issueco... is correct that it's not necessary, it's a choice.
Your line of reasoning appears to be "GPUs are inherently non-deterministic don't be quick to judge someone's code" which as far as I can tell is dead wrong.
Admittedly there are some cases and instructions that may result in non-determinism but they are inherently necessary. The author should thinking carefully before introducing non-determinism. There are many scenarios where it is irrelevant, but ultimately the issue we are discussing here isn't the GPU's fault.
Learn any GitHub repo in 59 seconds. Onboard AI learns any GitHub repo in minutes and lets you chat with it to locate functionality, understand different parts, and generate new code. Use it for free at www.getonboard.dev.
🤖 A PyTorch library of curated Transformer models and their composable components
Yeah. In curated transformers  we are seeing completely deterministic output across multiple popular transformer architectures on a single GPU (there can be variance between GPUs due to different kernels).
One non-determinism we see with a temperature of 0 is that once you have quantized weights, many predicted pieces will have the same probability, including multiple pieces with the highest probability. And then the sampler (if you are not using a greedy decoder) will sample from those pieces.
In other words, a temperature of 0 is a poor man’s greedy decoding. (It is totally possible that OpenAI’s implementation switches to a greedy decoder with a temperature of 0).
Is it even possible to design a ML model without using Python or MATLAB? Like using C++, C or Java?
1 project | /r/learnprogramming | 19 Jun 2023
How to do deep learning with Caffe?
4 projects | /r/SubSimulatorGPT2 | 6 Jun 2023
When the documentation has TODOs
1 project | /r/programminghorror | 3 Jun 2023
Deep Learning as Computational Graphs (2018)
1 project | news.ycombinator.com | 22 May 2023
The most sane TensorFlow user
1 project | /r/ProgrammerHumor | 17 May 2023