mup
yolov5
mup | yolov5 | |
---|---|---|
12 | 129 | |
1,186 | 47,202 | |
3.4% | 2.1% | |
2.7 | 8.8 | |
7 days ago | 5 days ago | |
Jupyter Notebook | Python | |
MIT License | GNU Affero General Public License v3.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
mup
-
Announcing xAI July 12th 2023
Our team is led by Elon Musk, CEO of Tesla and SpaceX. We have previously worked at DeepMind, OpenAI, Google Research, Microsoft Research, Tesla, and the University of Toronto. Collectively we contributed some of the most widely used methods in the field, in particular the Adam optimizer, Batch Normalization, Layer Normalization, and the discovery of adversarial examples. We further introduced innovative techniques and analyses such as Transformer-XL, Autoformalization, the Memorizing Transformer, Batch Size Scaling, and μTransfer. We have worked on and led the development of some of the largest breakthroughs in the field including AlphaStar, AlphaCode, Inception, Minerva, GPT-3.5, and GPT-4.
-
Bard is getting better at logic and reasoning
I believe tuning hyper parameters well without a lot of waste for the largest models was only figured out by Greg Yang/Microsoft Research around 2022 (cited in GPT-4 paper):
https://arxiv.org/abs/2203.03466
Also part of how they predicted the loss ahead of time so well.
-
Cerebras Open Sources Seven GPT models and Introduces New Scaling Law
This is the first time I have seen muP applied by the third party. See Cerebras Model Zoo, where muP models have scale-invariant constant LR.
-
OpenAI’s policies hinder reproducible research on language models
I guess, but its actually not simple to do that, in my experience. There’s another paper on that: https://arxiv.org/abs/2203.03466
Why isn’t chinchilla running google AI chat or whatever then?
-
[D] Anyone else witnessing a panic inside NLP orgs of big tech companies?
Well, but it isn't like this kind of research is new. Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer (2022) tuned hyperparameters in 40M model, transferred it to 6.7B model, and beat OpenAI's 6.7B run. It is likely what OpenAI did is perfecting this kind of research. I note that four authors of that paper (Igor Babuschkin, Szymon Sidor, David Farhi, Jakub Pachocki) are credited for pretraining optimization & architecture at https://openai.com/contributions/gpt-4.
-
[R] Greg Yang's work on a rigorous mathematical theory for neural networks
Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes: https://arxiv.org/abs/1910.12478 Tensor Programs II: Neural Tangent Kernel for Any Architecture: https://arxiv.org/abs/2006.14548 Tensor Programs III: Neural Matrix Laws: https://arxiv.org/abs/2009.10685 Tensor Programs IV: Feature Learning in Infinite-Width Neural Networks: https://proceedings.mlr.press/v139/yang21c.html Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer: https://arxiv.org/abs/2203.03466
- [D] How does one choose a learning rate schedule for models that take days or weeks to train?
- How to do meaningful work as an independent researcher? [Discussion]
-
DeepMind’s New Language Model,Chinchilla(70B Parameters),Which Outperforms GPT-3
I think there remains an immense amount of such suboptimality still hanging from the tree, so to speak.
For example, our recent paper "Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer"[1] shows that even learning rate and initialization used by existing models are deeply wrong. By just picking them correctly (which involves some really beautiful mathematics), we can effectively double the model size of the GPT-3 6.7B model (to be comparable in quality to the 13B model across the suite of benchmark tasks).
Large neural networks behave in a way we are only beginning to understand well just because each empirical probe of any such model is so much more expensive and time consuming than typical models. But principled theory here can have a lot of leverage by pointing out the right direction to look, as it did in our work.
[1] http://arxiv.org/abs/2203.03466
-
"Training Compute-Optimal Large Language Models", Hoffmann et al 2022 {DeepMind} (current LLMs are significantly undertrained)
On the hyperparameter front there seems to be some overlap with the recent hyperparameter transfer paper, which I get the impression Microsoft is going to try to scale, and which was referenced (and so is known) by the authors of this DeepMind paper. Which is to say, there's a good chance we'll be seeing models of this size trained with more optimal hyperparameters pretty soon.
yolov5
-
จำแนกสายพันธ์ุหมากับแมวง่ายๆด้วยYoLoV5
Ref https://www.youtube.com/watch?v=0GwnxFNfZhM https://github.com/ultralytics/yolov5 https://dev.to/gfstealer666/kaaraich-yolo-alkrithuemainkaartrwcchcchabwatthu-object-detection-3lef https://www.kaggle.com/datasets/devdgohil/the-oxfordiiit-pet-dataset/data
- How would i go about having YOLO v5 return me a list from left to right of all detected objects in an image?
-
Building a Drowsiness Detection Web App from scratch - pt2
!git clone https://github.com/ultralytics/yolov5.git ## Navigate to the model %cd yolov5/ ## Install requirements !pip install -r requirements.txt ## Download the YOLOv5 model !wget https://github.com/ultralytics/yolov5/releases/download/v6.0/yolov5s.pt
-
[Help: Project] Transfer Learning on YOLOv8
Specifically what I did was take the coco128.yaml, added 6 new classes from Dataset A (which have already been converted to YOLO Darknet TXT), from index 0-5 and subsequently adjusted the indices of the other COCO classes. The I proceeded to train and validate on Dataset A for 20 epochs.
-
Changing labels of default YOLOv5 model
I am using the default YOLOv5m6 model here with sahi/yolov5 library for my object detection project. I want to change just some of labels - for example when YOLO detects a human, I want it to label the human as "threat", not "person". Is there any way I can do it just changing some code, or I should train the model from scratch by just changing labels?
-
First time working with computer vision, need help figuring out a problem in my model
You should add them without annotations. Go through this.
-
AI Camera?
You are correct and if you check the firmware, it's yet another famous 3rd party project without attribution, namely https://github.com/ultralytics/yolov5
-
First non-default print on K1 - success
On one side, being a Linux user for 24 years now, it annoys me that they rip off code and claiming it as theirs again, thus violating licenses, but on the other thanks to k3d's exploit I'm able to tinker more with the machine and if needed do (selective) updates by hand then with a closed source system. It's not just "klipper", with klipper, fluidd and moonraker, it's also ffmpeg and mjpegstreamer. It's gonna be interesting since they also use a project that isn't just GPL, but APGL (in short "If your software gives service online, you have to publish the source code of it and any library that it borrows functions from.") - they use yolov5 (for AI).
- How does the background class work in object detection?
What are some alternatives?
com.openai.unity - A Non-Official OpenAI Rest Client for Unity (UPM)
mmdetection - OpenMMLab Detection Toolbox and Benchmark
NTK4A - Code for the paper: "Tensor Programs II: Neural Tangent Kernel for Any Architecture"
detectron2 - Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
gpt-3 - GPT-3: Language Models are Few-Shot Learners
darknet - YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
GP4A - Code for NeurIPS 2019 paper: "Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes"
Deep-SORT-YOLOv4 - People detection and optional tracking with Tensorflow backend.
cdx-index-client - A command-line tool for using CommonCrawl Index API at http://index.commoncrawl.org/
yolor - implementation of paper - You Only Learn One Representation: Unified Network for Multiple Tasks (https://arxiv.org/abs/2105.04206)
nn - 🧑🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠
OpenCV - Open Source Computer Vision Library