minGPT
darts
minGPT | darts | |
---|---|---|
35 | 47 | |
19,037 | 7,366 | |
- | 2.8% | |
0.0 | 9.1 | |
23 days ago | 5 days ago | |
Python | Python | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
minGPT
- FLaNK AI Weekly for 29 April 2024
-
Ask HN: Daily practices for building AI/ML skills?
minGPT (Karpathy): https://github.com/karpathy/minGPT
Next, some foundational textbooks for general ML and deep learning:
-
[D] What are some examples of being clever with batching for training efficiency?
Language Model novice here. I was going through the README section of minGPT and read this line.
-
LLM Visualization: 3D interactive model of a GPT-style LLM network running inference.
The first network displayed with working weights is a tiny such network, which sorts a small list of the letters A, B, and C. This is the demo example model from Andrej Karpathy's minGPT implementation.
- LLM Visualization
- Learn Machine Learning
-
Facebook Prophet: library for generating forecasts from any time series data
Tried it once. Its promise is to take the dataset's seasonal trend into account, which makes sense for Facebook's original use case.
We ran it on such a dataset and found out that directly using https://github.com/karpathy/minGPT consistently gives a better result. So we ended up using the output of Prophet as an input feature to a neural network, but the result was not improved in any significant way.
-
Tokenization of numerical series
Sure, im trying to regenerate a bunch of complex numbers based on their absolute value. So im trying to embed these absolute values and then using gpt model(probably mini gpt) try to recover the original comples numbers. There is a certain connection between these complex numbers and their order which im not capable of explaining yet. Im hoping the model would be capable of recognizing certain sequences of these absolute values and match them with the desired complex counterparts (by training the model).
-
Anyone know of any articles on training a LLM from scratch on a single GPU?
minGPT (https://github.com/karpathy/minGPT)
-
Understanding LLMs(to the best of our knowledge)
Check out minGPT and nanoGPT from Karpathy, he puts out some of the best machine learning tutorials and teaching content.
darts
- Darts: Python lib for forecasting and anomaly detection on time series
-
[D] Doubts on the implementation of LSTMs for timeseries prediction (like including weather forecasts)
Don't use an LSTM. Get up to date with SoTA methods and read the papers in the field. LSTMs are not the way forward. Read the papers I suggested. It would be very useful to come to grips with both the Time Series Repository (https://github.com/thuml/Time-Series-Library) and Darts (https://github.com/unit8co/darts) as these are widely used for research and in industry.
-
Facebook Prophet: library for generating forecasts from any time series data
As others have pointed out, Prophet is not a particularly good model for forecasting, and has been superseded by a multitude of other models. If you want to do time series forecasting, I'd recommend using Darts: https://github.com/unit8co/darts. Darts implements a wide range of models and is fairly easy to use.
The problem with time series forecasting in general is that they make a lot of assumptions on the shape of your data, and you'll find you're spending a lot of time figuring out mutating your data. For example, they expect that your data comes at a very regular interval. This is fine if it's, say, the data from a weather station. This doesn't work well in clinical settings (imagine a patient admitted into the ER -- there is a burst of data, followed by no data).
That said, there's some interesting stuff out there that I've been experimenting with that seems to be more tolerant of irregular time series and can be quite useful. If you're interested in exchanging ideas, drop me a line (email in my profile).
-
Elevate Your Python Skills: Machine Learning Packages That Transformed My Journey as ML Engineer
3. darts
-
Aeon: A unified framework for machine learning with time series
Looking forward to checking this out! How does this compare with darts[1]?
[1] https://unit8co.github.io/darts/
- [D] Hybrid forecasting framework ARIMA-LSTM
-
[D] Do any of you have experience using Darts for forecasting?
Darts is an open-source Python library by Unit8 for easy handling, pre-processing, and forecasting of time series. It contains an array of models, from standard statistical models such as ARIMA to deep neural networks. https://unit8co.github.io/darts/
-
gluonts VS darts - a user suggested alternative
2 projects | 13 Apr 2023
active support
-
A Simple Guide to Feature Engineering in the Forecast Menu
The new Forecast menu, featuring the open-source Darts Time Series library, offers script-friendly functionality. It's also easy to use. Don't have any data to load yet? Enter through the Stocks or Crypto menus.
-
Ask HN: Data Scientists, what libraries do you use for timeseries forecasting?
I would recommend Darts in Python [1]. It's easy to use (think fit()/predict()) and includes
* Statistical models (ETS, (V)ARIMA(X), etc)
* ML models (sklearn models, LGBM, etc)
* Many recent deep learning models (N-BEATS, TFT, etc)
* Seamlessly works on multi-dimensional series
* Models can be trained on multiple series
* Many models offer rich support for probabilistic forecasts
* Model evaluation is easy: Darts has many metrics, offers backtest etc
* Deep learning scales to large datasets, using GPUs, TPUs, etc
* There's even now an explainability module for some of the models - showing you what matters for computing the forecasts
* (coming soon): an anomaly detection module :)
* (also, it even include FB Prophet if you really want to use it)
Warning: I'm probably biased because I'm Darts creator.
[1] https://github.com/unit8co/darts
What are some alternatives?
nanoGPT - The simplest, fastest repository for training/finetuning medium-sized GPTs.
sktime - A unified framework for machine learning with time series
gpt-2 - Code for the paper "Language Models are Unsupervised Multitask Learners"
pytorch-forecasting - Time series forecasting with PyTorch
simpletransformers - Transformers for Information Retrieval, Text Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI
Prophet - Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.
Pytorch-Simple-Transformer - A simple transformer implementation without difficult syntax and extra bells and whistles.
Kats - Kats, a kit to analyze time series data, a lightweight, easy-to-use, generalizable, and extendable framework to perform time series analysis, from understanding the key statistics and characteristics, detecting change points and anomalies, to forecasting future trends.
nn-zero-to-hero - Neural Networks: Zero to Hero
tsai - Time series Timeseries Deep Learning Machine Learning Python Pytorch fastai | State-of-the-art Deep Learning library for Time Series and Sequences in Pytorch / fastai
huggingface_hub - The official Python client for the Huggingface Hub.
statsforecast - Lightning ⚡️ fast forecasting with statistical and econometric models.