Top 11 Python pretraining Projects

pretraining

• Add a project

1. LMOps

   1 6 4,065 6.7 Python

   General technology for enabling AI capabilities w/ LLMs and MLLMs

   

2. Stream

   getstream.io featured

   Stream - Scalable APIs for Chat, Feeds, Moderation, & Video. Stream helps developers build engaging apps that scale to millions with performant and flexible Chat, Feeds, Moderation, and Video APIs and SDKs powered by a global edge network and enterprise-grade infrastructure.

   

3. OFA

   2 3 2,498 2.8 Python

   Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

   

4. mPLUG-Owl

   3 2 2,493 6.3 Python

   mPLUG-Owl: The Powerful Multi-modal Large Language Model Family

   

5. SparK

   4 3 1,337 7.3 Python

   [ICLR'23 Spotlight🔥] The first successful BERT/MAE-style pretraining on any convolutional network; Pytorch impl. of "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling" (by keyu-tian)

   

6. ImageNet21K

   5 1 767 10.0 Python

   Official Pytorch Implementation of: "ImageNet-21K Pretraining for the Masses"(NeurIPS, 2021) paper

   

7. PITI

   6 1 499 0.6 Python

   PITI: Pretraining is All You Need for Image-to-Image Translation

   

8. LinkBERT

   7 2 438 1.8 Python

   [ACL 2022] LinkBERT: A Knowledgeable Language Model 😎 Pretrained with Document Links

   

9. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

10. llm-baselines

    8 1 99 2.4 Python

    nanoGPT-like codebase for LLM training

    

11. Revisiting-Contrastive-SSL

    9 1 89 0.0 Python

    Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations. [NeurIPS 2021]

    

12. awesome-clip-papers

    10 1 68 5.4 Python

    The most impactful papers related to contrastive pretraining for multimodal models!

    

13. tabular-dl-pretrain-objectives

    11 1 63 10.0 Python

    Revisiting Pretrarining Objectives for Tabular Deep Learning

    

Python pretraining related posts

• [D] Fine tuning LLM - VM requirements.

  1 project | /r/MachineLearning | 2 Jan 2023

Index

Sponsored

Stream - Scalable APIs for Chat, Feeds, Moderation, & Video.

Stream helps developers build engaging apps that scale to millions with performant and flexible Chat, Feeds, Moderation, and Video APIs and SDKs powered by a global edge network and enterprise-grade infrastructure.

getstream.io

Do not miss the trending Python projects with our weekly report!