Top 11 pretraining Open-Source Projects

• LMOps

  6 3,217 8.1 Python

  General technology for enabling AI capabilities w/ LLMs and MLLMs

  

• OFA

  3 2,337 2.8 Python

  Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

  

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• mPLUG-Owl

  2 1,966 7.6 Python

  mPLUG-Owl & mPLUG-Owl2: Modularized Multimodal Large Language Model

  

Project mention: Unleash the Power of Video-LLaMA: Revolutionizing Language Models with Video and Audio Understanding! | dev.to | 2023-06-12

We extend our deepest gratitude to the extraordinary projects that have influenced and contributed to the development of Video-LLaMA. We're indebted to MiniGPT-4, FastChat, BLIP-2, EVA-CLIP, ImageBind, LLaMA, VideoChat, LLaVA, WebVid, and mPLUG-Owl for their invaluable contributions. Special thanks to Midjourney for creating the stunning Video-LLaMA logo, encapsulating the essence of our groundbreaking project.

• SparK

  3 1,389 7.3 Python

  [ICLR'23 Spotlight🔥] The first successful BERT/MAE-style pretraining on any convolutional network; Pytorch impl. of "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling" (by keyu-tian)

• ImageNet21K

  1 695 10.0 Python

  Official Pytorch Implementation of: "ImageNet-21K Pretraining for the Masses"(NeurIPS, 2021) paper

  

• Entity

  2 669 6.9 Jupyter Notebook

  EntitySeg Toolbox: Towards Open-World and High-Quality Image Segmentation

  

• PITI

  1 471 2.0 Python

  PITI: Pretraining is All You Need for Image-to-Image Translation

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• LinkBERT

  2 389 1.8 Python

  [ACL 2022] LinkBERT: A Knowledgeable Language Model 😎 Pretrained with Document Links

• Revisiting-Contrastive-SSL

  1 86 0.0 Python

  Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations. [NeurIPS 2021]

• tabular-dl-pretrain-objectives

  1 58 10.0 Python

  Revisiting Pretrarining Objectives for Tabular Deep Learning

• awesome-clip-papers

  1 16 5.4 Python

  The most impactful papers related to contrastive pretraining for multimodal models!

Project mention: A History of CLIP Model Training Data Advances | dev.to | 2024-03-13

For a comprehensive catalog of papers pushing the state of CLIP models forward, check out this Awesome CLIP Papers Github repository. Additionally, the Zero-shot Prediction Plugin for FiftyOne allows you to apply any of the OpenCLIP-compatible models to your own data.

pretraining related posts

• [D] Fine tuning LLM - VM requirements.

  1 project | /r/MachineLearning | 2 Jan 2023

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com