[P] Cedille, the largest French language model (6b), released in open source

This page summarizes the projects mentioned and recommended in the original post on /r/MachineLearning

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • mesh-transformer-jax

    Model parallel transformers in JAX and Haiku

    We mostly stuck to the finetuning recommendations provided by GPT-J: https://github.com/kingoflolz/mesh-transformer-jax/blob/master/howto_finetune.md

  • cedille-ai

    ✒️ Cedille is a large French language model (6B), released under an open-source license

    We are proponents of “open AI” and as such have released a checkpoint for the world to use (MIT license) : https://github.com/coteries/cedille-ai

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  • allennlp

    Discontinued An open-source NLP research library, built on PyTorch.

    Another aspect we had fun with is dataset filtering. We have run the whole C4 French dataset through the Detoxify classifier to clean it up 🤬

  • lm-evaluation-harness

    A framework for few-shot evaluation of language models.

    We tried to overcome these problems to the best of our ability - Happy to answer if you have more specific questions! FYI: We used/adapted EleutherAI's eval harness (https://github.com/EleutherAI/lm-evaluation-harness) for most of this work.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts