Gemma.cpp: lightweight, standalone C++ inference engine for Gemma models

• gemma.cpp

  8 5,476 9.0 C++

  lightweight, standalone C++ inference engine for Google's Gemma models.

  

Yes - thanks for pointing that out. The README is being updated, you can see an updated WIP in the dev branch: https://github.com/google/gemma.cpp/tree/dev?tab=readme-ov-f...

• EasyDeL

  2 149 9.9 Python

  EasyDeL is an OpenSource Library to make your training faster and more Optimized With cool Options for training and serving Both in Python And Mojo🔥

This is not a production backend, as it says in the readme.

There are some very interesting efforts in JAX/TPU land like https://github.com/erfanzar/EasyDeL

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• llamafile

  34 13,765 9.6 C++

  Distribute and run LLMs with a single file.

  

llama.cpp has integrated gemma support. So you can use llamafile for this. It is a standalone executable that is portable across most popular OSes.

https://github.com/Mozilla-Ocho/llamafile/releases

So, download the executable from the releases page under assets. You want either just main or just server. Don't get the huge ones with the model inlined in the file. The executable is about 30MB in size,

https://github.com/Mozilla-Ocho/llamafile/releases/download/...

• highway

  66 3,645 9.8 C++

  Performance-portable, length-agnostic SIMD with runtime dispatch

  

Thanks so much!

Everyone working on this self-selected into contributing, so I think of it less as my team than ... a team?

Specifically want to call out: Jan Wassenberg (author of https://github.com/google/highway) and I started gemma.cpp as a small project just a few months ago + Phil Culliton, Dan Zheng, and Paul Chang + of course the GDM Gemma team.

• gemma-cpp-python

  1 34 7.9 Python

  A Python wrapper for gemma.cpp

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project