Meta: Code Llama, an AI Tool for Coding

• refact

  33 1,412 9.8 JavaScript

  WebUI for Fine-Tuning and Self-hosting of Open-Source Large Language Models for Coding

  

You can try this locally - https://github.com/smallcloudai/refact

This works well for me except the 15B+ don't run fast enough on a 4090 - hopefully exllama supports non-llama models, or maybe it'll support CodeLLaMa already I'm not sure.

For general testing/usage this works pretty well with lots of options -

• llama.cpp

  769 56,891 10.0 C++

  LLM inference in C/C++

“Normal laptop” is kind of hard to gauge but if you have a M series MacBook with 16GB+ RAM, you will be able to run 7B comfortably and 13B but stretching your RAM (cause of the unified RAM) at 4 bit quantization. These go all the way down to 2 bit but I personally I find the model noticeably deteriorate anything below 4 bit. You can see how much (V)RAM you need here [1].

[1] https://github.com/ggerganov/llama.cpp#quantization

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• godot-dodo

  16 504 3.1 Python

  Finetuning large language models for GDScript generation.

If you can find a large body of good, permissively licensed example code, you can finetune an LLM on it!

There was a similar attempt for Godot script trained a few months ago, and its reportedly pretty good:

https://github.com/minosvasilias/godot-dodo

I think more attempts havent been made because base llama is not that great at coding in general, relative to its other strengths, and stuff like Starcoder has flown under the radar.

• seamless_communication

  11 10,181 8.6 Jupyter Notebook

  Foundational Models for State-of-the-Art Speech and Text Translation

  

I wish that Meta would release models like SeamlessM4T[0] under the same license as llama, or an even better one.

There seem to be opportunities for people to use technology like this to improve lives, if it were licensed correctly, but I don't see how any commercial offering would compete with anything that Meta does.

Whisper is licensed more permissively and does a great job with speech to text in some languages, and it can translate to English only, but it can't translate between a large number of languages, and it doesn't have any kind of text to speech or speech to speech capabilities.

[0]: https://github.com/facebookresearch/seamless_communication

• codellama

  9 14,965 5.5 Python

  Inference code for CodeLlama models

  

• text-generation-webui

  876 36,293 9.9 Python

  A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

• ollama

  192 58,943 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

To run Code Llama locally, the 7B parameter quantized version can be downloaded and run with the open-source tool Ollama: https://github.com/jmorganca/ollama

   ollama run codellama "write a python function to add two numbers"

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• llama-cpp-python

  54 6,378 9.9 Python

  Python bindings for llama.cpp

LocalAI https://localai.io/ and LMStudio https://lmstudio.ai/ both have fairly complete OpenAI compatibility layers. llama-cpp-python has a FastAPI server as well: https://github.com/abetlen/llama-cpp-python/blob/main/llama_... (as of this moment it hasn't merged GGUF update yet though)

• tabby

  24 17,192 9.9 Rust

  Self-hosted AI coding assistant

  

There are a bunch of VSCode extensions that make use of local models. Tabby seems to be the most friendly right now, but I admittedly haven't tried it myself: https://tabbyml.github.io/tabby/

• ggml

  3 19 8.6

  Tensor library for machine learning (by philpax)

While we're at it, the GGML file format has been deprecated in favor of GGUF.

https://github.com/philpax/ggml/blob/gguf-spec/docs/gguf.md

https://github.com/ggerganov/llama.cpp/pull/2398

• smartcat

  1 2 6.4 JavaScript

• aider

  61 9,450 9.9 Python

  aider is AI pair programming in your terminal

I know about https://github.com/paul-gauthier/aider. Have you got a link to any others?

• lmdeploy

  3 2,324 9.8 Python

  LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

  

• ollama-ui

  2 522 7.3 JavaScript

  Simple HTML UI for Ollama

• ArchGPT

  1 11 7.8 TypeScript

  🐕 ArchGPT is a source-code-management framework to enable a new meta-programming paradigm specially designed for Language-Model-Driven-Development (LMDD) i.e. the utilization of Large Language Models for automated software development.

  

I’m working on a project related to that: https://github.com/0a-io/Arch-GPT

I think hypergraph is an overlooked concept in programming language theory

• SaaSHub

  www.saashub.com sponsored

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Suggest a related project