Simplest Guide to DIY Your Own LLM Toy in 2024

1. emqx

   1 9 14,547 10.0 Erlang

   The most scalable open-source MQTT broker for IoT, IIoT, and connected vehicles

   

   EMQX (optional): Open-source MQTT broker for IoT, IIoT, and connected vehicles. Used for managing your toys.

2. CodeRabbit

   coderabbit.ai featured

   CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

   

3. docker

   2 204 532 0.0 Go

   Discontinued Docker - the open-source application container engine (by microsoft)

   

   Docker (required): Understanding Docker is crucial for deploying software in containers, making your project portable and scalable. I use it for start Folo server.

4. ollama

   3 440 133,678 9.9 Go

   Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models.

   

   ollama (optional): If you want to run large language models locally, it's a good chooise.

5. mqtt.org

   4 32 5,088 6.0 SCSS

   The mqtt.org website

   

   MQTT (optional): If you're aiming for advanced customization, familiarity with MQTT (a lightweight messaging protocol) will be beneficial for communicating between the toy and the server.

Suggest a related project