Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers
Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.
www.nutrient.io
featured
CodeRabbit: AI Code Reviews for Developers
Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
coderabbit.ai
featured
  1. llm

    Access large language models from the command-line (by simonw)

    llm is Simon's command line front-end to a lot of the llm apis, local and cloud-based. Along with aider-chat, it's my main interface to any LLM work -- it works well with a chat model, one-off queries, and piping text or output into a llm chain. For people who live on the command line, or are just put-off by web interfaces, it's a godsend.

    About the only thing I need to look further abroad for is when I'm working multi-modally -- I know Simon and the community are mainly noodling over the best command line UX for that: https://github.com/simonw/llm/issues/331

  2. Nutrient

    Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers. Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.

    Nutrient logo
  3. ollama

    Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 2, and other large language models.

    Aren't the _0 quantizations considered deprecated and _K_S or _K_M preferable?

    https://github.com/ollama/ollama/issues/5425

  4. llama-models

    Utilities intended for use with Llama models.

    Full text:

    https://github.com/meta-llama/llama-models/blob/main/models/...

    https://github.com/meta-llama/llama-models/blob/main/models/...

    > With respect to any multimodal models included in Llama 3.2, the rights granted under Section 1(a) of the Llama 3.2 Community License Agreement are not being granted to you if you are an individual domiciled in, or a company with a principal place of business in, the European Union. This restriction does not apply to end users of a product or service that incorporates any such multimodal models.

  5. llm-ollama

    LLM plugin providing access to models running on an Ollama server

  6. ttok

    Count and truncate text based on tokens

    I use my https://github.com/simonw/ttok command for that - you can pipe stuff into it for a token count.

    Unfortunately it only uses the OpenAI tokenizers at the moment (via tiktoken), so counts for other models may be inaccurate. I find they tend to be close enough though.

  7. nexa-sdk

    Nexa SDK is a comprehensive toolkit for supporting GGML and ONNX models. It supports text generation, image generation, vision-language models (VLM), Audio Language Model, auto-speech-recognition (ASR), and text-to-speech (TTS) capabilities.

    Llama3.2 3B feels a lot better than other models with same size (e.g. Gemma2, Phi3.5-mini models).

    For anyone looking for a simple way to test Llama3.2 3B locally with UI, Install nexa-sdk(https://github.com/NexaAI/nexa-sdk) and type in terminal:

    nexa run llama3.2 --streamlit

    Disclaimer: I am from Nexa AI and nexa-sdk is an open-sourced. We'd love your feedback.

  8. ollama-docker

    Welcome to the Ollama Docker Compose Setup! This project simplifies the deployment of Ollama using Docker Compose, making it easy to run Ollama with all its dependencies in a containerized environment

    ive been using https://github.com/valiantlynx/ollama-docker which comes with https://github.com/open-webui/open-webui

  9. CodeRabbit

    CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

    CodeRabbit logo
  10. open-webui

    User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

    ive been using https://github.com/valiantlynx/ollama-docker which comes with https://github.com/open-webui/open-webui

  11. generative-ai-docs

    Documentation for Google's Gen AI site - including the Gemini API and Gemma

    This is great for experimentation, but as others have pointed out recently there are persistent issues with Gemini that prevent use in actual products. The recitation/self-sensoring issue results in random failures:

    https://github.com/google/generative-ai-docs/issues/257

  12. langtrace

    Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. 🚀💻📊

    Run test queries on all platforms using something like litellm [1] and langsmith [2] .

    You may not be able to match large queries but, testing will help you transition to other services.

    [1] https://github.com/BerriAI/litellm

    [2] https://langtrace.ai/

  13. litellm

    Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

    Run test queries on all platforms using something like litellm [1] and langsmith [2] .

    You may not be able to match large queries but, testing will help you transition to other services.

    [1] https://github.com/BerriAI/litellm

    [2] https://langtrace.ai/

  14. llama.cpp

    LLM inference in C/C++

    https://github.com/ggerganov/llama.cpp/blob/master/examples/...

  15. hallucination-leaderboard

    Leaderboard Comparing LLM Performance at Producing Hallucinations when Summarizing Short Documents

    Models just added to Hallucination Leaderboard: https://github.com/vectara/hallucination-leaderboard.

    TL;DR:

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • We no longer use LangChain for building our AI agents

    10 projects | news.ycombinator.com | 20 Jun 2024
  • Show HN: Abso – TypeScript LLM Client

    1 project | news.ycombinator.com | 12 Feb 2025
  • Effortlessly Share Ollama API and Open WebUI Online

    2 projects | dev.to | 6 Feb 2025
  • Install DeepSeek in Your Local Computer

    2 projects | dev.to | 2 Feb 2025
  • RamaLama

    4 projects | news.ycombinator.com | 31 Jan 2025

Did you know that Python is
the 2nd most popular programming language
based on number of references?