
-
llm is Simon's command line front-end to a lot of the llm apis, local and cloud-based. Along with aider-chat, it's my main interface to any LLM work -- it works well with a chat model, one-off queries, and piping text or output into a llm chain. For people who live on the command line, or are just put-off by web interfaces, it's a godsend.
About the only thing I need to look further abroad for is when I'm working multi-modally -- I know Simon and the community are mainly noodling over the best command line UX for that: https://github.com/simonw/llm/issues/331
-
Nutrient
Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers. Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.
-
ollama
Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 2, and other large language models.
Aren't the _0 quantizations considered deprecated and _K_S or _K_M preferable?
https://github.com/ollama/ollama/issues/5425
-
Full text:
https://github.com/meta-llama/llama-models/blob/main/models/...
https://github.com/meta-llama/llama-models/blob/main/models/...
> With respect to any multimodal models included in Llama 3.2, the rights granted under Section 1(a) of the Llama 3.2 Community License Agreement are not being granted to you if you are an individual domiciled in, or a company with a principal place of business in, the European Union. This restriction does not apply to end users of a product or service that incorporates any such multimodal models.
-
-
I use my https://github.com/simonw/ttok command for that - you can pipe stuff into it for a token count.
Unfortunately it only uses the OpenAI tokenizers at the moment (via tiktoken), so counts for other models may be inaccurate. I find they tend to be close enough though.
-
nexa-sdk
Nexa SDK is a comprehensive toolkit for supporting GGML and ONNX models. It supports text generation, image generation, vision-language models (VLM), Audio Language Model, auto-speech-recognition (ASR), and text-to-speech (TTS) capabilities.
Llama3.2 3B feels a lot better than other models with same size (e.g. Gemma2, Phi3.5-mini models).
For anyone looking for a simple way to test Llama3.2 3B locally with UI, Install nexa-sdk(https://github.com/NexaAI/nexa-sdk) and type in terminal:
nexa run llama3.2 --streamlit
Disclaimer: I am from Nexa AI and nexa-sdk is an open-sourced. We'd love your feedback.
-
ollama-docker
Welcome to the Ollama Docker Compose Setup! This project simplifies the deployment of Ollama using Docker Compose, making it easy to run Ollama with all its dependencies in a containerized environment
ive been using https://github.com/valiantlynx/ollama-docker which comes with https://github.com/open-webui/open-webui
-
CodeRabbit
CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
-
ive been using https://github.com/valiantlynx/ollama-docker which comes with https://github.com/open-webui/open-webui
-
This is great for experimentation, but as others have pointed out recently there are persistent issues with Gemini that prevent use in actual products. The recitation/self-sensoring issue results in random failures:
https://github.com/google/generative-ai-docs/issues/257
-
langtrace
Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. 🚀💻📊
Run test queries on all platforms using something like litellm [1] and langsmith [2] .
You may not be able to match large queries but, testing will help you transition to other services.
[1] https://github.com/BerriAI/litellm
[2] https://langtrace.ai/
-
litellm
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
Run test queries on all platforms using something like litellm [1] and langsmith [2] .
You may not be able to match large queries but, testing will help you transition to other services.
[1] https://github.com/BerriAI/litellm
[2] https://langtrace.ai/
-
https://github.com/ggerganov/llama.cpp/blob/master/examples/...
-
hallucination-leaderboard
Leaderboard Comparing LLM Performance at Producing Hallucinations when Summarizing Short Documents
Models just added to Hallucination Leaderboard: https://github.com/vectara/hallucination-leaderboard.
TL;DR: