Ask HN: How to get started with local language models?

• ollama

  229 72,781 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

You might want to start playing with with Ollama or LmStudio to get you started. The model's weights aren't normally inside your .exe . Typically you can address 128TB of virtual memory on Windows.

https://ollama.com/

https://lmstudio.ai/

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• text-generation-webui

  877 37,723 9.9 Python

  A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

You can use webui https://github.com/oobabooga/text-generation-webui

Once you get a version up and running I make a copy before I update it as several times updates have broken my working version and caused headaches.

a decent explanation of parameters outside of reading archive papers: https://github.com/oobabooga/text-generation-webui/wiki/03-%...

a news ai website:

• Local-LLM-User-Guideline

  1 168 8.3

• Cgml

  23 40 8.6 C++

  GPU-targeted vendor-agnostic AI library for Windows, and Mistral model implementation.

If you just want to run Mistral on Windows, you could try my port: https://github.com/Const-me/Cgml/tree/master/Mistral/Mistral...

The setup is relatively easy: install .NET runtime, download 4.5 GB model file from BitTorrent, unpack a small ZIP file and run the EXE.

• femtoGPT

  15 764 7.2 Rust

  Pure Rust implementation of a minimal Generative Pretrained Transformer

If you want to get deep in LLMs: https://github.com/keyvank/femtoGPT

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Suggest a related project