continue
llama-cpp-python
| continue | llama-cpp-python | |
|---|---|---|
| 18 | 55 | |
| 11,309 | 6,579 | |
| 14.7% | - | |
| 10.0 | 9.8 | |
| 3 days ago | 5 days ago | |
| TypeScript | Python | |
| Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
continue
-
Ask HN: Who is hiring? (February 2024)
Continue (YC S23) | Founding Engineer | ONSITE | Full-time | San Francisco | $130-$170K + 1-2% Equity
At Continue, we are on a mission to make building software feel like making music. We are creating the open-source autopilot for VS Code and JetBrains——the easiest way to code with any LLM (https://github.com/continuedev/continue).
You are likely a good fit if you
- have founded or want to found your own startup one day
- have experience with frontend, backend, ML technologies
- are enthusiastic about AI/LLMs, open source, developer tools
- get excited about supporting users and helping customers
- want to work in-person in SF the majority of the time
More info: https://www.ycombinator.com/companies/continue/jobs/smcxRnM-...
-
Meta AI releases Code Llama 70B
Continue doesn’t support tab completion like Copilot yet.
A pull/merge request is being worked on: https://github.com/continuedev/continue/pull/758
-
Show HN: Open-source, privacy oriented alternative to GitHub Copilot chat
Good job on the project, but it's unfortunately named. A privy also refers to a latrine.
Given that this project was started well after Continue.dev, I think it would be useful to include an FAQ or a comparison table on what exactly makes this project different.
https://github.com/continuedev/continue
- Continue will generate, refactor, and explain entire sections of code
-
VSC Continue.dev with own Rest API
In this Continue.dev file https://github.com/continuedev/continue/blob/preview/server/continuedev/libs/llm/llamacpp.py the request to llama.cpp is implemented.
- What is your motive for running open-source models, instead of just using a ready-made solution like GPT-4?
-
Ask HN: Who is hiring? (December 2023)
Continue | Founding Engineer | ONSITE | Full-time | San Francisco | $130-$170K + 1-2% Equity
At Continue, we are on a mission to make building software feel like making music. We are creating the open-source autopilot for software development—an IDE extension that brings the power of ChatGPT to VS Code and JetBrains (https://github.com/continuedev/continue).
You are likely a good fit if you
- have founded or want to found your own startup one day
- have experience with frontend, backend, and ML technologies
- are enthusiastic about AI/LLMs, open source, developer tools
- get excited about supporting users and helping customers
- want to work in-person in SF the majority of the time
More info: https://www.ycombinator.com/companies/continue/jobs/smcxRnM-...
-
How helpful are LLMs with MATLAB?
Original source: https://github.com/continuedev/continue/tree/main/docs/docs/languages/matlab.md
-
How are people using open source LLMs in production apps?
We are seeing developers deploy open-source LLMs for their teams to use while coding internally, which each developer then uses with Continue
-
Show HN: Continue – open-source coding autopilot, now in JetBrains
Hi HN!
Since launching Continue two months ago (https://news.ycombinator.com/item?id=36882146), we've received amazing feedback, added features, and greatly improved reliability. But one of the biggest things we heard was the desire for a JetBrains extension. My co-founder Ty and I are super excited to share that we've released an extension for PyCharm, Intellij, WebStorm, and most other JetBrains IDEs - ready for alpha users at https://plugins.jetbrains.com/plugin/22707-continue.
Perhaps the most exciting part is that this effort was kickstarted and in great part developed by a community contributor! If you're curious what it took to make this happen, check out the PR here (https://github.com/continuedev/continue/pull/457). We hope to eventually support every IDE, so we made adding a new extension as easy as implementing a single class. If you're curious why this is possible, you can read more about the Continue Server and the architectural decisions we made here: https://blog.continue.dev/how-we-made-continue-ide-agnostic.
llama-cpp-python
-
Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
There's a Python binding for llama.cpp which is actively maintained and has worked well for me: https://github.com/abetlen/llama-cpp-python
- FLaNK AI for 11 March 2024
-
OpenAI: Memory and New Controls for ChatGPT
I'll share the core bit that took a while to figure out the right format, my main script is a hot mess using embeddings with SentenceTransformer, so I won't share that yet. E.g: last night I did a PR for llama-cpp-python that shows how Phi might be used with JSON only for the author to write almost exactly the same code at pretty much the same time. https://github.com/abetlen/llama-cpp-python/pull/1184
-
TinyLlama LLM: A Step-by-Step Guide to Implementing the 1.1B Model on Google Colab
Python Bindings for llama.cpp
- Mistral-8x7B-Chat
-
Running Mistral LLM on Apple Silicon Using Apple's MLX Framework Is Much Faster
If the model could be made to work with llama.cpp, then https://github.com/abetlen/llama-cpp-python might be more compact. llama.cpp only supports a limited list of model types though.
- Run ChatGPT-like LLMs on your laptop in 3 lines of code
-
Code Llama, a state-of-the-art large language model for coding
https://github.com/abetlen/llama-cpp-python has a web server mode that replicates openai's API iirc and the readme shows it has docker builds already.
-
Meta: Code Llama, an AI Tool for Coding
LocalAI https://localai.io/ and LMStudio https://lmstudio.ai/ both have fairly complete OpenAI compatibility layers. llama-cpp-python has a FastAPI server as well: https://github.com/abetlen/llama-cpp-python/blob/main/llama_... (as of this moment it hasn't merged GGUF update yet though)
-
First steps with llama
I went with Python, llama-cpp-python, since my goal is just to get a small project up and running locally.
What are some alternatives?
aider - aider is AI pair programming in your terminal
LocalAI - :robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.
vscode-flexigpt - FlexiGPT plugin for VSCode. Interact with AI models as a power user
intel-extension-for-pytorch - A Python package for extending the official PyTorch that can easily obtain performance on Intel platform
ChatGPT.nvim - ChatGPT Neovim Plugin: Effortless Natural Language Generation with OpenAI's ChatGPT API
llama.cpp - LLM inference in C/C++
prompt - 🥝 A command line application to interact with OpenAI's ChatGPT API.
text-generation-inference - Large Language Model Text Generation Inference
openvsx - An open-source registry for VS Code extensions
mlc-llm - Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.
uniteai - Your AI Stack in Your Editor
FastChat - An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.