pkgx
Llama-2-Onnx
pkgx | Llama-2-Onnx | |
---|---|---|
47 | 3 | |
8,716 | 987 | |
0.7% | 2.0% | |
9.0 | 6.7 | |
7 days ago | 4 months ago | |
TypeScript | Python | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pkgx
-
Install Asdf: One Runtime Manager to Rule All Dev Environments
I’m liking pkgx over asdf as it can activate project tooling upon cd’ing into a project folder.
https://pkgx.sh
-
Show HN: Flox 1.0 – Open-source dev env as code with Nix
I saw some alternatives being suggested and wanted to do the same (Also, so that I can look back at this item, through my comments :) ). Started using https://pkgx.sh/ lately. I know it has some baggage with tea.xyz and crypto, but it is also easy to get started with.
-
Beginners Intro to Trunk Based Development
Secondly, our development environments must not drift, because then code may behave differently and a change could pass on our machine but fail in production. There are many tools for locking down environments, e.g nix, pkgx, asdf, containers, etc., and they all share the common goal of being able to lock down dependencies for an environment accurately and deterministically. And that needs to be enforced in our local workflow so we don't have to rely on CI environments for correctness. All developers must have environments that are effectively identical to what runs in CI (which itself should be representative of the production environment).
-
Practical Guide to Trunk Based Development
There are many ways this can be done (e.g nix, pkgx, asdf, containers, etc.), and we won’t get into which specific tools to use, because we'll instead cover the essential essence of preventing environment drift:
-
5 Developer CLI Essentials
1. pkgx
- FLaNK Stack Weekly for 14 Aug 2023
-
How to send a warm welcome email with Resend, Next-Auth and React-Email
Before diving in, it's a good idea to have a package manager handy, like tea. It'll handle your development environment and simplify your life!
-
Announcing tea/gui - The Open Store for Open-Source
Direct fast-track link to repo
-
Looking to help out on some open source projects
checkout https://github.com/teaxyz/cli and https://github.com/teaxyz/pantry
-
Run llama.cpp with tea – without the installation pain!
Install is tea: sh <(curl https://tea.xyz) and
Llama-2-Onnx
-
Show HN: Fine-tune your own Llama 2 to replace GPT-3.5/4
System: Here's some docs, answer concisely in a sentence.
YMMV on cost still, depends on cloud vendor, and my intuition & viewpoint agrees with yours, GPT-3.5 is priced low enough that there isn't a case where it makes sense to use another model.
It strikes me now that _very_ likely and not just our intuition: OpenAI's $/GPU hour is likely <= any other vendor's.
The next big step will come from formalizing the stuff rolling around the local LLM community, for months now it's either been one-off $X.c stunts that run on desktop, and the vast majority of the _actual_ usage and progress is coming from porn-y stuff, like all nascent tech.
Microsoft has LLaMa-2 ONNX available on GitHub[1]. There's budding but very small projects in different languages to wrap ONNX. Once there's a genuine cross-platform[2] ONNX wrapper that makes running LLaMa-2 easy, there will be a step change. It'll be "free"[3] to run your fine-tuned model that does as well as GPT-4 .
It's not clear to me exactly when this will occur. It's "difficult" now, but only because the _actual usage_ in the local LLM community doesn't have a reason to invest in ONNX, and it's extremely intimidating to figure out how exactly to get LLaMa-2 running in ONNX. Microsoft kinda threw it up on GitHub and moved on, the sample code even still needs a PyTorch model. I see at least one very small company on HuggingFace that _may_ have figured out full ONNX.
[1] https://github.com/microsoft/Llama-2-Onnx
- FLaNK Stack Weekly for 14 Aug 2023
- Llama 2 on ONNX runs locally
What are some alternatives?
nix - Nix, the purely functional package manager
vllm - A high-throughput and memory-efficient inference and serving engine for LLMs
litellm - Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)
onnx-coreml - ONNX to Core ML Converter
macports-base - The MacPorts command-line client
OpenPipe - Turn expensive prompts into cheap fine-tuned models
symmetric-ds - SymmetricDS is database replication and file synchronization software that is platform independent, web enabled, and database agnostic. It is designed to make bi-directional data replication fast, easy, and resilient. It scales to a large number of nodes and works in near real-time across WAN and LAN networks.
awesome-data-temporality - A curated list to help you manage temporal data across many modalities 🚀.
white-paper - how will the protocol work?
llama.cpp - LLM inference in C/C++
loxilb - eBPF based cloud-native load-balancer. Powering Kubernetes|Edge|5G|IoT|XaaS Apps.
gpt-llm-trainer