agency
simpleaichat
agency | simpleaichat | |
---|---|---|
5 | 22 | |
44 | 3,398 | |
- | - | |
7.0 | 8.7 | |
about 2 months ago | 5 months ago | |
Go | Python | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
agency
-
Show HN: LLaMA tokenizer that runs in browser
Tokenizers seem to be a massive pain in the neck if you are just calling into an API to use your model. The algorithm itself is non-trivial, and they need pretty sizable data to function: the vocabulary and the merges, which just sit there, using memory. I'm writing https://github.com/ryszard/agency in Go, and while there's a good library for the OpenAI tokenization, if you want a tokenizer for the HF models the best I found was a library calling HF's Rust implementation, which makes it horrible for distribution.
However, at some point I realized that I needed not really the tokens, but the token count, as my most important use was implementing a Token Buffer Memory (trim messages from the beginning in such a way that you never exceed a context size number of tokens). And in order to do that I don't need it to be exactly right, just mostly right, if I am ok with slightly suboptimal efficiency (keeping slightly less tokens than the model supports). So, I took files from Project Gutenberg, and compared the ratio of tokens I get using a proper tokenizer and just calling `strings.Split`, and it seems to be remarkably stable for a given model and language (multiply the length of the result of splitting on spaces by 1.55 for OpenAI and 1.7 for Claude, which leaves a tiny safety margin).
I'm not throwing shade at this project – just being able to call the tokenizer would've saved me a lot of time. But I hope that if I'm wrong about the estimates bring good enough some good person will point out the error of my ways :)
-
Understanding GPT Tokenizers
How I wish this post had appeared a few days earlier... I am writing on my own library for some agent experiments (in go, to make my life more interesting I guess), and knowing the number of tokens is important to implement a token buffer memory (as you approach the model's context window size, you prune enough messages from the beginning of the conversation that the whole thing keeps some given size, in tokens). While there's a nice native library in go for OpenAI models (https://github.com/tiktoken-go/tokenizer), the only library I found for Hugging Face models (and Claude, they published their tokenizer spec in the same JSON format) calls into HF's Rust implementation, which makes it challenging as a dependency in Go. What is more, any tokenizer needs to keep some representation of its vocabulary in memory. So, in the end I removed the true tokenizers, and ended up using an approximate version (just split it in on spaces and multiply by a factor I determined experimentally for the models I use using the real tokenizer, with a little extra for safety). If it turns out someone needs the real thing they can always provide their own token counter). I was actually rather happy with this result: I have less dependencies, and use less memory. But to get there I needed to do a deep dive too understand BPE tokenizers :)
(The library, if anyone is interested: https://github.com/ryszard/agency.)
-
[P] I got fed up with LangChain, so I made a simple open-source alternative for building Python AI apps as easy and intuitive as possible.
I completely agree about langchain being brittle; what I really hate is that it's really hard to make sense about what is going on by reading the code. I was similarly frustrated and rolled my own thing on go (shameless plug): https://github.com/ryszard/agency
- 🏢🤖Agency - An Idiomatic Go Interface for the OpenAI API🚀
- Agency - An Idiomatic Go Interface for the OpenAI API (request for feedback)
simpleaichat
- Efficient Coding Assistant with Simpleaichat
-
Please Don't Ask If an Open Source Project Is Dead
I checked both the issues mentioned, people have been respectful and showing empathy to author's situation
https://github.com/minimaxir/simpleaichat/issues/91
https://github.com/minimaxir/simpleaichat/issues/92
-
We Built an AI-Powered Magic the Gathering Card Generator
ChatGPT's June updated added support for "function calling", which in practice is structured data I/O marketed very poorly: https://openai.com/blog/function-calling-and-other-api-updat...
Here's an example of using structured data for better output control (lightly leveraging my Python package to reduce LoC: https://github.com/minimaxir/simpleaichat/blob/main/examples... )
-
LangChain Agent Simulation – Multi-Player Dungeons and Dragons
So what are the alternatives to LangChain that the HN crowd uses?
I see two contenders:
https://github.com/minimaxir/simpleaichat/tree/main/simpleai...
https://github.com/griptape-ai/griptape
There is also the llm command line utility that has a very thin underlying library, but which might grow eventually:
-
Custom Instructions for ChatGPT
A fun note is that even with system prompt engineering it may not give the most efficient solution: ChatGPT still outputs the avergage case.
I tested around it and doing two passes (generate code and "make it more efficient") works best, with system prompt engineering to result in less code output: https://github.com/minimaxir/simpleaichat/blob/main/examples...
-
The Problem with LangChain
I played around with simpleaichat for a few minutes just now, and I really like it. Unlike LangChain, I can understand what it does in minutes, and it looks like its primitives are fairly powerful. It looks like it's going to replace the `openai` library for me, it seems like a nice wrapper.
I'm especially looking forward to playing with the structured data models bit: https://github.com/minimaxir/simpleaichat/blob/main/examples...
Well done, Max!
-
How is Langchain's dev experience? Any alternatives?
https://github.com/minimaxir/simpleaichat bills itself as a simpler alternative to langchain. I have not tried it, but it looks interesting.
-
Stanford A.I. Courses
I think you are asking specifically about practical LLM engineering and not the underlying science.
Honestly this is all moving so fast you can do well by reading the news, following a few reddits/substacks, and skimming the prompt engineering papers as they come out every week (!).
https://www.latent.space/p/ai-engineer provides an early manifesto for this nascent layer of the stack.
Zvi writes a good roundup (though he is concerned mostly with alignment so skip if you don’t like that angle): https://thezvi.substack.com/p/ai-18-the-great-debate-debates
Simon W has some good writeups too: https://simonwillison.net/
I strongly recommend playing with the OpenAI APIs and working with langchain in a Colab notebook to get a feel for how these all fit together. Also, the tools here are incredibly simple and easy to understand (very new) so looking at, say, https://github.com/minimaxir/simpleaichat/tree/main/simpleai... or https://github.com/smol-ai/developer and digging in to the prompts, what goes in system vs assistant roles, how you gourde the LLM, etc.
-
Where is the engineering part in "prompt engineer"?
This notebook from the repo I linked to is a concise example, and the reason you would want to optimize prompts.
- Show HN: Python package for interfacing with ChatGPT with minimized complexity
What are some alternatives?
tokenizer - Pure Go implementation of OpenAI's tiktoken tokenizer
lmql - A language for constraint-guided and efficient LLM programming.
Constrained-Text-Genera
langroid - Harness LLMs with Multi-Agent Programming
Constrained-Text-Generation-Studio - Code repo for "Most Language Models can be Poets too: An AI Writing Assistant and Constrained Text Generation Studio" at the (CAI2) workshop, jointly held at (COLING 2022)
guidance - A guidance language for controlling large language models. [Moved to: https://github.com/guidance-ai/guidance]
llama-tokenizer-js - JS tokenizer for LLaMA 1 and 2
semantic-kernel - Integrate cutting-edge LLM technology quickly and easily into your apps
panml - PanML is a high level generative AI/ML development and analysis library designed for ease of use and fast experimentation.
gchain - Composable LLM Application framework inspired by langchain
feste - Feste is a free and open-source framework allowing scalable composition of NLP tasks using a graph execution model that is optimized and executed by specialized schedulers.
transynthetical-engine - Applied methods of analytical augmentation to build tools using large-language models.