cria
transmogrifier
cria | transmogrifier | |
---|---|---|
4 | 1 | |
77 | 0 | |
- | - | |
2.5 | 4.7 | |
about 1 year ago | about 1 year ago | |
Python | TypeScript | |
- | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
cria
-
Show HN: Speeding up LLM inference 2x times (possibly)
It originally started as a fork to Recmo’s cria pure numpy llama impl :)
https://github.com/recmo/cria
Took a whole night to compute a few
-
Jsonformer: A bulletproof way to generate structured output from LLMs
Not op, but I can share my approach - I went line by line by Recmo's Cria: https://github.com/recmo/cria - which is an implementation of Llama in Numpy - so very low level. Took me I think 3-4 days x 10 hours + 1-2 days of reading about Transformers to understand what's going on - but from that you can see how models generate text and have a deep understanding of what's going on.
- LLaMA for poor
transmogrifier
-
Jsonformer: A bulletproof way to generate structured output from LLMs
This is a useful pattern and seems to be discovered by devs after they've played with LLMs for a while.
I called it "transmogrifier" (thanks, Calvin!) and the maybe one interesting twist in my repo [1] is that you define the desired return type using Zod; when you call `transmogrify(...)` it validates the LLM response. If valid, data is returned (and you can use all the nice trappings of typescript from there); if not, an exception is raised.
[1] https://github.com/davepeck/transmogrifier
What are some alternatives?
clownfish - Constrained Decoding for LLMs against JSON Schema
jsonformer - A Bulletproof Way to Generate Structured JSON from Language Models
magic - AI functions for Typescript
effort - An implementation of bucketMul LLM inference
relm - ReLM is a Regular Expression engine for Language Models