Show HN: Magentic – Use LLMs as simple Python functions

• magentic

  9 1,362 9.1 Python

  Seamlessly integrate LLMs as Python functions

  1) The OpenAI API will be queried each time a "prompt-function" is called in python code. If you provide the `functions` argument in order to use function-calling then magentic will not execute the function the LLM has chosen, instead it returns a `FunctionCall` instance which you can validate before calling.

  2) I haven't measured additional latency but it should be negligible in comparison to the speed of generation of the LLM. And since it makes it easy to use streaming and async functions you might be able to achieve much faster generation speeds overall - see the Async section in the README. Token usage should also be a negligible change from calling the OpenAI API directly - the only "prompting" magentic does currently is in naming the functions sent to OpenAI, all other input tokens are written by the user. A user switching from explicitly defining the output schema in the prompt to using function-calling via magentic might actually save a few tokens.

  3) Functionality is not deterministic, even with `temperature=0`, but since we're working with python functions one option is to just add the `@cache` decorator. This would save you tokens and time when calling the same prompt-function with the same inputs.

  ---

  1) https://github.com/jackmpcollins/magentic#usage

• marvin

  15 3,912 9.8 Python

  ✨ Build AI interfaces that spark joy

  

  Yes, similar ideas. Marvin [asks the LLM to mimic the python function](https://github.com/PrefectHQ/marvin/blob/f37ad5b15e2e77dd998...), whereas in magentic the function signature just represents the inputs/outputs to the prompt-template/LLM, so the LLM “doesn’t know” that it is pretending to be a python function - you specify all the prompts.

• InfluxDB

  www.influxdata.com

  sponsored

  Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.

  

• lmql

  29 2,764 10.0 Python

  A language for constraint-guided and efficient LLM programming.

  

  This is also similar in spirit to LMQL

  https://github.com/eth-sri/lmql

• outlines

  16 3,437 9.4 Python

  Guided Text Generation

  

  Right now it just works with OpenAI chat models (gpt-3.5-turbo, gpt-4) but if there's interest I plan to extend it to have several backends. These would probably each be an existing library that implements generating structured output like https://github.com/outlines-dev/outlines or https://github.com/guidance-ai/guidance. If you have ideas how this should be done let me know - on a github issue would be great to make it visible to others.

• guidance

  18 14,930 9.4 Jupyter Notebook

  A guidance language for controlling large language models.

  

  Right now it just works with OpenAI chat models (gpt-3.5-turbo, gpt-4) but if there's interest I plan to extend it to have several backends. These would probably each be an existing library that implements generating structured output like https://github.com/outlines-dev/outlines or https://github.com/guidance-ai/guidance. If you have ideas how this should be done let me know - on a github issue would be great to make it visible to others.

• cria

  3 187 10.0 Rust

  OpenAI compatible API for serving LLAMA-2 model

• openai-functools

  4 47 8.7 Python

  openai-functools: Simplified Generation of OpenAI Functions JSON Metadata for OpenAI Function Calling

  Very cool! At first the title reminded me of a project me and my colleague are working on called OpenAI-Functools [1], but your concept is quite the opposite, combining LLMs in your code rather seamlessly instead of the other way around. Quite cool, and interesting examples :)

  I’ll definitely try to apply it in one of my pet projects. Good stuff

  [1] https://github.com/Jakob-98/openai-functools

• Onboard AI

  getonboard.dev

  sponsored

  Learn any GitHub repo in 59 seconds. Onboard AI learns any GitHub repo in minutes and lets you chat with it to locate functionality, understand different parts, and generate new code. Use it for free at www.getonboard.dev.

  

• vanna

  10 422 9.3 Jupyter Notebook

  🤖 AI-Powered Database Queries 📊. Accurate Text-to-SQL Conversion using RAG 🔄.

  

  Nice! I’m going to try it out and possibly integrate it into my Python package: https://vanna.ai

• LocalAI

  78 14,008 9.7 C++

  :robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs ggml, gguf, GPTQ, onnx, TF compatible models: llama, llama2, rwkv, whisper, vicuna, koala, cerebras, falcon, dolly, starcoder, and many others

  

  There's also LocalAI[0] which allows the use of local LLMs with an OpenAI compatible API.

  [0] https://github.com/go-skynet/LocalAI

• antiscope

  2 0 10.0 Python

  experimental model-based language structures

  

  See also: `antiscope`, an experiment in subjunctive programming

  https://github.com/MillionConcepts/antiscope

Suggest a related project