Show HN: Prompts as (WASM) Programs

• aici

  6 1,723 9.9 Rust

  AICI: Prompts as (Wasm) Programs

  

It's only the controller that runs in Wasm, not the inference.

The pyctrl is a just a sample controller, you can write a controller that takes any kind of grammar (eg., a yacc grammar [0] - the python code in that example is only used for glueing).

Llama.cpp grammars were quite slow in my testing (20ms per token or so, compared to 2ms for the yacc grammar referenced above).

[0] https://github.com/microsoft/aici/blob/main/controllers/pyct...

• ollama

  192 58,943 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

This is great generic idea.

It's also possible to wrap it in something user friendly a'la [0].

[0] https://github.com/ollama/ollama/issues/3019

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• transformers-CFG

  1 50 8.7 Python

  🤗 A specialized library for integrating context-free grammars (CFG) in EBNF with the Hugging Face Transformers

  

Does it support constrained generation during training?

This is what we need for the large language models I am training for health care use cases.

For example, constraining LLM output is currently done by masking, and having this rust based library would enable novel ways to train LLMs.

Relevant papers:

https://github.com/epfl-dlab/transformers-CFG

https://neurips.cc/virtual/2023/poster/70782

• outlines

  30 5,567 9.7 Python

  Structured Text Generation

  

> The most obvious usage of this is forcing a model to output valid JSON

Isn't this something that Outlines [0], Guidance [1] and others [2] already solve much more elegantly?

0. https://github.com/outlines-dev/outlines

1. https://github.com/guidance-ai/guidance

2. https://github.com/sgl-project/sglang

• guidance

  23 17,246 9.8 Jupyter Notebook

  A guidance language for controlling large language models.

  

> The most obvious usage of this is forcing a model to output valid JSON

Isn't this something that Outlines [0], Guidance [1] and others [2] already solve much more elegantly?

0. https://github.com/outlines-dev/outlines

1. https://github.com/guidance-ai/guidance

2. https://github.com/sgl-project/sglang

• sglang

  3 2,271 9.4 Python

  SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.

  

> The most obvious usage of this is forcing a model to output valid JSON

Isn't this something that Outlines [0], Guidance [1] and others [2] already solve much more elegantly?

0. https://github.com/outlines-dev/outlines

1. https://github.com/guidance-ai/guidance

2. https://github.com/sgl-project/sglang

• autogen

  31 24,917 9.9 Jupyter Notebook

  A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap

  

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project