marsha
cog
marsha | cog | |
---|---|---|
12 | 20 | |
461 | 7,328 | |
0.2% | 2.7% | |
8.4 | 9.4 | |
7 months ago | 2 days ago | |
Python | Python | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
marsha
-
LLMs as compilers
There is already a lot of hay to mow with the current state of affairs in generative AI. LLMs as proper compilers, compiLLMers if you will, can produce correct code reliably enough today given enough guidance. Getting an LLM to generate correct code requires providing various examples and descriptive instructions. The UX of a chat interface to an LLM inherently leads people to write prompts that do not meet these criteria. We need to make it easy for people to give LLMs precise descriptions and numerous examples as concisely as possible via syntaxes that are similar to English so they remain easy to learn and use. Coq is a great example of a functional programming syntax that is verbose and distant from English, but example-driven via assertions. David Ellis, Alejandro Guillen and I recently introduced Marsha as a proposal for what a syntax that meets the requirements outlined can look like. It is still early, but LLMs will increasingly give us the power to create more accessible representations of computer programs that look close to English. These representations will be distilled by LLMs into the complexities of the current high-level languages. Knowing Java or Python will become a rare skill, akin to individuals specializing in low-level optimizations using C or assembly language these days. Instead, the focus of developer experience will shift to the higher-level abstractions that are built on top of LLMs and composing these abstractions for different tasks. Compillmers will make programming more accessible in the near future such that writing software becomes part of the resume of most knowledge workers.
-
Show HN: Marsha – An LLM-Based Programming Language
> You're a bit too black-and-white on this situation.
While I agree with your other points, I feel this argument doesn't really hold water.
The output of the c compiler is deterministic.
I struggle very hard to believe that the floating point rounding errors when you compile C will cause it to occasionally emit a binary that is not byte-identical multiple sequential runs in a row.
What any program does at runtime is essentially non-deterministic, and that's 100% not what we're talking about here.
If you consider https://github.com/alantech/marsha/blob/main/examples/web/we... ...
The generated output of this file is a probability distribution with a sweet spot where the code does what you want; there are multiple outputs of code that sit in the sweet spot. You want one of these.
The actual output of this file is a probability distribution that includes the examples, but may or may not overlap the sweet spot of 'actually does the right thing'.
...in fact, and there's no specific reason to expect that, regardless of the number of examples you provide, the distribution that includes those examples also includes the sweet spot.
For common examples it will, but I'd argue that it's actually provable that there are times (eg. where the output length of a valid solution would be > the possible out of the model), that regardless of the examples / tests, it's not actually possible to generate a valid solution from. Just like how constraint solvers will sometimes tell you there's no solution that matches all the constraints.
So, that would be like a compiler error. "You've asked for something impossible".
...but I imagine it would be very very difficult to tell the difference between inputs that overlap the sweet spot and those that don't; the ones that don't will have solutions that look right, but actually only cover the examples; and there's literally no way of telling the difference between that and a correct solution without HFRL.
It seem like an intractable problem to me.
> Different tools for different scenarios, so if that is a huge problem, don't use Marsha as it currently is.
As you say~
- Marsha, a ChatGPT-based programming language
- Marsha is a functional, higher-level, English-based programming language that gets compiled into tested Python software more reliably by ChatGPT
-
Llama 2 – Meta AI
So this comment inspired me to write a Roman Numeral to Integer function in out LLM-based programming language, Marsha: https://github.com/alantech/marsha/blob/main/examples/genera...
cog
-
AI Grant Traction in OSS Startups
View on GitHub
- Insanely Fast Whisper: Transcribe 300 minutes of audio in less than 98 seconds
-
Talk-Llama
I'm in the same situation. I found this cog project to dockerise ML https://github.com/replicate/cog : you write just one python class and a yaml file, and it takes care of the "CUDA hell" and deps. It even creates a flask app in front of your model.
That helps keep your system clean, but someone with big $s please rewrite pytorch to golang or rust or even nodejs / typescript.
-
Llama 2 – Meta AI
https://github.com/replicate/cog
Our thinking was just that a bunch of folks will want to fine-tune right away, then deploy the fine-tunes, so trying to make that easy... Or even just deploy the models-as-is on their own infra without dealing with CUDA insanity!
-
Handling concurrent requests to ML model API
I have used this tool before: https://github.com/replicate/cog/tree/main
-
Opinions on Cog: Containers for machine learning
Then I discovered Cog: Containers for Machine Learning. Looks like a way more flexible solution to plug in the existing infrastructure: you write your custom code and Cog plugs it in a Docker image with FastAPI, no extra ecosystem complexity added.
-
can someone teach me how to install the new stable diffusion repo?
Highly recommend using cog https://github.com/replicate/cog
- Run Stable Diffusion on Your M1 Mac’s GPU
- replicate/cog: Containers for machine learning
-
Why companies move off Heroku (besides the cost)
Dokku Maintainer here.
Dokku also supports Dockerfiles, Docker Images, Tarballs (similar to heroku slugs), and Cloud Native Buildpacks. I'm also actively working on AWS Lambda support (both for simple usage without much config as well as SAM-based usage) and investigating Replicate's Cog[1] and Railways Nixpacks[2] functionalities for building apps.
There are quite a few options in the OSS space (as well as Commercial offerings from new startups and popular incumbents). It's an interesting space to be in, and its always fun to see how new offerings innovate on existing solutions.
[1] https://github.com/replicate/cog
What are some alternatives?
maccarone - AI-managed code blocks in Python ⏪⏩
nixpacks - App source + Nix packages + Docker = Image
llama2-chatbot - LLaMA v2 Chatbot
pytorch_wavelets - Pytorch implementation of 2D Discrete Wavelet (DWT) and Dual Tree Complex Wavelet Transforms (DTCWT) and a DTCWT based ScatterNet
OpenPipe - Turn expensive prompts into cheap fine-tuned models
piku - The tiniest PaaS you've ever seen. Piku allows you to do git push deployments to your own servers.
llama - Inference code for LLaMA models on CPU and Mac M1/M2 GPU
heroku-review-app-actions - GitHub action to automate managing review apps on your Heroku account
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
tvm - Open deep learning compiler stack for cpu, gpu and specialized accelerators
cog-llama-template - LLaMA Cog template
memray - Memray is a memory profiler for Python