modal-examples VS FastChat

We made a small app for SDXL Lightning, running your own Python code on GPUs. It generates images in real time.

https://potatoes.ai/

We know there was a fal.ai post yesterday, and that got a lot of interest, but we also made this demo yesterday and didn't share — just wanted to mention it as an alternative option for people who like running their own code and custom models instead of using a prebuilt API provider.

The backend code is open-source too and you can deploy it yourself: https://github.com/modal-labs/modal-examples/blob/main/06_gpu_and_ml/stable_diffusion/stable_diffusion_xl_lightning.py

The startup I work at is relatively pretty good at documentation engineering. We have written code to test the code snippets in docstrings (https://github.com/modal-labs/pytest-markdown-docs) and we have written code to do synthetic monitoring testing of the examples in our examples repo (https://github.com/modal-labs/modal-examples). We are also diligent about putting using Python's warnings library to handle API deprecation, and treat deprecation warnings as errors internally, ensuring our own code samples and examples are most up-to-date.

You can get it running with one Python script on Modal.com :)

https://github.com/modal-labs/modal-examples/blob/main/06_gp...

This demo lets you choose the podcast, and is open-source: https://modal-labs--whisper-pod-transcriber-fastapi-app.moda...

https://github.com/modal-labs/modal-examples/tree/main/06_gp...

Transcribes 1hr of audio in roughly 1min, using parallelisation across CPUs.

This demo is open-source: https://github.com/modal-labs/modal-examples/tree/main/06_gp....

https://modal-labs--whisper-pod-transcriber-fastapi-app.moda...

It's become so easy to stick together ML models, often without training most or all of them yourself.

*video demo:* https://youtu.be/mQsMuM8d4Qc

*cloud platform:* https://modal.com

*code*: https://github.com/modal-labs/modal-examples/tree/main/06_gp...

Transcription - OpenAI just released Whisper. Check out what it can do with podcasts

Here's the source code.

gpt2-chatbot isn't the only "mystery model" on LMSYS. Another is "deluxe-chat".

When asked about it in October last year, LMSYS replied [0] "It is an experiment we are running currently. More details will be revealed later"

One distinguishing feature of "deluxe-chat": although it gives high quality answers, it is very slow, so slow that the arena displays a warning whenever it is invoked

[0] https://github.com/lm-sys/FastChat/issues/2527

FastChat

How does it compare with something like FastChat? https://github.com/lm-sys/FastChat

Feature set seems like a decent amount of overlap. One limitation of FastChat, as far as I can tell, is that one is limited to the models that FastChat supports (though I think it would be minor to modify it to support arbitrary models?)

Looks like the Vicuna repo is Apache 2.0 also[1].

What's the interpretation of copyright law that would prevent the code being Apache 2.0 based on the source of the fine-tuning dataset?

[1] https://github.com/lm-sys/FastChat

Check how to start with FastChat. Support FastChat on GitHub ⭐

For something a little heavier but much more robust in terms of features/functionality I've been enjoying FastChat: https://github.com/lm-sys/FastChat

It allows you to plug in different backends so that you can use OpenAI compatible clients with various LLM's, selfhosted or otherwise.