buzz VS openai-cookbook

Easiest 1-click way to install and use Stable Diffusion on your computer."

https://github.com/easydiffusion/easydiffusion

And while Whisper is OpenAI, it is trivial to use locally and extremely usefull

https://github.com/chidiwilliams/buzz

Install this for Resolve

Just use this one: https://github.com/chidiwilliams/buzz

What utilities related to Whisper do you wish existed? What have you had to build yourself?

On the end user application side, I wish there was something that let me pick a podcast of my choosing, get it fully transcribed, and get an embeddings search plus answer q&a on top of that podcast or set of chosen podcasts. I've seen ones for specific podcasts, but I'd like one where I can choose the podcast. (Probably won't build it)

Also on the end user side, I wish there was an Otter alternative (still paid $30/mo, but unlimited minutes per month) that had longer transcription limits. (Started building this, not much interest from users though)

Things I've seen on the dev tool side:

Gladia (API call version of Whisper)

Whisper.cpp

Whisper webservice (https://github.com/ahmetoner/whisper-asr-webservice) - via this thread

Live microphone demo (not real time, it still does it in chunks) https://github.com/mallorbc/whisper_mic

Streamlit UI https://github.com/hayabhay/whisper-ui

Whisper playground https://github.com/saharmor/whisper-playground

Real time whisper https://github.com/shirayu/whispering

Whisper as a service https://github.com/schibsted/WAAS

Improved timestamps and speaker identification https://github.com/m-bain/whisperX

MacWhisper https://goodsnooze.gumroad.com/l/macwhisper

Crossplatform desktop Whisper that supports semi-realtime https://github.com/chidiwilliams/buzz

A pretrained LLM is a closed-book system: It can only access information that it was trained on. With domain fine-tuning, the system manifests additional material. An early prototype of this technique was shown in this OpenAi cookbook: For the target domain, text was embedded using an API, and then when using the LLM, embeddings were retrieved using semantic similarity search to formulate an answer. Although this approach evolved to retrieval-augmented generation, its still a technique to adapt a Gen2 (2020) or Gen3 (2022) LLM into a question-answering system.

https://github.com/openai/openai-cookbook/blob/main/examples...

It sounds like you are trying to tweak embeddings for similarity search. Rather than fine-tune the model's layers, you may want to try training a linear transformation the existing model's output embedding. Openai has a cookbook on how to do that. You will need some data though - but I think you can try it with ~20 pieces of synthetically generated data.

tutorial from OpenAI https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb

OpenAI Cookbook

OpenAI community repo with lots of examples: https://github.com/openai/openai-cookbook