PaLM-rlhf-pytorch VS whisper.cpp

ChatGPT isn't open sourced so we don't know what the actual implementation is. I think you can read Open Assistant's source code for application design. If that is too much, try Open Chat Toolkit's source code for developer tools . If you need very bare implementation, you should go for lucidrains/PaLM-rlhf-pytorch.

AFAIK, it is not. They are using the open-source re-implementation of Phil Wang (aka lucidrains), which is available here: https://github.com/lucidrains/PaLM-rlhf-pytorch

Not sure what do you mean by putting source code in double quote, but I don't think the source code is petabytes of text. GPT-2 implementation is few hundred lines of Python (in HuggingFace). PaLM + RLHF - Pytorch (Basically ChatGPT but with PaLM) is less than 1000 lines.

GitHub Link: https://github.com/lucidrains/PaLM-rlhf-pytorch

Well... Not necessarily. Actually, if you want to be extra thrifty, you could even go without an ML expert. Just use an open-source one, like LaMDA or PaLM. After that, use chatGPT to build you a basic front end (which would still be better than CAI lol).

and PaLM with RLHF from Phil Wang (open model, needs to be trained): https://github.com/lucidrains/PaLM-rlhf-pytorch

Closest you can get is probably with Google T5-Flan [1].

It is not the size of the model or the text it was trained on that makes ChatGPT so performant. It is the additional human assisted training to make it respond well to instructions. Open source versions of that are just starting to see the light of day [2].

[1] https://huggingface.co/google/flan-t5-xxl

[2] https://github.com/lucidrains/PaLM-rlhf-pytorch

whisper.cpp [1] has a karaoke example that uses ffmpeg's drawtext filter to display rudimentary karaoke-like captions. It also supports diarisation. Perhaps it could be a starting point to create a better script that does what you need.

--

1: https://github.com/ggerganov/whisper.cpp/blob/master/README....

The ggml library is one of the first library for local LLM interference. It’s a pure C library that converts models to run on several devices, including desktops, laptops, and even mobile device - and therefore, it can also be considered as a tinkering tool, trying new optimizations, that will then be incorporated into other downstream projects. This tool is at the heart of several other projects, powering LLM interference on desktop or even mobile phones. Subprojects for running specific LLMs or LLM families exists, such as whisper.cpp.

I'm not sure if it is _fully_ openai compatible, but whispercpp has a server bundled that says it is "OAI-like": https://github.com/ggerganov/whisper.cpp/tree/master/example...

I don't have any direct experience with it... I've only played around with whisper locally, using scripts.

unless i'm misunderstanding `whisper.cpp` seems to support streaming & the repository includes a native example[0] and a WASM example[1] with a demo site[2].

[0]: https://github.com/ggerganov/whisper.cpp/tree/master/example...

Usage 1: Good to transcribe audio. An example use case could be to summarize YouTube videos or long courses. Usage 2: You talk with voice to your AI that responds with text (later with audio too). - https://github.com/ggerganov/whisper.cpp

🗣️🎙️ whisper.cpp (offline speech-to-text transcription, models trained by OpenAI, CLI based, browser based)

Has anyone played around with Whisper C++ for swift? I'm hitting a snag even on the demo. I've downloaded the github repo and everything matches up with this video [ https://youtu.be/b10OHCDHDQ4 ] but when he hits the transcribe button, it actually prints out the captioning. When I do it, it skips that part and just says "Done...". But it, does everything else - plays the audio, says it's transcribing.. just doesn't show me the transcription: and it's not in the debug window either. But the demo isn't throwing any errors, and I haven't messed with the code really so this is their example. https://github.com/ggerganov/whisper.cpp