faster-whisper VS ROCm

Hi, I don't know what's SOTA, but I got good results with these (open source, on-device) :

https://github.com/SYSTRAN/faster-whisper (speech-to-text)

I've been using this:

https://github.com/bugbakery/transcribee

It's noticeably work-in-progress but it does the job and has a nice UI to edit transcriptions and speakers etc.

It's running on the CPU for me, would be nice to have something that can make use of a 4GB Nvidia GPU, which faster-whisper is actually able to [1]

https://github.com/SYSTRAN/faster-whisper?tab=readme-ov-file...

Faster-whisper (https://github.com/SYSTRAN/faster-whisper)

For our real-time STT needs, we'll employ a fantastic library called faster-whisper.

Thermostats: https://www.sinopetech.com/en/products/thermostat/

I haven't tried running a local text-to-speech engine backed by an LLM to control Home Assistant. Maybe someone is working on this already?

TTS: https://github.com/SYSTRAN/faster-whisper

LLM: https://github.com/Mozilla-Ocho/llamafile/releases

LLM: https://huggingface.co/TheBloke/Nous-Hermes-2-Mixtral-8x7B-D...

It would take some tweaking to get the voice commands working correctly.

Could someone elaborate how is this accomplished and is there any quality disparity compared to original whisper?

Repos like https://github.com/SYSTRAN/faster-whisper makes immediate sense about why it's faster than the original, but this one, not so much, especially considering it's even much faster.

Cool! I had the same project idea recently. You may be interested in this for the step of speech2text: https://github.com/SYSTRAN/faster-whisper

That's the implication. If the distil models are same format as original openai models then the Distil models can be converted for faster-whisper use as per the conversion instructions on https://github.com/guillaumekln/faster-whisper/

So then we'll see whether we get the 6x model speedup on top of the stated 4x faster-whisper code speedup.

guillaumekln/faster-whisper

Yep, did exactly that. IMO he threw a fit, even though AMD was working with him squashing bugs. https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuec...

Ok, I wonder what's wrong. maybe it's this? https://stackoverflow.com/questions/4959621/error-1001-in-cl...

Nope. Anything about this on the arch wiki? Nope

This bug report[2] from 2021? Maybe I need to update my groups.

[2]: https://github.com/RadeonOpenCompute/ROCm/issues/1411

    $ ls -la /dev/kfd

HMM is, I believe, a Linux feature.

AMD added HMM support in ROCm 5.0 according to this: https://github.com/RadeonOpenCompute/ROCm/blob/develop/CHANG...

Woot AMD now supports APU? I sold my notebook as i hit a wall when trying rocm [1] Is there a list oft Wirkung apu's ?

[1] https://github.com/RadeonOpenCompute/ROCm/issues/1587

Last I heard he's abandoned working with AMD products.

https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuec...

They're talking about the meltdown he had on stream [1] (in front of the mentioned pirate flag), that ended with him saying he'd stop using AMD hardware [2]. He recanted this two weeks after talking with AMD [3].

Maybe he'll succeed, but this definitely doesn't scream stability to me. I'd be wary of investing money into his ventures (but then I'm not a VC, so what do I know).

[1] https://www.youtube.com/watch?v=Mr0rWJhv9jU

[2] https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuec...

[3] https://twitter.com/realGeorgeHotz/status/166980346408248934...

As for rocm support on consumer devices, AMD wont even clarify what devices are supported. https://github.com/RadeonOpenCompute/ROCm/pull/1738

I tried to train another model using InceptionResNetV2 and the same issues happens. Also, this happens even using the model.predict() method if using the GPU. Probably this is an issue related to the AMD Radeon RX 6700 XT or some mine misconfiguration. System Inormation: ArchLinux 6.1.32-1-lts - AMD Radeon RX 6700 XT - gfx1031 Opened issues: - https://github.com/RadeonOpenCompute/ROCm/issues/2250 - https://github.com/ROCmSoftwarePlatform/tensorflow-upstream/issues/2125