lyra VS jukebox

Since Ballard's codec is "AI" based, can you add google's lyrav2 ( https://github.com/google/lyra ) and Facebook's/meta EnCodec ( https://github.com/facebookresearch/encodec ).

Also I don't seem to be able to access your page, so there might be error.

Finally, when doing opus comparison it's good now to denote if it is using Lace or NoLace decoder post processing filters that became available in opus 1.5 (note, this feature need to be enabled at compile time, and defying decode a new API call needs to be made to force higher complexity decoder) . See https://opus-codec.org/demo/opus-1.5/

I've thought about doing something similar for google's voice compression lyra https://github.com/google/lyra

edit: It seems Lyra is opensource https://github.com/google/lyra

1) https://github.com/google/lyra/releases/tag/v1.3.0

Very impressive.

It'd be interesting to see what the lift would be to get encoding & decoding running in webassembly/wasm. Further, it'd be really neat to try to take something like the tflife_model_wrapper[1] and to get it backed by something like tsjs-tflite[2] perhaps even atop for example tfjs-backend-webgpu[3].

Longer run, the web-nn[4] spec should hopefully simplify/bake-in some of these libraries to the web platform, make running inference much easier. But there's still an interesting challenge & question, that I'm not sure how to tackle; how to take native code, compile it to wasm, but to have some of the implementation provided else-where.

[1] https://github.com/google/lyra/pull/89/files#diff-ed2f131a63...

[2] https://www.npmjs.com/package/@tensorflow/tfjs-tflite

[3] https://www.npmjs.com/package/@tensorflow/tfjs-backend-webgp...

[4] https://www.w3.org/TR/webnn/

You can find an Android, Linux and macOS app here: https://github.com/google/lyra/actions/runs/3156735950

Here's the repository: https://github.com/google/lyra and it's licensed under Apache.

openai/jukebox: Music Generation

https://github.com/openai/jukebox The demo code is there.

I didn't even know about it until I heard Sam Altman casually mention it in an interview, I was expecting some basic tunes generator, but this is so amazing! I mean yeah the voices are not clear, it's muffled, but look at how far have image models progressed, if you applied the same amount of collaborative effort here, the results could be amazing! ElevenLabs showed how good and clear can AI-created voices sound. The only reason I can think of is that the Jukebox code is under view license only.

OpenAI had this figured out 3 years ago: https://openai.com/blog/jukebox/ . You could then even define your own text. Model is open source too.

They've had jukebox for a few years now, so I'm sure some new model will get released and explode overnight, like what chatGPT did.

That already exists

This model is designed to output raw audio.

However, there are many models which do output midi. That's actually much simpler, and has been done already a few years ago.

I thought OpenAI did this. But then, I might misremember, because their Jukebox actually also seems to produce raw audio (https://openai.com/blog/jukebox/).

However, midi generation is so easy, you even find it in some tutorials: https://www.tensorflow.org/tutorials/audio/music_generation