jukebox VS DALLE-mtf

openai/jukebox: Music Generation

https://github.com/openai/jukebox The demo code is there.

I didn't even know about it until I heard Sam Altman casually mention it in an interview, I was expecting some basic tunes generator, but this is so amazing! I mean yeah the voices are not clear, it's muffled, but look at how far have image models progressed, if you applied the same amount of collaborative effort here, the results could be amazing! ElevenLabs showed how good and clear can AI-created voices sound. The only reason I can think of is that the Jukebox code is under view license only.

OpenAI had this figured out 3 years ago: https://openai.com/blog/jukebox/ . You could then even define your own text. Model is open source too.

They've had jukebox for a few years now, so I'm sure some new model will get released and explode overnight, like what chatGPT did.

That already exists

This model is designed to output raw audio.

However, there are many models which do output midi. That's actually much simpler, and has been done already a few years ago.

I thought OpenAI did this. But then, I might misremember, because their Jukebox actually also seems to produce raw audio (https://openai.com/blog/jukebox/).

However, midi generation is so easy, you even find it in some tutorials: https://www.tensorflow.org/tutorials/audio/music_generation

This vision is in line with EleutherAI, a non-profit organization founded in July 2020 by a group of researchers. Driven by the perceived opacity and the challenge of reproducibility in AI, their goal was to create leading open-source language models.

My current idea is to utilize the EleutherAI pythia (Databricks Dolly). I would like to know whether translating the Dolly-15k dataset into the desired language using state-of-the-art translation techniques like DeepL would be a viable approach to fine-tune the Pythia base model. I want to use this model for semantic search, so perfection is not a necessity.

There are open source AI efforts, like EleutherAI. Needless to say, they are lagging behind big players, but it's better than nothing.

The new GPT 3.5 isn't aware what are GPT-3.5 or davinci-002 (repeatable) and claimed that it was designed by EleutherAI and has only 6 bil parameters (wasn't been able to repeat but didn't really try).

Hi, my name is Stella Biderman and I run EleutherAI, the one of the foremost non-profit research institutes in the world that trains and studies large language models. I have been involved with the majority of models to hold the title “largest open source GPT model in the world” and have dabbled in exploring using plagiarism detection tools to identify code written by GPT-J.

dolly-v2-12bis a 12 billion parameter causal language model created by Databricks that is derived from EleutherAI’s Pythia-12b and fine-tuned on a ~15K record instruction corpus generated by Databricks employees and released under a permissive license (CC-BY-SA)

It is true that Emad needs to find an appropriate business model. The good news is that the hype is still undergoing. I'm sure that Emad can grab another round of liquidity injection. He got plenty of resources. Remember he is also from the finance industry. He got https://www.eleuther.ai/ which can supply a secured, in-house custom LLM equivalent to bloombergGPT.

By promoting fully open-source AI, i.e. making datasets, models, methodology and codebases freely available and transparent. What OpenAI claimed to be aiming for, basically.