Whisper prompt tuning

1. peft

   1 27 18,737 9.5 Python

   🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

   

   Hi everyone. Recently I've been looking into the PEFT library (https://github.com/huggingface/peft) and I was wondering if it would be possible to do prompt tuning with OpenAI's Whisper model. They have an example notebook for tuning Whisper with LoRA (https://colab.research.google.com/drive/1vhF8yueFqha3Y3CpTHN6q9EVcII9EYzs?usp=sharing) but I'm not sure how to go about changing it to use prompt tuning instead.

2. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

3. transformers

   2 213 145,543 10.0 Python

   🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

   

   From what I know, Whisper already supports prompting (https://github.com/huggingface/transformers/pull/22496). Can I somehow freeze the whole model and tune exclusively the prompt or would I need to write an implementation from scratch?

Suggest a related project