fastT5 VS sparktorch

I've tried https://github.com/Ki6an/fastT5 but it works with CPU only.

I am working on a project where I fine-tuned a Pegasus model on the Reddit dataset. Now, I need to convert the fine-tuned model to ONNX for the deployment stage. I have followed this guide from Huggingface to convert to the ONNX model for unsupported architects. I got it done but the ONNX model can't generate text. Turned out that Pegasus is an encoder-decoder model and most guides are for either encoder-model (e.g. BERT) or decoder-model (e.g. GPT2). I found the only example of converting an encoder-decoder model to ONNX from here https://github.com/Ki6an/fastT5.

Microsoft Onnx Runtime T5 export tool / FastT5: to support caching, it exports 2 times the decoder part, one with cache, and one without (for the first generated token). So the memory footprint is doubled, which makes the solution difficult to use for these large transformer models.

Here is the fastT5 model code for reference code:https://github.com/Ki6an/fastT5/blob/master/fastT5/onnx_models.py

for more information on the project refer to the repository here.

Was reading the documentation of sparktorch (https://github.com/dmmiller612/sparktorch) which says you need spark >= 2.4.4. But to the best of my knowledge spark2 doesn't have gpu compute capabilities. Does that mean it can only use cpu compute? Am I missing something?