dropbox-sdk-python VS transformers

The creation of the dropbox-sdk-python repo was almost certainly overseen by Guido van Rossum since he was working at Dropbox at the time. There is a note in the Smartsheet SDK repo that parts of it were developed by Dropbox as well.

Or do you or should I just use this 'dropbox-sdk-python' (https://github.com/dropbox/dropbox-sdk-python)?

I'm not drawing conclusions from this, but I did some poking around..

I ran `git clone` on the repos of both rsync[0] and the official Dropbox SDK for Python[1]. I then ran sloccount on both of them.

---

rsync came in at a total of 51,410 physical source lines of code.

The Dropbox SDK came in at a total of 74,140 physical source lines of code.

---

[0] https://git.samba.org/rsync.git

[1] https://github.com/dropbox/dropbox-sdk-python

In this example, we will demonstrate how we can integrate with a third-party storage provider like Dropbox to create backups of files uploaded to Appwrite. For the sake of this example, we will be using Dropbox’s Python SDK. A similar concept applies to other API providers like Box or Google Drive. So let’s get started.

transformers is a package by Hugging Face, that helps you interact with models on HF Hub (GitHub)

Is t5x an encoder/decoder architecture?

Some more general options.

The Flax ecosystem

https://github.com/google/flax?tab=readme-ov-file

or dm-haiku

https://github.com/google-deepmind/dm-haiku

were some of the best developed communities in the Jax AI field

Perhaps the “trax” repo? https://github.com/google/trax

Some HF examples https://github.com/huggingface/transformers/tree/main/exampl...

Sadly it seems much of the work is proprietary these days, but one example could be Grok-1, if you customize the details. https://github.com/xai-org/grok-1/blob/main/run.py

The HuggingFace transformers library already has support for a similar method called prompt lookup decoding that uses the existing context to generate an ngram model: https://github.com/huggingface/transformers/issues/27722

I don't think it would be that hard to switch it out for a pretrained ngram model.

Most of this tutorial is based on Hugging Face course about Transformers and on Niels Rogge's Transformers tutorials: make sure to check their work and give them a star on GitHub, if you please ❤️

* Superconvergence + LR range finder + Fast AI's Ranger21 optimizer was the goto optimizer for CNNs, and worked fabulously well, but on transformers, the learning rate range finder sadi 1e-3 was the best, whilst 1e-5 was better. However, the 1 cycle learning rate stuck. https://github.com/huggingface/transformers/issues/16013

Thanks! :) I'm pushing them into transformers, pytorch-gemma and collabing with the Gemma team to resolve all the issues :)

The RoPE fix should already be in transformers 4.38.2: https://github.com/huggingface/transformers/pull/29285

My main PR for transformers which fixes most of the issues (some still left): https://github.com/huggingface/transformers/pull/29402

If you want to tinker with the architecture Hugging Face has a FOSS implementation in transformers: https://github.com/huggingface/transformers/blob/main/src/tr...

If you want to reproduce the training pipeline, you couldn't do that even if you wanted to because you don't have access to thousands of A100s.