bonito VS transformers

There is a technology called Nanopore. I’ve never used it myself but the concept was to be able to sequence samples of nucleic acid out in the field. A quick pubmed search indicated that it can detect miRNA, but maybe with some modifications. https://nanoporetech.com Best of luck with it!

> An unpopular opinion: there are too many other logos that look like the original logo

There are also too many logos that look like the new logo. The first blue circle logos that spring to mind are Blue Circle Cement/Tarmac/Lafarge [1] and Oxford Nanopore [2].

[1]: https://en.wikipedia.org/wiki/Tarmac_(company)

[2]: https://nanoporetech.com/

Oxford Nanopore Technologies (https://nanoporetech.com/) | Front end developer | Full-time | Oxford | Remote (UK)

Oxford Nanopore Technologies is headquartered at the Oxford Science Park outside Oxford, UK, with satellite offices and commercial presence in many global locations across the US, APAC and Europe. Our DNA/RNA sequencing platform is the only technology that offers real-time analysis (for rapid insights), in fully scalable formats from pocket to population scale. Our goal is to enable the analysis of any living thing, by anyone, anywhere.

Tech stack: Electron, Stencil, React, Typescript, RxJS, GRPC

For more details, please email: [email protected]

Ok this isn't really my area but I know that there are labs/companies performing these types of electrical current disturbance measurements of membrane-type proteins for both DNA sequencing (https://nanoporetech.com/) and protein sequencing (https://www.nature.com/articles/s41587-019-0401-y).

Now nanopore sequencing is solid state and gets much longer reads. https://nanoporetech.com/ and https://en.wikipedia.org/wiki/Nanopore_sequencing

Also best of luck with the basecaller, I will say that the latest guppy versions are very good, both in terms of accuracy and speed, as they are GPU accelerated and the best I've seen in accuracy. You may also be interested in Bonito, a tool to generate your own GPU basecalling model or tweak existing models to your data. https://github.com/nanoporetech/bonito.

transformers is a package by Hugging Face, that helps you interact with models on HF Hub (GitHub)

Is t5x an encoder/decoder architecture?

Some more general options.

The Flax ecosystem

https://github.com/google/flax?tab=readme-ov-file

or dm-haiku

https://github.com/google-deepmind/dm-haiku

were some of the best developed communities in the Jax AI field

Perhaps the “trax” repo? https://github.com/google/trax

Some HF examples https://github.com/huggingface/transformers/tree/main/exampl...

Sadly it seems much of the work is proprietary these days, but one example could be Grok-1, if you customize the details. https://github.com/xai-org/grok-1/blob/main/run.py

The HuggingFace transformers library already has support for a similar method called prompt lookup decoding that uses the existing context to generate an ngram model: https://github.com/huggingface/transformers/issues/27722

I don't think it would be that hard to switch it out for a pretrained ngram model.

Most of this tutorial is based on Hugging Face course about Transformers and on Niels Rogge's Transformers tutorials: make sure to check their work and give them a star on GitHub, if you please ❤️

* Superconvergence + LR range finder + Fast AI's Ranger21 optimizer was the goto optimizer for CNNs, and worked fabulously well, but on transformers, the learning rate range finder sadi 1e-3 was the best, whilst 1e-5 was better. However, the 1 cycle learning rate stuck. https://github.com/huggingface/transformers/issues/16013

Thanks! :) I'm pushing them into transformers, pytorch-gemma and collabing with the Gemma team to resolve all the issues :)

The RoPE fix should already be in transformers 4.38.2: https://github.com/huggingface/transformers/pull/29285

My main PR for transformers which fixes most of the issues (some still left): https://github.com/huggingface/transformers/pull/29402

If you want to tinker with the architecture Hugging Face has a FOSS implementation in transformers: https://github.com/huggingface/transformers/blob/main/src/tr...

If you want to reproduce the training pipeline, you couldn't do that even if you wanted to because you don't have access to thousands of A100s.