Transformers-Tutorials VS fiftyone

Most of this tutorial is based on Hugging Face course about Transformers and on Niels Rogge's Transformers tutorials: make sure to check their work and give them a star on GitHub, if you please ❤️

I'm pretraining on 850K grayscale spectrograms of birdsongs. I'm on epoch 400 out of 800 and the loss has declined from about 1.2 to 0.7. I don't really have a sense of what is "good enough" and I guess the only way I can judge is by looking at the reconstruction. I'm doing that using this notebook as a guide and right now it's doing quite badly.

More tutorials can be found at https://github.com/NielsRogge/Transformers-Tutorials.

Ive been looking for an off the shelf encoder-decoder document understanding model for key information extraction. I found a great Huggingface implementation with concise notebook examples. However, the token classification model outputs a list of token labels corresponding bounding boxes for the token, but, not the text contained within the labeled bounding boxes themselves. Am I missing something? LayoutLMv2 describes itself as being capable of information extraction but without extracting the text I feel like it's fallen short of that ambition.

Example Notebooks

My experience has been much like this. For twenty years, I’ve emphasized scientific and engineering discovery in my work as an academic researcher, publishing these findings at the top conferences in computer vision, AI, and related fields. Yet, at my company, we focus on infrastructure that enables others to unlock scientific discovery. We have built a software framework that enables its users to do better work when training models and curating datasets with large unstructured, visual data — it’s kind of like a PyTorch++ or a Snowflake for unstructured data. This software stack, called FiftyOne in its single-user open source incarnation and FiftyOne Teams in its collaborative enterprise version, has garnered millions of installations and a vibrant user community.

We will use the Hugging Face transformers and diffusers libraries for inference, FiftyOne for data management and visualization, and scikit-image for evaluation metrics.

With all that background out of the way, let’s turn theory into practice and learn how to use clustering to structure our unstructured data. We’ll be leveraging two open-source machine learning libraries: scikit-learn, which comes pre-packaged with implementations of most common clustering algorithms, and fiftyone, which streamlines the management and visualization of unstructured data:

FiftyOne is the leading open-source toolkit for the curation and visualization of unstructured data, built on top of MongoDB. It leverages the non-relational nature of MongoDB to provide an intuitive interface for working with datasets consisting of images, videos, point clouds, PDFs, and more.

Welcome to our weekly FiftyOne tips and tricks blog where we recap interesting questions and answers that have recently popped up on Slack, GitHub, Stack Overflow, and Reddit.

If you want to perform emoji searches locally with the same visual interface, you can do so with the Emoji Search plugin for FiftyOne.

kalpit-S contributed #2354 – added help link for Mapbox configuration in App