fastdup VS albumentations

Visualizing your dataset (especially large ones) in a low-dimensional embedding space can tell you a lot about the patterns and clusters in your dataset.

We recently release a notebook showing how you can visualize your dataset using DINOv2 models by running it on your CPU.

Yes! No GPUs needed.

We used it to find clusters of similar images, duplicates, and outliers in a subset of the LAION dataset

Try it on your own dataset:

Colab notebook - https://colab.research.google.com/github/visual-layer/fastdup/blob/main/examples/dinov2_notebook.ipynb

GitHub repo - https://github.com/visual-layer/fastdup

Run 1M images from the LAION dataset through the DINOv2 model and cluster the images using a free tool - fastdup.

Another option could be fastdup (https://github.com/visual-layer/fastdup) which is probably most helpful for analysis type objectives.

I recently stumbled upon fastdup a tool that lets you gain insights from a large image/video collection.

There is this fastdup framework which supposedly allows you to find duplicates and similar images. i haven't used it though

I came across fastdup recently https://github.com/visual-layer/fastdup

You can use albumentations if you are comfortable with using open source libraries https://github.com/albumentations-team/albumentations

One of the members of the core team of our open-source library https://albumentations.ai/

It was not the only reason he was hired; it was a solid addition to his already good performance at the interviews.

Or at least that is what the hiring manager later said.

I am one of the creators and maintainers of https://albumentations.ai/.

- 12800+ stars

Is something planned to support data augmentations? Something like https://albumentations.ai/

Here you go: https://albumentations.ai/

I think they should use some data augmentation techniques like I am using for Infinity AI if you wanna see more here. Note that most of these do not work for image generation.

Our second approach was a more automated one. Here the idea was to try out an image augmentation library, Albumentations, and use it for adversarial attacks. This time, instead of one-shot images, we applied the transformations at random time ranges. We chose for these transformations also to be more subtle than then one-shot images, such as vertical flips, grayscaling, downscaling, …

Check out albumentations. If your use case is segmentation, check out the offline augmentation of this project

I think this is the most popular open source project: https://github.com/albumentations-team/albumentations

You can also curate your own dataset by using open source images (https://universe.roboflow.com/search?q=faces%20images%3E1000) and open source augmentations (https://github.com/albumentations-team/albumentations). Or you can do use the augmentation UI (https://docs.roboflow.com/image-transformations/image-augmentation) to apply noise, blurring, shear, crop, etc.