DeepMind achieves SOTA image recognition with 8.7x less compute

• compare_gan

  4 1,803 0.0 Python

  Discontinued Compare GAN code.

  

  I'm surprised so many people want to see our BigGAN images. Thank you for asking :)

  You can watch the training process here: http://song.tensorfork.com:8097/#images

  It's been going on for a month and a half, but I leave it running mostly as a fishtank rather than to get to a specific objective. It's fun to load it up and look at a new random image whenever I want. Plus I like the idea of my little TPU being like "look at me! I'm doing work! Here's what I've prepared for you!" so I try to keep my little fella online all the time.

  https://i.imgur.com/0O5KZdE.png

  The model is getting quite good. I kind of forgot about it over the past few weeks. StyleGAN could never get anywhere close to this level of detail. I had to spend roughly a year tracking down a crucial bug in the implementation that prevented biggan from working very well until now: https://github.com/google/compare_gan/issues/54

  I've never seen conglomerate pictures like this used in AI training. Do you train models on these 4x4 images? What's the purpose vs a single picture at a time? Does the model know that you're feeding it 4x4 examples, or does it have to figure that out itself?

  Nah, the grid is just for convenient viewing for humans. Robots see one image at a time. (Or more specifically, a batch of images; we happen to use batch size 2 or 4, I forget, so each core sees two images at a time, and then all 8 cores broadcast their gradients to each other and average, so it's really seeing 16 or 32 images at a time.)

  I feel a bit silly plugging our community so much, but it's really true. If you like tricks like this, join the Tensorfork discord:

  https://discord.com/invite/x52Xz3y

  My theory when I set it up was that everyone has little tricks like this, but there's no central repository of knowledge / place to ask questions. But now that there are 1,200+ of us, it's become the de facto place to pop in and share random ideas and tricks.

  For what it's worth, https://thisanimedoesnotexist.ai/ was a joint collaboration of several Tensorfork discord members. :)

  If you want future updates about this specific BigGAN model, twitter is your best bet: https://twitter.com/search?q=(from%3Atheshawwn)%20biggan&src...

• xlnet

  1 153 0.0 Python

  XLNet: fine tuning on RTX 2080 GPU - 8 GB

• WorkOS

  workos.com

  sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project