Alias-Free GAN

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com
Machine Learning machine-translation Deep Learning reinforcement-learning tpu

InfluxDB
Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
Our great sponsors

  • StyleCLIP

    Official Implementation for "StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery" (ICCV 2021 Oral)

    • The first two demo videoes are interesting examples of using StyleCLIP's global directions to guide an image toward a "smiling face" as noted in that paper with smooth interpolation: https://github.com/orpatashnik/StyleCLIP

    I had ran a few chaotic experiments with StyleCLIP a few months ago which would work very well with smooth interpolation: https://minimaxir.com/2021/04/styleclip/

  • alias-free-gan

    Alias-Free GAN project website and code

    • This isn't true. I do ML every day. You are mistaken.

    I click the website. I search "model". I see two results. Oh no, that means no download link to model.

    I go to the github. Maybe model download link is there. I see zero code: https://github.com/NVlabs/alias-free-gan

    zero code. Zero model.

    You, and everyone like you, who are gushing with praise and hypnotized by pretty images and a nice-looking pdf, are doing damage by saying that this is correct and normal.

    The thing that's useful to me, first and foremost, is a model. Code alone isn't useful.

    Code, however, is the recipe to create the model. It might take 400 hours on a V100, and it might not actually result in the model being created, but it slightly helps me.

    There is no code here.

    Do you think that the pdf is helpful? Yeah, maybe. But I'm starting to suspect that the pdf is in fact a tech demo for nVidia, not a scientific contribution whose purpose is to be helpful to people like me.

    Okay? Model first. Code second. Paper third.

    Every time a tech demo like this comes out, I'd like you to check that those things exist, in that order. If it doesn't, it's not reproducible science. It's a tech demo.

    I need to write something about this somewhere, because a large number of people seem to be caught in this spell. You're definitely not alone, and I'm sorry for sounding like I was singling you out. I just loaded up the comment section, saw your comment, thought "Oh, awesome!" clicked through, and went "Oh no..."

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • NVAE

    The Official PyTorch Implementation of "NVAE: A Deep Hierarchical Variational Autoencoder" (NeurIPS 2020 spotlight paper)

    • non-commercially.

    It's a great example of the difference between open source (which it is) and free software which it is not. So we're back to square one where it is probably best to clean-room the implementation from the paper, which is nearly useless to reproduce the model.

    [1] https://github.com/NVlabs/NVAE/blob/master/LICENSE

  • compare_gan

    Discontinued Compare GAN code.

    • Roughly every assumption you've stated is mistaken.

    I would say that your view here, is what I thought ML would be when I got in. If I had your faith in the process still, I would be saying the same things you're saying here.

    The reason I'm saying the exact opposite, to ensure what you've said becomes the norm.

    Let's go through your points. I'll address each of them in detail.

    Think of playing pool at a bar together with your coworker. You've been on the job for some years; they just got their Github credentials, and are eager to get started.

    While you're playing pool, your friend starts trying to convince you of something you know isn't true. What do you do? You listen, chat, and keep playing pool.

    Your theory is that they'll learn on the job that what they're saying makes no sense, so, your best bet for now is to relax and keep playing pool.

    You're trying to convince me of your position. Unfortunately, based on the things that you've been saying, it indicates you haven't had a lot of experience doing what you're proposing. If you had, you'd be saying something approximate to what I'm saying now. Which of us should change their minds?

    I probably should. I spent two years trying to convince myself that none of what I was saying was true.

    That's called "gaslighting": https://en.wikipedia.org/wiki/Gaslighting

    I was reluctant to call explicit attention to that word, since I really was trying to chill with you and just talk.

    But if you're trying to understand why I was stressed, it's because I really felt that many of the papers I tried to reproduce, use for my own purposes, or integrate into my bag of tricks, were claiming things that I'd say are mistaken knowledge.

    You seem to be under the impression that, when Karras releases the contribution, that the science will be verified.

    The science doesn't get verified. Karras is already working on the next thing.

    The verification here is that the videos clearly show the results. That's nice. That gives me a target to aim for, if I wanted to try Karras' method.

    But it doesn't help me verify Karras' claims. Firstly, there's no way to know whether I've achieved "success," or something approximate to success. Maybe my model is mostly right, but there was some curious quirk during training that made it slightly different. I'm not worried about that case though.

    The real problem is that there aren't any tools when things go wrong. When I try to reproduce a paper by reading it, there's nothing to help you. Your position is "just be smarter." My position is, "I've tried. Many times."

    Either I'm very stupid, or the paper seems to be mistaken.

    That's how I end up feeling after most of the papers I tried to replicate. Many of these replication attempts lasted weeks. My longest replication lasted over a year, before I found the crucial bug preventing me from doing exactly what you want me to do. (BatchNorm layers were initialized to 0 instead of 1 in BigGan-Deep's "unofficial but closest we have" codebase: https://github.com/google/compare_gan/issues/54)

    If you haven't had this experience, you will. The only reason you're saying the things you're saying, is because you haven't spent a lot of time trying. I feel this in the core of my being; if it's mistaken, please, I'd love to know.

    Let's start with a simple example.

    https://github.com/tensorflow/tensor2tensor/pull/1883

    Now, here's a model in tensor2tensor, a pretty popular lib. I explain in the PR why this model was "catastrophically broken, from the beginning, but may have seemed like it worked."

    I would say that many, many ML papers have such an error.

    So when you're saying "reproduce the model," you mean "reproduce the errors in their code," if the code isn't available. Which it isn't here, until September. Therefore, it's not a scientific contribution until September.

    Now, from what I hear, your position seems to be that in September, the science will happen. That's true. The science may happen, because Karras.

    Most of us don't learn from Karras. Karras is impactful. But there's a whole long tail of followers that try to follow Karras' example. And those often don't release models: https://news.ycombinator.com/item?id=27127198

    The reply goes into detail about, why aren't model's released? Is it less frequent now than it was before? But my point is, in that case -- that thread -- I believe science wasn't happening. Do you agree?

    If we don't agree on that point, then I feel there's a fundamental difference of opinion. We'll have to agree to disagree, or persuade each other to change our minds. If you want more examples, I can give you many.

    My contention is that if I tried to reproduce the model in that thread -- which I did, successfully, with BigGAN-Deep in Tensorflow -- it will take me over a year. Which it did, for BigGAN-Deep.

    Your feelings are, well yes, but the paper gave you some useful ideas to use.

    I'm saying, the code didn't work at all. The model was the only thing that saved me. I reverse engineered the DeepMind public model release, including the tensorflow graph, looking for the difference between that model and the training code I was using.

    The final fix, was to change 0 to 1.

    The model worked.

    Either I am very stupid, or we're in territory where a certain scientific rigor is warranted.

    The reason that I'm speaking my mind now, here, on a Karras' release, is because most releases aren't Karras' quality. They leave parts out of the process, like Karras is doing here. Sometimes they come later. Most of the time, they don't.

    Now. As someone who has done what I've described above, for two years straight, which of my assumptions feel mistaken?

  • tensor2tensor

    Discontinued Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

    • Roughly every assumption you've stated is mistaken.

    I would say that your view here, is what I thought ML would be when I got in. If I had your faith in the process still, I would be saying the same things you're saying here.

    The reason I'm saying the exact opposite, to ensure what you've said becomes the norm.

    Let's go through your points. I'll address each of them in detail.

    Think of playing pool at a bar together with your coworker. You've been on the job for some years; they just got their Github credentials, and are eager to get started.

    While you're playing pool, your friend starts trying to convince you of something you know isn't true. What do you do? You listen, chat, and keep playing pool.

    Your theory is that they'll learn on the job that what they're saying makes no sense, so, your best bet for now is to relax and keep playing pool.

    You're trying to convince me of your position. Unfortunately, based on the things that you've been saying, it indicates you haven't had a lot of experience doing what you're proposing. If you had, you'd be saying something approximate to what I'm saying now. Which of us should change their minds?

    I probably should. I spent two years trying to convince myself that none of what I was saying was true.

    That's called "gaslighting": https://en.wikipedia.org/wiki/Gaslighting

    I was reluctant to call explicit attention to that word, since I really was trying to chill with you and just talk.

    But if you're trying to understand why I was stressed, it's because I really felt that many of the papers I tried to reproduce, use for my own purposes, or integrate into my bag of tricks, were claiming things that I'd say are mistaken knowledge.

    You seem to be under the impression that, when Karras releases the contribution, that the science will be verified.

    The science doesn't get verified. Karras is already working on the next thing.

    The verification here is that the videos clearly show the results. That's nice. That gives me a target to aim for, if I wanted to try Karras' method.

    But it doesn't help me verify Karras' claims. Firstly, there's no way to know whether I've achieved "success," or something approximate to success. Maybe my model is mostly right, but there was some curious quirk during training that made it slightly different. I'm not worried about that case though.

    The real problem is that there aren't any tools when things go wrong. When I try to reproduce a paper by reading it, there's nothing to help you. Your position is "just be smarter." My position is, "I've tried. Many times."

    Either I'm very stupid, or the paper seems to be mistaken.

    That's how I end up feeling after most of the papers I tried to replicate. Many of these replication attempts lasted weeks. My longest replication lasted over a year, before I found the crucial bug preventing me from doing exactly what you want me to do. (BatchNorm layers were initialized to 0 instead of 1 in BigGan-Deep's "unofficial but closest we have" codebase: https://github.com/google/compare_gan/issues/54)

    If you haven't had this experience, you will. The only reason you're saying the things you're saying, is because you haven't spent a lot of time trying. I feel this in the core of my being; if it's mistaken, please, I'd love to know.

    Let's start with a simple example.

    https://github.com/tensorflow/tensor2tensor/pull/1883

    Now, here's a model in tensor2tensor, a pretty popular lib. I explain in the PR why this model was "catastrophically broken, from the beginning, but may have seemed like it worked."

    I would say that many, many ML papers have such an error.

    So when you're saying "reproduce the model," you mean "reproduce the errors in their code," if the code isn't available. Which it isn't here, until September. Therefore, it's not a scientific contribution until September.

    Now, from what I hear, your position seems to be that in September, the science will happen. That's true. The science may happen, because Karras.

    Most of us don't learn from Karras. Karras is impactful. But there's a whole long tail of followers that try to follow Karras' example. And those often don't release models: https://news.ycombinator.com/item?id=27127198

    The reply goes into detail about, why aren't model's released? Is it less frequent now than it was before? But my point is, in that case -- that thread -- I believe science wasn't happening. Do you agree?

    If we don't agree on that point, then I feel there's a fundamental difference of opinion. We'll have to agree to disagree, or persuade each other to change our minds. If you want more examples, I can give you many.

    My contention is that if I tried to reproduce the model in that thread -- which I did, successfully, with BigGAN-Deep in Tensorflow -- it will take me over a year. Which it did, for BigGAN-Deep.

    Your feelings are, well yes, but the paper gave you some useful ideas to use.

    I'm saying, the code didn't work at all. The model was the only thing that saved me. I reverse engineered the DeepMind public model release, including the tensorflow graph, looking for the difference between that model and the training code I was using.

    The final fix, was to change 0 to 1.

    The model worked.

    Either I am very stupid, or we're in territory where a certain scientific rigor is warranted.

    The reason that I'm speaking my mind now, here, on a Karras' release, is because most releases aren't Karras' quality. They leave parts out of the process, like Karras is doing here. Sometimes they come later. Most of the time, they don't.

    Now. As someone who has done what I've described above, for two years straight, which of my assumptions feel mistaken?

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts