Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about batch normalization in WGAN_GP #17

Open
YukiRain opened this issue Nov 26, 2017 · 2 comments
Open

Question about batch normalization in WGAN_GP #17

YukiRain opened this issue Nov 26, 2017 · 2 comments

Comments

@YukiRain
Copy link

YukiRain commented Nov 26, 2017

Thank for your work! I'm confused about the batch normalization layer used in the discriminator of wgan_gp. I think there shouldn't be any batch normalization layer in the discriminator.
I think the reason is that the gradient penalty term directly otimizes the gradient loss of each vector sampled between data distribution and generated distribution, each of the vectors has its gradient and the gradient is independent of all other vectors, so the gradient penalty must be calculated separately w.r.t. each sampled vector. If batch normalization is applied in discriminator, the region constrained by 1-Lipchitz, I guess, would be somewhere else instead of "the region between data distribution and generated distribution".
I'm not sure whether i misinterpreted the idea of wgan-gp paper.

@njwfish
Copy link

njwfish commented Sep 29, 2018

You're correct, the bn layer should be removed from the discriminator.

@iskangkang
Copy link

You're correct, the bn layer should be removed from the discriminator.

when I removed the bn layer I find the results become worse,do you have met this problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants