Has anyone succeeded in reproducing the results?

[nashory]

I am still struggling with training VQ-GAN in the first stage, not even the conditional transformer which is a second stage.
The result looks fine before the discriminator loss is injected. BUT using the discriminator loss suddenly ruins the reconstructed images. disc_loss remains 1.0 during the training. Why??

[richcmwang]

The loss goes up with the default parameters/set up very early in the training even before the discriminator kicked in. I am puzzled by this.

[mhh0318]

The same, using discriminator loss makes my result worse.

[nicolasfischoeder]

once the discriminator kicks in everything goes crazy, does somebody have an advice?

[function2-llx]

once the discriminator kicks in everything goes crazy, does somebody have an advice?

Perhaps because the discriminator is not trained at all until the g_loss is introduced to the total loss. According to the code, both the training of discriminator and the introduction of g_loss starts at the self.discriminator_iter_start step. Perhaps we can try training discriminator from the very beginning.

taming-transformers/taming/modules/losses/vqperceptual.py

Lines 106 to 107 in 3ba01b2

	disc_factor = adopt_weight(self.disc_factor, global_step, threshold=self.discriminator_iter_start)
	loss = nll_loss + d_weight * disc_factor * g_loss + self.codebook_weight * codebook_loss.mean()

taming-transformers/taming/modules/losses/vqperceptual.py

Lines 129 to 130 in 3ba01b2

	disc_factor = adopt_weight(self.disc_factor, global_step, threshold=self.discriminator_iter_start)
	d_loss = disc_factor * self.disc_loss(logits_real, logits_fake)