Generative Adversarial Networks:
- generative model
- instead of MLE it uses an adversarial training model
- networks: deep learning
First we train the discriminator, we freeze the generator (so that the generated images are basically random noise) and gradient descent to update the discriminator weights.
Then we train the generator by fixing the discriminator.
The word adversarial should be clear. Using the standard notation we can view the Loss as negative the reward, then
this is a min max game. The Nash Equilibrium for this game is achieved at:
Mode collapse
- The generator settles for just one (or cycles few) generated examples that fool the discriminator: the model is almost indipendent from the noise .
- cat and mouse game: the generator produces a single mode. The discriminator quickly overfits to label as generated. The generator quickly finds a new mode. Repeat…