Visualizing Generative Adversarial Networks

$$ V(D, G)= \mathbb{E}_{x \sim p_{data}(x)}\left[\log D(x)\right] + \mathbb{E}_{z \sim p_{z}(z)}\left[\log(1 - D(G(z)))\right] $$

$$ \min_G \max_D V(D, G) $$

We assume a data distribution

Noise distribution

GAN-Definition

for number of training iterations do
- for $k$ steps do
  - Sample minibatch of $m$ noise samples $\{z^{(1)},\dots,z^{(m)}\}$ from noise prior $p_z(z)$
  - Sample minibatch of $m$ examples $\{x^{(1)},\dots,x^{(m)}\}$ from data generating distribution $p_{data}(x)$
  - Update the discriminator by ascending its stochastic gradient: $$ \nabla_{\theta_{d}} \frac{1}{m} \sum_{i=1}^{m}\left[\log D\left(x^{(i)}\right) + \log\left(1 - D\left(G\left(z^{(i)}\right)\right)\right)\right] $$
- Sample minibatch of $m$ noise samples $z^{(1)},\dots,z^{(m)}$ from noise prior $p_z(z)$
- Update the generator by descending its stochastic gradient: $$ \nabla_{\theta_{g}} \frac{1}{m} \sum_{i=1}^{m}\log\left(1 - D\left(G\left(z^{(i)}\right)\right)\right) $$