You might be wondering what the heck are Generative Adversarial Networks (GANs)? and, furthermore, why should you care about them in first place.

Let's start by defining a GAN as a computer program whose main goal is to simply generate fake (non-real) images that closely mimic and resemble anything you can think of, like cats, dogs, humans, hotdogs, snoop-dog, you name it!


This person does not exist

If you head to https://thispersondoesnotexist.com/ you'll see a portrait like the one below. Although this might be difficult to digest, I have to tell you that the person below (let's call her Katy, just for fun), does not exist and has never existed! As hard and surprising as it may sound, Katy's portrait below was generated by a computer program in a matter of seconds. To be more precise, it was generated by a very sophisticated Generative Adversarial Network.

How can something like this even be possible? What kind of witchcraft is this?


How GANs work

GANs have two main components (or agents) that are in a continuous battle of outplaying one another (this is the Adversarial part of it). These two components are the generator and the discriminator, which are a couple of neural networks that work as follows:

  • The task of the generator is to generate images that look like any object or agent we want. The generator gets trained by "seeing" and processing tens of thousands of pictures of the thing we want it to generate images of. Katy's generator was trained by processing a large number of real portraits.

  • The discriminator is another neural network that also gets trained on the same images used to train the generator. However, the discriminator's goal is to tell whether an image is real or not (and that's why it's called discriminator after all).

So, basically, we need to continuously train the generator to create more realistic images, whilst train the discriminator to easily tell a real image from a fake one.

A good comparison often used to explain the relationship between a generator and a discriminator is that of a forger and an art expert: the task of the former is to get better at producing precise forgeries, while the latter should continuously get better at identifying the traits and characteristics that make an original piece of art unique.

Does that mean that Katy in fact exists and the image above is just a replica of a real portrait? No. Katy does not exist. The image was produced by a forger (the generator). However, the generator tried to create the portrait of a woman by including all the features that it thinks a real woman has. Also, the generator is perfectly capable of producing fake images of any other made-up human being, regardless of their age, gender, race, or ethnicity.

One of these two American Gothic paintings is real and the other is fake.


For more technical details on how GANs work, scroll down to the last section: "notes for geeks".

How to revive the past

Several weeks ago, while browsing LinkedIn, I saw a post by one of my contacts that mentioned a paper by AI researchers at Tencent's Applied Research Center (Tencent is the largest gaming company in the world and the owners of WeChat). The paper explains how they created a state-of-the-art GAN that can restore images outstandingly well. They've named their model GFP-GAN, which stands for "Generative Facial Prior - GAN". 

In a nutshell, GFP-GAN leverages the same GAN used on https://thispersondoesnotexist.com website, but it's been adapted in a way that a degraded portrait (e.g. blurry, pixelated, noisy) is taken as reference and mapped to a GAN that can restore the image. Additionally, they've added several controls to the network so the restored images preserve as much as possible the person's facial identity as well as the quality and colocation of key facial features, such as the eyes and mouth.

This doesn't mean GFP-GAN will generate perfectly restored portraits, but it'll do its best to generate one that preserves a person's facial features based on both the blurry portrait and the knowledge that the original network acquired after having been trained on tens of thousands of real portraits.

The following is an image of my grandaunt when she was younger. The image on the left is the original portrait, whereas the image on the right is the restored portrait by the GFP-GAN network; pretty neat, huh?

Wanna try it yourself?

The simplest way to do it is by heading to https://huggingface.co/spaces/akhaliq/GFPGAN and then drag-and-drop the portrait you want to restore, then click on submit and wait for the magic to happen.

Notes for geeks

  • Clarification: the output of the discriminator is not really whether an image is fake or not, but the probability from 0 to 1 of the image being real or not.*
  • While it's perfectly possible to train GANs on vanilla deep neural networks architectures, they are most commonly trained on convolutional neural nets on both components, generator and the discriminator. Convolutional nets are a special type of neural network that are better suited for image processing applications.
  • You can find GFP-GAN published code on GitHub here.
  • This Colab link contains a step-by-step notebook that invokes the trained model for inference.
  • The name of the GAN used on https://thispersondoesnotexist.com/ is StyleGAN2 (an improved version of the original StyleGAN by NVIDIA engineers; this is the paper). 
  • GFP-GAN used a special type of convolutional net called U-net, plus StyleGAN2, plus a gazillion of other mappings and transformations.

* Assuming the discriminator uses a sigmoid activation function at its last layer, otherwise the network outputs a logit whose range goes from -∞ to ∞.