{"cells":[{"metadata":{},"cell_type":"markdown","source":"
Generate handwritten digits
\n
Generative Adversarial Networks with Keras
\n
\n
"},{"metadata":{},"cell_type":"markdown","source":"
The purpose of this notebook is to try to implement GANs for handwritten digit images generation. Starting from noise images which have same format than my handwritten digit images, I will try to get closer and closer to distribution of images for each class.
"},{"metadata":{},"cell_type":"markdown","source":"# The MNIST dataset"},{"metadata":{},"cell_type":"markdown","source":""},{"metadata":{},"cell_type":"markdown","source":"
MNIST, or Modified National Institute of Standards and Technology is a dataset of 70,000 small square 28×28 pixel grayscale images of handwritten digits between 0 and 9.
"},{"metadata":{},"cell_type":"markdown","source":"# Generative Adversarial networks"},{"metadata":{},"cell_type":"markdown","source":"
GAN, or Generative Adversarial Networks is an architecture for training generative models. Developing a GAN for generating images requires two essential models:\n
- A Generator model that uses inverse convolutional layers to transform an input to a full two-dimensional image of pixel values.\n
- A Discriminator model for classifying whether a given image is real or generated.
"},{"metadata":{},"cell_type":"markdown","source":""},{"metadata":{},"cell_type":"markdown","source":"How does it work ?\n\n
The first neural network is called the Generator. It generates fake data points and passes them to its opponent. That’s the second network, the Discriminator. Its job is to tell which data point is real and which is fake. First, we train the Discriminator on a set of real and fake data points. The discriminator guesses what is real and what is fake. After that, we unveil the real solution to it. Based on the feedback, the discriminator learns what is fake and what is not. Once the discriminator is trained, the Generator comes into play. We give him real and fake data points to train with. Based on these points, its job is to make new data points which fool the Discriminator.
"},{"metadata":{},"cell_type":"markdown","source":"Why not using only real data points ?\n\n
Indeed, it is easy to think that if you train the discriminator on fake images, then it will repeat the mistakes from those. However, the Generator must not get too good too fast. Otherwise the Discriminator can’t keep up. The Generator has to be misled from time to time so that the Discriminator has a chance to keep up. For example, if the Generator has made fake data. It will try to fool the Discriminator. The Discriminator will hopefully do a reasonable job at differentiating the real data points from the fake ones. It then informs the Generator about its decisions and based on this decision, the Generator will set to work again. It’ll make new fakes, but based on the feedback it will try to make them better so that they fool the Discriminator. Then it’s the Discriminator’s turn again, and so on and so forth. And at the end of the day, we’ve used a GAN to make a heap of new data points.
"},{"metadata":{},"cell_type":"markdown","source":"Why is it useful ?\n
\n\n\n
It can be used for all kinds of things. For example, make beautiful new pictures, based on others. Deepfake detection...
"},{"metadata":{},"cell_type":"markdown","source":"----\n
\n
It can be challenging to understand both how GANs work and how deep convolutional neural network models can be trained in a GAN architecture for image generation. A good starting point for beginners is to practice developing and using GANs on standard image datasets used in the field of computer vision, such as the Digit recognizer dataset (MNIST) that includes thousands of handwritten digit images. Using small datasets means that smaller models can be developed and trained quickly, allowing the focus to be put on the model architecture and image generation process itself.\n

\nIf you are already familair with GANs, I suggest you can go further implementing SOTA GANs models with these must-see research papers
"},{"metadata":{},"cell_type":"markdown","source":"# 1. Import and load data"},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom tqdm import tqdm\n%matplotlib inline\n\nfrom sklearn.model_selection import train_test_split\n\nfrom tensorflow.keras.layers import Dense, Dropout, Input, LeakyReLU\nfrom tensorflow.keras.models import Model,Sequential\nfrom tensorflow.keras.datasets import mnist\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.utils import plot_model, to_categorical","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"PATH_TO_DATA = '../input/digit-recognizer/'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def preprocessing(train, split_train_size = 1/7):\n\n X_train = train.drop([\"label\"],\n axis = 1)\n y_train = train[\"label\"]\n\n # Reshape into right format vectors\n X_train = X_train.values.reshape(-1,28,28)\n\n # Apply ohe on labels\n y_train = to_categorical(y_train, num_classes = 10)\n \n # Split the train and the validation set for the fitting\n X_train, X_test, y_train, y_test = train_test_split(X_train, y_train, test_size = split_train_size, random_state=42)\n \n return X_train, X_test, y_train, y_test","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_data(from_MNIST = True):\n \n \"\"\"\n Return ready to use train and test with images and targets\n from_MNIST = True: load data from keras mnist dataset\n from_MNIST = False: load data from digit-recognizer dataset\n \"\"\"\n \n if from_MNIST:\n # Load the data from mnist dataset (70k images)\n (x_train, y_train), (x_test, y_test) = mnist.load_data()\n else:\n # Load train from digit recognizer kaggle dataset (42k images)\n train = pd.read_csv(PATH_TO_DATA + 'train.csv')\n x_train, x_test, y_train, y_test = preprocessing(train)\n \n # Set pixel values between -1 and 1\n x_train = (x_train.astype(np.float32) - 127.5)/127.5\n \n nb_images_train = x_train.shape[0]\n # convert shape of x_train from (60000, 28, 28) to (60000, 784) \n # 784 columns per row\n x_train = x_train.reshape(nb_images_train, 784)\n return (x_train, y_train, x_test, y_test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 2. Define GAN architecture"},{"metadata":{"trusted":true},"cell_type":"code","source":"def adam_optimizer():\n return Adam(lr=0.0002, beta_1=0.5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_generator():\n \n \"\"\"\n Create generator architecture\n \"\"\"\n \n generator=Sequential()\n generator.add(Dense(units=256, input_dim=100))\n generator.add(LeakyReLU(0.2))\n \n generator.add(Dense(units=512))\n generator.add(LeakyReLU(0.2))\n \n generator.add(Dense(units=1024))\n generator.add(LeakyReLU(0.2))\n \n generator.add(Dense(units=784, activation='tanh'))\n \n generator.compile(loss = 'binary_crossentropy', \n optimizer = adam_optimizer())\n return generator\n\ng = create_generator()\ng.summary()\nplot_model(g, show_shapes=True, show_layer_names=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_discriminator():\n \n \"\"\"\n Create discriminator architecture\n \"\"\"\n \n discriminator = Sequential()\n discriminator.add(Dense(units = 1024, input_dim = 784))\n discriminator.add(LeakyReLU(0.2))\n discriminator.add(Dropout(0.3))\n \n \n discriminator.add(Dense(units = 512))\n discriminator.add(LeakyReLU(0.2))\n discriminator.add(Dropout(0.3))\n \n discriminator.add(Dense(units=256))\n discriminator.add(LeakyReLU(0.2))\n \n discriminator.add(Dense(units=1, activation='sigmoid'))\n \n discriminator.compile(loss = 'binary_crossentropy', \n optimizer = adam_optimizer())\n return discriminator\n\nd = create_discriminator()\nd.summary()\nplot_model(d, show_shapes=True, show_layer_names=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
We now create the GAN where we combine the Generator and Discriminator. When we train the generator we will freeze the Discriminator. We will input the noised image of shape 100 units to the Generator. The output generated from the Generator will be fed to the Discriminator.
"},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_gan(discriminator, generator):\n \n # Freeze the weights\n discriminator.trainable=False\n \n # Initialize random noise with generator\n gan_input = Input(shape=(100,))\n x = generator(gan_input)\n gan_output = discriminator(x)\n \n gan = Model(inputs = gan_input, outputs = gan_output)\n \n gan.compile(loss = 'binary_crossentropy', \n optimizer = 'adam')\n return gan\n\ngan = create_gan(d,g)\ngan.summary()\nplot_model(gan, show_shapes=True, show_layer_names=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
Before we start training the model, we will write a function plot_generated_images to plot the generated images. This way we can see how the images are generated. We save the generated images to file that we can view later
"},{"metadata":{},"cell_type":"markdown","source":"# 3. Training"},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_generated_images(epoch, generator, examples=100, dim=(10,10), figsize=(10,10)):\n \n noise = np.random.normal(loc=0, scale=1, size=[examples, 100])\n generated_images = generator.predict(noise)\n generated_images = generated_images.reshape(100,28,28)\n \n plt.figure(figsize=figsize)\n for i in range(generated_images.shape[0]):\n plt.subplot(dim[0], dim[1], i+1)\n plt.imshow(generated_images[i], \n interpolation = 'nearest', \n cmap = 'gray')\n plt.axis('off')\n plt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def training(epochs=1, batch_size=128):\n \n #Loading the data\n (X_train, y_train, X_test, y_test) = load_data(from_MNIST = False)\n batch_count = X_train.shape[0] / batch_size\n \n # Creating GAN\n generator= create_generator()\n discriminator= create_discriminator()\n gan = create_gan(discriminator, generator)\n \n for e in range(1,epochs+1 ):\n #print(\"Epoch %d\" %e)\n #tqdm()\n for _ in range(batch_size):\n #generate random noise as an input to initialize the generator\n noise= np.random.normal(0,1, [batch_size, 100])\n \n # Generate fake MNIST images from noised input\n generated_images = generator.predict(noise)\n \n # Get a random set of real images\n image_batch = X_train[np.random.randint(low=0,high=X_train.shape[0],size=batch_size)]\n \n #Construct different batches of real and fake data \n X= np.concatenate([image_batch, generated_images])\n \n # Labels for generated and real data\n y_dis=np.zeros(2*batch_size)\n y_dis[:batch_size]=0.9\n \n #Pre train discriminator on fake and real data before starting the gan. \n discriminator.trainable=True\n discriminator.train_on_batch(X, y_dis)\n \n #Tricking the noised input of the Generator as real data\n noise= np.random.normal(0,1, [batch_size, 100])\n y_gen = np.ones(batch_size)\n \n # During the training of gan, \n # the weights of discriminator should be fixed. \n #We can enforce that by setting the trainable flag\n discriminator.trainable=False\n \n #training the GAN by alternating the training of the Discriminator \n #and training the chained GAN model with Discriminator’s weights freezed.\n gan.train_on_batch(noise, y_gen)\n \n if e == 1 or e % 20 == 0:\n \n plot_generated_images(e, generator)\n\ntraining(400,128)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# References\n\n* GANs research papers\n* GAN tutorial\n* What is a GAN\n* Maths behind the GAN\n\n# My related articles\n\n* TOP 10 Papers to get started with GANs"},{"metadata":{},"cell_type":"markdown","source":"I am currently working on two notebooks to be released soon:\n- one for dog generation\n- one for abstract art images generation\nOn these, I will try to use SOTA GANs technics 'CycleGans etc...)\n\nTO BE CONTINUED..."},{"metadata":{},"cell_type":"markdown","source":"
\n
\n
Thank you for taking the time to read this notebook. I hope that I was able to answer your questions or your curiosity and that it was quite understandable. any constructive comments are welcome. They help me progress and motivate me to share better quality content. I am above all a passionate person who tries to advance my knowledge but also that of others. If you liked it, feel free to upvote and share my work.
\n
\n
Thank you and may passion guide you.
"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}