{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Memorizer GAN - pyTorch"},{"metadata":{},"cell_type":"markdown","source":"[](http://)This kernel is just a translation of memorizer GAN by chris from tf/keras to pytorch. It is based on the following two kernels:\n\n### UPDATE:\n- Memorizer GANs have been disallowed [here](https://www.kaggle.com/c/generative-dog-images/discussion/102701)\n- They don't serve any purpose other than filling one's ego of being at a high rank on the meaningless public LB\n\n\n1. [Dog Memorizer GAN by Chris](https://www.kaggle.com/cdeotte/dog-memorizer-gan)\n2. [Memorizer CGAN for dummies by Nanashi](https://www.kaggle.com/jesucristo/memorizer-cgan-for-dummies)\n\nFor a wonderful explanation of what's happening here, refer to: [Memorization GAN Explained by Chris](https://www.kaggle.com/c/generative-dog-images/discussion/99215)"},{"metadata":{},"cell_type":"markdown","source":"# Imports"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import gzip, pickle\nimport os\nimport numpy as np\nimport pandas as pd\nimport random\nimport shutil\nimport numpy as np\nimport os\n\nimport torch\nfrom torch import nn, optim\nimport torch.nn.functional as F\nfrom torchvision import datasets, transforms\nfrom torchvision.utils import save_image\nfrom torch.utils.data import Dataset, DataLoader\n\n\nfrom scipy import linalg\nimport pathlib\nimport urllib\nimport warnings\nfrom PIL import Image\nfrom tqdm import tqdm_notebook as tqdm","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","collapsed":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"markdown","source":"# Crop and display images\nClearly copy pasted from chris' kernel"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"ComputeLB = False\nDogsOnly = False #for some reason DogsOnly reduce my LB score (probably because of \"Mi\" of MiFID)\n\nimport numpy as np, pandas as pd, os\nimport xml.etree.ElementTree as ET \nimport matplotlib.pyplot as plt, zipfile \nfrom PIL import Image \n\nROOT = '../input/generative-dog-images/'\nif not ComputeLB: ROOT = '../input/'\nIMAGES = os.listdir(ROOT + 'all-dogs/all-dogs/')\nbreeds = os.listdir(ROOT + 'annotation/Annotation/') \n\nidxIn = 0; namesIn = []\nimagesIn = np.zeros((25000,64,64,3))\n\n# CROP WITH BOUNDING BOXES TO GET DOGS ONLY\n# https://www.kaggle.com/paulorzp/show-annotations-and-breeds\nif DogsOnly:\n    for breed in breeds:\n        for dog in os.listdir(ROOT+'annotation/Annotation/'+breed):\n            try: img = Image.open(ROOT+'all-dogs/all-dogs/'+dog+'.jpg') \n            except: continue           \n            tree = ET.parse(ROOT+'annotation/Annotation/'+breed+'/'+dog)\n            root = tree.getroot()\n            objects = root.findall('object')\n            for o in objects:\n                bndbox = o.find('bndbox') \n                xmin = int(bndbox.find('xmin').text)\n                ymin = int(bndbox.find('ymin').text)\n                xmax = int(bndbox.find('xmax').text)\n                ymax = int(bndbox.find('ymax').text)\n                w = np.min((xmax - xmin, ymax - ymin))\n                img2 = img.crop((xmin, ymin, xmin+w, ymin+w))\n                img2 = img2.resize((64,64), Image.ANTIALIAS)\n                imagesIn[idxIn,:,:,:] = np.asarray(img2)\n                namesIn.append(breed)\n                idxIn += 1\n    idx = np.arange(idxIn)\n    np.random.shuffle(idx)\n    imagesIn = imagesIn[idx,:,:,:]\n    namesIn = np.array(namesIn)[idx]\n    \n# RANDOMLY CROP FULL IMAGES\nelse:\n    IMAGES = np.sort(IMAGES)\n    np.random.seed(810)\n    x = np.random.choice(np.arange(20579),10000)\n    np.random.seed(None)\n    for k in range(len(x)):\n        img = Image.open(ROOT + 'all-dogs/all-dogs/' + IMAGES[x[k]])\n        w = img.size[0]; h = img.size[1];\n        if (k%2==0)|(k%3==0):\n            w2 = 100; h2 = int(h/(w/100))\n            a = 18; b = 0          \n        else:\n            a=0; b=0\n            if w<h:\n                w2 = 64; h2 = int((64/w)*h)\n                b = (h2-64)//2\n            else:\n                h2 = 64; w2 = int((64/h)*w)\n                a = (w2-64)//2\n        img = img.resize((w2,h2), Image.ANTIALIAS)\n        img = img.crop((0+a, 0+b, 64+a, 64+b))    \n        imagesIn[idxIn,:,:,:] = np.asarray(img)\n        namesIn.append(IMAGES[x[k]])\n        #if idxIn%1000==0: print(idxIn)\n        idxIn += 1\n    \n# DISPLAY CROPPED IMAGES\nx = np.random.randint(0,idxIn,25)\nfor k in range(5):\n    plt.figure(figsize=(15,3))\n    for j in range(5):\n        plt.subplot(1,5,j+1)\n        img = Image.fromarray( imagesIn[x[k*5+j],:,:,:].astype('uint8') )\n        plt.axis('off')\n        if not DogsOnly: plt.title(namesIn[x[k*5+j]],fontsize=11)\n        else: plt.title(namesIn[x[k*5+j]].split('-')[1],fontsize=11)\n        plt.imshow(img)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# DataLoader"},{"metadata":{"trusted":true},"cell_type":"code","source":"class dog_dataset(Dataset):\n    def __init__(self, train_y, train_X, zeros, device):\n        self.train_y = torch.Tensor(train_y).to(device)\n        self.train_X = torch.Tensor(train_X).to(device)\n        self.zeros = torch.Tensor(zeros).to(device)\n        \n    def __len__(self):\n        return len(train_y)\n\n    def __getitem__(self, idx):\n        return self.train_y[idx], self.train_X[idx], self.zeros[idx]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nbs = 256\n\ntrain_y = (imagesIn[:10000,:,:,:]/255.).reshape((-1,12288))\ntrain_X = np.zeros((10000,10000))\nfor i in range(10000): train_X[i,i] = 1\nzeros = np.zeros((10000,12288))\n\ndata_set = dog_dataset(train_y, train_X, zeros, device)    \ndata_loader = DataLoader(data_set, bs)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Sanity check\nprint(train_y.shape, train_X.shape, zeros.shape)\nprint(train_y[0].shape, train_X[0].shape, zeros[0].shape)\nprint(len(train_y), len(train_X), len(zeros))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Discriminator"},{"metadata":{"trusted":true},"cell_type":"code","source":"class Discriminator(nn.Module):\n\n    def __init__(self):\n        super(Discriminator, self).__init__()\n        self.fc1 = nn.Linear(10000, 12288)\n        self.conv1 = nn.Conv2d(1, 1, (2, 1), bias=False)\n        \n    def forward(self, imgs, imgnames):\n        x = self.fc1(imgnames)\n        x = torch.sigmoid(x)\n        x = torch.cat((imgs, x), dim=1).view(-1, 1, 2, 12288)\n        x = self.conv1(x)\n        return x.view(-1, 12288)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Train discriminator\nWe are training the discriminator ahead of generator here. It can be done together with the generator but I am imitating what Chris did."},{"metadata":{"trusted":true},"cell_type":"code","source":"lr = 0.005\nepochs = 500\nnetD = Discriminator().to(device)\noptimizerD = optim.Adam(netD.parameters(), lr=lr)\ncriteria = nn.BCELoss()\nnetD.conv1.weight = nn.Parameter(torch.Tensor([[[[ -1.0],\n                                    [1.0]]]]).to(device))    \nfor param in netD.conv1.parameters():\n    param.requires_grad = False ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# TRAIN DISCRIMINATOR NETWORK\nfor k in tqdm(range(epochs)):\n    for i, (y, X, Zeros) in enumerate(data_loader):\n        netD.zero_grad()\n        y_pred = netD(Zeros, X)\n        loss = criteria(y_pred, y)\n        loss.backward()\n        optimizerD.step()\n    if (k + 1) % 2 == 0:    \n        print(f\"Epoch: {k+1}/{epochs} | Loss: {loss}\")    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Discriminator recall from memory"},{"metadata":{"trusted":true},"cell_type":"code","source":"for k in range(5):\n    plt.figure(figsize=(15,3))\n    for j in range(5):\n        xx = torch.Tensor(np.zeros((10000))).to(device)\n        xx[np.random.randint(10000)] = 1\n        plt.subplot(1,5,j+1)\n#         img = netD([zeros[0,:].reshape((-1,12288)),xx.reshape((-1,10000))]).reshape((-1,64,64,3))\n        img = netD(torch.Tensor(zeros[0,:]).to(device).reshape((-1,12288)),xx.reshape((-1,10000))).reshape((-1,64,64,3))\n        img = img.detach().cpu().numpy()\n        img = Image.fromarray((255 * img).astype('uint8').reshape((64,64,3)))\n        plt.imshow(img)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Generator"},{"metadata":{"trusted":true},"cell_type":"code","source":"class Generator(nn.Module):\n    def __init__(self):\n        super(Generator, self).__init__()\n        self.fc1 = nn.Linear(10000, 12288)\n        \n    def forward(self, imgnames):\n        x = self.fc1(imgnames)\n        return x, imgnames.view(-1, 10000)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def show_image():\n    plt.figure(figsize=(15,3))\n    for j in range(5):\n        with torch.no_grad():\n            xx = np.zeros((10000))\n            xx[np.random.randint(10000)] = 1\n            plt.subplot(1,5,j+1)\n            inp = torch.Tensor(xx.reshape((-1,10000))).to(device)\n            img = netG(inp)[0].reshape((-1,64,64,3)).to(\"cpu\").clone().detach().numpy()\n            img = Image.fromarray( (img).astype('uint8').reshape((64,64,3)))\n            plt.axis('off')\n            plt.imshow(img)\n    plt.show()  \n    \ndef show_d_images(imgs, title):\n    with torch.no_grad():\n        imgs = imgs.reshape((-1,64,64,3))\n        plt.figure(figsize=(15,3))\n        for j in range(5):\n            plt.subplot(1,5,j+1)\n            img = imgs[j].detach().cpu().numpy()\n            img = Image.fromarray((255*img).astype('uint8').reshape((64,64,3)))\n            plt.title(title)\n            plt.imshow(img)\n        plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"lr = 0.01\nbeta1 = 0.5\nnetG = Generator().to(device)\noptimizerG = optim.Adam(netG.parameters(), lr=lr)\ncriterion = nn.MSELoss()\n\nnetD.conv1.weight = nn.Parameter(torch.Tensor([[[[ -1.],\n                                    [1.]]]]).to(device))\n\n# Discriminator is already trained\nfor param in netD.parameters():\n    param.requires_grad = False","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"epochs = 50\nfor epoch in tqdm(range(epochs)):\n    for i, (y, X, Zeros) in enumerate(data_loader):\n        ############################|\n        # (2) Train only the Generator\n        ############################\n        netG.zero_grad()\n        fake, seed = netG(X)\n        y_pred = netD(Zeros, seed)\n        errG = criterion(fake, y_pred)\n        errG.backward()\n        optimizerG.step()\n    if (epoch+1) % 5 == 0:    \n        print(f\"Epoch: {epoch+1}: G_Loss: {errG}\")\n        show_d_images(fake, \"Generator output\")                ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# DogGenerator class"},{"metadata":{"trusted":true},"cell_type":"code","source":"class DogGenerator():\n    index = 0   \n    t = [\n        transforms.RandomCrop((48, 48), padding=None, pad_if_needed=True, fill=0, padding_mode='symmetric'),\n        transforms.Scale((64,64))\n    ]\n    tfms = transforms.Compose([\n                                transforms.RandomHorizontalFlip(p=0.5),\n#                                 transforms.RandomApply(t, p=0.5),\n                                transforms.ColorJitter(brightness=(1,1.3), contrast=(1,1.3), saturation=0, hue=0)\n                               ])\n    def getDog(self,seed):\n        xx = torch.Tensor(np.zeros((10000))).to(device)\n        xx[self.index] = 0.999999\n        xx[np.random.randint(10000)] = 0.000001\n        img = netG(xx.reshape((-1,10000)))[0].reshape((64,64,3)).detach().cpu().numpy() * 255\n        self.index = (self.index+1)%10000\n        return self.tfms(Image.fromarray( img.astype('uint8')))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"d = DogGenerator()\nfor k in range(3):\n    plt.figure(figsize=(20,5))\n    for j in range(5):\n        plt.subplot(1,5,j+1)\n        img = d.getDog(seed = np.random.normal(0,1,100))\n        plt.axis('off')\n        plt.imshow(img)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Creat, zip and submit the images"},{"metadata":{"trusted":true},"cell_type":"code","source":"z = zipfile.PyZipFile('images.zip', mode='w')\nd = DogGenerator()\nfor k in range(10000):\n    img = d.getDog(np.random.normal(0,1,100))\n    f = str(k)+'.png'\n    img.save(f,'PNG'); z.write(f); os.remove(f)\nz.close()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Motivation\nI only started with GANs a few weeks ago and I wasn't able to understand chris' memorizer GAN kernel for a long time. Thus, to consolidate and make sure I finally understand his kernel (and the associated controversies) I decided to implement it using pytorch. I think I now understand it quite well and so I am making this kernel public, it might help someone.  "},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}