{"cells":[{"metadata":{},"cell_type":"markdown","source":"reference:<br>\nhttps://github.com/soumith/ganhacks\n"},{"metadata":{},"cell_type":"markdown","source":"<font size=4, color=Red>Version 30</font><br>\nVersion 2: Only changed epoches from 500 to 25, expected excute time about half an hour, in order to make comparision to verify some other methods.<br>\nVersion 3: Changed batch_size from 32 --> 64.<br>\n<font color=Green>Failed, Version 4</font>: Changed batch_size from 64 --> 16.<br>\nVersion 5: Changed batch_size back to 32, Changed transform function from RandomResizedCrop --> Resize.<br>\nVersion 6: Changed batch_size to 16, Changed transform function back to RandomResizedCrop, relized version 4.<br>\nVersion 7: Changed back to version 2, then add seed_everything.<br>\nVersion 8: Changed the seed of seed_everything.<br>\nVersion 9: Changed data_clean method, from 'abs(xdiff-ydiff)<100' --> '0.8<(xdiff/ydiff)<1.25'.<br>\nVersion 11: Changed data_clean method, from 'min(xdiff,ydiff)>=64' --> 'min(xdiff,ydiff)>=128'.<br>\nVersion 12: Changed data_clean back, changed seed, real_label'.<br>\nVersion 13: Epochs=500.<br>\nVersion 14: Epochs=25.<br>\nVersion 15: Changed the data_loader, crop into (xmin, ymin, xmin+min(xdiff, ydiff), ymin+min(xdiff, ydiff)).<br>\nVersion 16: Changed the data_loader back, add truncated_norm in the final generation.<br>\nVersion 17: Confirm truncated_norm works, at least it doesn't harm. Then used random label flip.<br>\nVersion 18: Changed back and value of label.<br>\nVersion 18: Changed value of label.<br>\nVersion 19-24: Changed value of label.<br>\nVersion 25: 500 EPOCHS.<br>\nVersion 26: 25 EPOCHS.<br>\nVersion 27: LeakyRelu in both G and D.<br>\nVersion 28: Train D more, change the lr.<br>\nVersion 29: LeakyRelu in D, Relu in G, change the lr both to 0.005(which is 0.0005 before).<br>\nVersion 30: Seems that lr would have impact on the LB, so change the lr both to 0.0001.<br>\nVersion 33: Change data_clean function to (abs(ydiff - xdiff)/min(ydiff, xdiff)<=0.25) and ((xdiff>=64) and (ydiff>=64))"},{"metadata":{},"cell_type":"markdown","source":"<font size=4, color=Red>Summary</font><br>\n500 epochs, V1: 43.85, <font color=Red>V13: 42.94</font>  V25: 42.95<br>\n25  epochs, V2: 92.21, V3: 122.95, V4: 124.52, V5: Failed. V6: 97.38, V7: 98.63, V8: 105.56, V9: 130.99, V11: 114.69, V12: 91.82, <br>\n25 epochs, V14: 91.82, V15: 104.70, V16: 91.22, V17: 185.94, V18: 102.24, V19: 96.71, V20: 89.67, V21: 93.90, V22: 91.29, V23: 96.05, V24: 93.83<br>\n25 epochs, <font color=Red>V26: 89.78,</font> "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport cv2\nimport random\nimport matplotlib.pyplot as plt\nfrom PIL import Image\nfrom skimage.util import random_noise\nimport xml.etree.ElementTree as ET\n\nimport torch\nfrom torch import nn, optim\nimport torch.nn.functional as F\nfrom torchvision import datasets, transforms\nfrom torchvision.utils import save_image\nfrom torch.autograd import Variable\nfrom collections import OrderedDict\nfrom torch.utils.data import TensorDataset, DataLoader,Dataset\n\nfrom tqdm import tqdm_notebook as tqdm","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def seed_everything(seed=19960720):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True\nseed_everything()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def summary(model, input_size, batch_size=-1, device=\"cuda\"):\n\n    def register_hook(module):\n\n        def hook(module, input, output):\n            class_name = str(module.__class__).split(\".\")[-1].split(\"'\")[0]\n            module_idx = len(summary)\n\n            m_key = \"%s-%i\" % (class_name, module_idx + 1)\n            summary[m_key] = OrderedDict()\n            summary[m_key][\"input_shape\"] = list(input[0].size())\n            summary[m_key][\"input_shape\"][0] = batch_size\n            if isinstance(output, (list, tuple)):\n                summary[m_key][\"output_shape\"] = [\n                    [-1] + list(o.size())[1:] for o in output\n                ]\n            else:\n                summary[m_key][\"output_shape\"] = list(output.size())\n                summary[m_key][\"output_shape\"][0] = batch_size\n\n            params = 0\n            if hasattr(module, \"weight\") and hasattr(module.weight, \"size\"):\n                params += torch.prod(torch.LongTensor(list(module.weight.size())))\n                summary[m_key][\"trainable\"] = module.weight.requires_grad\n            if hasattr(module, \"bias\") and hasattr(module.bias, \"size\"):\n                params += torch.prod(torch.LongTensor(list(module.bias.size())))\n            summary[m_key][\"nb_params\"] = params\n\n        if (\n            not isinstance(module, nn.Sequential)\n            and not isinstance(module, nn.ModuleList)\n            and not (module == model)\n        ):\n            hooks.append(module.register_forward_hook(hook))\n\n    device = device.lower()\n    assert device in [\n        \"cuda\",\n        \"cpu\",\n    ], \"Input device is not valid, please specify 'cuda' or 'cpu'\"\n\n    if device == \"cuda\" and torch.cuda.is_available():\n        dtype = torch.cuda.FloatTensor\n    else:\n        dtype = torch.FloatTensor\n\n    # multiple inputs to the network\n    if isinstance(input_size, tuple):\n        input_size = [input_size]\n\n    # batch_size of 2 for batchnorm\n    x = [torch.rand(2, *in_size).type(dtype) for in_size in input_size]\n    # print(type(x[0]))\n\n    # create properties\n    summary = OrderedDict()\n    hooks = []\n\n    # register hook\n    model.apply(register_hook)\n\n    # make a forward pass\n    # print(x.shape)\n    model(*x)\n\n    # remove these hooks\n    for h in hooks:\n        h.remove()\n\n    print(\"----------------------------------------------------------------\")\n    line_new = \"{:>20}  {:>25} {:>15}\".format(\"Layer (type)\", \"Output Shape\", \"Param #\")\n    print(line_new)\n    print(\"================================================================\")\n    total_params = 0\n    total_output = 0\n    trainable_params = 0\n    for layer in summary:\n        # input_shape, output_shape, trainable, nb_params\n        line_new = \"{:>20}  {:>25} {:>15}\".format(\n            layer,\n            str(summary[layer][\"output_shape\"]),\n            \"{0:,}\".format(summary[layer][\"nb_params\"]),\n        )\n        total_params += summary[layer][\"nb_params\"]\n        total_output += np.prod(summary[layer][\"output_shape\"])\n        if \"trainable\" in summary[layer]:\n            if summary[layer][\"trainable\"] == True:\n                trainable_params += summary[layer][\"nb_params\"]\n        print(line_new)\n\n    # assume 4 bytes/number (float on cuda).\n    total_input_size = abs(np.prod(input_size) * batch_size * 4. / (1024 ** 2.))\n    total_output_size = abs(2. * total_output * 4. / (1024 ** 2.))  # x2 for gradients\n    total_params_size = abs(total_params.numpy() * 4. / (1024 ** 2.))\n    total_size = total_params_size + total_output_size + total_input_size\n\n    print(\"================================================================\")\n    print(\"Total params: {0:,}\".format(total_params))\n    print(\"Trainable params: {0:,}\".format(trainable_params))\n    print(\"Non-trainable params: {0:,}\".format(total_params - trainable_params))\n    print(\"----------------------------------------------------------------\")\n    print(\"Input size (MB): %0.2f\" % total_input_size)\n    print(\"Forward/backward pass size (MB): %0.2f\" % total_output_size)\n    print(\"Params size (MB): %0.2f\" % total_params_size)\n    print(\"Estimated Total Size (MB): %0.2f\" % total_size)\n    print(\"----------------------------------------------------------------\")\n    # return summary","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Generator and Discriminator"},{"metadata":{"trusted":true},"cell_type":"code","source":"class Generator(nn.Module):\n    def __init__(self, nz=128, channels=3):\n        super(Generator, self).__init__()\n        \n        self.nz = nz\n        self.channels = channels\n        \n        def convlayer(n_input, n_output, k_size=4, stride=2, padding=0):\n            block = [\n                nn.ConvTranspose2d(n_input, n_output, kernel_size=k_size, stride=stride, padding=padding, bias=False),\n                nn.BatchNorm2d(n_output),\n                nn.ReLU(inplace=True),\n#                 nn.LeakyReLU(0.2, inplace=True),\n            ]\n            return block\n\n        self.model = nn.Sequential(\n            *convlayer(self.nz, 1024, 4, 1, 0), # Fully connected layer via convolution.\n            *convlayer(1024, 512, 4, 2, 1),\n            *convlayer(512, 256, 4, 2, 1),\n            *convlayer(256, 128, 4, 2, 1),\n            *convlayer(128, 64, 4, 2, 1),\n            nn.ConvTranspose2d(64, self.channels, 3, 1, 1),\n            nn.Tanh()\n        )\n\n    def forward(self, z):\n        z = z.view(-1, self.nz, 1, 1)\n        img = self.model(z)\n        return img","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class Discriminator(nn.Module):\n    def __init__(self, channels=3):\n        super(Discriminator, self).__init__()\n        \n        self.channels = channels\n\n        def convlayer(n_input, n_output, k_size=4, stride=2, padding=0, bn=False):\n            block = [nn.Conv2d(n_input, n_output, kernel_size=k_size, stride=stride, padding=padding, bias=False)]\n            if bn:\n                block.append(nn.BatchNorm2d(n_output))\n            block.append(nn.LeakyReLU(0.2, inplace=True))\n            return block\n\n        self.model = nn.Sequential(\n            *convlayer(self.channels, 64, 3, 1, 1),\n            *convlayer(64, 128, 4, 2, 1),\n            *convlayer(128, 256, 4, 2, 1, bn=True),\n            *convlayer(256, 512, 4, 2, 1, bn=True),\n            *convlayer(512, 1024, 4, 2, 1, bn=True),\n            nn.Conv2d(1024, 1, 4, 1, 0, bias=False),  # FC with Conv.\n        )\n        \n    def forward(self, imgs):\n        logits = self.model(imgs)\n        out = torch.sigmoid(logits)\n    \n        return out.view(-1, 1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Cleaning"},{"metadata":{"trusted":true},"cell_type":"code","source":"import glob\nimage = glob.glob('../input/all-dogs/all-dogs/*')\nbreed = glob.glob('../input/annotation/Annotation/*')\nannot = glob.glob('../input/annotation/Annotation/*/*')\nprint(len(image), len(breed), len(annot))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_bbox(annot):\n    \n    \"\"\"\n    This extracts and returns values of bounding boxes\n    \"\"\"\n    xml = annot\n    tree = ET.parse(xml)\n    root = tree.getroot()\n    objects = root.findall('object')\n    bbox = []\n    for o in objects:\n        bndbox = o.find('bndbox')\n        xmin = int(bndbox.find('xmin').text)\n        ymin = int(bndbox.find('ymin').text)\n        xmax = int(bndbox.find('xmax').text)\n        ymax = int(bndbox.find('ymax').text)\n        \n        bbox.append((xmin, ymin, xmax, ymax))\n    return bbox","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_image(annot):\n    \"\"\"\n    Retrieve the corresponding image given annotation file\n    \"\"\"\n    img_path = '../input/all-dogs/all-dogs/'\n    file = annot.split('/')\n    img_class = file[-2]\n    img_filename = img_path+file[-1]+'.jpg'\n    return img_filename, img_class","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"select_dogs = []\nselect_bbox = []","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_class_dict(breed):\n    class_dict = {}\n    for i in range(len(breed)):\n        class_ = breed[i].split('/')\n        class_dict[class_[-1]] = i\n    return class_dict\n\nclass_dict = get_class_dict(breed)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_cleaned_data(image, annot, class_dict):\n    select_dogs = []\n    select_bbox = []\n    select_labels = []\n    \n    for i in range(len(image)):\n        bbox = get_bbox(annot[i])\n        bbox = bbox[0]\n\n        dog, dog_class = get_image(annot[i])\n        if dog == '../input/all-dogs/all-dogs/n02105855_2933.jpg':   # this jpg is not in the dataset\n            continue\n        im = Image.open(dog)\n        \n        xdiff = abs(bbox[2] - bbox[0])\n        ydiff = abs(bbox[3] - bbox[1])\n        \n        if ((abs(ydiff - xdiff)/min(ydiff, xdiff)<=0.25) and ((xdiff>=64) and (ydiff>=64))):\n#         if (0.8<(ydiff/xdiff)<1.25) and (min(xdiff, ydiff)>=128):            \n            select_dogs.append(dog)\n            select_bbox.append(bbox)\n            select_labels.append(class_dict[dog_class])\n            \n    return select_dogs, select_bbox, select_labels\n\nselect_dogs, select_bbox, select_labels = get_cleaned_data(image, annot, class_dict)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(len(select_dogs), len(select_bbox), len(select_labels))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(select_bbox[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.preprocessing import MultiLabelBinarizer, LabelEncoder, OneHotEncoder\n\ndef prepare_labels(y):\n    values = np.array(y)\n    onehot_encoder = OneHotEncoder(sparse=False, categories='auto')\n    onehot_encoded = onehot_encoder.fit_transform(values.reshape(values.shape[0], 1))\n    y = onehot_encoded\n    return y\n\nlabels_encoded = prepare_labels(select_labels)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Loader"},{"metadata":{"trusted":true},"cell_type":"code","source":"class dogs_Dataset(Dataset):\n    def __init__(self, datafolder, datatype='train', index=[], \\\n                 transform = transforms.Compose([transforms.RandomResizedCrop(128),transforms.ToTensor()])):\n\n        self.datafolder = datafolder\n        self.datatype = datatype\n        \n        self.image_files_list = [select_dogs[i] for i in index]\n        self.bbox_list = [select_bbox[i] for i in index]\n        self.label_list = [labels_encoded[i] for i in index]\n        \n        self.transform = transform\n        \n        self.imgs = []\n        \n        for idx in range(len(self.image_files_list)):\n            img_name = self.image_files_list[idx]\n            bbox = self.bbox_list[idx]\n            \n            img = Image.open(img_name)\n            img = img.crop(bbox)\n            self.imgs.append(img)\n\n    def __len__(self):\n        return len(self.imgs)\n\n    def __getitem__(self, idx):\n        \n        image = self.imgs[idx]\n        image = self.transform(image)\n\n        if self.datatype == 'train':\n            label = self.label_list[idx]\n        else:\n            label = torch.zeros(1)\n        return image, label","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"batch_size = 32\nimage_size = 64\n\n\ntransform = transforms.Compose([transforms.RandomResizedCrop(image_size, scale=(0.8, 1.0)),\n#                                 transforms.Resize(image_size),\n                                transforms.RandomHorizontalFlip(p=0.5),\n                                transforms.ToTensor(),\n                                transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n\n\ntrain_idx = [i for i in range(len(select_dogs))]\ntrain_data = dogs_Dataset('../input/all-dogs/all-dogs/', index=train_idx, transform=transform)\ntrain_loader = torch.utils.data.DataLoader(train_data, shuffle=True,\n                                           batch_size=batch_size, num_workers=4)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"imgs, label = next(iter(train_loader))\nimgs = imgs.numpy().transpose(0, 2, 3, 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(20, 16))\nfor ii, img in enumerate(imgs):\n    ax = fig.add_subplot(4, 8, ii +1, xticks=[], yticks=[])\n    plt.imshow( (img+1.)/2. )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Parameters of GAN"},{"metadata":{"trusted":true},"cell_type":"code","source":"LR_G = 0.0005\nLR_D = 0.0005\n\nbeta1 = 0.5\n\nreal_label = 0.8\nfake_label = 0.0\nnz = 128\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Initialize models and optimizers"},{"metadata":{"trusted":true},"cell_type":"code","source":"netG = Generator(nz).to(device)\nnetD = Discriminator().to(device)\n\ncriterion = nn.BCELoss()\n\noptimizerD = optim.Adam(netD.parameters(), lr=LR_D, betas=(beta1, 0.999))\noptimizerG = optim.Adam(netG.parameters(), lr=LR_G, betas=(beta1, 0.999))\n\nfixed_noise = torch.randn(25, nz, 1, 1, device=device)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"summary(netG, (128,))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"summary(netD, (3, 64, 64))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def show_generated_img():\n    noise = torch.randn(1, nz, 1, 1, device=device)\n    gen_image = netG(noise).to(\"cpu\").clone().detach().squeeze(0)\n    gen_image = gen_image.numpy().transpose(1, 2, 0)\n    plt.imshow(gen_image)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Training loop"},{"metadata":{"trusted":true},"cell_type":"code","source":"epochs = 780","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for epoch in range(epochs):\n    for ii, (real_images, train_labels) in enumerate(train_loader):\n\n#         real_label = np.random.uniform(0.7, 0.9)\n        \n        netD.zero_grad()\n        real_images = real_images.to(device)\n        batch_size = real_images.size(0)\n        labels = torch.full((batch_size, 1), real_label, device=device)\n\n        output = netD(real_images)\n        errD_real = criterion(output, labels)\n        errD_real.backward()\n        D_x = output.mean().item()\n\n        # train with fake\n        noise = torch.randn(batch_size, nz, 1, 1, device=device)\n        fake = netG(noise)\n        labels.fill_(fake_label)\n        output = netD(fake.detach())\n        errD_fake = criterion(output, labels)\n        errD_fake.backward()\n        D_G_z1 = output.mean().item()\n        errD = errD_real + errD_fake\n        optimizerD.step()\n\n        ############################\n        # (2) Update G network: maximize log(D(G(z)))\n        ###########################\n        netG.zero_grad()\n        labels.fill_(real_label)  # fake labels are real for generator cost\n        output = netD(fake)\n        errG = criterion(output, labels)\n        errG.backward()\n        D_G_z2 = output.mean().item()\n        optimizerG.step()\n        \n        if (ii+1) % (len(train_loader)//2) == 0:\n            print('[%d/%d][%d/%d] Loss_D: %.4f Loss_G: %.4f D(x): %.4f D(G(z)): %.4f / %.4f'\n                  % (epoch + 1, epochs, ii+1, len(train_loader),\n                     errD.item(), errG.item(), D_x, D_G_z1, D_G_z2))\n            \n#             valid_image = netG(fixed_noise)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Generated results "},{"metadata":{"trusted":true},"cell_type":"code","source":"from scipy.stats import truncnorm\ndef truncated_normal(size, threshold=1):\n    values = truncnorm.rvs(-threshold, threshold, size=size)\n    return values","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"gen_z = torch.randn(32, nz, 1, 1, device=device)\ngen_images = netG(gen_z).to(\"cpu\").clone().detach()\ngen_images = gen_images.numpy().transpose(0, 2, 3, 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(25, 16))\nfor ii, img in enumerate(gen_images):\n    ax = fig.add_subplot(4, 8, ii + 1, xticks=[], yticks=[])\n    plt.imshow((img+1.)/2.)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Make predictions and submit"},{"metadata":{"trusted":true},"cell_type":"code","source":"if not os.path.exists('../output_images'):\n    os.mkdir('../output_images')\n    \nim_batch_size = 50\nn_images=10000\n\nfor i_batch in range(0, n_images, im_batch_size):\n    z = truncated_normal((im_batch_size, nz, 1, 1), threshold=1)\n    gen_z = torch.from_numpy(z).float().to(device)\n#     gen_z = torch.randn(im_batch_size, nz, 1, 1, device=device)\n    gen_images = (netG(gen_z)+1.)/2.\n    images = gen_images.to(\"cpu\").clone().detach()\n    images = images.numpy().transpose(0, 2, 3, 1)\n    for i_image in range(gen_images.size(0)):\n        save_image(gen_images[i_image, :, :, :], os.path.join('../output_images', f'image_{i_batch+i_image:05d}.png'))\n\nimport shutil\nshutil.make_archive('images', 'zip', '../output_images')","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}