{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Dog Memorizer GAN\n## DISCLAIMER: This may not be a valid comp solution\nIn this kernel we attempt to create a new kind of semi-supervised GAN. (Typical GAN design is presented in Nanashi's great tutorial [here][4]). Instead of having the Generator and Discriminator learn at the same time, we will first train the Discriminator to memorize all the training images. Next the Discriminator will teach the images to the Generator. We give the Generator poor memory (with a bottleneck) in hopes that it will learn to generalize. For our Kaggle submission, we will ask this Generator to output a mixture of (the generalized) images it has learned. Since the images are stored in a conv net, we hope to get a generalized conceptual mixture (versus a pixel blend). (This kernel is inspired by the tutorial [here][1] and by my previous kernels [here][2] and [here][3]).\n\n![image](http://playagricola.com/Kaggle/gan7419.png)\n\nA GAN consists of a Generator and Discriminator. After being trained, a Generator is a robot artist that draws dog images. During training the Discriminator teaches the Generator how to draw a dog. (And typically the G teaches the D to disguish real from fake dogs). The Generator never sees any images of dogs. Instead it continually attempts to draw a dog and is coached by the Discriminator. In this kernel, the **Memorizer** Generator is coached to memorize images from the training set. (We hope generalization arises from poor memory). In contrast, a **Generalizing** Generator is coached to generalize images!\n\n# Load and Crop Images\n\n[1]: https://medium.com/datadriveninvestor/generative-adversarial-network-gan-using-keras-ce1c05cfdfd3\n[2]: https://www.kaggle.com/cdeotte/dog-autoencoder\n[3]: https://www.kaggle.com/cdeotte/supervised-generative-dog-net\n[4]: https://www.kaggle.com/jesucristo/gan-introduction","metadata":{}},{"cell_type":"code","source":"import pandas as pd","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:18:19.504202Z","iopub.execute_input":"2023-01-03T06:18:19.504859Z","iopub.status.idle":"2023-01-03T06:18:19.511917Z","shell.execute_reply.started":"2023-01-03T06:18:19.504803Z","shell.execute_reply":"2023-01-03T06:18:19.509888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# initialize list of lists\npath1 = '../input/fgvc-aircraft/fgvc-aircraft-2013b/fgvc-aircraft-2013b/data/images_manufacturer_val.txt'\npath2 = '../input/fgvc-aircraft/fgvc-aircraft-2013b/fgvc-aircraft-2013b/data/images_manufacturer_test.txt'\npath3 = '../input/fgvc-aircraft/fgvc-aircraft-2013b/fgvc-aircraft-2013b/data/images_manufacturer_train.txt'\ndata = []\nwith open(path1) as f:\n    lines = f.readlines()\n    for line in lines:\n        name = line.split()[0]+'.jpg'\n        manu = ' '.join(line.split()[1:])\n        data.append([name, manu])\nwith open(path2) as f:\n    lines = f.readlines()\n    for line in lines:\n        name = line.split()[0]+'.jpg'\n        manu = ' '.join(line.split()[1:])\n        data.append([name, manu])\nwith open(path3) as f:\n    lines = f.readlines()\n    for line in lines:\n        name = line.split()[0]+'.jpg'\n        manu = ' '.join(line.split()[1:])\n        data.append([name, manu])\n\n# Create the pandas DataFrame\nlabels = pd.DataFrame(data, columns=['filename', 'Manufacturer'])\nlabels","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:18:19.520055Z","iopub.execute_input":"2023-01-03T06:18:19.520343Z","iopub.status.idle":"2023-01-03T06:18:19.694801Z","shell.execute_reply.started":"2023-01-03T06:18:19.520286Z","shell.execute_reply":"2023-01-03T06:18:19.693881Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(labels.loc[labels['filename'] == \"2042721.jpg\"].Manufacturer.values)\nprint(len(labels.loc[labels['filename'] == \"wrong\"].Manufacturer.values))","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:18:19.698948Z","iopub.execute_input":"2023-01-03T06:18:19.699369Z","iopub.status.idle":"2023-01-03T06:18:19.721059Z","shell.execute_reply.started":"2023-01-03T06:18:19.699191Z","shell.execute_reply":"2023-01-03T06:18:19.720361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np, pandas as pd, os\nimport xml.etree.ElementTree as ET \nimport matplotlib.pyplot as plt, zipfile \nfrom PIL import Image \n\nROOT = '../input/fgvc-aircraft/fgvc-aircraft-2013b/fgvc-aircraft-2013b/data/'\nIMAGES = os.listdir(ROOT + 'images')\n\nlabel_dict = {}\nimage_count = {}\nk = 0\nfor i in IMAGES:\n    if labels.loc[labels['filename'] == i].Manufacturer.values[0] not in label_dict:\n        label_dict[labels.loc[labels['filename'] == i].Manufacturer.values[0]] = k\n        image_count[labels.loc[labels['filename'] == i].Manufacturer.values[0]] = 0\n        k += 1\n    image_count[labels.loc[labels['filename'] == i].Manufacturer.values[0]] += 1\n# label_dict\nimage_count","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:18:19.722197Z","iopub.execute_input":"2023-01-03T06:18:19.722633Z","iopub.status.idle":"2023-01-03T06:18:45.731249Z","shell.execute_reply.started":"2023-01-03T06:18:19.722585Z","shell.execute_reply":"2023-01-03T06:18:45.730222Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(label_dict.keys())","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:18:45.735228Z","iopub.execute_input":"2023-01-03T06:18:45.737398Z","iopub.status.idle":"2023-01-03T06:18:45.747526Z","shell.execute_reply.started":"2023-01-03T06:18:45.735482Z","shell.execute_reply":"2023-01-03T06:18:45.746693Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"idxIn = [0 for i in range(30)]\nimagesIn = [np.zeros((image_count[key],64,64,3)) for key in label_dict.keys()]\n\ncount = 0\n# RANDOMLY CROP FULL IMAGES\nfor i in IMAGES:\n    i_label = labels.loc[labels['filename'] == i].Manufacturer.values[0]\n    img = Image.open(ROOT + 'images/' + i)\n    w = img.size[0]\n    h = img.size[1]\n    sz = np.min((w,h))\n    a=0; b=0\n    if w<h: b = (h-sz)//2\n    else: a = (w-sz)//2\n    img = img.crop((0+a, 0+b, sz+a, sz+b))  \n    img = img.resize((64,64), Image.ANTIALIAS)   \n    label_count = label_dict[i_label]\n    imagesIn[label_count][idxIn[label_count],:,:,:] = np.asarray(img)\n    idxIn[label_count] += 1\n    if (count %1000 == 0): print(count)\n    count += 1\n\nprint(idxIn)\n# print(namesIn)\n# print(labelsIn)","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-01-03T06:18:45.753362Z","iopub.execute_input":"2023-01-03T06:18:45.755379Z","iopub.status.idle":"2023-01-03T06:24:07.436802Z","shell.execute_reply.started":"2023-01-03T06:18:45.755330Z","shell.execute_reply":"2023-01-03T06:24:07.435894Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import random\n# DISPLAY CROPPED IMAGES\nfor k in range(5):\n    plt.figure(figsize=(15,3))\n    for j in range(5):\n        rand2 = random.randint(0, 99)\n        plt.subplot(1,5,j+1)\n        img = Image.fromarray( imagesIn[0][rand2,:,:,:].astype('uint8') )\n        plt.axis('off')\n        plt.title(0,fontsize=11)\n        plt.imshow(img)\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:24:07.438561Z","iopub.execute_input":"2023-01-03T06:24:07.439043Z","iopub.status.idle":"2023-01-03T06:24:09.940274Z","shell.execute_reply.started":"2023-01-03T06:24:07.438986Z","shell.execute_reply":"2023-01-03T06:24:09.939495Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Build Discriminator","metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a"}},{"cell_type":"code","source":"from keras.models import Model\nfrom keras.layers import Input, Dense, Conv2D, Reshape, Flatten, concatenate, UpSampling2D\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.callbacks import LearningRateScheduler\nfrom keras.optimizers import SGD, Adam","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:24:09.941725Z","iopub.execute_input":"2023-01-03T06:24:09.942178Z","iopub.status.idle":"2023-01-03T06:24:10.793883Z","shell.execute_reply.started":"2023-01-03T06:24:09.942123Z","shell.execute_reply":"2023-01-03T06:24:10.793048Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"discriminator = []\n# BUILD DISCRIMINATIVE NETWORK\nfor man in label_dict.keys():\n    plane = Input((12288,))\n    planeName = Input((image_count[man],))\n    x = Dense(12288, activation='sigmoid')(planeName) \n    x = Reshape((2,12288,1))(concatenate([plane, x]))\n    x = Conv2D(1,(2,1),use_bias=False,name='conv')(x)\n    discriminated = Flatten()(x)\n    discriminator.append(Model([plane,planeName], discriminated))\n\n# DISPLAY ARCHITECTURE\ndiscriminator[0].summary()","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:24:10.795290Z","iopub.execute_input":"2023-01-03T06:24:10.795599Z","iopub.status.idle":"2023-01-03T06:24:12.092955Z","shell.execute_reply.started":"2023-01-03T06:24:10.795537Z","shell.execute_reply":"2023-01-03T06:24:12.091311Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in range(30): \n    discriminator[i].get_layer('conv').trainable = False\n    discriminator[i].get_layer('conv').set_weights([np.array([[[[-1.0 ]]],[[[1.0]]]])])\n    discriminator[i].compile(optimizer='adam', loss='binary_crossentropy')\n    print(\"-\",end=\"\")","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:24:12.094227Z","iopub.execute_input":"2023-01-03T06:24:12.094521Z","iopub.status.idle":"2023-01-03T06:24:22.421831Z","shell.execute_reply.started":"2023-01-03T06:24:12.094472Z","shell.execute_reply":"2023-01-03T06:24:22.421000Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train Discriminator\nWe will train the Discriminator to memorize the training images. (Typically you don't train the Discriminator ahead of time. The D learns as the G learns. But this GAN is special).","metadata":{}},{"cell_type":"code","source":"ic_list = [image_count[key] for key in label_dict.keys()]\n\nfor i in range(30):\n    ic = ic_list[i]\n    # TRAINING DATA\n    train_y = (imagesIn[i][:ic,:,:,:]/255.0).reshape((-1,12288))\n    train_X = np.zeros((ic,ic))\n    for j in range(ic): train_X[j,j] = 1\n    zeros = np.zeros((ic,12288))\n\n    # TRAIN NETWORK\n    lr = 0.5\n    if (ic == 100): bs = 32\n    elif (ic < 300): bs = 64\n    else: bs = 128\n    \n    for k in range(5):\n        annealer = LearningRateScheduler(lambda x: lr)\n        h = discriminator[i].fit([zeros,train_X], train_y, epochs = 10, batch_size=bs, callbacks=[annealer], verbose=0)\n        if h.history['loss'][-1]<0.533: lr = 0.1\n    print(\"-\", end=\"\")","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:24:22.423394Z","iopub.execute_input":"2023-01-03T06:24:22.424000Z","iopub.status.idle":"2023-01-03T06:26:59.582796Z","shell.execute_reply.started":"2023-01-03T06:24:22.423949Z","shell.execute_reply":"2023-01-03T06:26:59.582009Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Delete Training Images\nOur Discriminator has memorized all the training images. We will now delete the training images. Our Generator will never see the training images. It will only be coached by the Discriminator. Below are examples of images that the Discriminator memorized.","metadata":{}},{"cell_type":"code","source":"del train_X, train_y, imagesIn","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:26:59.585386Z","iopub.execute_input":"2023-01-03T06:26:59.585899Z","iopub.status.idle":"2023-01-03T06:26:59.651109Z","shell.execute_reply.started":"2023-01-03T06:26:59.585842Z","shell.execute_reply":"2023-01-03T06:26:59.650144Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Discriminator Recalls from Memory Planes')    \n\nplt.figure(figsize=(15,3))\nfor j in range(5):\n    xx = np.zeros((100))\n    xx[np.random.randint(100)] = 1\n    plt.subplot(1,5,j+1)\n    img = discriminator[0].predict([zeros[0,:].reshape((-1,12288)),xx.reshape((-1,100))]).reshape((-1,64,64,3))\n    img = Image.fromarray( (255*img).astype('uint8').reshape((64,64,3)))\n    plt.axis('off')\n    plt.imshow(img)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-01-03T06:26:59.652517Z","iopub.execute_input":"2023-01-03T06:26:59.653074Z","iopub.status.idle":"2023-01-03T06:27:00.595397Z","shell.execute_reply.started":"2023-01-03T06:26:59.652845Z","shell.execute_reply":"2023-01-03T06:27:00.594457Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Build Generator and GAN\nWe will purposely give our Generator a bottleneck in its memory. Using poor memory forces the Generator to learn a generalization of images and not memorize the images exactly.","metadata":{}},{"cell_type":"code","source":"# BUILD GENERATOR NETWORK\n\ngenerator = []\nfor man in label_dict.keys():\n    seed = Input((image_count[man],))\n    x = Dense(2048, activation='elu')(seed)\n    x = Reshape((8,8,32))(x)\n    x = Conv2D(128, (3, 3), activation='elu', padding='same')(x)\n    x = UpSampling2D((2, 2))(x)\n    x = Conv2D(64, (3, 3), activation='elu', padding='same')(x)\n    x = UpSampling2D((2, 2))(x)\n    x = Conv2D(32, (3, 3), activation='elu', padding='same')(x)\n    x = UpSampling2D((2, 2))(x)\n    x = Conv2D(3, (3, 3), activation='linear', padding='same')(x)\n    generated = Flatten()(x)\n    generator.append(Model(seed, [generated,Reshape((image_count[man],))(seed)]))\n\n# DISPLAY ARCHITECTURE\ngenerator[0].summary()","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:27:00.596736Z","iopub.execute_input":"2023-01-03T06:27:00.597202Z","iopub.status.idle":"2023-01-03T06:27:03.541943Z","shell.execute_reply.started":"2023-01-03T06:27:00.596990Z","shell.execute_reply":"2023-01-03T06:27:03.541225Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# BUILD GENERATIVE ADVERSARIAL NETWORK\ngan = []\nfor i in range(30):\n    discriminator[i].trainable=False    \n    gan_input = Input(shape=(ic_list[i],))\n    x = generator[i](gan_input)\n    gan_output = discriminator[i](x)\n\n    # COMPILE GAN\n    gan.append(Model(gan_input, gan_output))\n\n    # DISPLAY ARCHITECTURE\ngan[0].summary()","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:27:03.543948Z","iopub.execute_input":"2023-01-03T06:27:03.544457Z","iopub.status.idle":"2023-01-03T06:27:05.617931Z","shell.execute_reply.started":"2023-01-03T06:27:03.544317Z","shell.execute_reply":"2023-01-03T06:27:05.614543Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in range(30):\n    gan[i].get_layer('model_' + str(i+1)).get_layer('conv').set_weights([np.array([[[[-1 ]]],[[[255.]]]])])\n    gan[i].compile(optimizer=Adam(5), loss='mean_squared_error')\n    print(\"-\", end='')\ngan[0].summary()","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:27:05.619280Z","iopub.execute_input":"2023-01-03T06:27:05.619765Z","iopub.status.idle":"2023-01-03T06:27:54.434451Z","shell.execute_reply.started":"2023-01-03T06:27:05.619713Z","shell.execute_reply":"2023-01-03T06:27:54.433560Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Discriminator Coaches Generator\nIn a typical GAN, the discriminator does not memorize the training images beforehand. Instead it learns to distinquish real images from fake images at the same time that the Generator learns to make fake images. In this GAN, we taught the Discriminator ahead of time and it will now teach the Generator.","metadata":{}},{"cell_type":"code","source":"for i in range(30):\n    \n    # TRAIN NETWORKS\n    ep = 1; it = 75\n    lr = 0.005\n    ic = ic_list[i]\n    if (ic == 100): bs = 32\n    elif (ic < 300): bs = 64\n    else: bs = 128\n    \n    # TRAINING DATA\n    train = np.zeros((ic,ic))\n    for j in range(ic): train[j,j] = 1\n    zeros = np.zeros((ic,12288))\n    \n    for k in range(it):  \n        \n        # BEGIN DISCRIMINATOR COACHES GENERATOR\n        annealer = LearningRateScheduler(lambda x: lr)\n        h = gan[i].fit(train, zeros, epochs = ep, batch_size=bs, callbacks=[annealer], verbose=0)\n\n        # DISPLAY GENERATOR LEARNING PROGRESS\n#         if k%10 == 0:\n#             plt.figure(figsize=(15,3))\n#             for j in range(5):\n#                 xx = np.zeros((ic))\n#                 xx[np.random.randint(ic)] = 1\n#                 plt.subplot(1,5,j+1)\n#                 img = generator[i].predict(xx.reshape((-1,ic)))[0].reshape((-1,64,64,3))\n#                 img = Image.fromarray( (img).astype('uint8').reshape((64,64,3)))\n#                 plt.axis('off')\n#                 plt.imshow(img)\n#             plt.show()  \n        print('i:', i, ' Epoch',(k+1),'/'+str(it)+' - loss =',h.history['loss'][-1] )\n        # ADJUST LEARNING RATES\n\n        ep *= 2\n        if ep>=32: lr = 0.001\n        if ep>256: ep = 256\n            \n        if h.history['loss'][-1] < 50 or k == 74: \n            plt.figure(figsize=(15,3))\n            for j in range(5):\n                xx = np.zeros((ic))\n                xx[np.random.randint(ic)] = 1\n                plt.subplot(1,5,j+1)\n                img = generator[i].predict(xx.reshape((-1,ic)))[0].reshape((-1,64,64,3))\n                img = Image.fromarray( (img).astype('uint8').reshape((64,64,3)))\n                plt.axis('off')\n                plt.imshow(img)\n            plt.show() \n            break","metadata":{"execution":{"iopub.status.busy":"2023-01-03T06:27:54.435759Z","iopub.execute_input":"2023-01-03T06:27:54.436039Z","iopub.status.idle":"2023-01-03T10:38:22.190548Z","shell.execute_reply.started":"2023-01-03T06:27:54.435991Z","shell.execute_reply":"2023-01-03T10:38:22.189675Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Build Generator Class\nOur Generative Network has now learned all the training images from our Discriminative Network. With its poor memory, we hope that it has learned to generalize somewhat. Now let's build a Generator Class that accepts any random 100 dimensional vector and outputs an image. Our class will return 70% of one \"memorized\" image mixed with 30% another. Since the images are stored in a convolutional network, we hope that it makes a generalized conceptual mixture (versus a pixel blend).","metadata":{}},{"cell_type":"code","source":"class PlaneGenerator:\n    index = 0   \n    def getPlane(self, ptype):\n        ic = ic_list[ptype]\n        xx = np.zeros((ic))\n        xx[self.index] = 0.7\n        xx[np.random.randint(ic)] = 0.3\n        img = generator[ptype].predict(xx.reshape((-1,ic)))[0].reshape((64,64,3))\n        self.index = (self.index+1)%ic\n        return Image.fromarray( img.astype('uint8') ) ","metadata":{"execution":{"iopub.status.busy":"2023-01-03T10:38:22.191741Z","iopub.execute_input":"2023-01-03T10:38:22.192143Z","iopub.status.idle":"2023-01-03T10:38:22.200295Z","shell.execute_reply.started":"2023-01-03T10:38:22.192077Z","shell.execute_reply":"2023-01-03T10:38:22.199380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Examples of Generated Planes","metadata":{}},{"cell_type":"code","source":"# DISPLAY EXAMPLE PLANES\np = [PlaneGenerator() for i in range(30)]\nfor k in range(3):\n    plt.figure(figsize=(15,3))\n    for j in range(5):\n        plt.subplot(1,5,j+1)\n        img = p[1].getPlane(1)\n        plt.axis('off')\n        plt.imshow(img)\n    plt.show() ","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-01-03T10:40:18.433015Z","iopub.execute_input":"2023-01-03T10:40:18.433889Z","iopub.status.idle":"2023-01-03T10:40:20.099164Z","shell.execute_reply.started":"2023-01-03T10:40:18.433417Z","shell.execute_reply":"2023-01-03T10:40:20.098402Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submit to Kaggle\nIn this kernel we learned how to make an experimental GAN. Currently it scores around LB 100. We must be careful as we try to improve its score. If we give this GAN excellent memory and request a mixture of 99.9% one image and 0.1% another, then it can score LB 7 but then it is returning \"altered versions\" of images and violates the rules [here][1]\n\n[1]: https://www.kaggle.com/c/generative-dog-images/discussion/98183","metadata":{}},{"cell_type":"code","source":"# SAVE TO ZIP FILE NAMED IMAGES.ZIP\nz = zipfile.PyZipFile('images.zip', mode='w')\ni = 0\nfor ptype in image_count.keys() :\n    for k in range(1, 21):\n        img = p[i].getPlane(i)\n        f = f\"{ptype}\\\\{str(k)}.png\"\n        img.save(f,'PNG'); z.write(f); os.remove(f)\n        #if k % 1000==0: print(k)\n    i = i+1\nz.close()","metadata":{"execution":{"iopub.status.busy":"2023-01-03T10:40:32.756230Z","iopub.execute_input":"2023-01-03T10:40:32.756563Z","iopub.status.idle":"2023-01-03T10:40:34.994857Z","shell.execute_reply.started":"2023-01-03T10:40:32.756503Z","shell.execute_reply":"2023-01-03T10:40:34.993989Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}