{"nbformat_minor": 0, "metadata": {"language_info": {"codemirror_mode": {"version": 3, "name": "ipython"}, "version": "3.6.1", "name": "python", "pygments_lexer": "ipython3", "mimetype": "text/x-python", "nbconvert_exporter": "python", "file_extension": ".py"}, "kernelspec": {"language": "python", "display_name": "Python 3", "name": "python3"}}, "nbformat": 4, "cells": [{"source": "#Paint your catdog in TensorFlow\n\nReference:\nhttps://github.com/pkmital/CADL/blob/master/session-2/lecture-2.ipynb\n\n###Predict color to paint based on row/column position of data.", "metadata": {"_uuid": "60b5cc783f745fa92eeea91d2e1480152fd77cb5", "_execution_state": "idle", "collapsed": false, "_cell_guid": "5a15b45e-3aa6-4c94-b680-511ed37de354"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "TRAIN_DIR = '../input/train/'\nimport os\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom skimage.transform import resize\nfrom skimage import data\nimport tensorflow as tf\nimport warnings\nwarnings.filterwarnings(\"ignore\", category=UserWarning) ", "metadata": {"_uuid": "5af1782d88ef71326914f1d9b54cf971d7a49e07", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "13dbecdb-211c-405e-93f6-ba3d1dfd5866"}, "cell_type": "code", "outputs": [], "execution_count": 50}, {"source": "Create \"files\" variable which is list of path names for the files in a directory. Choose only the first 100 files. Read all rows, columns and 3 dimensions in Z axis for the first 100 path names (files) and make it into a new list (imgs). If you prefer dogs, you can play around with the index to choose a file or group of files further down the \"files\" list. Resize all the images in the imgs variable to certain size (depending on the quality you want and time you are willing to wait you can increase/decrease this size). Finally, create a numpy array out of the \"imgs\" variable.", "metadata": {"_uuid": "88022ccdad67727d388308a07853ca37ec9710ea", "_execution_state": "idle", "collapsed": false, "_cell_guid": "824f4121-b0fb-467a-9633-948b93a2566f"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "files = [os.path.join(TRAIN_DIR, fname)\n        for fname in os.listdir(TRAIN_DIR)]\n\ncats = files[:3]\ndogs = files[-4:-1]\n\ncat_imgs = [plt.imread(fname)[..., :3] for fname in cats]\ncat_imgs = [resize(img_i, (100, 100)) for img_i in cat_imgs]\ncat_imgs = np.array(cat_imgs).astype(np.float32)\n\ndog_imgs = [plt.imread(fname)[..., :3] for fname in dogs]\ndog_imgs = [resize(img_i, (100, 100)) for img_i in dog_imgs]\ndog_imgs = np.array(dog_imgs).astype(np.float32)", "metadata": {"_uuid": "cffee64c13be3937012d485f0543b124d1466b1b", "_execution_state": "idle", "trusted": false, "_cell_guid": "29356031-5c1c-401c-a165-5494187d015c"}, "cell_type": "code", "outputs": [], "execution_count": 51}, {"cell_type": "code", "metadata": {"_uuid": "66757075af48eaa1bf817a253eacf46d3c086f90", "_execution_state": "idle", "collapsed": false}, "source": "plt.imshow(cat_imgs[1])", "outputs": [], "execution_count": 52}, {"source": "plt.imshow(dog_imgs[1])", "metadata": {"_uuid": "17488471ee0eb6e9c353410224c5d190e0cb2cea", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "f64bc2a3-2ed4-4079-a855-190529bec2e1"}, "cell_type": "code", "outputs": [], "execution_count": 53}, {"cell_type": "code", "metadata": {"_uuid": "54062ccf68fbb04c7391a59507b1670625459e07", "_execution_state": "idle", "collapsed": false}, "source": "imgs = np.concatenate((cat_imgs[1], dog_imgs[1]), axis=1)\nplt.imshow(imgs)", "outputs": [], "execution_count": 54}, {"source": "Checking chosen image shape", "metadata": {"_uuid": "e2f8532743d204826114607bfa38271976d6e934", "_execution_state": "idle", "collapsed": false, "_cell_guid": "13e0cd8a-da1e-4800-825b-e3513561fafe"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "Function that loops over rows (img.shape[0]) and columns (img.shape[1]) of an image and creates inputs (xs) and outputs (ys) where inputs are the row/column coordinate and outputs are the three RGB color channel numbers at that coordinate (img[row_i, col_i])", "metadata": {"_uuid": "88df11c0904c4f896d5476bae3ab0deb7a45ea2e", "_execution_state": "idle", "collapsed": false, "_cell_guid": "f377ca9f-a01e-4c16-8ac2-4dd425845820"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "def split_image(img):\n    # positions, ie row/column tuple\n    xs = []\n\n    # 3 rgb colors\n    ys = []\n\n    for row_i in range(img.shape[0]):\n        for col_i in range(img.shape[1]):\n            xs.append([row_i, col_i])\n            ys.append(img[row_i, col_i])\n            \n    xs = np.array(xs)\n    ys = np.array(ys)\n    return xs, ys", "metadata": {"_uuid": "f3787228fd2ee259b1d61c55cff52f4f8ea31d84", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "f8b5ea2f-f0d3-40d2-a583-8f8fc7d90619"}, "cell_type": "code", "outputs": [], "execution_count": 55}, {"source": "Splitting chosen image into inputs(coordinates) and targets(RGB color numbers).", "metadata": {"_uuid": "d8d3bf9bca664e8d764a43ab34941744236ac905", "_execution_state": "idle", "collapsed": false, "_cell_guid": "5444c4a7-ab43-4c0b-aa2c-38e725da7ff9"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "xs, ys = split_image(imgs)\n\nxs.shape, ys.shape", "metadata": {"_uuid": "2dc2cfbf5a1b802c1055782d687c8ef55e2716ec", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "c4b5bd5a-93cf-4c75-bc88-58aa74920c60"}, "cell_type": "code", "outputs": [], "execution_count": 56}, {"source": "Normalize input incase image array numbers in 0-255 range.", "metadata": {"_uuid": "a4a672710af78e445797d540df9e308654fac36e", "_execution_state": "idle", "collapsed": false, "_cell_guid": "55fa5524-80c0-4135-a5ce-98f3acf7378f"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "xs = ((xs - np.mean(xs)) / np.std(xs))", "metadata": {"_uuid": "e6d5d57414c36daeab2ebb53d0a57fcff72487bf", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "86ecab45-15ae-4582-aa0a-700fa668ac93"}, "cell_type": "code", "outputs": [], "execution_count": 57}, {"source": "print(np.min(ys), np.max(ys))", "metadata": {"_uuid": "b88f953b93d29a44fb486b41765ae0deb8d60297", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "93c00bf7-caab-4d1c-89f7-a840671091c4"}, "cell_type": "code", "outputs": [], "execution_count": 58}, {"source": "Input tensor placeholder X(coordinates) needs arbitrary amount of 2D inputs.", "metadata": {"_uuid": "dc6100e8ddf1b64321b71d74f5c2c963209d12f9", "_execution_state": "idle", "collapsed": false, "_cell_guid": "5380f088-7e75-4c42-843b-d8562fb2c670"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "X = tf.placeholder(name='X', shape=(None, 2), dtype=tf.float32)", "metadata": {"_uuid": "67d924da1c734f4f4398b6721d744731441d14a0", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "653f3bee-86df-40d0-8c3b-c0a35d6a818b"}, "cell_type": "code", "outputs": [], "execution_count": 59}, {"source": "Linear function does matrix multiply with n_input (row, columns coordinates in this case) and n_outputs (size of second dimension in matrix multiply) and adds a bias value to the matrix and then optionally applies an activation function, returning the output(\"h\" whilst moving through the neural network and Y_pred on its final step as well as the resulting weight matrix \"W\".", "metadata": {"_uuid": "2018ef265092491d829414e976599c1ca947ef69", "_execution_state": "idle", "collapsed": false, "_cell_guid": "84cea4ab-eac9-4922-ba23-2dede39f5678"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "def linear(x, n_output, name=None, activation=None, reuse=None):\n\n    n_input = x.get_shape().as_list()[1]\n\n    with tf.variable_scope(name or \"fully_connected\", reuse=reuse):\n        W = tf.get_variable(\n            name='W',\n            shape=[n_input, n_output],\n            dtype=tf.float32,\n            initializer=tf.contrib.layers.xavier_initializer())\n\n        b = tf.get_variable(\n            name='b',\n            shape=[n_output],\n            dtype=tf.float32,\n            initializer=tf.constant_initializer(0.0))\n\n        h = tf.nn.bias_add(\n            name='h',\n            value=tf.matmul(x, W),\n            bias=b)\n\n        if activation:\n            h = activation(h)\n\n        return h, W", "metadata": {"_uuid": "edd20319b964aa056f26de25b1fb55ea016dd161", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "e34c1e55-9fd6-40be-8b9f-3b4d2ae9177a"}, "cell_type": "code", "outputs": [], "execution_count": 60}, {"source": "Return \"h\" output and \"W\" weight matrix, the result of passing tensor \"X\" through the linear function with 20 outputs per 1 input, giving the operation the name 'linear' and using the tf.nn.relu activation function.", "metadata": {"_uuid": "7961057f9ef3bdb89d79a6c10c26297d8444de76", "_execution_state": "idle", "collapsed": false, "_cell_guid": "d241e872-b45c-4715-88ce-a3920d9bed48"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "h, W = linear(x=X, n_output=20, name='linear', activation=tf.nn.relu)", "metadata": {"_uuid": "383ae075339f1a4b344829897ce1930f83b9a2ad", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "6814093c-1031-44cb-8e7d-3de8211b8f04"}, "cell_type": "code", "outputs": [], "execution_count": 61}, {"source": "Create input X and output Y tensor placeholders that correspond to 2 coordinate inputs for X and 3 RGB color outputs for Y.", "metadata": {"_uuid": "d5f8d3da3fe9b0624883a36bfaca3c04bf15e9c3", "_execution_state": "idle", "collapsed": false, "_cell_guid": "2cb51bf5-268f-4051-adb2-9c7aacd4b34f"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "tf.reset_default_graph()\nX = tf.placeholder(name='X', shape=(None, 2), dtype=tf.float32)\nY = tf.placeholder(name='Y', shape=(None, 3), dtype=tf.float32)", "metadata": {"_uuid": "f4555da0eba2448b853641ac2c45a057108286e4", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "c6133192-795b-4817-b2bc-c0b0a87f3cb1"}, "cell_type": "code", "outputs": [], "execution_count": 62}, {"source": "Specify our network structure, which will take in X and pass it through a number of layers. On each layer a multiplication with a weight matrix takes place, an addition of the bias and a pass through an activation function (except for the final layer where there is no activation and instead of passing \"n_neurons\" we specify the number of outputs we want for our predicted Y, which is 3 numbers corresponding to 3 RGB color values. Different activation functions might yield different final results.", "metadata": {"_uuid": "5317216bbc9a13a45d8e5a3b85cfa740fa0f182a", "_execution_state": "idle", "collapsed": false, "_cell_guid": "dabf8319-d7d6-45c6-b3f0-0232c7b3bb79"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "n_neurons = 100\nh1, W1 = linear(X, n_neurons, name='layer1', activation=tf.nn.relu)\nh2, W2 = linear(h1, n_neurons, name='layer2', activation=tf.nn.relu)\nh3, W3 = linear(h2, n_neurons, name='layer3', activation=tf.nn.relu)\nh4, W4 = linear(h3, n_neurons, name='layer4', activation=tf.nn.relu)\nh5, W5 = linear(h4, n_neurons, name='layer5', activation=tf.nn.relu)\n\nY_pred, W6 = linear(h5, 3, activation=None, name='pred')", "metadata": {"_uuid": "7da743f3a7bef52c19b63b1ec44d71936e399674", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "4c4c1d79-c0b9-4a14-b8ec-967734a032b8"}, "cell_type": "code", "outputs": [], "execution_count": 63}, {"source": "We define our error as the squared difference between our true Y and our predicted, which gives us an absolute error metric. We sum all the error values for our three RGB values to get a single error value per image. We then define our cost as the mean summed error value across all images.", "metadata": {"_uuid": "9f85efe7ee98ab85a3c7707ebbaf5954e7c92020", "_execution_state": "idle", "collapsed": false, "_cell_guid": "5e750207-065e-4cec-8da1-8b95480e6934"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "error = tf.squared_difference(Y, Y_pred)\nsum_error = tf.reduce_sum(input_tensor=error, axis=1)\ncost = tf.reduce_mean(input_tensor=sum_error)", "metadata": {"_uuid": "17579045465e1fdec514e895e4d719ef0c356247", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "ba42efd5-0cad-4b78-9173-0d84a297501e"}, "cell_type": "code", "outputs": [], "execution_count": 64}, {"source": "Define the optimizer to use, giving it a learning rate and specifying what loss function (cost in this case) it should minimize. Specify number of iterations and batch size, which can be used to tune the model and get different varieties of final result. We also initialize the session.", "metadata": {"_uuid": "b2abc6f53f8046282b66f49ef63895606cc8b0f2", "_execution_state": "idle", "collapsed": false, "_cell_guid": "2f3a2bcd-abfc-4872-9f9f-c77435985271"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "optimizer = tf.train.AdamOptimizer(learning_rate=0.003).minimize(cost)\nn_iterations = 300\nbatch_size = 50\nsess = tf.Session()", "metadata": {"_uuid": "175b78cebae53e0f493f6e3eaba18941dc4adcbb", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "c245acc4-4aae-4f1c-be24-d3d9edc0ca44"}, "cell_type": "code", "outputs": [], "execution_count": 65}, {"source": "Check the shape of the image we chose to paint and store that shape in a variable.", "metadata": {"_uuid": "3525f835d7da2206ac13fa7f345983172f82fb38", "_execution_state": "idle", "collapsed": false, "_cell_guid": "32b62469-058f-4b47-80ce-e4e2d2bb8071"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "print(imgs.shape)\nimg_shape = imgs.shape", "metadata": {"_uuid": "a377cf3fd6720b7bc4ea828001725c945a7d82d6", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "35cb2a30-3a50-4275-b87c-12431d712786"}, "cell_type": "code", "outputs": [], "execution_count": 66}, {"source": "Initialize all the variables we created. Create imgs list for storing resulting images during training. Loop through iterations, choosing random xs for training. Loop through batches splitting chosen xs into batches and train on that data, trying to determine what xs(row/column) coordinates should return what 3 RGB color values. \n\nIf our current iteration is divisible by our display step, make a Y prediction(3 RGB color values) for current xs(row/column coordinates), clip those values to be between 0 and 1 and reshape it to be the same shape as our chosen input. We append that image to our \"imgs\" list variable which can later be used to access images produced during training.\n\nIf training takes long, decrease your network size/number of neurons, increase learning rate or decrease image resize in the second cell to something smaller. To get nice results at higher resolution requires larger initial images, larger number of neurons and longer training time usually.", "metadata": {"_uuid": "47014bcf87a2a6c28427b285a632fc3903435296", "_execution_state": "idle", "collapsed": false, "_cell_guid": "fbe76a0e-5bef-4707-9d79-d991e9aa25a1"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "sess.run(tf.global_variables_initializer())\n\nimgs = []\ndisplay_step = n_iterations // 10\n\nfor it_i in range(n_iterations):\n    \n    idxs = np.random.permutation(range(len(xs)))\n    \n    n_batches = len(idxs) // batch_size\n    \n    for batch_i in range(n_batches):\n        \n        idxs_i = idxs[batch_i * batch_size: (batch_i + 1) * batch_size]\n        \n        training_cost = sess.run([cost, optimizer],\n                                feed_dict={X: xs[idxs_i], Y: ys[idxs_i]})[0]\n    \n    if (it_i + 1) % display_step == 0:\n        \n        ys_pred = Y_pred.eval(feed_dict={X: xs}, session=sess)\n        img = np.clip(ys_pred.reshape(img_shape), 0, 1)\n        imgs.append(img)\n        \n        ax = plt.imshow(img)\n        plt.title('Iteration {}'.format(it_i))\n        plt.show()", "metadata": {"_uuid": "5fda1bc4c7077c83c4ec6dfcfd47a3d4c7cd5a91", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "5421a9e1-c6fd-4a9e-a194-d9a2de9a1e0e"}, "cell_type": "code", "outputs": [], "execution_count": 67}, {"source": "Using -1 we select the final image produced by our training process, we can use 0, 1 etc to view images from earlier in the training process.", "metadata": {"_uuid": "5abfa10dd7cdb1ab01dd7f4eed0c8732ee16974c", "_execution_state": "idle", "collapsed": false, "_cell_guid": "7801146a-cbb4-4e4e-8fc5-cdab489eef88"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "plt.figure(figsize=(8, 8))\nplt.imshow(imgs[-1])", "metadata": {"_uuid": "c0863f7d70e97633c4ba7108196d0ad2f8bedb6a", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "c80d9ef6-22a0-4b91-8b26-38423a52aa6d"}, "cell_type": "code", "outputs": [], "execution_count": 69}, {"source": "We save the final image produced by our model as a .png", "metadata": {"_uuid": "3a20b852ef0ffd65d6bf31c9461aa00ad40f673a", "_execution_state": "idle", "collapsed": false, "_cell_guid": "28405ac5-f7a3-486c-9b8d-1bb23e15762c"}, "cell_type": "markdown", "outputs": [], "execution_count": null}, {"source": "plt.imsave(fname='my_catdog.png', arr=imgs[-1])", "metadata": {"_uuid": "ce4802957d1e3e856fa38c0df84e078ce7d6a4f4", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "ccb674d0-bd47-492b-a43c-177f5346d676"}, "cell_type": "code", "outputs": [], "execution_count": 70}, {"source": "Reference:\nhttps://github.com/pkmital/CADL/blob/master/session-2/lecture-2.ipynb", "metadata": {"_uuid": "19d7c82cf997eef00aba65bdf4463cb2f7cec509", "_execution_state": "idle", "collapsed": false, "_cell_guid": "fa650eb3-b16b-4ead-a183-34f85556245f"}, "cell_type": "markdown", "outputs": [], "execution_count": 19}, {"cell_type": "code", "metadata": {"_uuid": "328e7d33c8512c2262443ba4cf8573a40dce20ac", "_execution_state": "idle", "collapsed": false, "trusted": false, "_cell_guid": "05391fb7-9b8a-4534-a131-57c4a3ca4c3c"}, "source": "", "outputs": [], "execution_count": null}]}