{"cells":[{"metadata":{"_uuid":"2af9b615f4bf20eac4c1dce058551d9000eac181"},"cell_type":"markdown","source":"* ## This Notebook Demonstrates 'Transfer Learning' in Keras\n\n* Press \"Fork\" at the top-right of this screen to run this notebook yourself.\n* Change some values in the fit command\n* Let's see who can achieve the best predictions by class on Monday.\n\nThanks to abhiksark for the original version of this notebook:\nhttps://www.kaggle.com/abhiksark/introduction-to-transfer-learning-cats-dogs"},{"metadata":{"_uuid":"2a25d06d3c778d4272b808099c45690058447c74"},"cell_type":"markdown","source":"## Transfer learning\nIt is a machine learning method where a model developed for a task is reused as the starting point for a model on a second task. \n\n<img src=\"https://elearningindustry.com/wp-content/uploads/2016/09/5-tips-improve-knowledge-transfer-elearning-e1475138920743.jpeg\" width=\"400px\"/>\n\nThe above image exactly represets what the model does for you.\nIt remembers the learning from a fairly related problem and applies it to the problem having new data.\n\n\nPre-trained Model Approach :-\n\n   1.  Select Source Model. A pre-trained source model is chosen from available models. Many research institutions release models on large and challenging datasets that may be included in the pool of candidate models from which to choose from.\n   <br/><br/>\n   2.  Reuse Model. The model pre-trained model can then be used as the starting point for a model on the second task of interest. This may involve using all or parts of the model, depending on the modeling technique used.  \n   <br/>\n   3.  Tune Model. Optionally, the model may need to be adapted or refined on the input-output pair data available for the task of interest.\n   <br/>\n![](http://https://3qeqpr26caki16dnhd19sv6by6v-wpengine.netdna-ssl.com/wp-content/uploads/2017/09/Three-ways-in-which-transfer-might-improve-learning.png)    \n\nPre-trained Models <br/>\n<a href=\"https://github.com/KaimingHe/deep-residual-networks\">Microsoft ResNet Model</a> <br/>\n<a href=\"https://github.com/tensorflow/models/tree/master/inception\">Google Inception Model</a> <br/>\n<a href=\"https://github.com/BVLC/caffe/wiki/Model-Zoo\">Caffe Model Zoo</a>\n\n"},{"metadata":{"_uuid":"bf0ba281ef46d9c17d618bc7ec1f0102179132f1"},"cell_type":"markdown","source":"## Libraries ##\nImporting the Required libraries. libraries allow programmers to share their solutions, so we don't have to keep re-inventing the wheel. While the \"language\" defines the core requirements of a Turing compete computing device, additional features often emerge as libraries that use the language to perform specific functions. - bonus question: What does turing complete mean?"},{"metadata":{"trusted":true},"cell_type":"code","source":"print( \"hello world\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\"hello world\"","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"import os, cv2, random\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\nfrom tqdm import tqdm    #Helps in visualization\nfrom random import shuffle #to shuffle the images \n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## settings ##\nOur code will often reuse a number of string value, or we may want to change that value depending on circumstances. putting these values near the top and together makes it easier to find and change these values. Imagesize is a good example."},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\nTRAIN_DIR = '../input/paper-training-images/TrainingResized/'\nTEST_DIR = '../input/paper-training-images/TrainingResized/'\nIMG_SIZE = 128  \n\nSHORT_LIST_TRAIN = os.listdir(TRAIN_DIR) #using a subset of data as resouces as limited. \nSHORT_LIST_TEST = os.listdir(TEST_DIR)\n\nSHORT_LIST_TRAIN","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Run a command line call ##\n\nThis line is a bit tricky - called a magic function - it's purpose is to set the \"inline\" flag in the matplotlib already imported into this kernal you can remove it and observe the difference, but you would need to restart the kernal because this \"inline\" flag is persistant.\n\nhttps://stackoverflow.com/questions/43027980/purpose-of-matplotlib-inline\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"%matplotlib inline\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Methods ##\n\"Methods\" provide a means of \"encapsulating\" a series of commands. In this case \"label_im\" can be used as a short-hand for the trnslation between the words \"dog\" and \"cat\", and a numeric representation of 1 and 0 in a \"one hot\" array."},{"metadata":{"_uuid":"78bec5a9ce267aa7d910f44ce22839fbc9e163b9","trusted":true},"cell_type":"code","source":"def label_img(img): \n    if \"Food container\" in img: \n        return [1,0,0,0]\n    elif \"food tray\" in img: \n        return [0,1,0,0]\n    elif \"paper cup\" in img: \n        return [0,0,1,0]\n    elif \"paper plate\" in img: \n        return [0,0,0,1]\n    ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e8ae1548ea6e2d6167785fda2ee2168446560110","trusted":true},"cell_type":"code","source":"#returns an numpy array of train and test data\ndef create_train_data():\n    training_data = []\n    for img in tqdm(SHORT_LIST_TRAIN):\n        label = label_img(img)\n        path = os.path.join(TRAIN_DIR,img)\n        img = cv2.imread(path,cv2.IMREAD_COLOR)\n        img = cv2.resize(img, (IMG_SIZE,IMG_SIZE))\n        training_data.append([np.array(img),np.array(label)])\n    shuffle(training_data)\n    return training_data\n\ndef process_test_data():\n    testing_data = []\n    for img in tqdm(os.listdir(TEST_DIR)):\n        path = os.path.join(TEST_DIR,img)\n        img_num = img.split('.')[0]\n        img = cv2.imread(path,cv2.IMREAD_COLOR)\n        img = cv2.resize(img, (IMG_SIZE,IMG_SIZE))\n        testing_data.append([np.array(img), img_num])\n    shuffle(testing_data)\n    return testing_data","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c0ca9ae75c6ee2da2791212b1d3790e5c30ab78c"},"cell_type":"markdown","source":"## Visualizing Classes ##\nVisulization is how we interact with complex datasets - we measure and sample them in many ways.\n"},{"metadata":{"_uuid":"7112c62ae337855b5c17180970f37bff526dbf37","trusted":true},"cell_type":"code","source":"\nlabels = [] #make an empty list.\nfor i in SHORT_LIST_TRAIN: \n    labels.append(str(label_img(i)))\n\nsns.countplot(labels) # show the list as a graph\nplt.title('Recycling Classes')\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"012a523c1b1dee44f6bed9885d563cff5f4295a3"},"cell_type":"markdown","source":"## Creating a Training Set Data##"},{"metadata":{"_uuid":"f1aa5d1aef17d7a50e3163fd3076456fb9b47d70","trusted":true},"cell_type":"code","source":"train = create_train_data() #This is a method defined above","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets have a quick look at some of the training images"},{"metadata":{"trusted":true},"cell_type":"code","source":"\n#import matplotlib.pyplot as plt\n\nfig , ax = plt.subplots(3, 3, figsize=(30, 25))\nfor i, axis in enumerate(ax.flat):\n    axis.imshow(train[i][0], cmap='gray')\n    axis.set_title(f'Label:  {train[i][1]}', fontsize=20)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"92570dba082df8054b188acb31a38e2b39ea6dce"},"cell_type":"markdown","source":"## Specify Model##\n\nThis code will result in a deperecation warning.\nThings change, and in software, when the developers of the language realize that some feature is causing problems, they may choose to provide a different features, in which case, they will create a warning message to people using the old feature, and then, in a yet later version, they may remove the offending feature completely."},{"metadata":{"_uuid":"d6c9d86a101db152f9e431fd2ebe9ca8503a12de","trusted":true},"cell_type":"code","source":"from tensorflow.python.keras.applications import ResNet50\nfrom tensorflow.python.keras.models import Sequential\nfrom tensorflow.python.keras.layers import Dense, Flatten, GlobalAveragePooling2D\n\nNUM_CLASSES = 4\nRESNET_WEIGHTS_PATH = '../input/resnet50/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5' #importing a pretrained model\nmy_new_model = Sequential()\nmy_new_model.add(ResNet50(include_top=False, pooling='max', weights=RESNET_WEIGHTS_PATH))\nmy_new_model.add(Dense(NUM_CLASSES, activation='softmax'))\n\n# Say not to train first layer (ResNet) model. It is already trained\nmy_new_model.layers[0].trainable = True","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"50af9a9bed2bc9e5ac493db6b84bd9b76db5bc64","trusted":true},"cell_type":"code","source":"my_new_model.compile(optimizer='sgd', loss='categorical_crossentropy', metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0425612ac8e9bd947cc8f27772cca79890916289"},"cell_type":"markdown","source":"## Compile Model##"},{"metadata":{"_kg_hide-input":true,"_uuid":"780be7320cc68646c105e4eb4196f40d76bde419","trusted":true},"cell_type":"code","source":"my_new_model.summary()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3548879f48ccfb0b73f1515d02a2fb4e28c14aa7"},"cell_type":"markdown","source":"## From Train Dividing X and Y##\nbecause \"fit\" requires labels and data to be in seperated lists, we seperate the lists here.\n"},{"metadata":{"_uuid":"0171a2515458e1f6194e18ec48ab44aa041eb6bf","trusted":true},"cell_type":"code","source":"X = np.array([i[0] for i in train]).reshape(-1,IMG_SIZE,IMG_SIZE,3)\nY = np.array([i[1] for i in train])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"baa535766e60046b4d25cecba8fc5d3e1347c925"},"cell_type":"markdown","source":"## Fit Model##\n\nthis is machine learing!\n\nhttps://keras.io/models/sequential/#fit\nTrains the model for a given number of epochs (iterations on a dataset).\n\nReturns\n\nA History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).\n\n"},{"metadata":{"_kg_hide-input":true,"_uuid":"dc4a2d4933a0b7fbab3ffe15216479c6bb2a6875","trusted":true},"cell_type":"code","source":"history = my_new_model.fit(X, Y, validation_split=0.20, epochs=4, batch_size=64)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"74de353ffaf5e86c8828c85baad4dc5cb3d62dda"},"cell_type":"markdown","source":"## Plotting loss and accuracy for the model##\n\nnow we're going to look at the history object returned by Training"},{"metadata":{"_kg_hide-input":true,"_uuid":"2d96ecaee49e71e14c86d293aff3c032b18519e8","trusted":true},"cell_type":"code","source":"plt.plot(history.history['acc'])\nplt.plot(history.history['val_acc'])\nplt.title('model accuracy')\nplt.ylabel('accuracy')\nplt.xlabel('epoch')\nplt.legend(['train', 'test'], loc='upper left')\nplt.show()\n# summarize history for loss\nplt.plot(history.history['loss'])\nplt.plot(history.history['val_loss'])\nplt.title('model loss')\nplt.ylabel('loss')\nplt.xlabel('epoch')\nplt.legend(['train', 'test'], loc='upper left')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Repeat ##\nHere it all is again in one section:\nCreate a list,\nbreak it X&Y\nTrain - this time with different values\nAnd plot the learning curve so we can compare how changing the values affects learning\n\nDiscussion: What Values are different in this test vs. the previous test?\n"},{"metadata":{"_uuid":"707fb10f7a890e3880ae37aadeb6e28e37bf50c5","trusted":true},"cell_type":"code","source":"\nSHORT_LIST_TRAIN = os.listdir(TRAIN_DIR) #[0:500]\ntrain = create_train_data()\nX = np.array([i[0] for i in train]).reshape(-1,IMG_SIZE,IMG_SIZE,3)\nY = np.array([i[1] for i in train])\nhistory = my_new_model.fit(X, Y, validation_split=0.5, epochs=20, batch_size=64)\n\nplt.plot(history.history['acc'])\nplt.plot(history.history['val_acc'])\nplt.title('model accuracy')\nplt.ylabel('accuracy')\nplt.xlabel('epoch')\nplt.legend(['train', 'test'], loc='upper left')\nplt.show()\n# summarize history for loss\nplt.plot(history.history['loss'])\nplt.plot(history.history['val_loss'])\nplt.title('model loss')\nplt.ylabel('loss')\nplt.xlabel('epoch')\nplt.legend(['train', 'test'], loc='upper left')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Use the model to predict the labels of some Images ##\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"testing_data = []\nfor img in tqdm(os.listdir(TEST_DIR)[0:100]):\n    path = os.path.join(TEST_DIR,img)\n    img_num = img.split('.')[0]\n    img = cv2.imread(path,cv2.IMREAD_COLOR)\n    img = cv2.resize(img, (IMG_SIZE,IMG_SIZE))\n    testing_data.append([np.array(img), img_num])\n    \nshuffle(testing_data)    \ntest_data = testing_data \n\n\nfig , ax = plt.subplots(6, 4, figsize=(30, 25))\nfor i, axis in enumerate(ax.flat):\n    axis.imshow(test_data[i][0], cmap='gray')\n    img_data = test_data[i][0]\n    orig = img_data\n    data = img_data.reshape(-1,IMG_SIZE,IMG_SIZE,3)\n    model_out = my_new_model.predict([data])[0]    \n\n    #axis.set(title=f'{im_pred[i].max()} => {category[im_pred[i].argmax()]}')\n    axis.set_title(f'Predict: {model_out.max()} => {model_out.argmax()}', fontsize=20)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f44e0e1ff6ff7fe011f1d6dc5413fa6f9d35c5b1"},"cell_type":"markdown","source":"## Sources\n1. https://machinelearningmastery.com/transfer-learning-for-deep-learning/ <br/>\n2. http://cs231n.github.io/transfer-learning/ <br/>\n3. https://arxiv.org/abs/1411.1792 <br/>\nIf you are Interested in Research.\n\n\n"},{"metadata":{"_uuid":"be56ce986f213a15f65bba387e37da1d61d8cf6a","collapsed":true},"cell_type":"markdown","source":"Please upvote this kernel so that it reaches the top of the chart and is easily locatable by new users. Your comments on how we can improve this kernel is welcome. Thanks You For your Support."}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}