{"cells":[{"metadata":{},"cell_type":"markdown","source":"**[Deep Learning Micro-Course Home Page](https://www.kaggle.com/learn/deep-learning)**\n\n---\n"},{"metadata":{},"cell_type":"markdown","source":"# Introduction\n\nYou've built a model to identify clothing types in the **MNIST for Fashion** dataset.  Now you will make your model bigger, specify larger stride lengths and apply dropout. These changes will make your model faster and more accurate.\n\nThis is a last step in the **[Deep Learning Track](https://www.kaggle.com/learn/deep-learning)**.\n\n\n## Data Preparation\n**Run this cell of code.**\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np\nfrom sklearn.model_selection import train_test_split\nfrom tensorflow import keras\n\n# Set up code checking\nfrom learntools.core import binder\nbinder.bind(globals())\nfrom learntools.deep_learning.exercise_8 import *\nprint(\"Setup Complete\")\n\nimg_rows, img_cols = 28, 28\nnum_classes = 10\n\ndef prep_data(raw):\n    y = raw[:, 0]\n    out_y = keras.utils.to_categorical(y, num_classes)\n    \n    x = raw[:,1:]\n    num_images = raw.shape[0]\n    out_x = x.reshape(num_images, img_rows, img_cols, 1)\n    out_x = out_x / 255\n    return out_x, out_y\n\nfashion_file = \"../input/fashionmnist/fashion-mnist_train.csv\"\nfashion_data = np.loadtxt(fashion_file, skiprows=1, delimiter=',')\nx, y = prep_data(fashion_data)","execution_count":1,"outputs":[{"output_type":"stream","text":"Setup Complete\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"# 1) Increasing Stride Size in A Layer\n\nBelow is a model without strides (or more accurately, with a stride length of 1)\n\nRun it. Notice it's accuracy and how long it takes per epoch. Then you will change the stride length in one of the layers."},{"metadata":{"trusted":true},"cell_type":"code","source":"from tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense, Flatten, Conv2D, Dropout\n\nbatch_size = 16\n\nfashion_model = Sequential()\nfashion_model.add(Conv2D(16, kernel_size=(3, 3),\n                 activation='relu',\n                 input_shape=(img_rows, img_cols, 1)))\nfashion_model.add(Conv2D(16, (3, 3), activation='relu'))\nfashion_model.add(Flatten())\nfashion_model.add(Dense(128, activation='relu'))\nfashion_model.add(Dense(num_classes, activation='softmax'))\n\nfashion_model.compile(loss=keras.losses.categorical_crossentropy,\n              optimizer='adam',\n              metrics=['accuracy'])\n\nfashion_model.fit(x, y,\n          batch_size=batch_size,\n          epochs=3,\n          validation_split = 0.2)\n","execution_count":2,"outputs":[{"output_type":"stream","text":"WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\nInstructions for updating:\nColocations handled automatically by placer.\nTrain on 48000 samples, validate on 12000 samples\nWARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\nInstructions for updating:\nUse tf.cast instead.\nEpoch 1/3\n48000/48000 [==============================] - 15s 313us/sample - loss: 0.3966 - acc: 0.8584 - val_loss: 0.3479 - val_acc: 0.8668\nEpoch 2/3\n48000/48000 [==============================] - 12s 251us/sample - loss: 0.2526 - acc: 0.9081 - val_loss: 0.2861 - val_acc: 0.8977\nEpoch 3/3\n48000/48000 [==============================] - 12s 252us/sample - loss: 0.1866 - acc: 0.9308 - val_loss: 0.2540 - val_acc: 0.9122\n","name":"stdout"},{"output_type":"execute_result","execution_count":2,"data":{"text/plain":"<tensorflow.python.keras.callbacks.History at 0x7f882c9fa2b0>"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"You have the same code in the cell below, but the model is now called `fashion_model_1`.  Change the specification of `fashion_model_1` so the second convolutional layer has a stride length of 2.\n\nRun the cell after you have done that. How does the speed and accuracy change compared to the first model you ran above?"},{"metadata":{"trusted":true},"cell_type":"code","source":"fashion_model_1 = Sequential()\nfashion_model_1.add(Conv2D(16, kernel_size=(3, 3),\n                 activation='relu',\n                 input_shape=(img_rows, img_cols, 1)))\nfashion_model_1.add(Conv2D(16, (3, 3), activation='relu', strides=2))\nfashion_model_1.add(Flatten())\nfashion_model_1.add(Dense(128, activation='relu'))\nfashion_model_1.add(Dense(num_classes, activation='softmax'))\n\nfashion_model_1.compile(loss=keras.losses.categorical_crossentropy,\n              optimizer='adam',\n              metrics=['accuracy'])\n\nfashion_model_1.fit(x, y,\n          batch_size=batch_size,\n          epochs=3,\n          validation_split = 0.2)\nq_1.check()","execution_count":3,"outputs":[{"output_type":"stream","text":"Train on 48000 samples, validate on 12000 samples\nEpoch 1/3\n48000/48000 [==============================] - 13s 273us/sample - loss: 0.4277 - acc: 0.8465 - val_loss: 0.3185 - val_acc: 0.8883\nEpoch 2/3\n48000/48000 [==============================] - 12s 253us/sample - loss: 0.2843 - acc: 0.8966 - val_loss: 0.2929 - val_acc: 0.8958\nEpoch 3/3\n48000/48000 [==============================] - 12s 253us/sample - loss: 0.2266 - acc: 0.9161 - val_loss: 0.2688 - val_acc: 0.9057\n","name":"stdout"},{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 1.0, \"interactionType\": 1, \"learnTutorialId\": 83, \"questionId\": \"1_AddStrides\", \"learnToolsVersion\": \"0.2.13\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Correct","text/markdown":"<span style=\"color:#33cc33\">Correct</span>"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"For the solution, uncomment and run the cell below:"},{"metadata":{"trusted":true},"cell_type":"code","source":"q_1.solution()","execution_count":5,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"interactionType\": 3, \"learnTutorialId\": 83, \"questionId\": \"1_AddStrides\", \"learnToolsVersion\": \"0.2.13\", \"valueTowardsCompletion\": 0.0, \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\", \"outcomeType\": 4}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Solution: \n```python\n\nfashion_model_1 = Sequential()\nfashion_model_1.add(Conv2D(16, kernel_size=(3, 3),\n                 activation='relu',\n                 input_shape=(img_rows, img_cols, 1)))\n# This is the only line that changed\nfashion_model_1.add(Conv2D(16, (3, 3), activation='relu', strides=2))\nfashion_model_1.add(Flatten())\nfashion_model_1.add(Dense(128, activation='relu'))\nfashion_model_1.add(Dense(num_classes, activation='softmax'))\n\n# the fit and compile steps are not shown here since they did not change\n\n```","text/markdown":"<span style=\"color:#33cc99\">Solution:</span> \n```python\n\nfashion_model_1 = Sequential()\nfashion_model_1.add(Conv2D(16, kernel_size=(3, 3),\n                 activation='relu',\n                 input_shape=(img_rows, img_cols, 1)))\n# This is the only line that changed\nfashion_model_1.add(Conv2D(16, (3, 3), activation='relu', strides=2))\nfashion_model_1.add(Flatten())\nfashion_model_1.add(Dense(128, activation='relu'))\nfashion_model_1.add(Dense(num_classes, activation='softmax'))\n\n# the fit and compile steps are not shown here since they did not change\n\n```"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"You should notice that your model training ran about twice as fast, but the accuracy change was trivial.  \n\nIn addition to being faster to train, this model is also faster at making predictions. This is very important in many scenarios. In practice, you'll need to decide whether that type of speed is important in the applications where you eventually apply deep learning models.\n\nYou could experiment with more layers or more convolutions in each layer. With some fine-tuning, you can build a model that is both faster and more accurate than the original model."},{"metadata":{},"cell_type":"markdown","source":"# Congrats\nYou've finished the Deep Learning course.  You have the tools to create and tune computer vision models.  \n\nIf you feel like playing more with this dataset, you can open up a new code cell to experiment with different models (adding dropout, adding layers, etc.)  Or pick a new project and try out your skills.  \n\nA few fun datasets you might try include:\n- [Written letter recognition](https://www.kaggle.com/olgabelitskaya/classification-of-handwritten-letters)\n- [Flower Identification](https://www.kaggle.com/alxmamaev/flowers-recognition)\n- [Cats vs Dogs](https://www.kaggle.com/c/dogs-vs-cats-redux-kernels-edition)\n- [10 Monkeys](https://www.kaggle.com/slothkong/10-monkey-species)\n- [Predict Bone Age from X-Rays](https://www.kaggle.com/kmader/rsna-bone-age)\n\nYou have learned a lot, and you'll learn it as you practice. Have fun with it!"},{"metadata":{},"cell_type":"markdown","source":"---\n**[Deep Learning Micro-Course Home Page](https://www.kaggle.com/learn/deep-learning)**\n\n"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.5"}},"nbformat":4,"nbformat_minor":1}