{"cells":[{"metadata":{"_uuid":"f15ce6b398727e37f0451497d4c86e217293d679"},"cell_type":"markdown","source":"# Intro\n**This is Lesson 8 in the [Deep Learning](https://www.kaggle.com/learn/deep-learning) track**  \n\nAt the end of this lesson, you will understand and know how to use\n- **Stride lengths** to make your model faster and reduce memory consumption\n- **Dropout** to combat overfitting\n\nBoth of these techniques are especially useful in large models.\n\n# Lesson"},{"metadata":{"_kg_hide-input":false,"collapsed":true,"trusted":false,"_uuid":"66e086318e16c8dbe41d34e6729ac2b4dd3e34ab"},"cell_type":"code","source":"from IPython.display import YouTubeVideo\nYouTubeVideo('fwNLf4t7MR8', width=800, height=450)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"38ca0e1bd2e173e4d3acbac0cc4eb7a0494f7fe9"},"cell_type":"markdown","source":"# Sample Code"},{"metadata":{"trusted":true,"_uuid":"f0d0b2f7340fdaa55afaeeffd2dc9338ff5c9980"},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nfrom sklearn.model_selection import train_test_split\nfrom tensorflow.python import keras\nfrom tensorflow.python.keras.models import Sequential\nfrom tensorflow.python.keras.layers import Dense, Flatten, Conv2D, Dropout\n\nimg_rows, img_cols = 28, 28\nnum_classes = 10\n\ndef data_prep(raw):\n    out_y = keras.utils.to_categorical(raw.label, num_classes)\n\n    num_images = raw.shape[0]\n    x_as_array = raw.values[:,1:]\n    x_shaped_array = x_as_array.reshape(num_images, img_rows, img_cols, 1)\n    out_x = x_shaped_array / 255\n    return out_x, out_y\n\ntrain_size = 30000\ntrain_file = \"../input/digit-recognizer/train.csv\"\nraw_data = pd.read_csv(train_file)\n\nx, y = data_prep(raw_data)\n\nmodel = Sequential()\nmodel.add(Conv2D(30, kernel_size=(3, 3),\n                 strides=2,\n                 activation='relu',\n                 input_shape=(img_rows, img_cols, 1)))\nmodel.add(Dropout(0.5))\nmodel.add(Conv2D(30, kernel_size=(3, 3), strides=2, activation='relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Flatten())\nmodel.add(Dense(128, activation='relu'))\nmodel.add(Dense(num_classes, activation='softmax'))\n\nmodel.compile(loss=keras.losses.categorical_crossentropy,\n              optimizer='adam',\n              metrics=['accuracy'])\nmodel.fit(x, y,\n          batch_size=128,\n          epochs=2,\n          validation_split = 0.2)","execution_count":null,"outputs":[{"output_type":"stream","text":"WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\nInstructions for updating:\nColocations handled automatically by placer.\nWARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/keras/layers/core.py:143: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\nInstructions for updating:\nPlease use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\nTrain on 33600 samples, validate on 8400 samples\nWARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\nInstructions for updating:\nUse tf.cast instead.\nEpoch 1/2\n 6016/33600 [====>.........................] - ETA: 9s - loss: 1.4110 - acc: 0.5391","name":"stdout"}]},{"metadata":{"_uuid":"32f6c0bc1dd713af40bfb476a54ecd8d469f9dd9"},"cell_type":"markdown","source":"# Exercise\n**[Apply dropout and strides](https://www.kaggle.com/kernels/fork/663261)** yourself while experimenting with larger models.\n\n---\n**[Deep Learning Course Home Page](https://www.kaggle.com/learn/deep-learning)**\n"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.5"}},"nbformat":4,"nbformat_minor":1}