{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Kaggle Starter Code for Minecraft Fakes Kaggle In-Class Competition\n\nThis workbook is a starter code for the [Minecraft Fakes Kaggle In-Class Competition](https://www.kaggle.com/c/applications-of-deep-learning-wustl-spring-2021)  This competition is one of the assignments for [T81-558: Applications of Deep Neural Netw1orks](https://sites.wustl.edu/jeffheaton/t81-558/) at [Washington University in St. Louis](https://www.wustl.edu).\n\nThis notebook is not a particularly high-scoring model; however, it does demonstrate how to begin the project entirely in Kaggle.  It is also possable to run this project from Google CoLab.  I have a separate starter project for CoLab.  "},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\n\nPATH = \"/kaggle/input/applications-of-deep-learning-wustl-spring-2021/mc/\"\nPATH_TRAIN = os.path.join(PATH, \"train.csv\")\nPATH_TEST = os.path.join(PATH, \"test.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next we check versions and if the GPU is available. A GPU or TPU will be very helpful."},{"metadata":{"trusted":true},"cell_type":"code","source":"# What version of Python do you have?\nimport sys\n\nimport tensorflow.keras\nimport pandas as pd\nimport sklearn as sk\nimport tensorflow as tf\n\nprint(f\"Tensor Flow Version: {tf.__version__}\")\nprint(f\"Keras Version: {tensorflow.keras.__version__}\")\nprint()\nprint(f\"Python {sys.version}\")\nprint(f\"Pandas {pd.__version__}\")\nprint(f\"Scikit-Learn {sk.__version__}\")\nprint(\"GPU is\", \"available\" if tf.test.is_gpu_available() \\\n      else \"NOT AVAILABLE\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next, we prepare to read the training data (that we have labels for) and the test data that we must predict and send to Kaggle."},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train = pd.read_csv(PATH_TRAIN)\ndf_test = pd.read_csv(PATH_TEST)\ndf_train['label'] = df_train['label'].astype(str)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Perform a basic balance plot.  This data is only slightly unbalanced."},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\n\ndf_train.label.value_counts().plot(kind='bar')\nplt.title('Labels counts')\nplt.xlabel('Fake')\nplt.ylabel('Count')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We want to use early stopping.  To do this, we need a validation set.  We will break the data into 80 percent test data and 20 validation.  Do not confuse this validation data with the test set provided by Kaggle.  This validation set is unique to your program and is just used for early stopping."},{"metadata":{"trusted":true},"cell_type":"code","source":"TRAIN_PCT = 0.9\nTRAIN_CUT = int(len(df_train) * TRAIN_PCT)\n\ndf_train_cut = df_train[0:TRAIN_CUT]\ndf_validate_cut = df_train[TRAIN_CUT:]\n\nprint(f\"Training size: {len(df_train_cut)}\")\nprint(f\"Validate size: {len(df_validate_cut)}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next, we create the generators that will provide the images to the neural network as it is trained.  We normalize the images so that the RGB colors between 0-255 become ratios between 0 and 1.  We also use the **flow_from_dataframe** generator to connect the Pandas dataframe to the actual image files. We see here a straightforward implementation; you might also wish to use some of the image transformations provided by the data generator.\n\nThe **HEIGHT** and **WIDTH** constants specify the dimensions that the image will be scaled (or expanded) to. It is probably not a good idea to expand the images."},{"metadata":{"trusted":true},"cell_type":"code","source":"import tensorflow as tf\nimport keras_preprocessing\nfrom keras_preprocessing import image\nfrom keras_preprocessing.image import ImageDataGenerator\n\nWIDTH = 150\nHEIGHT = 150\n\ntraining_datagen = ImageDataGenerator(\n  rescale = 1./255,\n  #horizontal_flip=True,\n  #vertical_flip=True,\n  fill_mode='nearest')\n\ntrain_generator = training_datagen.flow_from_dataframe(\n        dataframe=df_train_cut,\n        directory=PATH,\n        x_col=\"filename\",\n        y_col=\"label\",\n        target_size=(HEIGHT, WIDTH),\n        batch_size=126,\n        class_mode='binary')\n\nvalidation_datagen = ImageDataGenerator(rescale = 1./255)\n\nval_generator = validation_datagen.flow_from_dataframe(\n        dataframe=df_validate_cut,\n        directory=PATH,\n        x_col=\"filename\",\n        y_col=\"label\",\n        target_size=(HEIGHT, WIDTH),\n        batch_size=126,\n        class_mode='binary')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We now create the neural network and fit it.  Some essential concepts are going on here.\n\n* **Batch Size** - The number of training samples that should be evaluated per training step.  Smaller batch sizes, or mini-batches, are generally preferred.\n* **Step** - A training step is one complete run over the batch.  At the end of a step, the weights are updated, and the neural network learns.\n* **Epoch** - An arbitrary point at which to measure results or checkpoint the model.  Generally, an epoch is one complete pass over the training set.  However, when generators are used, the training set size is theoretically infinite. Because of this, we set a **steps_per_epoch** parameter.\n* **validation steps** - The validation set may also be infinite; because of this, we must specify how many steps we wish to validate at the end of each Epoch."},{"metadata":{"trusted":true},"cell_type":"code","source":"from tensorflow.keras.callbacks import EarlyStopping\n\nmodel = tf.keras.models.Sequential([\n    # Note the input shape is the desired size of the image 300x300 with 3 bytes color\n    # This is the first convolution\n    tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(HEIGHT, WIDTH, 3)),\n    tf.keras.layers.MaxPooling2D(2, 2),\n    # The second convolution\n    tf.keras.layers.Conv2D(32, (3,3), activation='relu'),\n    tf.keras.layers.Dropout(0.5),\n    tf.keras.layers.MaxPooling2D(2,2),\n    # Flatten the results to feed into a DNN\n    \n    tf.keras.layers.Flatten(),\n    tf.keras.layers.Dropout(0.5),\n    # 512 neuron hidden layer\n    tf.keras.layers.Dense(512, activation='relu'),\n    # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('horses') and 1 for the other ('humans')\n    tf.keras.layers.Dense(1, activation='sigmoid')\n])\n\nmodel.summary()\nvalidation_steps = len(df_validate_cut)\nmodel.compile(loss = 'binary_crossentropy', optimizer='adam')\nmonitor = EarlyStopping(monitor='val_loss', min_delta=1e-3, patience=5, verbose=1, mode='auto',\n        restore_best_weights=True)\n\nhistory = model.fit(train_generator, epochs=50, steps_per_epoch=25, \n                    validation_data = val_generator, \n                    verbose = 1, validation_steps=25)\n\n\n","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}