{"metadata": {"kernelspec": {"language": "python", "name": "python3", "display_name": "Python 3"}, "language_info": {"file_extension": ".py", "version": "3.6.4", "mimetype": "text/x-python", "name": "python", "codemirror_mode": {"version": 3, "name": "ipython"}, "pygments_lexer": "ipython3", "nbconvert_exporter": "python"}}, "nbformat_minor": 1, "cells": [{"metadata": {"_cell_guid": "065c3988-746c-4ce0-912c-371bcaf43f3e", "_uuid": "06a8a373da4ec321a0f28723ec3241b9472fd980"}, "cell_type": "markdown", "source": ["This is inspired by [Ceshine Lee](https://www.kaggle.com/ceshine/lgbm-starter?scriptVersionId=1852107) and [LingZhi's](https://www.kaggle.com/vrtjso/lgbm-one-step-ahead?scriptVersionId=1965435) LGBM kernel. \n", "\n", "This kernel tackles the problem using a 2-layer dense neural network that looks something like this:\n", "![](https://www.pyimagesearch.com/wp-content/uploads/2016/08/simple_neural_network_header.jpg)\n", "\n", "Technically, Tensorflow is used to build this neural network. Before feeding the data into the second layer, batch normalization is used for faster learning(quicker convergent in gradient descent) and Dropout layer is used for regularisation to prevent overfitting.  Instead of a constant learning rate, I have used AdamOptimizer that decays the learning rate over time so that the whole training of network takes much lesser time in my experiment.\n", "\n", "I'm sorry that the naming conventions is a little confusing but feel free to ask questions!"]}, {"metadata": {"_cell_guid": "d003536a-8a8a-439b-a620-789b6ff7d191", "collapsed": true, "_uuid": "522482df3a6bae436e281de4a49e8e5cbdcf3ca0"}, "source": ["import numpy as np # linear algebra\n", "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n", "from sklearn.metrics import mean_squared_error\n", "import gc\n", "import tensorflow as tf"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "6ddab71b-69ed-4fcf-b6e6-218cb5f8a4aa", "_uuid": "3eec70f23ba17467f2055b8e18ef75931e882d25"}, "cell_type": "markdown", "source": ["Some standard data reading, pre-processing, etc"]}, {"metadata": {"_cell_guid": "a3d3b694-7e57-4b25-87f6-b6450d54bd1b", "collapsed": true, "_uuid": "5956bf20052fe9ac934ec58d98edf6b1e21c793e"}, "source": ["df_train_X = pd.read_csv('../input/fork-of-lgbm-one-step-ahead-xgb/x_train.csv')\n", "df_train_Y = pd.read_csv('../input/fork-of-lgbm-one-step-ahead-xgb/y_train.csv')\n", "df_test_X = pd.read_csv('../input/fork-of-lgbm-one-step-ahead-xgb/x_test.csv')\n", "df_test_Y = pd.read_csv('../input/fork-of-lgbm-one-step-ahead-xgb/y_test.csv')\n", "df_Submission_X = pd.read_csv('../input/fork-of-lgbm-one-step-ahead-xgb/submissionX.csv')"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "1e3fa4e1-c3c2-47bb-851e-0db24a411c51", "collapsed": true, "_uuid": "6c89665a02e288b8d9509c31101d66a4219725a4"}, "source": ["itemsDF = pd.read_csv('../input/fork-of-lgbm-one-step-ahead-xgb/items_reindex.csv')"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "145a60c4-8f2a-4d98-9b95-c86a673f1415", "collapsed": true, "_uuid": "276e5c3a6aa8a5f8dfd80dc09d4fca5720715c59"}, "source": ["def NWRMSLE(y, pred, w):\n", "    return mean_squared_error(y, pred, sample_weight=w)**0.5"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "17159c6c-c508-4cae-b28c-cb108e34b87d", "collapsed": true, "_uuid": "7b7daf5ba2d445e32d7b792a74de2d737d701cb4"}, "source": ["df_train_X.drop(['Unnamed: 0'], inplace=True,axis=1)\n", "df_test_X.drop(['Unnamed: 0'], inplace=True,axis=1)\n", "df_train_Y.drop(['Unnamed: 0'], inplace=True,axis=1)\n", "df_test_Y.drop(['Unnamed: 0'], inplace=True,axis=1)\n", "df_Submission_X.drop(['Unnamed: 0'], inplace=True,axis=1)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "a1726396-7595-4fd5-9b54-22f55a7d8549", "_uuid": "72d8c75299368ca06c7a6da188817d6eeb5473ee"}, "cell_type": "markdown", "source": ["This is the start of building the computation graph of TensorFlow NN model.\n", "\n", "Let's declare some constant values for our TF NN model."]}, {"metadata": {"_cell_guid": "61743a41-dfbb-464c-b5d7-8d958c0af263", "collapsed": true, "_uuid": "2bc4e17b1f8ac66abe87b4e4fe609a834e3bb326"}, "source": ["numFeatures = df_train_X.shape[1]\n", "numLabels = 1\n", "hiddenUnit = 20\n", "learningRate = 0.01\n", "numEpochs = 1000"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "cfc350ce-64e1-47d1-b905-adb2e15aef0a", "_uuid": "74f794c2ae4ada83f59095612e3ff6defa24f8c1"}, "cell_type": "markdown", "source": ["Declare the placeholders for the input(x) and output(y_) layer."]}, {"metadata": {"_cell_guid": "66fc1d4e-7033-4bb7-b8a7-1db74ed42b21", "collapsed": true, "_uuid": "ff0f411d54ee04375e3e61e070cb96325054e293"}, "source": ["x = tf.placeholder(tf.float64, [None, numFeatures],name=\"X_placeholder\")\n", "y_ = tf.placeholder(tf.float64, [None, numLabels],name=\"Y_placeholder\")"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "079d111f-a51e-47d7-ac6c-f7a7c0503193", "_uuid": "78f93fb5086020a9d6ce88aae44f3c72fab50b87"}, "cell_type": "markdown", "source": ["Declare the first and second hidden layer by initializing the weights to a range of random normally distributed values."]}, {"metadata": {"_cell_guid": "955e8f22-85b1-4697-8a0c-bb5f87d1a919", "collapsed": true, "_uuid": "49e1644eb6f0b5046d43bb3208e3175216cefa08"}, "source": ["weights = tf.Variable(tf.random_normal([numFeatures,hiddenUnit],stddev=0.1,name=\"weights\", dtype=tf.float64))\n", "weights2 = tf.Variable(tf.random_normal([hiddenUnit,1],name=\"weights2\", dtype=tf.float64))"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "d99c17ff-a5f6-49a1-9eac-8733ab33abd4", "_uuid": "1a5772f78582aa342ed590b3c040f9650954060e"}, "cell_type": "markdown", "source": ["Declare the bias that will be multiplied together with the weights later. Similarly,  we'll initializing the bias to a range of random normally distributed values."]}, {"metadata": {"_cell_guid": "b1dcc2bf-3c26-4368-87cc-660c3f9d42bd", "collapsed": true, "_uuid": "017f52bf6f8f99f971ebeff16542e3fcab79c4c8"}, "source": ["bias = tf.Variable(tf.random_normal([1,hiddenUnit],stddev=0.1,name=\"bias\", dtype=tf.float64))\n", "bias2 = tf.Variable(tf.random_normal([1,1],stddev=0.1,name=\"bias2\", dtype=tf.float64))"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "cc0c9101-4a54-4d07-ad98-03a14a14a9c3", "_uuid": "de75a7ed665e055f9dc87edfa93dd11e9b1f267d"}, "cell_type": "markdown", "source": ["We'll define a placeholder for inputting the \"perishable\" feature which is used to compute the weighted loss"]}, {"metadata": {"_cell_guid": "d8167f29-64d7-4b1c-af4b-ea9bfa7c2755", "collapsed": true, "_uuid": "e70841ab022b0c2eb229b5cb84c4b26dde24c6f9"}, "source": ["weightsNWR = tf.placeholder(tf.float32, [None, 1],name=\"weightsNWR\")"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "202f9adb-5b56-4493-9eb4-adbf1cc0b733", "_uuid": "6bb9982fdcfa9c9c0e9d824168837a7aebffcd63"}, "cell_type": "markdown", "source": ["Take this chance to populate the weight variables which will be used to pass to the placeholder during the training phase."]}, {"metadata": {"_cell_guid": "bfed1df4-9665-4e0a-8013-255ddb21f779", "collapsed": true, "_uuid": "8315380e0dea54a16e5b19964c5aab4c137fb414"}, "source": ["itemWeightsTrain = pd.concat([itemsDF[\"perishable\"]] * 6) * 0.25 + 1\n", "itemWeightsTrain = np.reshape(itemWeightsTrain,(itemWeightsTrain.shape[0], 1))"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "5ad1d7f2-21ea-4b4a-9e45-c301660223cd", "collapsed": true, "_uuid": "14a9f1788d7de802354e861f0a0c9024afa7cd38"}, "source": ["itemWeightsTest = itemsDF[\"perishable\"]* 0.25 + 1\n", "itemWeightsTest = np.reshape(itemWeightsTest,(itemWeightsTest.shape[0], 1))"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "8986930f-89a0-4c9f-b1f0-57a6e03864a9", "_uuid": "2b0641e5c667c6577575ab0ad46e8d9bd1f4cabd"}, "cell_type": "markdown", "source": ["First hidden layer is composed of multiplication of input, weights and the bias we have declared above"]}, {"metadata": {"_cell_guid": "d72c11ec-773d-4a30-bd11-33c042fb2122", "collapsed": true, "_uuid": "8d04207b8c340d8855cd9b57746bc3d58dc674b6"}, "source": ["y = tf.matmul(x,weights) + bias"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "bd2aa89e-4f99-4e0f-82a9-e5d1bf1110ad", "_uuid": "b76d8dda54e47fb56c41473f884ff99cb45fc521"}, "cell_type": "markdown", "source": ["We'll pass the results of the first layer to a relu activation function to convert the linear values into a non-linear one."]}, {"metadata": {"_cell_guid": "32ae7a94-4133-4ca9-8f51-43655bae9dfd", "collapsed": true, "_uuid": "f6a84f46af35adc80fede2d436adba95fcbc6460"}, "source": ["y = tf.nn.relu(y)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "3654c147-5fbc-4e73-a680-2e8f0e087132", "_uuid": "3851fbadabac130ee201a45f2977cee7e857390c"}, "cell_type": "markdown", "source": ["Next, we'll set up a batch normalization function that normalize the values that comes from out the relu function."]}, {"metadata": {"_cell_guid": "2604b90b-a4ce-4cc0-9118-690addd744ea", "_uuid": "019bdc443cf6dbdd0e2804d4488faffe26aa8fbb"}, "cell_type": "markdown", "source": ["Normalization can improve learning speed because the path to the global minimum is reduced:\n", "![](http://cs231n.github.io/assets/nn2/prepro1.jpeg)"]}, {"metadata": {"_cell_guid": "032a3c31-e7ee-4932-bc33-297ebc9b5b10", "_uuid": "0fff27eab2bc6ea71f3ffd4f773322fa4e5ac9cc"}, "cell_type": "markdown", "source": ["Although many literatures say that batch norm is applied **before** activation function, I believe that it would be more beneficial if batch normalization is applied **after** the activation function so that the range of linear values will not be restricted to a down-sized range."]}, {"metadata": {"_cell_guid": "a14d274b-5a38-49bb-900f-3b5342bbd714", "collapsed": true, "_uuid": "17d11f198d77e02ae78bf61d7dfe617fbbb60a6d"}, "source": ["epsilon = 1e-3\n", "batch_mean2, batch_var2 = tf.nn.moments(y,[0])\n", "scale2 = tf.Variable(tf.ones([hiddenUnit],dtype=tf.float64),dtype=tf.float64)\n", "beta2 = tf.Variable(tf.zeros([hiddenUnit],dtype=tf.float64),dtype=tf.float64)\n", "y = tf.nn.batch_normalization(y,batch_mean2,batch_var2,beta2,scale2,epsilon)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "08953c7b-d7f2-4b3f-aa55-19dccdc7e70b", "_uuid": "1dc2a59d37e7b752b79318aed7a17f651abf1dd7"}, "cell_type": "markdown", "source": ["We set up a dropout layer to intentionally deactivate certain units. This will improve generalization and reduce overfitting(better validation set score) because it force your layer to learn with different neurons the same \"concept\".\n", "\n", "![](http://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/image_folder_5/dropout.jpeg)\n", "\n", "Note that during the prediction phase, the dropout is deactivated."]}, {"metadata": {"_cell_guid": "4612e4c6-52b7-448d-b863-a875d381b4e8", "collapsed": true, "_uuid": "43558918348b8d7c7869d977115b531906aec1b3"}, "source": ["dropout_placeholder = tf.placeholder(tf.float64,name=\"dropout_placeholder\")\n", "y=tf.nn.dropout(y,dropout_placeholder)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "4fefc685-0a1b-4386-b3e6-15e83fdc2a83", "_uuid": "3081872e7fa2a902b90abef919e7c0c3c3f3ca97"}, "cell_type": "markdown", "source": ["Next we'll build the second hidden layer. As usual, it's the multiplication of input, weights and the bias we have declared above"]}, {"metadata": {"_cell_guid": "2cdf2313-f535-4115-890a-37b032944b16", "collapsed": true, "_uuid": "c7a88bff0883061b3b02eeebeab14dac4512409b"}, "source": ["#create 1 more hidden layer\n", "y = tf.matmul(y,weights2)+bias2"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "437c4cd3-002a-4d64-a31b-51fcd9cbaad0", "_uuid": "cef23f8c2a1f426ac8898f003ed5843d1f4f9eb9"}, "cell_type": "markdown", "source": ["Pass the results to another relu activation function"]}, {"metadata": {"_cell_guid": "e30e1b54-5466-49a0-8380-9a3bd77d0339", "collapsed": true, "_uuid": "8603c1e88d4c81e1dc7c9a6e941c933533b19f0d"}, "source": ["y = tf.nn.relu(y)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "53a09cab-c113-4ed7-a8a9-fd008fe9bf17", "_uuid": "8d1beb310e21730fc6ea75b8db38d30f90f90c60"}, "cell_type": "markdown", "source": ["The loss function that are trying to optimize, or the goal of training, is to minimize the weighted mean squared error. \n", "\n", "Perishable items are given a weight of 1.25 where all other items are given a weight of 1.00, as described in the competition details. \n"]}, {"metadata": {"_cell_guid": "ce15bd4b-06eb-4b11-aa75-8656071abcb5", "collapsed": true, "_uuid": "020a80c5762308da25860503ceaad732f5c39f44"}, "source": ["loss = tf.losses.mean_squared_error(predictions=y,labels=y_,weights=weightsNWR)\n", "cost = tf.reduce_mean(loss)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "c667017a-5ab7-4b78-915e-c874d5145bc2", "_uuid": "71905a6855bb5378f98e0118d8104d529ce0b55c"}, "cell_type": "markdown", "source": ["As stated above, I have found AdamOptimizer, which decays the learning rate over time to be better than the GradientOptimizer option in terms of training speed. Beside that, AdamOptimizer also dampens the oscillations in the direction that do not point to the minimal so that the back-and-forth between these walls will be reduced and at the same time, we'll build up momentum in the direction of the minimum."]}, {"metadata": {"_cell_guid": "7a8bd7a3-5d6f-4389-8cc9-226e61f178a5", "collapsed": true, "_uuid": "451368b68e63a92cf8288c939ce4b96e1eedf252"}, "source": ["optimizer = tf.train.AdamOptimizer(learning_rate=learningRate).minimize(cost)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "2c0b60f7-ed5f-401f-ae75-35b3f5d38717", "_uuid": "c394a683da2e534bc8bbd95170af74f230c73c7c"}, "cell_type": "markdown", "source": ["Finally, we'll create a TF session for training our model."]}, {"metadata": {"_cell_guid": "20b68216-96aa-46bf-a3c3-c7b68c145de7", "collapsed": true, "_uuid": "bce8df6e20c494837dce0aff1142db7c18ab86ca"}, "source": ["sess = tf.Session()"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "24460efc-253c-4163-a6a8-4a88d25f0372", "_uuid": "6c661e05e68931fbe7d45620d4e932fb2c39213d"}, "cell_type": "markdown", "source": ["Initilize the variables that we have been setting up"]}, {"metadata": {"_cell_guid": "4dcbd39e-9b2c-4789-8d5d-f9a2427ab5fa", "collapsed": true, "_uuid": "ac77ba7c8866bcb601b6110d89d3a0cfe4417d38"}, "source": ["sess.run(tf.global_variables_initializer())"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "02a1ef0e-13e3-4a06-bf46-a4f9ede1c5ca", "_uuid": "d2add8e3facce88f231fb1558bba6afd6dfa7a0f"}, "cell_type": "markdown", "source": ["Finally, it's time to train our NN model! We are actually training 16 NN model (1 for each column of y values). There are 16 columns in Y train and Y test, which represents prediction for 16 days.\n", "\n", "We are training for 1000 epoch. At every 100 epoch, we do an output in terms of weighted mse to see if it overfits for test set. After 1000 epoch is done, we'll use the trained model for prediction by feeding the X submission data.\n", "\n", "Note that Dropout rate is set at 0.6(deactivate 40% of units) during training and back at 1.0(no deactivation) during prediction. Check these values when you are building your own drop out layers to ensure that you are not throwing away results during prediction.\n", "\n", "Also, training will be longer than Kaggle's allowable timeout limit so run this at your own local machine."]}, {"metadata": {"_cell_guid": "008e2100-1485-4e7b-9ac3-6d82dff9c8f1", "collapsed": true, "_uuid": "4617f9b58905434e800eb7da3328087e7ee65d4e"}, "source": ["val_pred_nn = []\n", "test_pred_nn = []\n", "cate_vars_nn = []\n", "submit_pred_nn=[]\n", "\n", "trainingLoss=[]\n", "validationLoss=[]\n", "\n", "\n", "#step through all the dates(16)\n", "for i in range(16):\n", "    print(\"Step %d\" % (i+1))\n", "    \n", "    trainY_NN = np.reshape(df_train_Y.iloc[:,i],(df_train_Y.shape[0], 1))\n", "    testY_NN = np.reshape(df_test_Y.iloc[:,i],(df_test_Y.shape[0], 1))\n", "    \n", "    for epoch in range(numEpochs):\n", "        _,loss = sess.run([optimizer,cost], feed_dict={x: df_train_X, y_: trainY_NN,weightsNWR:itemWeightsTrain,dropout_placeholder:0.6})\n", "\n", "        if epoch%100 == 0:\n", "            print('Epoch', epoch, 'completed out of',numEpochs,'loss:',loss)\n", "            #trainingLoss.append(loss)\n", "            #check against test dataset\n", "            test_pred = sess.run(cost, feed_dict={x:df_test_X,y_: testY_NN,weightsNWR:itemWeightsTest,dropout_placeholder:1.0})\n", "            print('Acc for test dataset ',test_pred)\n", "            #validationLoss.append(test_pred)\n", "    \n", "    tf_pred = sess.run(y,feed_dict={x:df_test_X,weightsNWR:itemWeightsTest,dropout_placeholder:1.0})\n", "    tf_predY = np.reshape(tf_pred,(tf_pred.shape[0],))\n", "    test_pred_nn.append(tf_predY)\n", "    print('score for step',(i+1))\n", "    print(\"Validation mse:\", mean_squared_error(df_test_Y.iloc[:,i], tf_predY))\n", "    print('NWRMSLE:',NWRMSLE(df_test_Y.iloc[:,i], tf_predY,itemsDF[\"perishable\"]*0.25+1))\n", "\n", "    #predict for submission set\n", "    nn_submit_predY = sess.run(y,feed_dict={x:df_Submission_X,dropout_placeholder:1.0})\n", "    nn_submit_predY = np.reshape(nn_submit_predY,(nn_submit_predY.shape[0],))\n", "    submit_pred_nn.append(nn_submit_predY)\n", "    \n", "    gc.collect()\n", "    sess.run(tf.global_variables_initializer())"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "1b8d7c02-3374-4abe-bde4-a1ee97892975", "collapsed": true, "_uuid": "7ff65b082c6c5941bcac83dbaeca997a3ae614c6"}, "source": ["nnTrainY= np.array(test_pred_nn).transpose()\n", "pd.DataFrame(nnTrainY).to_csv('nnTrainY.csv')\n", "nnSubmitY= np.array(submit_pred_nn).transpose()\n", "pd.DataFrame(nnSubmitY).to_csv('nnSubmitY.csv')"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "af614e2e-6c15-4971-b1db-74633d560bd8", "_uuid": "39daccd2c2999f89026552fd48053214ae807b8b"}, "cell_type": "markdown", "source": ["You can use the below NWRMSLE to compare test set score with other benchmarks, or finding out the optimal weights for your ensemble."]}, {"metadata": {"_cell_guid": "1e411ffc-f27c-49ae-9f92-147ae59715e5", "collapsed": true, "_uuid": "b488f53945d0b877d9463b3c96d9a08a24783b40"}, "source": ["print('NWRMSLE:',NWRMSLE(df_test_Y,nnTrainY,itemsDF[\"perishable\"]* 0.25 + 1))"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "76b81bbf-4bba-4208-b7b4-27b0e3b7dbe3", "_uuid": "5441d55891ee5b1602981683b730db33984048cc"}, "cell_type": "markdown", "source": ["With the prediction values from the NN model, prepare for submission. The following cells are pretty self-explantory."]}, {"metadata": {"_cell_guid": "3da860c8-7200-47f3-9d51-9d2b337047a6", "collapsed": true, "_uuid": "85774f61bd3e7f4ed62299c527e74b6e0edce990"}, "source": ["#to reproduce the testing IDs\n", "df_train = pd.read_csv(\n", "    '../input/favorita-grocery-sales-forecasting/train.csv', usecols=[1, 2, 3, 4, 5],\n", "    dtype={'onpromotion': bool},\n", "    converters={'unit_sales': lambda u: np.log1p(\n", "        float(u)) if float(u) > 0 else 0},\n", "    parse_dates=[\"date\"],\n", "    skiprows=range(1, 66458909)  # 2016-01-01\n", ")\n", "\n", "df_2017 = df_train.loc[df_train.date>=pd.datetime(2017,1,1)]\n", "del df_train\n", "\n", "df_2017 = df_2017.set_index(\n", "    [\"store_nbr\", \"item_nbr\", \"date\"])[[\"unit_sales\"]].unstack(\n", "        level=-1).fillna(0)\n", "df_2017.columns = df_2017.columns.get_level_values(1)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "4d578f15-731a-45f3-b137-b5ae8d8f0c04", "collapsed": true, "_uuid": "ccefa47b9973cd8a55b637b303146a546f5b450f"}, "source": ["#submitDF = pd.read_csv('../input/testforsubmit/testForSubmit.csv',index_col=False)\n", "df_test = pd.read_csv(\n", "    \"../input/favorita-grocery-sales-forecasting/test.csv\", usecols=[0, 1, 2, 3, 4],\n", "    dtype={'onpromotion': bool},\n", "    parse_dates=[\"date\"]  # , date_parser=parser\n", ").set_index(\n", "    ['store_nbr', 'item_nbr', 'date']\n", ")"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "00f05b5a-aba4-4184-90bf-f85e39fc2c76", "collapsed": true, "_uuid": "d6c3cae11657cf1ce9ae9139b8459cf7a1fd35b6"}, "source": ["print(\"Making submission...\")\n", "combinedSubmitPredY = nnSubmitY\n", "df_preds = pd.DataFrame(\n", "    combinedSubmitPredY, index=df_2017.index,\n", "    columns=pd.date_range(\"2017-08-16\", periods=16)\n", ").stack().to_frame(\"unit_sales\")\n", "df_preds.index.set_names([\"store_nbr\", \"item_nbr\", \"date\"], inplace=True)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "141d87f2-cbf6-4494-91fc-50f59d955785", "collapsed": true, "_uuid": "51366ddf4c72948b3688c695c92ef0ad6c272094"}, "source": ["submission = df_test[[\"id\"]].join(df_preds, how=\"left\").fillna(0)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "97a27a7c-07f0-4ca2-a166-7f469d28f1fe", "collapsed": true, "_uuid": "f7b6c796940e5f7dfe7687497cc8efaaaf0021c0"}, "source": ["submission[\"unit_sales\"] = np.clip(np.expm1(submission[\"unit_sales\"]), 0, 1000)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "18283e83-b4de-49a1-bda0-814fce9100b5", "collapsed": true, "_uuid": "95f435c6f795e46c8b8f8fdf5587bb4a7c2a95f4"}, "source": ["submission[['id','unit_sales']].to_csv('submit_nn.csv',index=None)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_cell_guid": "22c96a61-0109-42c5-a0b0-038fad4e667f", "_uuid": "4f6cd75564a0c7f7cc0bd4956c403e82343b3630"}, "cell_type": "markdown", "source": ["**TODO/Areas to improve/To be Updated:**\n", "* Normalize X data before inputting into input layer.\n", "* Use Tensorboard to graphically visualize the model to see if there's any bottlenecks or areas that could improve the robustness of the model."]}, {"metadata": {"_cell_guid": "ce98a7e2-a365-47f7-82d0-45b65bf3e9e6", "collapsed": true, "_uuid": "d25d88345e370a57c8aeb81f8c57fa98edb2632c"}, "source": [], "cell_type": "code", "outputs": [], "execution_count": null}], "nbformat": 4}