{"cells": [{"source": ["# **WORK IN PROGRESS**\n", "\n", "- To reduce the database in order to avoid any crash from the Kernel, we will select the 10 most frequent products in the train data base.\n"], "cell_type": "markdown", "metadata": {"_uuid": "efa14b2ddf9cd56eb7a2d4950d9e0122be56827a", "_cell_guid": "23de8d51-b15b-48bc-b359-5c3e734e9f11"}}, {"source": ["import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from scipy.stats import itemfreq\n", "import seaborn as sns\n", "from sklearn.model_selection import train_test_split\n", "%matplotlib inline\n", "\n", "import datetime\n", "from datetime import date, timedelta"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "ab46f472418091c00d924254f3bf4d638e0e804e", "_cell_guid": "0dc94141-6d73-4e7f-9ddd-c797cc9ff18c", "collapsed": true}}, {"source": ["# **Preparation & Initial Study**\n", "\n", "## 0. Load the Data\n", "\n", "Let's use a part of the code from [inversion's Kernel](https://www.kaggle.com/inversion/dataframe-with-all-date-store-item-combinations)."], "cell_type": "markdown", "metadata": {"_uuid": "a14b76823df87a6c14c32f63d7a995ff90233cfe", "_cell_guid": "507d45a7-f1ea-4367-b8bb-f166cb0351cf"}}, {"source": ["dtypes = {'store_nbr': np.dtype('int64'),\n", "          'item_nbr': np.dtype('int64'),\n", "          'unit_sales': np.dtype('float64'),\n", "          'onpromotion': np.dtype('O')}\n", "\n", "train = pd.read_csv('../input/train.csv', dtype=dtypes)\n", "test = pd.read_csv('../input/test.csv', dtype=dtypes)\n", "stores = pd.read_csv('../input/stores.csv')\n", "items = pd.read_csv('../input/items.csv')\n", "trans = pd.read_csv('../input/transactions.csv')\n", "#oil = pd.read_csv('../input/oil.csv') #we upload this database later\n", "holidays = pd.read_csv('../input/holidays_events.csv')"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "289bc3fd5e48aa096abc4789d0476302e9b9632a", "_cell_guid": "5c377747-bed6-402b-8705-d3c0baac678a", "collapsed": true}}, {"source": ["date_mask = (train['date'] >= '2017-07-15') & (train['date'] <= '2017-08-15')\n", "pd_train = train[date_mask]\n", "\n", "#Print the size\n", "len(pd_train)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "2810c2bc6b1d2f2dd2205a7c4388c1a587b0bcc5", "_cell_guid": "97b53b08-53f9-4661-8177-0c84284c1a03"}}, {"source": ["## 1. Feature engineering\n", "\n", "### **Oil.csv - Replace missing values**\n", "\n", "- We can observe some missing values in the Oil prices database.\n"], "cell_type": "markdown", "metadata": {"_uuid": "fe55c4746b0f407d180b61fe07dd3f4908337fc8", "_cell_guid": "a41298a2-10c8-4575-a45c-2c20c28525c2"}}, {"source": ["#Load the data\n", "oil = pd.read_csv('../input/oil.csv')\n", "\n", "#add missing date\n", "min_oil_date = min(pd_train.date)\n", "max_oil_date = max(pd_train.date)\n", "\n", "calendar = []\n", "\n", "d1 = datetime.datetime.strptime(min_oil_date, '%Y-%m-%d')  # start date date(2008, 8, 15)\n", "d2 = datetime.datetime.strptime(max_oil_date, '%Y-%m-%d')  # end date\n", "\n", "delta = d2 - d1         # timedelta\n", "\n", "for i in range(delta.days + 1):\n", "    calendar.append(datetime.date.strftime(d1 + timedelta(days=i), '%Y-%m-%d'))\n", "\n", "calendar = pd.DataFrame({'date':calendar})\n", "\n", "oil = calendar.merge(oil, left_on='date', right_on='date', how='left')"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "522b904785e4159801287e785366728d66b2c26c", "_cell_guid": "1714fafd-c758-4281-9923-49ccc5dbf3f4", "collapsed": true}}, {"source": ["#Check how many NA\n", "print(oil.isnull().sum(), '\\n')\n", "\n", "#Type\n", "print('Type : ', '\\n', oil.dtypes)\n", "\n", "#Print the 3 first line\n", "oil.head(5)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "63a76a6661adefd0230afd7da8bb7c3948b79589", "_cell_guid": "e4e9c514-53f2-4aeb-9aa4-9f004d055453"}}, {"source": ["We will replace the missing value with the following formula : \n", "\n", "$$\\frac{(dcoilwtico[t-1] + dcoilwtico[t+1])} {2}$$"], "cell_type": "markdown", "metadata": {"_uuid": "225e7e516073e3dfb90a0f2a70958a5a0f61d0a6", "_cell_guid": "a23fc595-9bcd-4607-8ee8-546284cbd71b"}}, {"source": ["#Check index to apply the formula\n", "na_index_oil = oil[oil['dcoilwtico'].isnull() == True].index.values\n", "\n", "#Define the index to use to apply the formala\n", "na_index_oil_plus = na_index_oil.copy()\n", "na_index_oil_minus = np.maximum(0, na_index_oil-1)\n", "\n", "for i in range(len(na_index_oil)):\n", "    k = 1\n", "    while (na_index_oil[min(i+k,len(na_index_oil)-1)] == na_index_oil[i]+k):\n", "        k += 1\n", "    na_index_oil_plus[i] = min(len(oil)-1, na_index_oil_plus[i] + k )\n", "\n", "#Apply the formula\n", "for i in range(len(na_index_oil)):\n", "    if (na_index_oil[i] == 0):\n", "        oil.loc[na_index_oil[i], 'dcoilwtico'] = oil.loc[na_index_oil_plus[i], 'dcoilwtico']\n", "    elif (na_index_oil[i] == len(oil)):\n", "        oil.loc[na_index_oil[i], 'dcoilwtico'] = oil.loc[na_index_oil_minus[i], 'dcoilwtico']\n", "    else:\n", "        oil.loc[na_index_oil[i], 'dcoilwtico'] = (oil.loc[na_index_oil_plus[i], 'dcoilwtico'] + oil.loc[na_index_oil_minus[i], 'dcoilwtico'])/ 2    "], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "860de56c0162249bf8eefe74b9564e5b3f56a9af", "_cell_guid": "7e3b8290-07ff-4c83-8147-d24613052cf2", "collapsed": true}}, {"source": ["#Plot the oil values\n", "oil_plot = oil['dcoilwtico'].copy()\n", "oil_plot.index = oil['date'].copy()\n", "oil_plot.plot()\n", "plt.show()"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "6094db980d5666ccfad050fbbe0ad1de284b1a19", "_cell_guid": "907fbbac-488a-463f-be3f-d22ee8060d63"}}, {"source": ["## 2. Merge all the database"], "cell_type": "markdown", "metadata": {"_uuid": "5d6b4e51a926f2edd2d5ba4c153c98c4acf57d2a", "_cell_guid": "1fc07bfb-7b80-462f-a8a6-2cb451eb19a3"}}, {"source": ["#Merge train\n", "pd_train = pd_train.drop('id', axis = 1)\n", "pd_train = pd_train.merge(stores, left_on='store_nbr', right_on='store_nbr', how='left')\n", "pd_train = pd_train.merge(items, left_on='item_nbr', right_on='item_nbr', how='left')\n", "pd_train = pd_train.merge(holidays, left_on='date', right_on='date', how='left')\n", "pd_train = pd_train.merge(oil, left_on='date', right_on='date', how='left')\n", "pd_train = pd_train.drop(['description', 'state', 'locale_name', 'class'], axis = 1)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "efa0e2ab829c7bcab788f2b170c325e778052c6b", "_cell_guid": "fb46b57e-dc11-4c7f-ba1a-5e4e0fb8a213", "collapsed": true}}, {"source": ["#Merge test - here is the code\n", "test = test.drop('id', axis = 1)\n", "test = test.merge(stores, left_on='store_nbr', right_on='store_nbr', how='left')\n", "test = test.merge(items, left_on='item_nbr', right_on='item_nbr', how='left')\n", "test = test.merge(oil, left_on='date', right_on='date', how='left')\n", "test = test.merge(holidays, left_on='date', right_on='date', how='left')\n", "test = test.drop(['description', 'state', 'locale_name', 'class'], axis = 1)"], "cell_type": "markdown", "metadata": {"_uuid": "ee8b1dd30d5da60a45a4d38bf8e4e3d1f3ed3d4d", "_cell_guid": "e3acd70c-4823-40e4-9092-e75b7e01db3e", "collapsed": true}}, {"source": ["## 3. Quick look and modification on the data\n", "\n", "### **- Newly created Train DataBase**"], "cell_type": "markdown", "metadata": {"_uuid": "49c5b0336e486d356fdc953c3edc71f3b9f92804", "_cell_guid": "47f87c5a-8580-47ee-9aa8-67aad1b00f44"}}, {"source": ["#Shape\n", "print('Shape : ', pd_train.shape, '\\n')\n", "\n", "#Type\n", "print('Type : ', '\\n', pd_train.dtypes)\n", "\n", "#Summary\n", "pd_train.describe()"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "bbb603d929d4061fee812264b599a76e8e21b00c", "_cell_guid": "dd6f2914-10ed-479e-b6aa-3607ae1fdd86", "scrolled": true}}, {"source": ["#5 random lines\n", "pd_train.sample(10)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "9218ef6bb90189b2163f88fa771ac2c349fce125", "_cell_guid": "60871ace-5f4e-4f1d-8bb1-7dc267a4624e"}}, {"source": ["sns.countplot(x='store_nbr', data=pd_train);"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "521e692c54808b5a7f62cb73f18d78adede66c5c", "_cell_guid": "c0981dee-a736-4040-9cb9-62ed4df4a6a5"}}, {"source": ["### **- Let's extract only the 10 most purchased product**"], "cell_type": "markdown", "metadata": {"_uuid": "ba5a8f4e97590f757a4599f53f945306929ef3cf", "_cell_guid": "67fe482c-5a09-4132-94f9-e92570620ee6"}}, {"source": ["#######Get the N most purchased products########\n", "def N_most_labels(data, variable , N , all='TRUE'):\n", "    labels_freq_pd = itemfreq(data[variable])\n", "    labels_freq_pd = labels_freq_pd[labels_freq_pd[:, 1].argsort()[::-1]] #[::-1] ==> to sort in descending order\n", "    \n", "    if all == 'FALSE':\n", "        main_labels = labels_freq_pd[:,0][0:N]\n", "    else: \n", "        main_labels = labels_freq_pd[:,0][:]\n", "        \n", "    labels_raw_np = data[variable].as_matrix() #transform in numpy\n", "    labels_raw_np = labels_raw_np.reshape(labels_raw_np.shape[0],1)\n", "\n", "    labels_filtered_index = np.where(labels_raw_np == main_labels)\n", "    \n", "    return labels_freq_pd, labels_filtered_index\n", "\n", "label_freq, labels_filtered_index = N_most_labels(data = pd_train, variable = \"item_nbr\", N = 10, all='FALSE')\n", "print(\"labels_filtered_index[0].shape = \", labels_filtered_index[0].shape)\n", "\n", "pd_train_filtered = pd_train.loc[labels_filtered_index[0],:]\n", "print(\"pd_train_filtered.shape = \", pd_train_filtered.shape)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "74c49cb6fad24ee35622f19804a2778dfd4da766", "_cell_guid": "e6d54d7e-a042-4fae-9a93-cc74d84ba997"}}, {"source": ["label_freq[0:10]"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "c0dc38fcd2c85e6be684c4bd72977fc41d3fbb4f", "_cell_guid": "1ec08c1c-35bc-4530-aef8-4c63db2646c8"}}, {"source": ["pd_train_filtered.sample(3)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "efca604818edc41e90ce49c0d5122ecba993dc7b", "_cell_guid": "9e7d07ff-8f75-4e04-8c29-e2a9a4480e81"}}, {"source": ["### **- Replace NA for \"holydays\" variables**"], "cell_type": "markdown", "metadata": {"_uuid": "c59fcbd075c6efae668507613b503524e4650820", "_cell_guid": "f32f094b-63de-427c-802b-97239dad8ae0"}}, {"source": ["#Fill in cells if there is no holyday by the value : \"no_holyday\"\n", "na_index_pd_train = pd_train_filtered[pd_train_filtered['type_y'].isnull() == True].index.values\n", "print(\"Size of na_index_pd_train : \", len(na_index_pd_train), '\\n')\n", "\n", "pd_train_filtered.loc[pd_train_filtered['type_y'].isnull(), 'type_y'] = \"no_holyday\"\n", "pd_train_filtered.loc[pd_train_filtered['locale'].isnull(), 'locale'] = \"no_holyday\"\n", "pd_train_filtered.loc[pd_train_filtered['transferred'].isnull(), 'transferred'] = \"no_holyday\"\n", "    \n", "#check is there is NA\n", "pd_train_filtered.isnull().sum()"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "85c099a04b847ce78e3e9c6c02080af03be91d9e", "_cell_guid": "86e24dfb-7fef-45ea-8ae2-86def096c04e"}}, {"source": ["### **- Reformat the date**\n", "\n", "We will extract the day, the month and the year from the 'date' variable."], "cell_type": "markdown", "metadata": {"_uuid": "51fc57c13f727982884e835e03393e704d60beec", "_cell_guid": "e09f4e86-4613-4436-9f59-d52b6c668ae4"}}, {"source": ["def get_month_year(df):\n", "    df['month'] = df.date.apply(lambda x: x.split('-')[1])\n", "    df['year'] = df.date.apply(lambda x: x.split('-')[0])\n", "    \n", "    return df\n", "\n", "get_month_year(pd_train_filtered);"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "ff10325cabb5eba659e1c3b53b40c0971a2e5c8f", "_cell_guid": "f619b819-2026-49c0-ad76-12c2f3613eb7", "collapsed": true}}, {"source": ["pd_train_filtered['date'] = pd.to_datetime(pd_train_filtered['date'])\n", "pd_train_filtered['day'] = pd_train_filtered['date'].dt.weekday_name\n", "pd_train_filtered = pd_train_filtered.drop('date', axis=1)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "d8932f34cbd7aad75a7535a1470b20884c37b792", "_cell_guid": "8b115148-dbaa-4b72-98d3-a00eb8b7a33e", "collapsed": true}}, {"source": ["pd_train_filtered.sample(10)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "70640e56a2235cb239a95cc8007343e2e63bbb5a", "_cell_guid": "14cfd785-667f-4354-8357-bbc9f4dcabb2"}}, {"source": ["## 3. Dummy variables\n", "\n", "We will create binary variables."], "cell_type": "markdown", "metadata": {"_uuid": "849b8714c2205f153601e434af90c26bd6e8b87a", "_cell_guid": "7a617138-098b-41d5-9a4a-8f9ce800d92b"}}, {"source": ["dummy_variables = ['onpromotion','city','type_x','cluster','store_nbr','item_nbr',\n", "                'family','perishable','type_y', 'locale', 'transferred', 'month', 'day']\n", "\n", "for var in dummy_variables:\n", "    dummy = pd.get_dummies(pd_train_filtered[var], prefix = var, drop_first = False)\n", "    pd_train_filtered = pd.concat([pd_train_filtered, dummy], axis = 1)\n", "\n", "pd_train_filtered = pd_train_filtered.drop(dummy_variables, axis = 1)\n", "pd_train_filtered = pd_train_filtered.drop(['year'], axis = 1)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "1e1cf808f465f54b8dcacb2ad20398c84832202e", "_cell_guid": "96a95bbd-aed4-416d-a22c-09d28f4f2426", "collapsed": true}}, {"source": ["print('Shape : ', pd_train_filtered.shape)\n", "pd_train_filtered.sample(10)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "6489ebf5ef1d6ee469b8d121253462da52f9af59", "_cell_guid": "cef02c82-b32f-4418-83bf-eccec861c97e"}}, {"source": ["## 4. Scale variables\n", "\n", "We can scale the variables so they are normalized with 0 mean and with a standard deviation equal to 1."], "cell_type": "markdown", "metadata": {"_uuid": "110bf34feb9f3328d7ad665d4326e2bdfe6ef5b1", "_cell_guid": "d6f1c8c1-672f-4f49-95af-5d3f1f9aeab4"}}, {"source": ["#Re-scale\n", "#We keep this value to re-scale the predicted unit_sales values in the following lines of code.\n", "min_train, max_train = pd_train_filtered['unit_sales'].min(), pd_train_filtered['unit_sales'].max()"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "7457b262971b7a4c749374d52317c3ecf78b7605", "_cell_guid": "9011bd16-73f0-4510-80ea-6d11cdb17c20", "collapsed": true}}, {"source": ["scalable_variables = ['unit_sales','dcoilwtico']\n", "\n", "for var in scalable_variables:\n", "    mini, maxi = pd_train_filtered[var].min(), pd_train_filtered[var].max()\n", "    pd_train_filtered.loc[:,var] = (pd_train_filtered[var] - mini) / (maxi - mini)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "4bc6e479ce8755b8d2de960ecfcc0c17b680bd5a", "_cell_guid": "6ef2f234-4edc-4824-89af-361169e0649a", "collapsed": true}}, {"source": ["print('Shape : ', pd_train_filtered.shape)\n", "pd_train_filtered.sample(10)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "b23d7fab4b6ecabd0dbccab1db0b42bd945c53a7", "_cell_guid": "94e1ff53-ea16-4058-8a9e-cb67a8c3d84c"}}, {"source": ["## **Split the data into a train and a validation database**"], "cell_type": "markdown", "metadata": {"_uuid": "a13a34f919715664195da0a9d3638a91abd96804", "_cell_guid": "13cdfe7a-c056-4a5a-96dc-7be2cb003698"}}, {"source": ["#train database without uni_sales\n", "pd_train_filtered = pd_train_filtered.reset_index(drop=True)  #we reset the index\n", "y_labels = pd_train_filtered['unit_sales']\n", "X_train_filtered = pd_train_filtered.drop(['unit_sales'], axis = 1)\n", "\n", "print('Shape X :', X_train_filtered.shape)\n", "print('Shape y :', y_labels.shape)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "6deacb4d2fc6500f3cebed2e5bd22ae9d4d60702", "_cell_guid": "0616dd0f-3996-41bb-b678-a2358b7f5972"}}, {"source": ["We split the train database with the function train_test_split (very useful and easy to use function)."], "cell_type": "markdown", "metadata": {"_uuid": "ff2656986d3973fbf5389b7b31bec25783e31a21", "_cell_guid": "fff998fe-b963-45d0-8f4a-d2964d7cba4b"}}, {"source": ["num_test = 0.20\n", "X_train, X_validation, y_train, y_validation = train_test_split(X_train_filtered, y_labels, test_size=num_test, random_state=15)\n", "print('X_train shape :', X_train.shape)\n", "print('y_train shape :', y_train.shape)\n", "print('X_validation shape :', X_validation.shape)\n", "print('y_validation shape :', y_validation.shape)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "a4372c8ed50c25a064fa350d980f71b292327197", "_cell_guid": "5aac0db5-4690-41b7-a3a6-bdf334924972"}}, {"source": ["# **Random Forest**\n", "\n", "Before trying to build a neural network, we will try to predict the sales with a random forest (for the fun). We already know that because we cannot use all the data in this kaggle kernel, the results might not be good. Nevertheless, it might be interesting to compare different method."], "cell_type": "markdown", "metadata": {"_uuid": "8135eb2ba55de068e0e798f7ae9f2bd09a6add73", "_cell_guid": "82849055-8861-49c9-a5cf-8ebdaa589cf3"}}, {"source": ["from sklearn.ensemble import RandomForestRegressor\n", "from sklearn.model_selection import GridSearchCV\n", "\n", "# Choose the type of classifier. \n", "RFR = RandomForestRegressor()\n", "\n", "# Choose some parameter combinations to try\n", "#YOU CAN TRY DIFFERENTS PARAMETERS TO FIND THE BEST MODEL\n", "parameters = {'n_estimators': [5, 10, 100],\n", "              #'criterion': ['mse'],\n", "              #'max_depth': [5, 10, 15], \n", "              #'min_samples_split': [2, 5, 10],\n", "              'min_samples_leaf': [1,5]\n", "             }\n", "\n", "# Type of scoring used to compare parameter combinations\n", "#We have to use RandomForestRegressor's own scorer (which is R^2 score)\n", "\n", "# Run the grid search\n", "grid_obj = GridSearchCV(RFR, parameters,\n", "                        cv=5, #Determines the cross-validation splitting strategy /to specify the number of folds in a (Stratified)KFold\n", "                        n_jobs=-1, #Number of jobs to run in parallel\n", "                        verbose=1)\n", "grid_obj = grid_obj.fit(X_train, y_train)\n", "\n", "# Set the clf to the best combination of parameters\n", "RFR = grid_obj.best_estimator_\n", "\n", "# Fit the best algorithm to the data. \n", "RFR.fit(X_train, y_train)"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "9b7a0d7264d666884be14ae4ce2f1227c1942e31", "_cell_guid": "6644f639-fed5-4eae-af38-161ea74cf20c"}}, {"source": ["from sklearn.metrics import r2_score\n", "from sklearn.metrics import mean_squared_error\n", "\n", "predictions = RFR.predict(X_validation)\n", "\n", "#if we want to Re-scale, use this lines of code :\n", "#predictions = predictions * (max_train - min_train) + min_train\n", "#y_validation_RF = y_validation * (max_train - min_train) + min_train\n", "\n", "#if not, keep this one:\n", "y_validation_RF = y_validation\n", "\n", "print('R2 score = ',r2_score(y_validation_RF, predictions), '/ 1.0')\n", "print('MSE score = ',mean_squared_error(y_validation_RF, predictions), '/ 0.0')"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "df03e65b25c6ba49fd3ed09039d3f89da06ef935", "_cell_guid": "1e11b4f1-348b-4421-82e3-7fdf50915eac"}}, {"source": ["#Check and plot the 50 first predictions\n", "plt.plot(y_validation_RF.as_matrix()[0:50], '+', color ='blue', alpha=0.7)\n", "plt.plot(predictions[0:50], 'ro', color ='red', alpha=0.5)\n", "plt.show()"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "83f105b98f2ac3cda3c571d6ab6f0d93506c970d", "_cell_guid": "7231350e-0101-461e-966b-c638671cefac"}}, {"source": ["The results are nice ! We can do better by using more observations."], "cell_type": "markdown", "metadata": {"_uuid": "5222997fa072781925041d67f3ac293c0f6d862f", "_cell_guid": "a8ebfd78-f1e8-4ab7-96a4-4e75c5713a61"}}, {"source": ["# **Neural Network with Keras**\n", "\n", "## KERAS REGRESSION - NEURAL NETWORK"], "cell_type": "markdown", "metadata": {"_uuid": "8d4f3f7eaba1a37f6c858d44609bd5f0475e1957", "_cell_guid": "6575fe9b-4b86-471f-90e4-611eb4b73ead"}}, {"source": ["import keras\n", "\n", "# Convert data as np.array\n", "features = np.array(X_train)\n", "targets = np.array(y_train.reshape(y_train.shape[0],1))\n", "features_validation= np.array(X_validation)\n", "targets_validation = np.array(y_validation.reshape(y_validation.shape[0],1))\n", "\n", "print(features[:10])\n", "print(targets[:10])"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "e30928e25e887cc5e29150fe5625fb9a6eac8374", "_cell_guid": "b5065fe5-263a-4fe8-a951-cf0d0fe41ab4"}}, {"source": ["from keras.models import Sequential\n", "from keras.layers.core import Dense, Dropout, Activation\n", "\n", "# Building the model\n", "model = Sequential()\n", "model.add(Dense(32, activation='relu', input_shape=(X_train.shape[1],)))\n", "model.add(Dropout(.2))\n", "model.add(Dense(16, activation='relu'))\n", "model.add(Dropout(.1))\n", "model.add(Dense(1))\n", "\n", "# Compiling the model\n", "model.compile(loss = 'mse', optimizer='adam', metrics=['mse']) #mse: mean_square_error\n", "model.summary()"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "48dbec21f160c18ba6e75786d482ac3d51bdac9a", "_cell_guid": "d83f314b-8f40-4e38-9d89-e1bc86e94469"}}, {"source": ["# Training the model\n", "epochs_tot = 1000\n", "epochs_step = 250\n", "epochs_ratio = int(epochs_tot / epochs_step)\n", "hist =np.array([])\n", "\n", "for i in range(epochs_ratio):\n", "    history = model.fit(features, targets, epochs=epochs_step, batch_size=100, verbose=0)\n", "    \n", "    # Evaluating the model on the training and testing set\n", "    print(\"Step : \" , i * epochs_step, \"/\", epochs_tot)\n", "    score = model.evaluate(features, targets)\n", "    print(\"Training MSE:\", score[1])\n", "    score = model.evaluate(features_validation, targets_validation)\n", "    print(\"Validation MSE:\", score[1], \"\\n\")\n", "    hist = np.concatenate((hist, np.array(history.history['mean_squared_error'])), axis = 0)\n", "    \n", "# plot metrics\n", "plt.plot(hist)\n", "plt.show()"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "819151431dd610ae0a107fe4e92d0a96caf72894", "_cell_guid": "940f9d4c-aefb-4cbd-a1e9-23f9d24d632e"}}, {"source": ["predictions = model.predict(features_validation, verbose=0)\n", "\n", "print('R2 score = ',r2_score(y_validation, predictions), '/ 1.0')\n", "print('MSE score = ',mean_squared_error(y_validation_RF, predictions), '/ 0.0')"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "ee52c011057118bf70b516d7ffa37d2c6fbb16bd", "_cell_guid": "e06066d1-0fa7-4d00-a31f-86fcde3074a6"}}, {"source": ["#Check and plot the 50 first predictions\n", "plt.plot(y_validation.as_matrix()[0:50], '+', color ='blue', alpha=0.7)\n", "plt.plot(predictions[0:50], 'ro', color ='red', alpha=0.5)\n", "plt.show()"], "execution_count": null, "outputs": [], "cell_type": "code", "metadata": {"_uuid": "a95623376509cc3b79ecc0fdbe94dbaf1bedf1ed", "_cell_guid": "68ecf414-19b0-4c33-817d-abc864610fbd"}}, {"source": ["I'm quite surprised the predictions are \"okay\" even if we did not take a huge number of data. By gathering all the data from the database, we might obtain much more accurate results."], "cell_type": "markdown", "metadata": {"_uuid": "63fbee859397cdadba31795e61ca73ddb84bad4f", "_cell_guid": "83db44c2-c888-4b54-b47e-52749c67c40b"}}], "nbformat": 4, "nbformat_minor": 1, "metadata": {"kernelspec": {"language": "python", "name": "python3", "display_name": "Python 3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "name": "python", "pygments_lexer": "ipython3", "mimetype": "text/x-python", "file_extension": ".py", "version": "3.6.3", "nbconvert_exporter": "python"}}}