{"cells":[{"metadata":{"_uuid":"e091775361969f39c91b9647f2e6b3cf94ae3c3c"},"cell_type":"markdown","source":"# Elo Merchant Category Recommendation | Kaggle\n## By Yonela Nuba\n\n\n### What files do I need?\nYou will need, at a minimum, the train.csv and test.csv files. These contain the card_ids that we'll be using for training and prediction.\n\nThe historical_transactions.csv and new_merchant_transactions.csv files contain information about each card's transactions. historical_transactions.csv contains up to 3 months' worth of transactions for every card at any of the provided merchant_ids. new_merchant_transactions.csv contains the transactions at new merchants (merchant_ids that this particular card_id has not yet visited) over a period of two months.\n\nmerchants.csv contains aggregate information for each merchant_id represented in the data set.\n\n### What should I expect the data format to be?\nThe data is formatted as follows:\n\ntrain.csv and test.csv contain card_ids and information about the card itself - the first month the card was active, etc. train.csv also contains the target.\n\nhistorical_transactions.csv and new_merchant_transactions.csv are designed to be joined with train.csv, test.csv, and merchants.csv. They contain information about transactions for each card, as described above.\n\nmerchants can be joined with the transaction sets to provide additional merchant-level information.\n\n### What am I predicting?\nYou are predicting a loyalty score for each card_id represented in test.csv and sample_submission.csv.\n\n\n# This what we are going to do:\n\n1. Import Libraries and Data\n2. Data Visualization\n3. Data Cleaning\n4. Feature Engineering\n5. More Data Cleaning\n6. Models\n7. Predictions\n8. Output Data"},{"metadata":{"_uuid":"869d7fd85d5299df8931e79fb65eaec1147cc4ad"},"cell_type":"markdown","source":"# 1. Import libraries and Data"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nplt.style.use('fivethirtyeight') \n%matplotlib inline\n\n\nimport warnings\nimport datetime\nimport calendar\nfrom datetime import time\nfrom dateutil.relativedelta import relativedelta\n\n# to ignore future warnings\nwarnings.simplefilter(action = 'ignore', category = FutureWarning)\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\ntrain = pd.read_csv('../input/train.csv')\ntest = pd.read_csv('../input/test.csv')\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"merchants = pd.read_csv('../input/merchants.csv')\nhistorical_trans = pd.read_csv('../input/historical_transactions.csv')\nnew_merchants = pd.read_csv('../input/new_merchant_transactions.csv')\ndata_dict = pd.read_excel('../input/Data_Dictionary.xlsx')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1080ba13d1c1bcfe2d4dcdb087681105166f1044"},"cell_type":"markdown","source":"# 2. Data visualization "},{"metadata":{"trusted":true,"_uuid":"8194d9ce44292a60949ffa878d9d15318741cb2e"},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ac416a68c4eedb4b4c03c090163d410804859dd1"},"cell_type":"code","source":"test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f9f182dbe57f2eae875bd3d6555cf4ba8bc8c7d9"},"cell_type":"code","source":"merchants.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5b283ed364e8deb8f68fcdc044d0861fa9181356"},"cell_type":"code","source":"new_merchants.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"edd9619cf3d0ffbe0b7e260e36c13767c9aed9df"},"cell_type":"code","source":"data_dict.head(10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cdae606555511c4b8a9e1ac6cdedabecb4c757a6"},"cell_type":"code","source":"historical_trans.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3998fb3594a3bf7d63dbdbf7c06f8304e3dae070"},"cell_type":"markdown","source":"## SHAPE OF THE DATASETS"},{"metadata":{"trusted":true,"_uuid":"ef4bd9a49e7050379cd973ddad8193243a4e2c56"},"cell_type":"code","source":"print('train Data shape: {}'.format(train.shape))\nprint('test Data shape: {}'.format(test.shape))\nprint('merchants data shape: {}'.format(merchants.shape))\nprint('New merchants data shape: {}'.format(new_merchants.shape))\nprint('historical data shape: {}'.format(historical_trans.shape))\nprint('Data dictionary data shape: {}'.format(data_dict.shape))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"aa531d187bb2f9736b49e90b01bdf0088b9ab78c"},"cell_type":"code","source":"sns.distplot(train.target.values, kde = False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"3044e142daa87429b4a8923740c9c5c296518ec3"},"cell_type":"code","source":"sns.violinplot(x = \"target\", data=train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5fe6aaec38204678b150854c4194f04c5e2aed50"},"cell_type":"code","source":"# Target value outliers\nprint(f'There are: {train[train.target <- 10].target.shape[0]} outliers')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ed51ac1e19760f2d7c2807680324e05636c0202c"},"cell_type":"code","source":"train.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2ac4287dee7ee7bbcdce4d4226ea187c86e35ccc"},"cell_type":"code","source":"train.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2df295740cd5e227540c09e383aa13efdbac4064"},"cell_type":"code","source":"# Lets change the First_active_month column from string to date.\n\ntrain['first_active_month'] = pd.to_datetime(train['first_active_month'])\n\n# Also do this to the test dataset\n\ntest['first_active_month'] = pd.to_datetime(test['first_active_month'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bafe66a606d21718aec2f2816ffc2f726e93a447"},"cell_type":"code","source":"train_cat_feats = train.dtypes[train.dtypes == 'object'].index\ntrain_num_feats = train.dtypes[train.dtypes != 'object'].index","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0065e802520dbfcf8edc1ada3d44272064db6a99"},"cell_type":"code","source":"print('train Categorical features {}'.format(len(train_cat_feats)))\nprint('train Numerical features {}'.format(len(train_num_feats)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6a5eb8e6767507a57fdb862c16956b015761fc1f"},"cell_type":"code","source":"sns.set()\nsns.pairplot(train, kind='scatter', size = 2, diag_kind='kde')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"72109b2c1b605fbeaf951517777721d79302f454"},"cell_type":"code","source":"plt.figure(figsize = (7,5))\nplt.scatter(x= train.feature_1, y = train.target)\nplt.title('Feature_1', size = 15)\n\nplt.figure(figsize = (7,5))\nplt.scatter(x = train.feature_2, y = train.target)\nplt.title('Feature_2')\n\nplt.figure(figsize = (7,5))\nplt.scatter(x = train.feature_3, y = train.target)\nplt.title('Feature_3')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"907989db2237c418e518caa3b97f71b628d72be2"},"cell_type":"code","source":"train.drop(train[train['target'] < -10].index, inplace = True)\n# train.drop(train[train['feature_2'] < -10].index, inplace = True)\n# train.drop(train[train['feature_3'] < -10].index, inplace = True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6af0ca158196e22b5575309f3a6e06e38f82ec6c"},"cell_type":"code","source":"plt.figure(figsize = (7,5))\nplt.scatter(x= train.feature_1, y = train.target)\nplt.title('Feature_1', size = 15)\n\nplt.figure(figsize = (7,5))\nplt.scatter(x = train.feature_2, y = train.target)\nplt.title('Feature_2')\n\nplt.figure(figsize = (7,5))\nplt.scatter(x = train.feature_3, y = train.target)\nplt.title('Feature_3')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a2ac2f975a4a3e8ac2cd54cc46b370169a990b08"},"cell_type":"code","source":"correlation = train.corr()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"33d7312677368ab013051f82cac7a6e41ec5e5ca"},"cell_type":"code","source":"correlation","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5ddfc8850ca3afb2aa057fb14a4c817a59b3d421"},"cell_type":"code","source":"def missing_values(df):\n    total_miss = df.isnull().sum()\n    perc_miss = 100 * total_miss / len(df)\n    table_miss = pd.concat([total_miss, perc_miss], axis = 1)\n    ren_table = table_miss.rename(columns = {0:'Total Miss Values', 1: '% of miss values'})\n    ren_table = ren_table[ren_table.iloc[:,1]!=0].sort_values('% of miss values', ascending = False).round(2)\n    \n    print('You data contains {}'.format(df.shape[1]) + ' columns and has {}'.format(ren_table.shape[0]) + ' colums with missing values' )\n    \n    return ren_table","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"38fef8850cc8c92e222e4721c4e61867c152c062"},"cell_type":"code","source":"missing_values(train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b23ce5c975421c322da3c9ffefe9fe1df4d98c62"},"cell_type":"code","source":"train = train[train.target > -10]\nsns.pairplot(train[['feature_1', 'feature_2', 'feature_3', 'target']])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"13d08fce495650c799bd3733e1fc7a9c9a845d80"},"cell_type":"markdown","source":"### Lets check the distribution of our training datasets"},{"metadata":{"trusted":true,"_uuid":"51d15fb77895b5fcdd2832db5ba9b40afe8d1111"},"cell_type":"code","source":"f, axes = plt.subplots(1,figsize = (12,6))\nsns.distplot(train.feature_1, ax = axes, kde = False, color = 'green', bins = 10).set_title('Train data')\naxes.set(ylabel = 'Card Counts')\naxes.set_xticks(np.arange(1,6,1))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"181d8e56d035b0e452502f22e47137ca6dbbd21c"},"cell_type":"code","source":"f, axes = plt.subplots(1,figsize = (12,6))\nsns.distplot(train.feature_2, ax = axes, kde = False, color = 'green', bins = 10).set_title('Train data')\naxes.set(ylabel = 'Card Counts')\naxes.set_xticks(np.arange(1,6,1))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c7a0ad6e8d027d186cccd4e063e4bd40c840933e"},"cell_type":"code","source":"f, axes = plt.subplots(1,figsize = (12,6))\nsns.distplot(train.feature_3, ax = axes, kde = False, color = 'green', bins = 10).set_title('Train data')\naxes.set(ylabel = 'Card Counts')\naxes.set_xticks(np.arange(1,6,1))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2a7cf26db68f68fbdadcf3bbbc0ea602d0457deb"},"cell_type":"code","source":"# f, axes = plt.subplots(1,figsize = (12,6))\n# sns.distplot(train.first_active_month, ax = axes, color = 'green', kde = False, bins = 10).set_title('Train data')\n# axes.set(ylabel = 'Card Counts')\n# axes.set_xticks(np.arange(1,6,1))\n# plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1597d864ff75a2d25820fc9b470bfee3d150a9cd"},"cell_type":"code","source":"train.groupby('first_active_month').count()['card_id'].plot(figsize = (15,5), title = 'Count of First Active Month in Train set', color = 'r')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"066ef4f94b8ff80110f97c9fd939ab1ff9e63311"},"cell_type":"code","source":"train_cat_feats = train.dtypes[train.dtypes == 'object'].index\ntrain_num_feats = train.dtypes[train.dtypes != 'object'].index\n\nprint('Train categories: {}'.format(len(train_cat_feats)))\nprint('Train Numerical values: {}'.format(len(train_num_feats)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f7b8bf1ed0091d2d3d298ab567cb5d23e796ada1"},"cell_type":"code","source":"for col in train_cat_feats:\n    cols = train[col].value_counts().index.shape[0]\n    print(f'{cols} \\t unique values in \\t {col}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ec992367706ff5f3e255d544c225459cfdf44a36"},"cell_type":"code","source":"for col in train_num_feats:\n    cols = train[col].value_counts().index.shape[0]\n    print(f'{cols} \\t unique values in \\t {col}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a6548ec1c6fc39b22840a84b928f15fe534ff53b"},"cell_type":"code","source":"target = train.target\ntarget.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d0f5fa0fd037b9964791ab884299715fb9c1b854"},"cell_type":"markdown","source":"## Now lets see how other datasets look like\n\n### Lets start with merchants dataset"},{"metadata":{"trusted":true,"_uuid":"df1a3d3d1d018741c171a39666eee6a05a0d41bb"},"cell_type":"code","source":"merchants.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"40c4efc885f83dc7e4afae53528a9ab9581e5871"},"cell_type":"code","source":"missing_values(merchants)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4db12b4cecce1199cb9a138e187b44bd9743b5e3"},"cell_type":"code","source":"merchants.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8cb7f6820c5b406fe6bbb36a3d1bac14bf040fb9"},"cell_type":"code","source":"merchant_cat_feats = merchants.dtypes[merchants.dtypes == 'object'].index\nmerchant_cat_feats","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"de6ad5909a1e7292e50eedea64fdd09cc8ea04e1"},"cell_type":"code","source":"merchant_num_feats = merchants.dtypes[merchants.dtypes != 'object'].index\nmerchant_num_feats","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cc903c312fb4b8cc4c26e8898272c59489700919"},"cell_type":"code","source":"print('Merchant categorical features {}'.format(len(merchant_cat_feats)))\nprint('Merchant Numerical features {}'.format(len(merchant_num_feats)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ba65761af241dbdb87dd40cf9422cbe0a93afdc6"},"cell_type":"code","source":"for col in merchant_cat_feats:\n    cols = merchants[col].value_counts().index.shape[0]\n    print(f'{cols} \\t unique values in \\t {col}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"08dbd93e7ed1b001ddacc91152fb3bcea8a34701"},"cell_type":"code","source":"for col in merchant_num_feats:\n    cols = merchants[col].value_counts().index.shape[0]\n    print(f'{cols} \\t unique values in \\t {col}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b662e1751578fcce586754d1affbf4a2240051dc"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"13ee3d192c8859e3e7fc2842eb7c676a5f747e9c"},"cell_type":"markdown","source":"## Now lets observe Historical transactions"},{"metadata":{"trusted":true,"_uuid":"20e272872a748c757b26c4621a251673ea0435af"},"cell_type":"code","source":"historical_trans.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1923d13b7f2b4b286ad859db7be4b0e9a8876544"},"cell_type":"code","source":"historical_trans.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d28459a7756de08dbbea182f301c209e7e076c11"},"cell_type":"code","source":"historical_trans.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d2ddf55f94058e5edd855dffedd9c8670efbf754"},"cell_type":"code","source":"missing_values(historical_trans)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c707b84eb0070706294979e82da4db8726691dd7"},"cell_type":"code","source":"hist_cat_feats = historical_trans.dtypes[historical_trans.dtypes == 'object'].index\nhist_num_feats = historical_trans.dtypes[historical_trans.dtypes != 'object'].index\n\nprint('Historical Transactions categorical features {}'.format(len(hist_cat_feats)))\nprint('Historical Transactions Numerical features {}'.format(len(hist_num_feats)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cdbf567dc2b52af16036a3c88ef23bd7a2d148da"},"cell_type":"code","source":"for col in hist_cat_feats:\n    cols = historical_trans[col].value_counts().index.shape[0]\n    print(f'{cols} \\t unique values in \\t {col}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9248378d7b5764e72e4f899da17fd5517615c5e4"},"cell_type":"code","source":"for col in hist_num_feats:\n    cols = historical_trans[col].value_counts().index.shape[0]\n    print(f'{cols} \\t unique values in \\t {col}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5936f6c76278265ec7037d429165f80ab4274729"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cbf6c631dddef2d947ccbecdb2552ac43f32e110"},"cell_type":"markdown","source":"## Now lets check how does the New Merchant dataset looks like"},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"74cb5cc0cea26cd3564446e8439720f97b949fb7"},"cell_type":"code","source":"new_merchants.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"e7ad510a8fa80594b95299810aaaf1b316879dd5"},"cell_type":"code","source":"new_merchants.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cdd71c3107872ca3900b7861876d546dd769faa6"},"cell_type":"code","source":"new_merchants.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"283fe80a51e6390ac6469f662d09c57575ad43a4"},"cell_type":"code","source":"missing_values(new_merchants)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2e753e1d2efbdd8d7990b3c7ca2faecfd1df3a17"},"cell_type":"code","source":"new_merch_cat_feats = new_merchants.dtypes[new_merchants.dtypes == 'object'].index\nnew_merch_num_feats = new_merchants.dtypes[new_merchants.dtypes != 'object'].index\n\nprint('New Merchant categorical features {}'.format(len(new_merch_cat_feats)))\nprint('New Merchant numerical features {}'.format(len(new_merch_num_feats)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ec9c1947a9aab10334310ecc70bb79fa9e99d0e6"},"cell_type":"code","source":"for col in new_merch_cat_feats:\n    cols = new_merchants[col].value_counts().index.shape[0]\n    print(f'{cols} \\t unique values in \\t {col}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c445f0a357892f971a19e36400a52e8cd0d0d708"},"cell_type":"code","source":"for col in new_merch_num_feats:\n    cols = new_merchants[col].value_counts().index.shape[0]\n    print(f'{cols} \\t unique values in \\t {col}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"04a0bc73b40f9679f352e99361f77c27025bc822"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b69f9bea68d4d13abe03fb9743aaf15bc5989292"},"cell_type":"markdown","source":"# 3. Feature Engineering"},{"metadata":{"trusted":true,"_uuid":"fe7fb54ba2f629cb785bcb7260d288bb8dff5b82"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"23c44599d5fc2bff95980af606029e0ec8c10cfd"},"cell_type":"markdown","source":"### Let's work with the Merchants datasets (both merchants and new_merchants data)\n\nfor more info on this check this kernel out [By Robin Denz](https://www.kaggle.com/denzo123/a-closer-look-at-date-variables)"},{"metadata":{"trusted":true,"_uuid":"de24860d1a2d23a8ef25c2fd25e1c7392c495913"},"cell_type":"code","source":"print(f'Merchants data shape: {merchants.shape}, new merchants data shape: {new_merchants.shape}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d51ad06d58a1e09e3ae4b4ee42452b645602f8b4"},"cell_type":"code","source":"print(merchants.columns)\nprint('=='*18)\nprint(new_merchants.columns)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"74ddbb0527a53261984c3ce0ebb512e9e5701df0"},"cell_type":"code","source":"# Lets merge our datasets\n# But first we need to delete duplicate columns\ntodrop = ['merchant_category_id', 'subsector_id', 'city_id', 'state_id', 'category_2', 'category_1']\n\nfor col in todrop:\n    merchants = merchants.drop(col, axis = 1)\n    \nmerchants_data = pd.merge(merchants, new_merchants, on = 'merchant_id')\nmerchants_data[:5]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e9144f4c9f821f637a902f85b7f6f1b63b1cba4c"},"cell_type":"code","source":"# Lets merge our merchant data with the training data.\n\ndata = pd.merge(merchants_data, train, on = 'card_id')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"64e6092d16cfef018556114f133d59e7cd8d091b"},"cell_type":"code","source":"data.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"776f394190100a79e5474f0b1a99e492c56d5dfa"},"cell_type":"code","source":"data[:5]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e7a30fe3c5d5a74143b7fda421b18a461719b741"},"cell_type":"markdown","source":"If you look at our data column - purchase_date, it had date and time. To better work with this data we need to split the two to two columns. "},{"metadata":{"trusted":true,"_uuid":"c5059897a515b06529416bd97a6f018ce0f29ba5"},"cell_type":"code","source":"data['purchase_time'] = data['purchase_date'].str.split(' ')\ndata['purchase_date'] = data['purchase_time'].str[0]\ndata['purchase_time'] = data['purchase_time'].str[1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5ac39aeda278db4ebacbcb9726f3e21039d55676"},"cell_type":"code","source":"# Let's also do what we have done to the testing dataset\ntest_data = pd.merge(merchants_data, test, on = 'card_id')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bfd60d4967a69e99497e9698afcf955384032138"},"cell_type":"code","source":"test_data['purchase_time'] = test_data['purchase_date'].str.split(' ')\ntest_data['purchase_date'] = test_data['purchase_time'].str[0]\ntest_data['purchase_time'] = test_data['purchase_time'].str[1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"959ae9075f6fc57c9b77b33ce8183eab29bb2547"},"cell_type":"code","source":"print(f'Train data shape: {data.shape} Test data shape: {test_data.shape}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5792f7745d9096dbb10dcb1519d35381e56f73a4"},"cell_type":"code","source":"test_data[:5]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7ac5b9bfe0f1b0ecf409194f44388d06b9107441"},"cell_type":"code","source":"#We were not suppose to have converted these to date at first But now we fix that so that we can easily manipulate them\ndata['first_active_month'] = data['first_active_month'].astype(str)\ntest_data['first_active_month'] = test_data['first_active_month'].astype(str)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"38190b8f4d24451c74840c35f1b20a3c0fd5c435"},"cell_type":"code","source":"data.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cdef3bf5b2694184b55ade7899e5790756493257"},"cell_type":"code","source":"# The main business with the purchase_date and purchase_time. Lets see if we can't generate more columns out of them\ndef get_weekday(date_string):\n    date = datetime.datetime.strptime(date_string, '%Y-%m-%d')\n    return calendar.day_name[date.weekday()]\n\n# get weekday for date variable\ndata['purchase_weekday'] = data['purchase_date'].apply(lambda x: get_weekday(x))\n\n# for plotting recode to ordered categorical\nday_labels = ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday']\ndata['purchase_weekday'] = pd.Categorical(data['purchase_weekday'], categories = day_labels, \n                                          ordered = True)\n\ndef get_month(date_string, kind = 'month'):\n    if kind == 'month':\n        date = datetime.datetime.strptime(date_string, '%Y-%m-%d')\n    elif kind == 'day':\n        date = datetime.datetime.strptime(date_string, '%Y-%m-%d')\n    return date.strftime(\"%B\")\n\n# Creating new columns and their values\n\ndata['purchase_month'] = data['purchase_date'].apply(lambda x: get_month(x, kind = 'day'))\ndata['first_active_month2'] = data['first_active_month'].apply(lambda x: get_month(x))\ndata['first_active_year'] = data['first_active_month'].str[:4]\n\nmonth_labels = ['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August',\n                'September', 'October', 'November', 'December']\ndata['purchase_month'] = pd.Categorical(data['purchase_month'], categories = month_labels, \n                                          ordered = True)\ndata['first_active_month2'] = pd.Categorical(data['first_active_month2'], categories = month_labels, \n                                          ordered = True)\n\nyear_labels = ['2011', '2012', '2013', '2014', '2015', '2016', '2017', '2018']\ndata['first_active_year'] = pd.Categorical(data['first_active_year'], categories = year_labels, \n                                          ordered = True)\n\n# get time of the day\ndata['temp'] = data['purchase_time'].str.split(':')\n\ndef get_session(time_list):\n    time_list[0] = int(time_list[0])\n    if time_list[0] > 4 and time_list[0] < 12:\n        return 'Morning'\n    elif time_list[0] >= 12 and time_list[0] < 17:\n        return 'Afternoon'\n    elif time_list[0] >= 17 and time_list[0] < 21:\n        return 'Evening'\n    else:\n        return 'Night'\n    \ndata['purchase_session'] = data['temp'].apply(lambda x: get_session(x))\n\nsession_labels = ['Morning', 'Afternoon', 'Evening', 'Night']\ndata['purchase_session'] = pd.Categorical(data['purchase_session'], categories = session_labels, \n                                          ordered = True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ec71bb10cf84f60ec4cfa92c3afaffe01cc4d9e2"},"cell_type":"code","source":"## time of month\n# as categorical variable, thressholds are arbitrary and could be different\ndef get_time_of_month_cat(date):\n    date_temp = date.split('-')\n    if int(date_temp[2]) < 10:\n        time_of_month = 'Beginning'\n    elif int(date_temp[2]) >= 10 and int(date_temp[2]) < 20:\n        time_of_month = 'Middle'\n    else:\n        time_of_month = 'End'\n    return time_of_month\n\ndata['time_of_month_cat'] = data['purchase_date'].apply(lambda x: get_time_of_month_cat(x))\n\ntof_labels = ['Beginning', 'Middle', 'End']\ndata['time_of_month_cat'] = pd.Categorical(data['time_of_month_cat'], categories = tof_labels, \n                                           ordered = True)\n\ndata['time_of_month_num'] = data['purchase_date'].str[8:].astype(int)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b9d403136c62cd77db9cfdb58355ac5d5b4b23f4"},"cell_type":"code","source":"ax = sns.lineplot(x = \"purchase_month\", y = \"target\", \n                  markers = True, dashes = False, data = data)\nplt.xticks(rotation = 45)\nax.set_title('Target Variable Changes over Purchase Month')\nax.set_xlabel('Purchase Month')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ec33086ae1564d46ab3105e98018b17fcd2acd57"},"cell_type":"code","source":"ax = sns.lineplot(x = \"first_active_month2\", y = \"target\", \n                  markers = True, dashes = False, data = data)\nplt.xticks(rotation = 45)\nax.set_title('Target Variable Changes over the First Active Month')\nax.set_xlabel('First Active Month')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b269e9e2b397090144bd106b3e594bc01f08924b"},"cell_type":"code","source":"print(f'Train data shape: {data.shape} , Test data shape: {test_data.shape}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3b0d21f3e52138692145b5ada3b7dae021dc7533"},"cell_type":"markdown","source":"## Now let's repeat what we have done on Training data to the Testing dataset"},{"metadata":{"trusted":true,"_uuid":"b1e5581ca0c86bc468a17c94e6d39ba449ceadad"},"cell_type":"code","source":"# The main business with the purchase_date and purchase_time. Lets see if we can't generate more columns out of them\ndef get_weekday(date_string):\n    date = datetime.datetime.strptime(date_string, '%Y-%m-%d')\n    return calendar.day_name[date.weekday()]\n\n# get weekday for date variable\ntest_data['purchase_weekday'] = test_data['purchase_date'].apply(lambda x: get_weekday(x))\n\n# for plotting recode to ordered categorical\nday_labels = ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday']\ntest_data['purchase_weekday'] = pd.Categorical(test_data['purchase_weekday'], categories = day_labels, ordered = True)\n\ndef get_month(date_string, kind = 'month'):\n    if kind == 'month':\n        date = datetime.datetime.strptime(date_string, '%Y-%m-%d')\n    elif kind == 'day':\n        date = datetime.datetime.strptime(date_string, '%Y-%m-%d')\n    return date.strftime(\"%B\")\n\ntest_data['purchase_month'] = test_data['purchase_date'].apply(lambda x: get_month(x, kind = 'day'))\ntest_data['first_active_month2'] = test_data['first_active_month'].apply(lambda x: get_month(x))\ntest_data['first_active_year'] = test_data['first_active_month'].str[:4]\n\nmonth_labels = ['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August',\n                'September', 'October', 'November', 'December']\ntest_data['purchase_month'] = pd.Categorical(test_data['purchase_month'], categories = month_labels, ordered = True)\ntest_data['first_active_month2'] = pd.Categorical(test_data['first_active_month2'], categories = month_labels, ordered = True)\n\nyear_labels = ['2011', '2012', '2013', '2014', '2015', '2016', '2017', '2018']\ntest_data['first_active_year'] = pd.Categorical(test_data['first_active_year'], categories = year_labels, ordered = True)\n\n# get time of the day\ntest_data['temp'] = test_data['purchase_time'].str.split(':')\n\ndef get_session(time_list):\n    time_list[0] = int(time_list[0])\n    if time_list[0] > 4 and time_list[0] < 12:\n        return 'Morning'\n    elif time_list[0] >= 12 and time_list[0] < 17:\n        return 'Afternoon'\n    elif time_list[0] >= 17 and time_list[0] < 21:\n        return 'Evening'\n    else:\n        return 'Night'\n    \ntest_data['purchase_session'] = test_data['temp'].apply(lambda x: get_session(x))\n\nsession_labels = ['Morning', 'Afternoon', 'Evening', 'Night']\ntest_data['purchase_session'] = pd.Categorical(test_data['purchase_session'], categories = session_labels, ordered = True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5fcf75410e2a3b1273f93667ccdfac49a83169a7"},"cell_type":"code","source":"## time of month\n# as categorical variable, thressholds are arbitrary and could be different\ndef get_time_of_month_cat(date):\n    date_temp = date.split('-')\n    if int(date_temp[2]) < 10:\n        time_of_month = 'Beginning'\n    elif int(date_temp[2]) >= 10 and int(date_temp[2]) < 20:\n        time_of_month = 'Middle'\n    else:\n        time_of_month = 'End'\n    return time_of_month\n\ntest_data['time_of_month_cat'] = test_data['purchase_date'].apply(lambda x: get_time_of_month_cat(x))\n\ntof_labels = ['Beginning', 'Middle', 'End']\ntest_data['time_of_month_cat'] = pd.Categorical(test_data['time_of_month_cat'], categories = tof_labels, ordered = True)\n\ntest_data['time_of_month_num'] = test_data['purchase_date'].str[8:].astype(int)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"698dcefd30c908219b9528ab3974135c9530e631"},"cell_type":"code","source":"# Now the shape should be the same in terms of columns except that Train data will have one extra column because of the target value\nprint(f'Train data shape: {data.shape} , Test data shape: {test_data.shape}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"69f4888b0f83d5abfbb830a44750921803730b62"},"cell_type":"code","source":"def get_end_of_month(date):\n    date_temp = date.split('-')\n    if int(date_temp[2]) >= 25:\n        end_of_month = 'Y'\n    else:\n        end_of_month = 'N'\n    return end_of_month\ndata['end_of_month'] = data['purchase_date'].apply(lambda x: get_end_of_month(x))\ntest_data['end_of_month'] = test_data['purchase_date'].apply(lambda x: get_end_of_month(x))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f2b0c75a20cd98803828ee5635e220770019aff0"},"cell_type":"code","source":"print(f'Train data shape: {data.shape} , Test data shape: {test_data.shape}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"fd0a1b45f758153988b5dae0e0c2a778d9a5c755"},"cell_type":"code","source":"data[:5]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1eee79548f5442fa5519f4c39c7deaaeb2aad5da"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"78afea22dbd40454c513164dbf08ef1d07a17ae4"},"cell_type":"markdown","source":"# 4. Data Cleaning"},{"metadata":{"trusted":true,"_uuid":"3487e6b68c18cffc683dbbf569a082f91048c860"},"cell_type":"code","source":"correlation_data = data.corr()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ab3eee3091a515262aff52fb51369a8be6850fc1","scrolled":true},"cell_type":"code","source":"correlation_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"238674e1a24fa6e772cb6790a050b14a7de316c7"},"cell_type":"code","source":"print(correlation_data['target'].sort_values(ascending = False))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2a4764e37814d54450c675833c0d7c7ed4eee79b"},"cell_type":"code","source":"data_cat_feats = data.dtypes[data.dtypes == 'object'].index\ndata_num_feats = data.dtypes[data.dtypes != 'object'].index\n\nprint(f'Data categrical: {len(data_cat_feats)}')\nprint(f'Data Numerical: {len(data_num_feats)}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c3406cec709ebda8e624bd1b7528a969cd44dea0"},"cell_type":"code","source":"# for col in data_cat_feats:\n#     cols = data[col].value_counts().index.shape[0]\n#     print(f'{cols} \\t unique values in \\t {col}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"92a1a9020e478090f99431825fc2cecbcd015d9c"},"cell_type":"code","source":"# for col in data_num_feats:\n#     cols = data[col].value_counts().index.shape[0]\n#     print(f'{cols} \\t unique values in \\t {col}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1d5efb414ece141feb6e714640e6b86083bec6c6"},"cell_type":"code","source":"missing_values(data.select_dtypes(['object']))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c1b0b259dc8cfc44594d1954aa483943a3f1193d"},"cell_type":"code","source":"missing_values(data.select_dtypes(include = [np.number]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1216a8a825d960b6c7c704aae1a9297fb07deae2"},"cell_type":"code","source":"missing_values(data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5c80cdbe280841f375dfc64694ed18ff2b84c81f"},"cell_type":"code","source":"data['category_3'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d5263dcc78b408e7e30c6ca9d57de43a5a574c5f"},"cell_type":"code","source":"data['category_3'] = data['category_3'].fillna(data['category_3'].mode()[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"fee0ccf2fdb064d8e0f14d1049772ed6e3e97e2f"},"cell_type":"code","source":"missing_values(data.select_dtypes(['object']))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"84527ed17e8eb41922e5f49dea840f80d690d7f0"},"cell_type":"code","source":"data['category_3'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f60c92ee59a59a44ab5ac8328261afe37e76ccff"},"cell_type":"code","source":"cols = ['category_2', 'avg_sales_lag3', 'avg_sales_lag6', 'avg_sales_lag12']\n\nfor col in cols:\n    data[col] = data[col].fillna(data[col].mean())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0168c019b8f0013cff86dc4b3f2ee8fe1354152d"},"cell_type":"code","source":"missing_values(data)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a16c10cfb139b1b666ebfc3d329ef0d80f04bd45"},"cell_type":"markdown","source":"# Now lets look at the Test data"},{"metadata":{"trusted":true,"_uuid":"cc8e877d589ca5114009f6b67708648af5d7681c"},"cell_type":"code","source":"missing_values(test_data.select_dtypes(['object']))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b1770abeb27a30b3502477f5f7cfa4cc00907a93"},"cell_type":"code","source":"test_data['category_3'] = test_data['category_3'].fillna(test_data['category_3'].mode()[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8b5cc6e1fc8ac89416fafe433e6c1cbe769042a3"},"cell_type":"code","source":"missing_values(test_data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ce34a2cc779f148ac50be8d1658a7db3cdac0ef9"},"cell_type":"code","source":"cols = ['category_2', 'avg_sales_lag3', 'avg_sales_lag6', 'avg_sales_lag12']\n\nfor col in cols:\n    test_data[col] = test_data[col].fillna(test_data[col].mean())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"fe305d8f8c9146921a35f282a0f266998e36012b"},"cell_type":"code","source":"missing_values(test_data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c32c430097488517e13cf26cd4d9aedcb499dbd0"},"cell_type":"markdown","source":"Now that we have filled all the nulls both in training data and testing dataset, we can then work with categorical features, map the ones that have few categorical features and create dummy features for those that have lots of uniques categorical features"},{"metadata":{"trusted":true,"_uuid":"844e260b54176f576a56586dc6229ca5803bc429","scrolled":true},"cell_type":"code","source":"# Lets change the First_active_month column from string to date.\n\ndata['first_active_month'] = pd.to_datetime(data['first_active_month'])\n\n# Also do this to the test dataset\n\ntest_data['first_active_month'] = pd.to_datetime(test_data['first_active_month'])\n\ndata['purchase_date'] = pd.to_datetime(data['purchase_date'])\ntest_data['purchase_date'] = pd.to_datetime(test_data['purchase_date'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2540ec11f445dbe1341208f49bb9f33dc0093879"},"cell_type":"code","source":"data['purchase_time'] = pd.to_timedelta(data['purchase_time'])\ntest_data['purchase_time'] = pd.to_timedelta(test_data['purchase_time'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0cc86d8122409f41774f9627b2b85be2c8e626a0"},"cell_type":"code","source":"data[:5]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"07bc404d4b97de37faf010d125701c407e0e897b"},"cell_type":"code","source":"test_data[:5]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"93ab668167c7a76dc95b8320ee6f7ef489626ee3"},"cell_type":"code","source":"print(f'train: {data.shape}, Test: {test_data.shape}')\n\ncols = ['merchant_id', 'card_id', 'temp']\n\nfor col in cols:\n    data.drop([col], axis = 1, inplace = True)\n    test_data.drop([col], axis = 1, inplace = True)\n    \nprint(f'train: {data.shape}, test: {test_data.shape}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"72aa8944cbc1511169d3883d06bab11d65544511"},"cell_type":"code","source":"catcols = data.select_dtypes(['object'])\n\nfor col in catcols:\n    print(col)\n    print(data[col].value_counts())\n    print('==' * 20)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2124fbae5b2db16b95995ea8eb8baf6609045f90"},"cell_type":"code","source":"data['most_recent_sales_range'] = data['most_recent_sales_range'].map({'A':0, 'B':1, 'C':2, 'D':3, 'E':4})\ndata['most_recent_purchases_range'] = data['most_recent_purchases_range'].map({'A':0, 'B':1, 'C':2, 'D':3, 'E':4})\ndata['category_4'] = data['category_4'].map({'Y':1, 'N':0})\ndata['authorized_flag'] = data['authorized_flag'].map({'Y':0})\ndata['category_1'] = data['category_1'].map({'Y':1, 'N':0})\ndata['category_3'] = data['category_3'].map({'A':0, 'B':1, 'C':2})\ndata['end_of_month'] = data['end_of_month'].map({'Y':1, 'N':0})\n\n\ntest_data['most_recent_sales_range'] = test_data['most_recent_sales_range'].map({'A':0, 'B':1, 'C':2, 'D':3, 'E':4})\ntest_data['most_recent_purchases_range'] = test_data['most_recent_purchases_range'].map({'A':0, 'B':1, 'C':2, 'D':3, 'E':4})\ntest_data['category_4'] = test_data['category_4'].map({'Y':1, 'N':0})\ntest_data['authorized_flag'] = test_data['authorized_flag'].map({'Y':0})\ntest_data['category_1'] = test_data['category_1'].map({'Y':1, 'N':0})\ntest_data['category_3'] = test_data['category_3'].map({'A':0, 'B':1, 'C':2})\ntest_data['end_of_month'] = test_data['end_of_month'].map({'Y':1, 'N':0})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"65b868ef288d2bc6b80459d499a5a04f60b373a2"},"cell_type":"code","source":"catcols = data.select_dtypes(['object'])\n\nfor col in catcols:\n    print(col)\n    print(data[col].value_counts())\n    print('==' * 20)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"48acbb696ab45767463c7dd7dc7eb112b693dbfa"},"cell_type":"code","source":"data[:5]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"dd08b79e0a7a53f17ed83ffb40af28718f9735a1"},"cell_type":"code","source":"data.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"102bcb3217422ee5ca59e2c6878518c22fae7f69"},"cell_type":"code","source":"data_app = pd.get_dummies(data)\ntest_app = pd.get_dummies(test_data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"fc8c1ce35d6a69247aeb71ad4744800f50fdda4e"},"cell_type":"code","source":"data_app[:5]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"245fc5aa210a6ffb9e469db3fa35e5ca55bc8364"},"cell_type":"code","source":"missing_values(test_app)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"472c10c2e5082422e0afa5d78995c49b617581b2"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bb3f2b842175de1e4bbc4788a8007bd8eeb5ee9c"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"78c284f570eebd4943849ce531ce30cb20492dbc"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"85a7f15f3c1274b1641ea4eae346b44f0cad3b1b"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6da6d5f14e7dc02825a104cabbeeadb70edf5164"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"dfc7ec4d1dbb4a82ae76143d646cb181f0c95a72"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"66afe5ec411bb789721b1734714ca9047649a98e"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5b59430e78ab85c087c1e5274d114a2d9e822ac5"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8a1b819bd13b7cad5fecb4cb9c64702eea4ddd36"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"84cf024f5243860f2921682e5d2027b72c152b99"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3362fac8502b987bd5fc43adae5aae03cfe45ed6"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c7258f9777453cb4a29bca6feb582c5fe8be7d84"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d71124dd08e98f39ca1144f8c7f52bad95a35fe4"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"81b76609158f33a949a48b93a339e802301b3929"},"cell_type":"markdown","source":"# Models"},{"metadata":{"trusted":true,"_uuid":"23f402ad80f0f5eb93b7bb0c4bfff75555d3290f"},"cell_type":"code","source":"from xgboost import XGBRegressor","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"570ffb9bfb328d86637e48944588748dc46fe6af"},"cell_type":"code","source":"y = data_app.target\nX = data_app.drop(['target'], axis = 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7ed67420932076e2862c3dfbdef7b4a4600231e6"},"cell_type":"code","source":"X.shape, test_app.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5f128cee20d186ddd31b546dbc356e68a4eb1198"},"cell_type":"code","source":"y[:5]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"acaa5005353104bff1b453adb1fdac33dd1f1274"},"cell_type":"code","source":"xgb = XGBRegressor()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c4684eceda303dc422410294ce0c433a4f11bd46"},"cell_type":"code","source":"X['purchase_date'] = X['purchase_date'].astype(int)\nX['first_active_month'] = X['first_active_month'].astype(int)\nX['purchase_time'] = X['purchase_time'].astype(int)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cc030c82dacc9b0fac8ec951bd937201399fe997"},"cell_type":"code","source":"test_app['purchase_date'] = test_app['purchase_date'].astype(int)\ntest_app['first_active_month'] = test_app['first_active_month'].astype(int)\ntest_app['purchase_time'] = test_app['purchase_time'].astype(int)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1fd89a460ba6457d646d4f0431ad8c4c8579194f"},"cell_type":"code","source":"# xgb_model = xgb.fit(X,y)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3a5a37ee9838d0ea676e01f6ca9955b03de0ee9d"},"cell_type":"code","source":"# test, X = test.align(X, join = 'inner', axis = 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"dd311eb554d8bb9123f12ecf7d3e0d2eec80020c"},"cell_type":"code","source":"test, X = test.align(X, join = 'left', axis = 0)\nX, test_app = X.align(test_app, join = 'left', axis = 0)\nX, y = X.align(y, join = 'left', axis = 0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"388dd727aed3f54a7096652e8310627201922ef0"},"cell_type":"code","source":"test.shape, X.shape, test_app.shape, y.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9f255a03b4a66b4754f48764c52f55680ad618fa"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c925e6c94f68a578070d4ed0d0736c822b06ff21"},"cell_type":"code","source":"from sklearn.linear_model import Lasso","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cc2d313d00d23e072bd5a58c7213ca0b76d25954"},"cell_type":"code","source":"lassoreg = Lasso(alpha = 1.0, normalize = True, max_iter=1e5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f099f97cb339eaf0082030c8c977e503a5f31bf1"},"cell_type":"code","source":"model_lasso = lassoreg.fit(X,y)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9aaa05de501a7da1cfa6cfc876b3b7c63f7f6968"},"cell_type":"code","source":"pred_lasso = model_lasso.predict(test_app)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c3759ffdcb3351d605b2bf6c600da5e904a15362"},"cell_type":"code","source":"test['card_id'].shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5125adbd32f1edcdedad0127c73a4dca58ac2e95"},"cell_type":"code","source":"pred_lasso.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"517d41754100646cc0b754f667d00445a95b0941"},"cell_type":"code","source":"# pred_lasso, test['card_id'] = pred_lasso.align(test['card_id'], join = 'inner', axis = 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2449c35f788ab6856dbfbb173da2290fd43ce6db"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"072e3681388c23deea732fcf310bda0ebc9a65c8"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c7b9004e3e318938d969e5f5321a0239ee217a4a"},"cell_type":"code","source":"sub_lasso = pd.DataFrame({'card_id': test['card_id'], 'target': pred_lasso})\nsub_lasso.to_csv('Lasso_submission.csv', index = False)\nsub_lasso.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ad9987fb4d628afd206105a0f9c8435f35c32c7f"},"cell_type":"markdown","source":"Score : 3.933"},{"metadata":{"trusted":true,"_uuid":"7fc2416f0f0c01cebb7b4b5377296dae583f88c1"},"cell_type":"code","source":"import lightgbm as lgb","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a3836d73ef324fb8474eecefe5d0009dd7560e45"},"cell_type":"code","source":"params = {\n        \"objective\" : \"regression\",\n        \"metric\" : \"rmse\",\n        \"num_leaves\" : 30,\n        \"min_child_weight\" : 50,\n        \"learning_rate\" : 0.05,\n        \"bagging_fraction\" : 0.7,\n        \"feature_fraction\" : 0.7,\n        \"bagging_seed\" : 2018,\n        \"verbosity\" : -1\n    }","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"fa85fe9376ec466baca368ba7c2f2fd344ab1cd1"},"cell_type":"code","source":"train_set = lgb.Dataset(X, label = y.values)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4ed6727d9975761248b953bdb1236f0414b6b762"},"cell_type":"code","source":"val_set = lgb.Dataset(X)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"003a329270d28d4d8ab0fc029fd349824dc24ae8"},"cell_type":"code","source":"lgb_model = lgb.train(params, train_set,valid_sets= val_set, early_stopping_rounds=1000,verbose_eval=1000, num_boost_round=1000)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"03274a887394a662cb1e50f7db9c38cf77222bdb"},"cell_type":"code","source":"lgb_pred = lgb_model.predict(test_app)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cc046401ecbb01a4c3697853ee53c8368cd1b837"},"cell_type":"code","source":"lgb_pred","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"090eedffd720b4c2aabd3bb23938f3ca162dda92"},"cell_type":"code","source":"sub_lgb = pd.DataFrame({'card_id':test['card_id'], 'target': lgb_pred})\nsub_lgb.to_csv('lgb_submission.csv', index = False)\nsub_lgb.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d43bf84dcb44cda6707144ff3a031c5b9195d931"},"cell_type":"code","source":"import xgboost as xgb","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d10a5bedcfc4bf8ff9fa6d9f0fafc5ee10c130ae"},"cell_type":"code","source":"dtrain = xgb.DMatrix(X, label = y.values)\ndtest = xgb.DMatrix(test_app)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2406524bf014ca75b600e0d3d8ed09aa97179bbd"},"cell_type":"code","source":"xgb_params = {'learning_rate':0.001, \n              'n_estimators':1000,\n              'max_depth':4,\n              'min_child_weight':6,\n              'gamma':0,\n              'subsample':0.8,\n              'colsample_bytree':0.8,\n              'objective':'binary:logistic',\n              'nthread':4,\n              'scale_pos_weight':1,\n              'seed':27,\n              'silent':1\n              }","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"481bb0a6ea4a7437abd3082c75bee1278b8d45db"},"cell_type":"code","source":"xgb_model = xgb.train(params = {'silent':1, 'learning_rate':0.1, 'n_estimators':1000, 'min_child_weight':5},dtrain=dtrain, verbose_eval = False, num_boost_round=100)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9dad63584661324fb15eab3b4f93c3840bd99fc9"},"cell_type":"code","source":"xgb_pred = xgb_model.predict(dtest)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6c8690b9db4c1e27d982d577b1236b8b6d64972c"},"cell_type":"code","source":"xgb_pred","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2c1d0e2cf4479d45d4b90da6837904d6f08cf4c7"},"cell_type":"code","source":"xgb_sub = pd.DataFrame({'card_id':test['card_id'], 'target':xgb_pred})\nxgb_sub.to_csv('xgb_submission.csv', index = False)\nxgb_sub.tail(20)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e20510834f80c020f59050e6ea5111e254237376"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}