{"cells":[{"metadata":{"_uuid":"5bc4e2697650c1b8ba570def9bf94f2482b9f705"},"cell_type":"markdown","source":"## Objective:\n\npredict a loyalty score for each card_id represented in test.csv and sample_submission.csv.\n\n\n## Introduction about company and data provided\n\n**Elo**, one of the largest payment brands in Brazil, has built partnerships with merchants in order to offer promotions or discounts to cardholders. \n\nWe need to find out\n\n### Do these promotions work for either the consumer or the merchant? \n\n### Do customers enjoy their experience? \n\n### Do merchants see repeat business? \n\nPersonalization is key.\n\n## File descriptions\n\n**train.csv** - the training set\n\n**test.csv** - the test set\n\n**sample_submission.csv** - a sample submission file in the correct format - contains all card_ids you are expected to predict for.\n\n**historical_transactions.csv** - up to 3 months' worth of historical transactions for each card_id\n\n**merchants.csv** - additional information about all merchants / merchant_ids in the dataset.\n\n**new_merchant_transactions.csv**  - two months' worth of data for each card_id containing ALL purchases that card_id made at merchant_ids that were not visited in the historical data."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n%matplotlib inline\nimport warnings\nwarnings.simplefilter(action='ignore')\nimport gc\nimport os\nimport time\n\nimport plotly\nimport plotly.graph_objs as go\nplotly.offline.init_notebook_mode(connected = True)\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"43c4e42d493e4c1411ac5782e51ca66df0547122"},"cell_type":"markdown","source":"### 1. Data Exploration"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"5506921aae1ecd58b658a3e4b2840b7a8af2cfbb"},"cell_type":"code","source":"# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"fad7813565da7996abf0d967cd89bf5c16a16552"},"cell_type":"markdown","source":"### 2. Data Analysis\n\n#### 2.1 Train and Test Data Analysis"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_df = pd.read_csv('../input/train.csv')\ntest_df = pd.read_csv('../input/test.csv')\nprint(\"{} observations and {} features in train set.\".format(train_df.shape[0],train_df.shape[1]))\nprint(\"{} observations and {} features in test set.\".format(test_df.shape[0],test_df.shape[1]))","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"9170ce149a875e19bff6c91cedcd94f9c1d02160"},"cell_type":"code","source":"train_df.head(5)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"99782303f44afeadb1e9b58a678cc1ad0668c5cc"},"cell_type":"code","source":"test_df = pd.read_csv('../input/test.csv')\ntest_df.head(5)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"26e4a873410fa04f1b4eb866d054eaaec30ef328"},"cell_type":"markdown","source":"#### 2.2.1 Analysing first active month data in train and test dataset\n"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"68e8e26a466bc94e8a3b9c40515a029376ad5cfc"},"cell_type":"code","source":"active_month_series_traindf = train_df['first_active_month'].value_counts()\nactive_month_series_testdf = test_df['first_active_month'].value_counts()\n\ntrace0 = go.Scatter(\n        x = active_month_series_traindf.index,\n        y = active_month_series_traindf.values,\n        name = 'train set')\n\ntrace1 = go.Scatter(\n        x = active_month_series_testdf.index,\n        y = active_month_series_testdf.values,\n        name = 'test set')\n\nplotly.offline.iplot({\n    \"data\": [trace0, trace1],\n    \"layout\": go.Layout(title=\"First Active month in Train & Test data\")\n})","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a0217ec34215eb99ee0aa1ae5f0ece6ea6886812"},"cell_type":"markdown","source":"### Observation: \n\n#### This shows overall business is growing. To be more precise Number of Loyalt card holders are overall increasing every year. "},{"metadata":{"_uuid":"0be68fba54cdc3c07efb5de48febd2a324c100d2"},"cell_type":"markdown","source":"### 2.2.2 Analysing Target values in train dataset"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"cfd23a5535b2f827db56ae325cd975facbe374fa"},"cell_type":"code","source":"train_df['target'] = pd.to_numeric(train_df['target'], errors = 'ignore')\ntrain_df['target'].dtypes","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"e77c0387f26c27f5d188f99b4d0f882734d1d275"},"cell_type":"code","source":"print(\"maximum loyalt score is {}.\".format(train_df['target'].max()))\nprint(\"minimum loyalt score is {}.\".format(train_df['target'].min()))\nprint(\"mean loyalt score is {}.\".format(train_df['target'].mean()))\nprint(\"std loyalt score is {}.\".format(train_df['target'].std()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ac7db9484bf2f1d8f562fb43af004b6820e81143","_kg_hide-input":true},"cell_type":"code","source":"train_df['target'].describe()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"c83d6132f48f06dbddf5248b18418c2e477901a6"},"cell_type":"code","source":"train_df.describe()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"20b1516a8384369dbce626aed9383fc74b075db8"},"cell_type":"code","source":"target_bins = pd.cut(train_df['target'],[-35,-30,-20,-10,-5,0,5,10,20,30])\nLoyalt_score = train_df.groupby(target_bins)['target'].agg(['count']).reset_index()\n\n\nfig, ax = plt.subplots(1,2,figsize=(15,5))\nax[0].set_title('Bar plot of Loyalt Score distribution')\nvis1 = sns.countplot(x=target_bins, data=Loyalt_score, ax = ax[0])\n\n\nax[1].set_title('Histogram of Loyalt Score distribution')\nvis2 = sns.distplot(train_df['target'].values, bins=50, kde=False, color=\"red\", ax = ax[1])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"163c274884d071fc4e3b707dfa2995a7b58847ed"},"cell_type":"markdown","source":"### Observations: \n\n* Loyalt score shows normal distribution between -10 and 10.\n* Majority Loyalt score is between -5 and 5. \n* few data has <-30 values which are suspecious.\n\nHow they have calculated Loyalt Score or Target values, can have various parameters.\n\nHowever, Lets check given feature_1, feature_2 and feature_3 in train and test set."},{"metadata":{"_uuid":"624c220e0401e90d8e015e3b210984c53aff9743"},"cell_type":"markdown","source":"### 2.2.3 Analysing feature_1, feature_2 and feature_3 distribution in train and test dataset"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"30537abca3d6d8cc974145fae134115725bbe2c5"},"cell_type":"code","source":"fig, ax = plt.subplots(1,3, figsize=(20,5))\nvis1 = sns.countplot(x = 'feature_1', data = train_df, ax = ax[0])\nvis2 = sns.countplot(x = 'feature_2', data = train_df, ax = ax[1])\nvis2 = sns.countplot(x = 'feature_3', data = train_df, ax = ax[2])\nax[0].set_title('Feature_1 in train set')\nax[1].set_title('Feature_2 in train set')\nax[2].set_title('Feature_3 in train set')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"3ef3441c18b0d6036c28f2a40471b25c931daa95"},"cell_type":"code","source":"from plotly import tools \n\nfeature1_traindf = train_df['feature_1'].value_counts()\nfeature1_testdf = test_df['feature_1'].value_counts()\nfeature2_traindf = train_df['feature_2'].value_counts()\nfeature2_testdf = test_df['feature_2'].value_counts()\nfeature3_traindf = train_df['feature_3'].value_counts()\nfeature3_testdf = test_df['feature_3'].value_counts()\n\n\nfig = tools.make_subplots(rows=1, cols=3)\n\ntrace0 = go.Bar(x = feature1_traindf.index, y = feature1_traindf.values, name = 'feature1 train set')\ntrace1 = go.Bar(x = feature1_testdf.index, y = feature1_testdf.values, name = 'feature1 test set')\ntrace2 = go.Bar(x = feature2_traindf.index, y = feature2_traindf.values, name = 'feature2 train set')\ntrace3 = go.Bar(x = feature2_testdf.index, y = feature2_testdf.values, name = 'feature2 test set')\ntrace4 = go.Bar(x = feature3_traindf.index, y = feature3_traindf.values, name = 'feature3 train set')\ntrace5 = go.Bar(x = feature3_testdf.index, y = feature3_testdf.values, name = 'feature3 test set')\n\n\nfig.append_trace(trace0, 1, 1)\nfig.append_trace(trace1, 1, 1)\nfig.append_trace(trace2, 1, 2)\nfig.append_trace(trace3, 1, 2)\nfig.append_trace(trace4, 1, 3)\nfig.append_trace(trace5, 1, 3)\n\n\nfig['layout'].update(height=400, width=1100, title='Feature 1, 2 3 in train and test set')\nplotly.offline.iplot(fig, filename='stacked-subplots')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6da9d0cdb1e50ce152a5a3270a6632c12b740bb1"},"cell_type":"markdown","source":"Not getting much detail information, just that feature_1, feature_2 and feature_3 distribution pattern in test and train data is overall same."},{"metadata":{"_uuid":"13955e7f9db55680e73e6eeb29c68c637555cce7"},"cell_type":"markdown","source":"### 2.2 Merchant Data Analysis"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"9a05ec49984a106f5586ec20c3d36d06ef52cbaf"},"cell_type":"code","source":"merchants_df = pd.read_csv('../input/merchants.csv')\nprint('merchant datasets')\nmerchants_df.head(5)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"d01629ea995ba46a88cd726b783b12431d786ad5"},"cell_type":"code","source":"print(\"{} observations and {} features in merchants data.\".format(merchants_df.shape[0],merchants_df.shape[1]))","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"90828e9e71d8a6bf25da53d32a1a1f5c068c55ae"},"cell_type":"code","source":"sale_range = merchants_df['most_recent_sales_range'].value_counts()\npurchanse_range = merchants_df['most_recent_purchases_range'].value_counts()\n\nplotly.offline.iplot({\n    \"data\": [go.Scatter(x = sale_range.index, y = sale_range.values, name = 'sales range'),\n            go.Scatter(x = purchanse_range.index, y = purchanse_range.values, name = 'purchases range')],\n    \"layout\": go.Layout(title=\"most_recent_sales_purchases_range\")\n})","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"22c203061baa81297a8a0ee8798639a2e832098d"},"cell_type":"markdown","source":"### Observation :\n\n#### From given set of merchants data, most recent sale and purchase ratio is quite similar. Which is a good sign for any business.\n    "},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"1c099d5746e62577f7306de04d49ce305da76fbc"},"cell_type":"code","source":"lag3 = merchants_df['active_months_lag3'].value_counts()\nlag6 = merchants_df['active_months_lag6'].value_counts()\nlag12 = merchants_df['active_months_lag12'].value_counts()\n\nplotly.offline.iplot({\n    \"data\": [go.Scatter(x = lag3.index, y = lag3.values, name = 'lag3'),\n            go.Scatter(x = lag6.index, y = lag6.values, name = 'lag6'),\n            go.Scatter(x = lag12.index, y = lag12.values, name = 'lag12')],\n    \"layout\": go.Layout(title=\"active_months_lag3, lag6, lag12 data\")\n})","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7db16fcf427842322c08ab04100f6b53a3063379"},"cell_type":"markdown","source":"### Observations: \n\nBusiness is gradually increasing every quater for individual merchants.\n\nit means promotion offers are working well for loyalt cards. Lets check how good the repeate business from history data."},{"metadata":{"_uuid":"dc16454567e25c104a6b97a608f2c0f76d6e7335"},"cell_type":"markdown","source":"### 2.3 History Merchant Data Analysis"},{"metadata":{"trusted":true,"_uuid":"3841c7c654f4f3f469871b901968116d1d3ce8fa","_kg_hide-input":true},"cell_type":"code","source":"hist_df = pd.read_csv('../input/historical_transactions.csv')\nhist_df.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ef7dc7de8ba6b994bdd8958e3d3041ba10cc48ee","_kg_hide-input":true},"cell_type":"code","source":"print(\"{} observations and {} features in history merchant data.\".format(hist_df.shape[0],hist_df.shape[1]))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"aed59211682c441f9ee65b7bb843fd406b146702"},"cell_type":"markdown","source":"As we can see, this file is resonably big and took much memory space. thus to do further analysis, it's important to reduce memory usage.\n\nThere's a way to optimise for the reading issue\n\n* Load objects as categories.\n* Binary values are switched to int8\n* Binary values with missing values are switched to float16 (int does not understand nan)\n* 64 bits encoding are all switched to 32, or 16 of possible"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"94f2e95500ad92270981218b26f730c2e4d88308"},"cell_type":"code","source":"def reduce_mem_usage(df, verbose=True):\n    numerics = ['int16', 'int32', 'int64', 'float16', 'float32', 'float64']\n    start_mem = df.memory_usage().sum() / 1024**2    \n    for col in df.columns:\n        col_type = df[col].dtypes\n        if df[col].dtypes == 'object':\n            df[col] = df[col].astype('category')\n        if col_type in numerics:\n            c_min = df[col].min()\n            c_max = df[col].max()\n            if str(col_type)[:3] == 'int':\n                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n                    df[col] = df[col].astype(np.int8)\n                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n                    df[col] = df[col].astype(np.int16)\n                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n                    df[col] = df[col].astype(np.int32)\n                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n                    df[col] = df[col].astype(np.int64)  \n            else:\n                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n                    df[col] = df[col].astype(np.float16)\n                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n                    df[col] = df[col].astype(np.float32)\n                else:\n                    df[col] = df[col].astype(np.float64)    \n    end_mem = df.memory_usage().sum() / 1024**2\n    if verbose: print('Mem. usage decreased to {:5.2f} Mb ({:.1f}% reduction)'.format(end_mem, 100 * (start_mem - end_mem) / start_mem))\n    return df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4d0408f347e73cee02da27bde42460016817cce7","scrolled":true},"cell_type":"code","source":"new_trans_df = pd.read_csv(\"../input/new_merchant_transactions.csv\")\nnew_trans_df.head()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"7b8688c7b86de992bb1915bc8c160ea6a0476fe7"},"cell_type":"code","source":"historical_transactions = reduce_mem_usage(hist_df)\nnewMerchant_transactions = reduce_mem_usage(new_trans_df)\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8a3ddc2996cc8247e357dc0a6a904ec18513ea4b"},"cell_type":"markdown","source":"Wow! with very less memory usage, we have dataframe ready to inspect. Thanks to [Ashish Patel's kernel](http://www.kaggle.com/ashishpatel26/lightgbm-gbdt-rf-baysian-ridge-reg-lb-3-61).\nGlad can continue my work now with less memory usage now."},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"c09a9239ce9743b02a62bc7b1a583c96a75dd22b"},"cell_type":"code","source":"historical_transactions.dtypes","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":false,"_uuid":"97e5bdff0bf88893865ae3240f178985d4cecf98"},"cell_type":"markdown","source":"####  2.3.1 Repeat Business"},{"metadata":{"_uuid":"a485c4f875dd6c7c810888966aa8cc909e2f7878"},"cell_type":"markdown","source":"To check repeate business, I tried calculating through monthly total transactions groupby card-Id. however I have experienced more memory consumtion. lets analyze impact of authorized_flag, number of installments, purchase amount."},{"metadata":{"_uuid":"126f44b3482c704d08c503817acfaa734746aa1e"},"cell_type":"markdown","source":"#### 2.3.2 (a) Analysing authorised_flag fields and their impact on business. "},{"metadata":{"trusted":true,"_uuid":"e43b495f6baab2b573f89c8ba4c3359af5083a32","_kg_hide-input":true},"cell_type":"code","source":"histPurchase_amount = pd.DataFrame({'hist_total_transactions' : historical_transactions.groupby(['card_id','authorized_flag'])['purchase_amount'].size()}).reset_index()\nhistPurchase_amount.head(5)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"218e56e44baef623e1df9bf47840d3fc4f11ea02"},"cell_type":"markdown","source":"### Observations: \n\n#### Looks like each card having authorised and unauthorised transactions entries. so, now its important to check weather any card holder has done only unauthorised transactions?"},{"metadata":{"trusted":true,"_uuid":"a34c5abc58657e182a74fdb9eafbb0d5eb70fda4","_kg_hide-input":true},"cell_type":"code","source":"authorised_count = pd.DataFrame({'hist_authorised_count' : histPurchase_amount.groupby('card_id')['authorized_flag'].count()}).reset_index()\nhistPurchase_amount = pd.merge(authorised_count, histPurchase_amount, on = 'card_id', how = 'left')\nhistPurchase_amount.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ec763def17437f3a6fc0db5f5d7a01fdd81ed06f","_kg_hide-input":true},"cell_type":"code","source":"newPurchase_amount = pd.DataFrame({'new_total_transactions' : newMerchant_transactions.groupby(['card_id','authorized_flag'])['purchase_amount'].size()}).reset_index()\nnewPurchase_amount.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"84ac736244cd804d27ebfadeb6f4decea41e4756","_kg_hide-input":true},"cell_type":"code","source":"newAuthorised_count = pd.DataFrame({'new_authorised_count' : newPurchase_amount.groupby('card_id')['authorized_flag'].count()}).reset_index()\nnewPurchase_amount = pd.merge(newAuthorised_count, newPurchase_amount, on = 'card_id', how = 'left')\nnewPurchase_amount.head(5)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7d0a84e9a7dae858b094c6db1e6d762eeb3d4018"},"cell_type":"markdown","source":"Lets check there are any card holder which has only Unauthorised entry. or it is just that in history transactions there were\nno such authorised condition for transaction or loyalt score."},{"metadata":{"trusted":true,"_uuid":"00af72c9859d58cfe03ffed4c5bdee4f41a9b081","_kg_hide-input":true},"cell_type":"code","source":"print(\"Unauthorised entries in new Transactions are: {}\\nOnly Unauthorised entries in history Transactions are: {}\"\n      .format(newPurchase_amount.loc[newPurchase_amount['authorized_flag'] == 'N'].shape[0],\n               histPurchase_amount.loc[((histPurchase_amount['authorized_flag'] == 'N') & (histPurchase_amount['hist_authorised_count'] != 2))].shape[0]))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c80664ba2a0fc8ecdbb4cda3ce0373c91bb388a1"},"cell_type":"markdown","source":"### Observation: \n   #### In History transaction data there are individual card holder and their entry as Authorised as well Unauthorised flag shows there were no restrictions or conditions for authorised transactions only. where in new merchant transaction has might be that condition and thus all entries are authorised only.\n   \n   #### So, we can total the hist_total_transactions of individual card holder and ignore their authorised flag. \n\n#### All card holders has atleast once authorised entires so."},{"metadata":{"trusted":true,"_uuid":"9c8b91d0ec0a990b95b83759090ac7bab850b027","_kg_hide-input":true},"cell_type":"code","source":"# All card holders has atleast once authorised entires so.\ndel histPurchase_amount\ndel newPurchase_amount\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"fde1aa3777f6b363e05fd328125655c40f4f5d12"},"cell_type":"markdown","source":"### 2.3.2 (b) Analysing installments field."},{"metadata":{"trusted":true,"_uuid":"9ad7edc6bb3873a32179c332ac0f06a67a83a8c5","_kg_hide-input":true},"cell_type":"code","source":"historical_transactions.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e5867e40d37481681b3c92f49932e7c7d78a9bb6","_kg_hide-input":true},"cell_type":"code","source":"historical_installments = pd.DataFrame({'hist_total_installments': historical_transactions.groupby(['card_id'])['installments'].value_counts()}).reset_index()\nhistorical_installments.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d469356c1f5d21a7f69b3a99a4136a700202403f","_kg_hide-input":true},"cell_type":"code","source":"newM_installments = pd.DataFrame({'newM_total_installments': newMerchant_transactions.groupby(['card_id'])['installments'].value_counts()}).reset_index()\nnewM_installments.head(5)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"a5c5b8cfb6b51eb5f4d8dab49f715cc4f876f224"},"cell_type":"code","source":"train_df = pd.merge(train_df, historical_installments, on=\"card_id\", how=\"left\")\ntest_df = pd.merge(test_df, historical_installments, on=\"card_id\", how=\"left\")\ntrain_df = pd.merge(train_df, newM_installments, on=\"card_id\", how=\"left\")\ntest_df = pd.merge(test_df, newM_installments, on=\"card_id\", how=\"left\")\ntrain_df.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2ca442f80131772f9546ec3bcd513e5aa16637ee","_kg_hide-input":true},"cell_type":"code","source":"bins = pd.cut(train_df['installments_x'],[0,1,2,3,4,5,6,7,10,20,50,100,900,1000])\n#as the highest number of transaction is 1137\nhist_installments_data = train_df.groupby(bins)['installments_x'].agg(['count','sum','mean','std','min','max']).reset_index()\nhist_installments_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6b634dff138c862cd14e29f161880cca65ee6ba2","_kg_hide-input":true,"scrolled":true},"cell_type":"code","source":"plt.figure(figsize=(18,5))\nsns.boxplot(x=bins, y=train_df['target'], data=hist_installments_data, showfliers=False)\nplt.xticks(rotation='90')\nplt.ylabel('target/loyalt score')\nplt.title('loyalt score based on number of installments in history transactions data')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d9bd7f83a55737153caf77f734d10cfbe4100ec8","_kg_hide-input":true},"cell_type":"code","source":"bins = pd.cut(train_df['installments_y'],[0,1,2,3,4,5,6,7,10,20,50,100,900,1000])\n#as the highest number of transaction is 1137\nnewM_installments_data = train_df.groupby(bins)['installments_y'].agg(['count','sum','mean','std','min','max']).reset_index()\nnewM_installments_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"15df2fe2734b9875bd29d144e2b314eb3fa0d662","_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=(18,5))\nsns.boxplot(x=bins, y=train_df['target'], data=newM_installments_data, showfliers=False)\nplt.xticks(rotation='90')\nplt.ylabel('target/loyalt score')\nplt.title('loyalt score based on number of installments in new Merchant Data')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"fc1b88df2ada9f192649b7a3de0d5014db3d4557"},"cell_type":"markdown","source":"### Observations\n\n#### There is nearly equal distribution of target values based on installments upto 20 installments.\n#### 999 installment and their counts are bit strange."},{"metadata":{"_uuid":"d13d146154de8df1e4b6b90f9deec138db3c0244"},"cell_type":"markdown","source":"### 2.3.2 (c) Analysing Loyalt Score based on purchased amount field."},{"metadata":{"trusted":true,"_uuid":"e6b719afb51815a29faefc9775855776c55794e2","_kg_hide-input":true},"cell_type":"code","source":"histPurchase_amount = pd.DataFrame({'hist_total_transactions' : historical_transactions.groupby(['card_id'])['purchase_amount'].size()}).reset_index()\nhistPurchase_amount.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"03798c075127d8b96099f061aac7e91f8f7eafc7","_kg_hide-input":true},"cell_type":"code","source":"newPurchase_amount = pd.DataFrame({'new_total_transactions' : newMerchant_transactions.groupby(['card_id'])['purchase_amount'].size()}).reset_index()\nnewPurchase_amount.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"81e73b299ababca14a1d2201ee7dc0c34e173f8c","_kg_hide-input":true},"cell_type":"code","source":"histPurchase_amount.columns = ['card_id','hist_total_transactions']\nnewPurchase_amount.columns = ['card_id','new_total_transactions']\n\ntrain_df = pd.merge(train_df, histPurchase_amount, on = 'card_id', how = 'left')\ntest_df = pd.merge(test_df, histPurchase_amount, on = 'card_id', how = 'left')\ntrain_df = pd.merge(train_df, newPurchase_amount, on = 'card_id', how = 'left')\ntest_df = pd.merge(test_df, newPurchase_amount, on = 'card_id', how = 'left')\n\ntrain_df.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7a7ddda9e7538449fc47d1b8347fd1d13bdbc434","_kg_hide-input":true},"cell_type":"code","source":"s1 = train_df.groupby(['hist_total_transactions'])['target'].mean()\ns2 = train_df.groupby(['new_total_transactions'])['target'].mean()\nplotly.offline.iplot({\n    \"data\": [go.Scatter(x = s1.index, y = s1.values, name = 'hist_transactions'),\n            go.Scatter(x = s2.index, y = s2.values, name = 'new_transactions')],\n    \"layout\": go.Layout(title=\"Number of Transactions in History and New Data vs. Loyalt score based on Purchase amount\")\n})","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"3957ec45210c8928c852cb283015860b52f22108"},"cell_type":"code","source":"bins = pd.cut(train_df['hist_total_transactions'],[0,10,20,30,40,50,60,70,80,100,120,150,300, 500,1000,1500,2000,3000])\n#as the highest number of transaction is 1137\nhist_data = train_df.groupby(bins)['hist_total_transactions'].agg(['count','sum','mean','std','min','max']).reset_index()\nhist_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d62e05209fe2962107fa485ab024d636086f8969","_kg_hide-input":true},"cell_type":"code","source":"hist_data.columns = ['card_id', 'count_hist_transaction', \"sum_hist_transaction\", \"mean_hist_transaction\", \"std_hist_transaction\",\n                     \"min_hist_transaction\", \"max_hist_transaction\"]\ntrain_df = pd.merge(train_df, hist_data, on=\"card_id\", how=\"left\")\ntest_df = pd.merge(test_df, hist_data, on=\"card_id\", how=\"left\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1bee6d8f7428b57ba14fa2688d6d4064ff7a0995","_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=(18,5))\nsns.boxplot(x=bins, y=train_df['target'], data=train_df)\nplt.xticks(rotation='90')\nplt.ylabel('target/loyalt score')\nplt.title('loyalt score based on number of purchased amount in new history Data')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1c8aff30ffb93b42f4ec613612173f09bbef0588"},"cell_type":"markdown","source":"### Lets understand this graph bit more. \nLoyalt score or target is based on many parameters. i.e\n\n* Number of transactions ,\n* Purchase amount for each transactions ,\n* Category they buy,\n* Time period they activated and offers that time. etc,\n\nHere, loyalt score for bin range >1000 is different then usual. lets see what difference it speaks and decide weather this information or features are useful for prediction or not?"},{"metadata":{"trusted":true,"_uuid":"0745e8abdbeb16760de87acb4784c1090ed45c9b","_kg_hide-input":true},"cell_type":"code","source":"(train_df.loc[train_df['hist_total_transactions'] > 1500]).head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"92871870020079eabd502ce3882485458ba4c45a","_kg_hide-input":true},"cell_type":"code","source":"bins = pd.cut(train_df['hist_total_transactions'],[1000,1500,2000,3000])\n#as the highest number of transaction is 1137\nhist_data = train_df.groupby(bins)['hist_total_transactions'].agg(['mean']).reset_index()\nplt.figure(figsize=(15,5))\nsns.boxplot(x=bins, y=train_df['target'], data=train_df, showfliers=False)\nplt.xticks(rotation='90')\nplt.ylabel('target/loyalt score')\nplt.title('loyalt score based on purchase amount')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6b913baa4ecf42a9a1256c5a115a7a9a26d2f103"},"cell_type":"markdown","source":"### Observation\n\nAs expected aggregation by 'count','mean' or 'std' their distribution across loyalt score remains same based on the purchase amount range here."},{"metadata":{"_uuid":"b9dbdbeda8f636952afe270cfde79bc316f39c47"},"cell_type":"markdown","source":"Card holders whose first active month is 2017, generally has higher loyalt score. they have feature variations too. however their purchase amount count is average."},{"metadata":{"trusted":true,"_uuid":"f4c68496e1d87a2f7b318211e6e4c34c42bbeee6","_kg_hide-input":true},"cell_type":"code","source":"(train_df.loc[(train_df['first_active_month'] == '2017-01')]).head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a5b5036fe75cedc159c40949c989b848b4880a93"},"cell_type":"markdown","source":"We can clearly see here monthly repete business by observing multiple transactions for individual card holders. \nTarget value or Loyalt score is generally high here in 2017."},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"056a05505016cb06b0bb47b1481327c7a2a46ba8"},"cell_type":"code","source":"historical_transactions['purchase_date'] = pd.to_datetime(historical_transactions['purchase_date'], infer_datetime_format=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4032f242fb374f4e9ab5c582ffe3045915062955","_kg_hide-input":true},"cell_type":"code","source":"historical_transactions['month_yr'] = historical_transactions['purchase_date'].apply(lambda x: x.strftime('%B-%Y'))\nhistorical_transactions.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"478ead3779ccba518ca970482f9ea8296068bff2","_kg_hide-input":true},"cell_type":"code","source":"history_repeat_business = pd.DataFrame({'hist_transactions' : historical_transactions.groupby('card_id')['month_yr'].value_counts()}).reset_index()\nhistory_repeat_business.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6a488cb67b3f84dcf4d0f46cbb40794bdfa459e5","_kg_hide-input":true},"cell_type":"code","source":"bins1 = pd.cut(history_repeat_business['hist_transactions'],[0,10,20,30,40,50,60,70,80,100,120,150,1000,1500])\n#as the highest number of transaction is 1137\nhist_data = history_repeat_business.groupby(bins1)['hist_transactions'].agg(['count','sum'])\nhist_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ebcb017d70e63ff106329328e4bca13db0552ab3","_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=(15,5))\nsns.countplot(x = bins1, data = hist_data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"032fc198b9f2160b5d4b7a452ad7204f5ae9a513","_kg_hide-input":true},"cell_type":"code","source":"cnt_srs = history_repeat_business['month_yr'].value_counts()\ncnt_srs","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-output":true,"_kg_hide-input":true,"trusted":true,"_uuid":"a128504b2dc95db7dc7a00fed9ac5827ed20b1e9"},"cell_type":"code","source":"#month_year data sorting.\nmonth_yr = [\"February-2018\",\"January-2018\",\"January-2017\", \"February-2017\", \"March-2017\", \"April-2017\", \"May-2017\", \"June-2017\", \n          \"July-2017\", \"August-2017\", \"September-2017\", \"October-2017\", \"November-2017\", \"December-2017\"]\nhistory_repeat_business['month_yr'] = pd.Categorical(history_repeat_business['month_yr'], categories=month_yr, ordered=True)\nhistory_repeat_business.sort_values(by='month_yr')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9d453cad629418826e86dce489fba9543a8b953d","_kg_hide-input":true},"cell_type":"code","source":"plotly.offline.iplot({\n    \"data\": [go.Scatter(x = cnt_srs.index[::-1], y = cnt_srs.values[::-1], name = 'transactions')],\n    \"layout\": go.Layout(title=\"Number of transactions in year 2017\")\n})","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"86770ce9ec324a478dd5c9bf26104e4f7a9b0687"},"cell_type":"markdown","source":"Business is growing overall in year 2017-2018. Greate news promotion offers and discount schemes are working reasonably good."},{"metadata":{"trusted":true,"_uuid":"5b675a7c91fd2afa0190c1290987fccfcb2e42eb","_kg_hide-input":true},"cell_type":"code","source":"#history_repeat_business = pd.DataFrame({'hist_transactions' : historical_transactions.groupby('card_id')['month_yr'].value_counts()}).reset_index()\nhistory_repeat_business.loc[history_repeat_business['hist_transactions'] >= 500]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"701a842d1d1b04a69dde64823f0baf4807777d56"},"cell_type":"markdown","source":"Mercent can offer special discounts or promotional offers to the card holders based on number of transactions or based on amount spend."},{"metadata":{"trusted":true,"_uuid":"4d02d8876b84a45c5d5bbe1873310f888b4a04de","_kg_hide-input":true},"cell_type":"code","source":"'''L = ['year', 'month', 'day', 'dayofweek', 'dayofyear', 'weekofyear', 'quarter']\nhist_date_data = historical_transactions.join(pd.concat((getattr(historical_transactions['purchase_date'].dt, i).rename(i) for i in L), axis=1))\n#new_date_data = new_date_data.join(pd.concat((getattr(new_date_data['purchase_date'].dt, i).rename(i) for i in L), axis=1))\nhist_date_data.head(5)'''","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"71de0c92f9e673ea42252e46fb0068ad746a2acd","_kg_hide-input":true},"cell_type":"code","source":"'''months_map = {1: 'Jan', 2: 'Feb', 3:'March', 4:'Apr', 5:'May', 6:'June', 7:'July', 8:'Aug', 9:'Sep', 10:'Oct', 11:'Nov', 12:'Dec'}\nday_map = {0: 'Monday', 1: 'Tuesday', 2: 'Wednesday', 3: 'Thursday', 4:'Friday', 5:'Saturday', 6: 'Sunday'}\nhist_date_data['month'] = hist_date_data['month'].apply(lambda x: months_map[x])\nhist_date_data['dayofweek'] = hist_date_data['dayofweek'].apply(lambda x: day_map[x])\nhist_day_data = pd.DataFrame({'hist_transactions' : hist_date_data.groupby('card_id')['dayofweek'].value_counts()}).reset_index()\n\n#day_srs = hist_date_data.groupby('card_id')['dayofweek'].value_counts()\nhist_day_data.head(5)'''","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d57c8fb5d8c77be5d213b171e65ef1e0806fc61f"},"cell_type":"markdown","source":"### 2.4 New Merchant Data Analysis"},{"metadata":{"trusted":true,"_uuid":"f3113ff84878804ffe2dab6a1ed4f47571283b90","_kg_hide-input":true},"cell_type":"code","source":"'''new_trans_df = pd.read_csv(\"../input/new_merchant_transactions.csv\")\nnew_trans_df.head()'''","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"85e92b3d99d33fd4cb9db3dd54a4bea8912a66d8","_kg_hide-input":true},"cell_type":"code","source":"'''newMerchant_transactions = reduce_mem_usage(new_trans_df)\ngc.collect()'''","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1dee41644612836ce86fc56cbcb1a4e94081e002","_kg_hide-input":true},"cell_type":"code","source":"newMerchant_transactions['purchase_date'] = pd.to_datetime(newMerchant_transactions['purchase_date'], infer_datetime_format=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6aeafc68226935fad24f5522539c875de8c4d7b2","_kg_hide-input":true},"cell_type":"code","source":"newMerchant_transactions['month_yr'] = newMerchant_transactions['purchase_date'].apply(lambda x: x.strftime('%B-%Y'))\nnewMerchant_transactions.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"71269fa18a80d1a6d6c44e430d04de73c385c3d0","_kg_hide-input":true},"cell_type":"code","source":"newMerchant_transactions = pd.DataFrame({'new_merchant_transactions' : newMerchant_transactions.groupby('card_id')['month_yr'].size()}).reset_index()\nnewMerchant_transactions.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"addd2ecb1496aaf4c447e4fdf11489f965abf93e","_kg_hide-input":true},"cell_type":"code","source":"train_df = pd.merge(train_df, newMerchant_transactions, on = 'card_id', how = 'left')\ntest_df = pd.merge(test_df, newMerchant_transactions, on = 'card_id', how = 'left')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3bfd1217f5ccab08c447bfdc71a52af32c1bc7b9","_kg_hide-input":true},"cell_type":"code","source":"train_df.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"81d797262025a865da197c2c7a345e36ffe74b78","_kg_hide-input":true},"cell_type":"code","source":"bins = pd.cut(newMerchant_transactions['new_merchant_transactions'],[0,10,20,30,40,50,60,70,80])\n#as the highest number of transaction is 1137\nnewM_data = newMerchant_transactions.groupby(bins)['new_merchant_transactions'].agg(['count','sum','mean','std','min','max']).reset_index()\nnewM_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c7cbe07c979f78509f89dff8fd229cbd8f3cfaca","_kg_hide-input":true},"cell_type":"code","source":"bins = pd.cut(newMerchant_transactions['new_merchant_transactions'],[0,10,20,30,40,50,60,70,80])\n#as the highest number of transaction is 1137\nnewM_data = newMerchant_transactions.groupby(bins)['new_merchant_transactions'].agg(['count','sum','mean','std','min','max']).reset_index()\nnewM_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8735492dbd1e2aaa1d2b7fb8e5f1b42c1af631c6","_kg_hide-input":true},"cell_type":"code","source":"newM_data.columns = [\"card_id\", \"count_newM_trans\", \"sum_newM_trans\", \"mean_newM_trans\", \"std_newM_trans\", \"min_newM_trans\", \"max_newM_trans\"]\ntrain_df = pd.merge(train_df, newM_data, on=\"card_id\", how=\"left\")\ntest_df = pd.merge(test_df, newM_data, on=\"card_id\", how=\"left\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"424b8059f57942970bc6fd4577ae714d1a9c7fdf","_kg_hide-input":true},"cell_type":"code","source":"train_df.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"354ca6709eaa173587f006336faa5745c572ecc1","_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=(15,5))\nsns.countplot(x = bins, data = train_df)\nplt.ylabel('target/loyalt score')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"0960ce070b031c995a47d628a379f2a6a2cfb2e0"},"cell_type":"code","source":"bins = pd.cut(train_df['new_merchant_transactions'],[0,10,20,30,40,50,60,70,80,100,120])\nnewM_srs = train_df.groupby(bins)['target'].value_counts()\nplt.figure(figsize=(15,5))\nsns.boxplot(x=bins, y='target', data=train_df, showfliers=False)\nplt.ylabel('target/loyalt score')\nplt.xticks(rotation = '90')\nplt.title('new Merchant transaction distribution by target/loyalty score values')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9736f55b5278463359de16be51d7b761fbf23e06"},"cell_type":"markdown","source":"### 3. Baseline Model\n\nLet us build a baseline model using the new features."},{"metadata":{"trusted":true,"_uuid":"72fe047f67b94502cb56dc43e18e4b18ed1cbe07","_kg_hide-input":true},"cell_type":"code","source":"train_df['first_active_month'] = pd.to_datetime(train_df['first_active_month'], infer_datetime_format=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f7a28a5368d283b983a9c8a9fee3320f51d9a924","_kg_hide-input":true},"cell_type":"code","source":"test_df['first_active_month'] = pd.to_datetime(test_df['first_active_month'], infer_datetime_format=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7b2ee6a3786f87abf2a4ef7a398448435fdbc793","_kg_hide-input":true},"cell_type":"code","source":"train_df['year'] = train_df['first_active_month'].dt.year\ntest_df['year'] = test_df['first_active_month'].dt.year\ntrain_df['month'] = train_df['first_active_month'].dt.year\ntest_df['month'] = test_df['first_active_month'].dt.month","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d4cbd190c0e49effc26bd57825f5f4d24ed69fa0","_kg_hide-input":true},"cell_type":"code","source":"cols = ['feature_1', 'feature_2', 'feature_3','installments_x', 'hist_total_installments', 'installments_y',\n       'newM_total_installments', 'hist_total_transactions',\n       'new_total_transactions', 'count_hist_transaction',\n       'sum_hist_transaction', 'mean_hist_transaction', 'std_hist_transaction',\n       'min_hist_transaction', 'max_hist_transaction',\n       'new_merchant_transactions', 'count_newM_trans', 'sum_newM_trans',\n       'mean_newM_trans', 'std_newM_trans', 'min_newM_trans', 'max_newM_trans',\n       'year', 'month']","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"44914a847a6b2bec115eb96762136a35fa25559a"},"cell_type":"markdown","source":"#### For LightGBM theory Understanding followed [Pushkar Mandot](http://https://medium.com/@pushkarmandot/https-medium-com-pushkarmandot-what-is-lightgbm-how-to-implement-it-how-to-fine-tune-the-parameters-60347819b7fc).\n\n#### For implementation of LightGBM baseline model, took help from [SRK](http://https://www.kaggle.com/sudalairajkumar/simple-exploration-notebook-elo) kernel.\n"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"5637edc1879d6a395f8f18d817f8e7eda5ceea58"},"cell_type":"code","source":"from sklearn import model_selection, preprocessing, metrics\nimport lightgbm as lgb\n\nx_train = train_df[cols]\nx_test = test_df[cols]\ny_train = train_df[cols].values\ny_test = test_df[cols].values\n\n# Feature Scaling\nfrom sklearn.preprocessing import StandardScaler\nsc = StandardScaler()\n\n#---Build LGB Model-----\nx_train = sc.fit_transform(x_train)\nx_test = sc.transform(x_test)\n\n\n\ndef run_lgb(train_X, train_y, val_X, val_y, test_X):\n    params = {\n        \"objective\" : \"regression\",\n        \"metric\" : \"rmse\",\n        \"num_leaves\" : 30,\n        \"min_child_weight\" : 50,\n        \"learning_rate\" : 0.05,\n        \"bagging_fraction\" : 0.7,\n        \"feature_fraction\" : 0.7,\n        \"bagging_frequency\" : 5,\n        \"bagging_seed\" : 2018,\n        \"verbosity\" : -1,\n        'max_depth' : 10,\n        'n_estimators' : 200,\n        'min_child_samples': 399, \n        'min_child_weight': 0.1\n    }\n    \n    train_data=lgb.Dataset(train_X,label=train_y)\n    valid_data=lgb.Dataset(val_X,label=val_y)\n \n    evals_result = {}\n    model = lgb.train(params, train_data, 1000, valid_sets=[valid_data], early_stopping_rounds=100, verbose_eval=100, evals_result=evals_result)\n    \n    pred_test_y = model.predict(test_X, num_iteration=model.best_iteration)\n    return pred_test_y, model, evals_result\n\ntrain_X = train_df[cols]\ntest_X = test_df[cols]\ntrain_y = train_df['target'].values\n\npred_test = 0\nkf = model_selection.KFold(n_splits=5, random_state=2018, shuffle=True)\nfor dev_index, val_index in kf.split(train_df):\n    dev_X, val_X = train_X.loc[dev_index,:], train_X.loc[val_index,:]\n    dev_y, val_y = train_y[dev_index], train_y[val_index]\n    \n    pred_test_tmp, model, evals_result = run_lgb(dev_X, dev_y, val_X, val_y, test_X)\n    pred_test += pred_test_tmp\npred_test /= 5.","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"33b30636c19a260788df315e91f4422afddb1a43"},"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(12,10))\nlgb.plot_importance(model, max_num_features=50, height=0.8, ax=ax)\nax.grid(False)\nplt.title(\"LightGBM - Feature Importance\", fontsize=15)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"139caebbfec9083d39aad733693451bba2a5f24f"},"cell_type":"code","source":"sub_df = pd.DataFrame({\"card_id\":test_df[\"card_id\"].values})\nsub_df[\"target\"] = pred_test\nsub_df.to_csv(\"lgb_baseline.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bcec3ddffe887469550f82c711dea407a53f654f"},"cell_type":"markdown","source":"Thanks for stopping by. If you like my kernel, please upvote."},{"metadata":{"trusted":true,"_uuid":"a0e678f692996adfd08f60d772a5706afefb2b66"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}