{"cells":[{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","collapsed":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"markdown","source":"An important part when applying Machine Learning is picking an approriate training and test set. In particular they should have the same characteristics. In this competition this could mean that the same kinds of customers and merchants appear in both train and set set. If that is not the case then we'd do a poor job at predicting outside the training set. We would be training the model on a fundamentally different kind of data than the one against which we will be testing it. We wouldn't want to train the model on young professionals and then test our predictions on pensioners. \n\nWhat I want to do in this kernel is to test this assumption by using a simple RandomForestClassifier to check whether there is something particular about the train and test set. If the two data sets are similar then I would expect the classification prediction to be random, i.e. the ROC_AUC score would be around 0.5. "},{"metadata":{"_uuid":"2d3807f83fd70e942bc615499f05e5ec08664563"},"cell_type":"markdown","source":"# Import Libraries"},{"metadata":{"trusted":true,"_uuid":"db7213d8ebf3e491de19926f70c338a0e2adbbcb"},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport re","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"91065abcb118cb1ffe9b281dcd86e3bb11818483"},"cell_type":"markdown","source":"# Load Data"},{"metadata":{"trusted":true,"_uuid":"01dce9a15508d6bd2fc5e969be24e2722d231877"},"cell_type":"code","source":"train = pd.read_csv(\"../input/train.csv\")\ntest = pd.read_csv(\"../input/test.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7de99ccb8728204a2a7cb31e8df113ac138d54e9"},"cell_type":"code","source":"train.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b22c385fa53d5aa0b2c32cdff8a0d68ab878e908"},"cell_type":"code","source":"test.info()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3a55331da94c61455d576137aceb3eefecc5a282"},"cell_type":"markdown","source":"# Clean Data"},{"metadata":{"_uuid":"d9dabe9744fdaf27163d1a79d15f14ab2fae0579"},"cell_type":"markdown","source":"Let's delete the single row with the missing date. It doesn't matter for our purposes. When submitting tp the Kaggle challenge then obviously you can't delete rows from the test set."},{"metadata":{"trusted":true,"_uuid":"4e6594e7e7b8c6a0e10326f855e8e2ffe4bcfbbd"},"cell_type":"code","source":"test.dropna(axis=0,how='any', inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"aae86f74e6127c02d2c4ac4929809dbf38e61dcc"},"cell_type":"markdown","source":"Let's drop 'target' from train as I won't be needing it. Instead I will set up a new target variable, which I call 'train'. 'train' indicates whether or not the row belongs to the training set."},{"metadata":{"trusted":true,"_uuid":"ecccf11cb08076dffd85da700754467dad727354"},"cell_type":"code","source":"# Drop target\ntrain.drop('target', axis=1, inplace=True)\n\n# Create the new target\ntrain['train'] = 1\ntest['train'] = 0","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"227c6821d5a3ca3cfed572324789325896b59a46"},"cell_type":"markdown","source":"Next, I concatenate the two data sets."},{"metadata":{"trusted":true,"_uuid":"d1883b5e8e974a622dbd34752fe0bc753e2510c6"},"cell_type":"code","source":"train_test = pd.concat([train, test], axis=0)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7819fcead6aa826b2edb9420e1e9279976125554"},"cell_type":"markdown","source":"What does the ratio of train to test set look like? Turns out that 62% of the combined data set is made up of train data."},{"metadata":{"trusted":true,"_uuid":"7934a21eadae99f47092dc2591aad2bc7d5c6ac1"},"cell_type":"code","source":"print(np.mean(train_test['train']))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f8893c7c295730023671867b9a3142d2eda704c"},"cell_type":"markdown","source":"## Feature Enginerring"},{"metadata":{"_uuid":"ba569ff76bc0c6503d8a320d388ebc8c6a8ba862"},"cell_type":"markdown","source":"The two features 'first_active_month' and 'card_id' are not in a usable format for our classification yet. So, I will have to make some adjustments. \n\n1. I first convert 'first_active_month' to datetime and then extract each subcomponent from it such as year, month etc.\n2. I convert 'card_id' to a categorical variable so that it becomes a number, which I can actually use."},{"metadata":{"trusted":true,"_uuid":"88acb2bc373d466704866a8dc903889e67620c26"},"cell_type":"code","source":"# Convert to datetime\ntrain_test['first_active_month'] = pd.to_datetime(train_test['first_active_month'])\n\n# Set up the function to extract the subcomponents of date\ndef add_datepart(df, fldname, drop=True, time=False):\n    fld = df[fldname]\n    #if not np.issubdtype(fld.dtype, np.datetime64):\n    #    df[fldname] = fld = pd.to_datetime(fld, infer_datetime_format=True)\n    targ_pre = re.sub('[Dd]ate$', '', fldname)\n    attr = ['Year', 'Month', 'Week', 'Day', 'Dayofweek', 'Dayofyear', 'Hour', 'Minute',\n            'Is_month_end', 'Is_month_start', 'Is_quarter_end', 'Is_quarter_start', 'Is_year_end', 'Is_year_start']\n    if time: attr = attr + ['Hour', 'Minute', 'Second']\n    for n in attr: df[targ_pre + n] = getattr(fld.dt, n.lower())\n    #df[targ_pre + 'Elapsed'] = fld.astype(np.int64) // 10 ** 9\n    if drop: df.drop(fldname, axis=1, inplace=True)\n        \n# Extract subcomponent of date\nadd_datepart(train_test, 'first_active_month')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a73b2a7ab593aa05bcc5edc6a1baf077db64a76e"},"cell_type":"code","source":"# Convert 'card_id' to categorical, i.e. number\ntrain_test['card_id'] = train_test['card_id'].astype('category').cat.as_ordered()\ntrain_test['card_id'] = train_test['card_id'].cat.codes","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1af105a0e23d82642fac206f4b2ffd60c1af4faa"},"cell_type":"markdown","source":"# Modelling"},{"metadata":{"_uuid":"92b322fbdb134d8969b95a66fdfe101adee1dfc3"},"cell_type":"markdown","source":"## Prepare Data for Modelling"},{"metadata":{"_uuid":"d9bf05bb3fecafc301ef898f6ac4173f016cfa13"},"cell_type":"markdown","source":"Here I split the data into dependent and independent variables. There is no need to split this into a new train and validation set as cross_val_score automatically does stratified K-Folds. In other words, the ratio of 62% of training to test data will be preserved when doing the cross-validation. "},{"metadata":{"trusted":true,"_uuid":"b9d16316c92dad1f5d076aaade327e0ce62e2faa"},"cell_type":"code","source":"from sklearn.ensemble import RandomForestClassifier\nfrom sklearn.model_selection import cross_val_score","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1879710e3b65a6315926d0b12edc3968c2fed9c5"},"cell_type":"code","source":"# Split up the dependent and independent variables\nX = train_test.drop('train', axis=1)\ny= train_test['train']","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ad332af84e63524cea04655ad44e2dec3caf0fd7"},"cell_type":"markdown","source":"## Run the Model"},{"metadata":{"_uuid":"da1da8cdcc9e731f91d001e93af2f2437a8c3526"},"cell_type":"markdown","source":"I will run a simple RandomForestClassifier using cross-validation."},{"metadata":{"trusted":true,"_uuid":"9d8617bc0f5c876150e3f12b87a78020757bd2e1"},"cell_type":"code","source":"rfc = RandomForestClassifier(n_estimators=10,\n                            random_state=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7f449b29ad5b068a22ca6ebdf2dbe6c4e3ea3184"},"cell_type":"code","source":"cv_results = cross_val_score(rfc, X, y, cv=5, scoring='roc_auc')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a3348ba3992361a0b215bb6001dbca2480025d9a"},"cell_type":"code","source":"print(cv_results)\nprint(np.mean(cv_results))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6fd6432aaf309935f27dd33fcb7f5a4ad3f5638e"},"cell_type":"markdown","source":"# Evaluation"},{"metadata":{"_uuid":"808f41814bfe265ac4b99eee360053af43f5ebe8"},"cell_type":"markdown","source":"And indeed the distributions/behaviours of train and test set are very similar. The ROC_AUC score is 0.5 indicating that the Random Forest is as good as randomly guessing whether a row belongs to train or test.\n\nThe results might look different if we joined all tables together and do the same analysis. But doing this quick-and-dirty exercise goes a long way of sense checking whether or not train and test data are similar. We wouldn't want to train the model on youn professionals and then test our predictions on pensioners. We would most likely get a pretty bad prediction.\n\n**I hope you found this useful and I would like to hear from whether you use a similar approach or whether there are bits that I can improve.**"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}