{"cells":[{"metadata":{},"cell_type":"markdown","source":"#  (OHE + Logit)  + (Target, Cyclic Encoding+ Xgboost)\n\nIdeas:\n* Replace missing values with constant\n* Add number of missing values in row as a feature\n* Apply StandardScaler to created feature\n* Apply Target to features that have many unique values\n* Apply OHE for other features + Logit\n* Apply Target and Cyclic Encoding + Xgboost\n* Blend Logit and Xgboost"},{"metadata":{},"cell_type":"markdown","source":"\nIt's just some minor changes to the great kernel Cat in dat 2: Embeddings,Target + Keras\n<br>I just encode the date and month features with [cyclic encoding method](https://www.kaggle.com/avanwyk/encoding-cyclical-features-for-deep-learning) and change Keras to XGboost Model, and also add my learning note which is Chinese.\n<br>Please Upvote the [original kernel](https://www.kaggle.com/pavelvpster/cat-in-dat-2-embeddings-target-keras). \n<p><font color=\"blue\">Don't hesitate to give your suggestions in the comment section</font></p>\n<p><font color=\"blue\">Thank you...</font></p>\n\n"},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\n\nimport warnings\nwarnings.simplefilter('ignore')\nfrom sklearn.metrics import roc_auc_score\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.model_selection import GridSearchCV\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Load data"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"train = pd.read_csv('/kaggle/input/cat-in-the-dat-ii/train.csv', index_col='id')\ntest = pd.read_csv('/kaggle/input/cat-in-the-dat-ii/test.csv', index_col='id')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head(3).T","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def summary(df):\n    summary = pd.DataFrame(df.dtypes, columns=['dtypes'])\n    summary = summary.reset_index()\n    summary['Name'] = summary['index']\n    summary = summary[['Name', 'dtypes']]\n    summary['Missing'] = df.isnull().sum().values    \n    summary['Uniques'] = df.nunique().values\n    summary['First Value'] = df.loc[0].values\n    summary['Second Value'] = df.loc[1].values\n    summary['Third Value'] = df.loc[2].values\n    return summary\n\nsummary(train)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Handle missing values"},{"metadata":{},"cell_type":"markdown","source":"Add number of missing values in row as a feature"},{"metadata":{"trusted":true},"cell_type":"code","source":"# 看这一行中有多少个NULL值\ntrain['missing_count'] = train.isnull().sum(axis=1)\ntest['missing_count'] = test.isnull().sum(axis=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Replace missing values with constants"},{"metadata":{"trusted":true},"cell_type":"code","source":"missing_number = -99999\nmissing_string = 'MISSING_STRING'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"numerical_features = [\n    'bin_0', 'bin_1', 'bin_2',\n    'ord_0',\n    'day', 'month'\n]\n\nstring_features = [\n    'bin_3', 'bin_4',\n    'ord_1', 'ord_2', 'ord_3', 'ord_4', 'ord_5',\n    'nom_0', 'nom_1', 'nom_2', 'nom_3', 'nom_4', 'nom_5', 'nom_6', 'nom_7', 'nom_8', 'nom_9'\n]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def impute(train, test, columns, value):\n    for column in columns:\n        train[column] = train[column].fillna(value)\n        test[column] = test[column].fillna(value)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"impute(train, test, numerical_features, missing_number)\nimpute(train, test, string_features, missing_string)","execution_count":null,"outputs":[]},{"metadata":{"heading_collapsed":true},"cell_type":"markdown","source":"## Feature engineering"},{"metadata":{"hidden":true},"cell_type":"markdown","source":"Split 'ord_5' preserving missing values"},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"#ord_5 是由两个字母组成的，这段的目的是将这两个字母拆分成两列，仍然保留用 MISSING_STRING 去填充\ntrain['ord_5_1'] = train['ord_5'].str[0]\ntrain['ord_5_2'] = train['ord_5'].str[1]\n\ntrain.loc[train['ord_5'] == missing_string, 'ord_5_1'] = missing_string\ntrain.loc[train['ord_5'] == missing_string, 'ord_5_2'] = missing_string\n\ntrain = train.drop('ord_5', axis=1)\n\n\ntest['ord_5_1'] = test['ord_5'].str[0]\ntest['ord_5_2'] = test['ord_5'].str[1]\n\ntest.loc[test['ord_5'] == missing_string, 'ord_5_1'] = missing_string\ntest.loc[test['ord_5'] == missing_string, 'ord_5_2'] = missing_string\n\ntest = test.drop('ord_5', axis=1)","execution_count":null,"outputs":[]},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"# 这里将 Feature 作为 3类，simple_features， ohe_features 和 target_features \n# Apply Target to features that have many unique values 这里仅是讲 这一列 Unique Value如果很大的话，才做Target Encoding\nsimple_features = [\n    'missing_count'\n]\n\noe_features = [\n    'bin_0', 'bin_1', 'bin_2', 'bin_3', 'bin_4',\n    'nom_0', 'nom_1', 'nom_2', 'nom_3', 'nom_4',\n    'ord_0', 'ord_1', 'ord_2', 'ord_3', 'ord_4', 'ord_5_1', 'ord_5_2'\n]\n\ntarget_features = [\n    'nom_5', 'nom_6', 'nom_7', 'nom_8', 'nom_9'\n]\n\ncyc_features = ['day', 'month']\n\n# ohe的含义是 One Hot Encoding\nohe_features = [\n    'bin_0', 'bin_1', 'bin_2', 'bin_3', 'bin_4',\n    'nom_0', 'nom_1', 'nom_2', 'nom_3', 'nom_4',\n    'nom_5', 'nom_6', 'nom_7', 'nom_8', 'nom_9', # 这一行的这些Feature包含了大量的unique values\n    'ord_0', 'ord_1', 'ord_2', 'ord_3', 'ord_4', 'ord_5_1', 'ord_5_2',\n    'day', 'month'\n]\n# 用于 Logit 的是 ohe_features + simple_features\n# 用于 Xgboost 的是 oe_features + cyc_features + simple_features + target_features","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Extract target variable"},{"metadata":{"trusted":true},"cell_type":"code","source":"y_train = train['target'].copy()\nx_train = train.drop('target', axis=1)\ndel train\n\nx_test = test.copy()\n\ndel test","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Standard scaler for simple_features"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.preprocessing import StandardScaler\n\n\nscaler = StandardScaler()\nsimple_x_train = scaler.fit_transform(x_train[simple_features])\nsimple_x_test = scaler.transform(x_test[simple_features])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"type(simple_x_train)","execution_count":null,"outputs":[]},{"metadata":{"heading_collapsed":true},"cell_type":"markdown","source":"## OHE"},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"from sklearn.preprocessing import OneHotEncoder\n\n\nohe = OneHotEncoder(dtype='uint16', handle_unknown=\"ignore\")\nohe_x_train = ohe.fit_transform(x_train[ohe_features])\nohe_x_test = ohe.transform(x_test[ohe_features])","execution_count":null,"outputs":[]},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"# OneHotEncoder transfer 之后的默认的是一个稀疏矩阵，可以通过 to_array 或者设置 sparse=False 等转化成 正常的 Array\ntype(ohe_x_train)","execution_count":null,"outputs":[]},{"metadata":{"hidden":true,"scrolled":false,"trusted":true},"cell_type":"code","source":"ohe_x_train.shape","execution_count":null,"outputs":[]},{"metadata":{"hidden":true,"scrolled":true,"trusted":true},"cell_type":"code","source":"ohe_x_train[:,0]","execution_count":null,"outputs":[]},{"metadata":{"heading_collapsed":true},"cell_type":"markdown","source":"## Encoding cyclic features "},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"def encode(data, col, max_val):\n    data[col + '_sin'] = np.sin(2 * np.pi * data[col]/max_val)\n    data[col + '_cos'] = np.cos(2 * np.pi * data[col]/max_val)\n    return data\n#直接生成2列\n\nx_train = encode(x_train, 'month', 12)\nx_train = encode(x_train, 'day', 7)\n\nx_test = encode(x_test, 'month', 12)\nx_test = encode(x_test, 'day', 7)","execution_count":null,"outputs":[]},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"cyclic_x_train = x_train[['month_sin','month_cos','day_sin','day_cos']]\ncyclic_x_test = x_test[['month_sin','month_cos','day_sin','day_cos']]","execution_count":null,"outputs":[]},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"cyclic_x_test.head(1).T","execution_count":null,"outputs":[]},{"metadata":{"heading_collapsed":true},"cell_type":"markdown","source":"## Ordinal encoder"},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"from sklearn.preprocessing import OrdinalEncoder\n\noe = OrdinalEncoder()\noe_x_train = oe.fit_transform(x_train[oe_features])\noe_x_test = oe.transform(x_test[oe_features])","execution_count":null,"outputs":[]},{"metadata":{"heading_collapsed":true},"cell_type":"markdown","source":"## Target encoder"},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"from category_encoders import TargetEncoder\nfrom sklearn.model_selection import StratifiedKFold\n\n# 很高级的一种做法，做 Target Encoding 的时候拆分开来做\n# oof 的含义是 out of fold\ndef transform(transformer, x_train, y_train, cv):\n    oof = pd.DataFrame(index=x_train.index, columns=x_train.columns)\n    for train_idx, valid_idx in cv.split(x_train, y_train):\n        x_train_train = x_train.loc[train_idx]\n        y_train_train = y_train.loc[train_idx]\n        x_train_valid = x_train.loc[valid_idx]\n        transformer.fit(x_train_train, y_train_train)\n        oof_part = transformer.transform(x_train_valid)\n        oof.loc[valid_idx] = oof_part\n    return oof\n\ntarget = TargetEncoder(drop_invariant=True, smoothing=0.2)\n\ncv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)\n# shuffle：在每次划分时，是否进行洗牌\n# ①若为Falses时，其效果等同于random_state等于整数，每次划分的结果相同\n# ②若为True时，每次划分的结果都不一样，表示经过洗牌，随机取样的\n# target_x_train 用的是 oof 技术计算的\ntarget_x_train = transform(target, x_train[target_features], y_train, cv).astype('float')\n# target_x_test 用的是 train 的全量target去计算的\ntarget.fit(x_train[target_features], y_train)\n# 用 fit 完成后的对象，去transfer x_test\ntarget_x_test = target.transform(x_test[target_features]).astype('float')\n\n#生成的是 DF 格式\n# type(target_x_test)","execution_count":null,"outputs":[]},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"target.get_params()","execution_count":null,"outputs":[]},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"#生成的是 DF\ntype(target_x_test)","execution_count":null,"outputs":[]},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"#生成的是 DF\ntype(target_x_train)","execution_count":null,"outputs":[]},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"target_x_test.head(1).T","execution_count":null,"outputs":[]},{"metadata":{"heading_collapsed":true},"cell_type":"markdown","source":"## Merge for Logit"},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"import scipy\nx_train = scipy.sparse.hstack([ohe_x_train, simple_x_train]).tocsr()\nx_test = scipy.sparse.hstack([ohe_x_test, simple_x_test]).tocsr()","execution_count":null,"outputs":[]},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"x_train.shape","execution_count":null,"outputs":[]},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"type(x_train)","execution_count":null,"outputs":[]},{"metadata":{"hidden":true,"trusted":true},"cell_type":"code","source":"type(ohe_x_train)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Logistic regression"},{"metadata":{"trusted":true},"cell_type":"code","source":"# 对 logit 做网格搜索\n\nfrom sklearn.linear_model import LogisticRegression\n\nlogit_param_grid = {\n    'C': list(np.linspace(start = 0, stop = 0.1, num = 11))\n}\n\nlogit_grid = GridSearchCV(LogisticRegression(solver='lbfgs'), logit_param_grid,\n                          scoring='roc_auc', cv=5)\nlogit_grid.fit(x_train, y_train)\n\nbest_C = logit_grid.best_params_['C']\nbest_Score = logit_grid.best_score_\nprint('Best C:', best_C)\nprint('Best Score:', best_Score)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# 用 Best C predict x_test\nlogit = LogisticRegression(\n    C=best_C, \n    solver='lbfgs', \n    max_iter=10000)\nlogit.fit(x_train, y_train)\ny_pred_logit = logit.predict_proba(x_test)[:, 1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# 看在 Train Set 上的AUC结果如何\ny_train_pred_logit = logit.predict_proba(x_train)[:, 1]\ntrain_auc_logit = roc_auc_score(y_train, y_train_pred_logit)\ntrain_auc_logit","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Merge for XGB"},{"metadata":{"trusted":true},"cell_type":"code","source":"x_train = np.concatenate((oe_x_train, simple_x_train, target_x_train, cyclic_x_train), axis=1)\nx_test = np.concatenate((oe_x_test, simple_x_test, target_x_test, cyclic_x_test), axis=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## XGB classifier"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.model_selection import KFold\nfrom sklearn.model_selection import StratifiedKFold\nfrom sklearn.metrics import roc_auc_score\nfrom xgboost import plot_importance\nfrom sklearn.metrics import make_scorer\n\nimport xgboost as xgb\n\n## Hyperopt modules\nfrom hyperopt import fmin, hp, tpe, Trials, space_eval, STATUS_OK, STATUS_RUNNING\nfrom functools import partial\nimport gc\n\nimport time\ndef objective(params):\n    time1 = time.time()\n    params = {\n        'max_depth': int(params['max_depth']),\n        'gamma': \"{:.3f}\".format(params['gamma']),\n        'subsample': \"{:.2f}\".format(params['subsample']),\n        'reg_alpha': \"{:.3f}\".format(params['reg_alpha']),\n        'reg_lambda': \"{:.3f}\".format(params['reg_lambda']),\n        'learning_rate': \"{:.3f}\".format(params['learning_rate']),\n        'num_leaves': '{:.3f}'.format(params['num_leaves']),\n        'colsample_bytree': '{:.3f}'.format(params['colsample_bytree']),\n        'min_child_samples': '{:.3f}'.format(params['min_child_samples']),\n        'feature_fraction': '{:.3f}'.format(params['feature_fraction']),\n        'bagging_fraction': '{:.3f}'.format(params['bagging_fraction'])\n    }\n\n    print(\"\\n############## New Run ################\")\n    print(f\"params = {params}\")\n    FOLDS = 12\n    count=1\n    kf = StratifiedKFold(n_splits=FOLDS, shuffle=False, random_state=42)\n\n    # tss = TimeSeriesSplit(n_splits=FOLDS)\n#     y_preds = np.zeros(submission.shape[0]) #这句看起来没有用\n    # y_oof = np.zeros(X_train.shape[0])\n    score_mean = 0 #初始化 mean\n    for tr_idx, val_idx in kf.split(x_train, y_train):\n        clf = xgb.XGBClassifier(\n            n_estimators=1000, random_state=4, \n            verbose=True, \n            \n            tree_method='gpu_hist', # GPU加速\n            **params #这个用法需要注意\n        )\n\n        X_tr, X_vl = x_train[tr_idx, :], x_train[val_idx, :] # 需要根据 x_train 的类型来判断是否用 iloc\n        y_tr, y_vl = y_train.iloc[tr_idx], y_train.iloc[val_idx]\n        \n        clf.fit(X_tr, y_tr) #Training Set内部的K Fold # 用fit方法没有早停\n        \n        #y_pred_train = clf.predict_proba(X_vl)[:,1]\n        #print(y_pred_train)\n        \n        score = make_scorer(roc_auc_score, needs_proba=True)(clf, X_vl, y_vl) # 自定义一个score，别的项目需要替换掉这个\n        # plt.show()\n        score_mean += score # 先把这几次Fold的mean加起来\n        print(f'{count} CV - score: {round(score, 4)}')\n        count += 1 # count 单纯是为了用来计数 表示是第几次cv\n    time2 = time.time() - time1\n    print(f\"Total Time Run: {round(time2 / 60,2)}\")\n    gc.collect() #内存回收机制，检查是否有内存泄漏\n    print(f'Mean ROC_AUC: {score_mean / FOLDS}') #得到最终的mean\n    del X_tr, X_vl, y_tr, y_vl, clf, score\n    \n    return -(score_mean / FOLDS) #这里是由于要取损失函数的最小值，因为score前面要有负号\n\nspace = {\n    # The maximum depth of a tree, same as GBM.\n    # Used to control over-fitting as higher depth will allow model \n    # to learn relations very specific to a particular sample.\n    # Should be tuned using CV.\n    # Typical values: 3-10\n    'max_depth': hp.quniform('max_depth', 2, 8, 1),\n    \n    # reg_alpha: L1 regularization term. L1 regularization encourages sparsity \n    # (meaning pulling weights to 0). It can be more useful when the objective\n    # is logistic regression since you might need help with feature selection.\n    'reg_alpha':  hp.uniform('reg_alpha', 0.01, 0.4),\n    \n    # reg_lambda: L2 regularization term. L2 encourages smaller weights, this\n    # approach can be more useful in tree-models where zeroing \n    # features might not make much sense.\n    'reg_lambda': hp.uniform('reg_lambda', 0.01, .4),\n    \n    # eta: Analogous to learning rate in GBM\n    # Makes the model more robust by shrinking the weights on each step\n    # Typical final values to be used: 0.01-0.2\n    'learning_rate': hp.uniform('learning_rate', 0.01, 0.15),\n    \n    # colsample_bytree: Similar to max_features in GBM. Denotes the \n    # fraction of columns to be randomly samples for each tree.\n    # Typical values: 0.5-1\n    'colsample_bytree': hp.uniform('colsample_bytree', 0.3, 1),\n    \n    # A node is split only when the resulting split gives a positive\n    # reduction in the loss function. Gamma specifies the \n    # minimum loss reduction required to make a split.\n    # Makes the algorithm conservative. The values can vary depending on the loss function and should be tuned.\n    'gamma': hp.uniform('gamma', 0.01, .7),\n    \n    # more increases accuracy, but may lead to overfitting.\n    # num_leaves: the number of leaf nodes to use. Having a large number \n    # of leaves will improve accuracy, but will also lead to overfitting.\n    'num_leaves': hp.choice('num_leaves', list(range(20, 200, 5))),\n    \n    # specifies the minimum samples per leaf node.\n    # the minimum number of samples (data) to group into a leaf. \n    # The parameter can greatly assist with overfitting: larger sample\n    # sizes per leaf will reduce overfitting (but may lead to under-fitting).\n    'min_child_samples': hp.choice('min_child_samples', list(range(100, 250, 10))),\n    \n    # subsample: represents a fraction of the rows (observations) to be \n    # considered when building each subtree. Tianqi Chen and Carlos Guestrin\n    # in their paper A Scalable Tree Boosting System recommend \n    'subsample': hp.choice('subsample', [.5, 0.6, 0.7, .8]),\n    \n    # randomly select a fraction of the features.\n    # feature_fraction: controls the subsampling of features used\n    # for training (as opposed to subsampling the actual training data in \n    # the case of bagging). Smaller fractions reduce overfitting.\n    'feature_fraction': hp.uniform('feature_fraction', 0.4, .8),\n    \n    # randomly bag or subsample training data.\n    'bagging_fraction': hp.uniform('bagging_fraction', 0.4, .9)\n    \n    # bagging_fraction and bagging_freq: enables bagging (subsampling) \n    # of the training data. Both values need to be set for bagging to be used.\n    # The frequency controls how often (iteration) bagging is used. Smaller\n    # fractions and frequencies reduce overfitting.\n}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"best = fmin(fn=objective,\n            space=space,\n            algo=tpe.suggest,\n            max_evals=50, \n            # trials=trials #trials 是为了后来绘图所用，详细课件 hyperopt 教程\n           )\n# 整个 fmin 之后返回的是一个参数空间","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"best_params = space_eval(space, best) #可能是由于索引问题，公式得到的best，需要重新加工一下才是最终的 best_params\nbest_params['max_depth'] = int(best_params['max_depth'])\nbest_params","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"clf = xgb.XGBClassifier(\n    n_estimators=500,\n    **best_params,\n    tree_method='gpu_hist'\n)\n\nclf.fit(x_train, y_train)\n\ny_pred_xgb = clf.predict_proba(x_test)[:,1] ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# 看在 Train Set 上的AUC结果如何\ny_train_pred_xgb = clf.predict_proba(x_train)[:, 1]\ntrain_auc_xgb = roc_auc_score(y_train, y_train_pred_xgb)\ntrain_auc_xgb","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Submit predictions"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Blending\ny_pred = np.add(y_pred_logit, y_pred_xgb) / 2","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission = pd.read_csv('../input/cat-in-the-dat-ii/sample_submission.csv', index_col='id')\nsubmission['target'] = y_pred\nsubmission.to_csv('logit_xgboost.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.head()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.3"}},"nbformat":4,"nbformat_minor":4}