{"cells":[{"metadata":{},"cell_type":"markdown","source":"## 1. Load the main data sets"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"#import libraries\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# load the data\ntrain = pd.read_csv('../input/train.csv')\ntest = pd.read_csv('../input/test.csv')\nsample_submission = pd.read_csv('../input/sample_submission.csv')\n\n# get the shape of the train and test sets\ntrain_shape = train.shape\ntest_shape = test.shape\n\nprint('Train shape: {}'.format(train_shape))\nprint('Test shape: {}'.format(test_shape))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2. Load the additional sets"},{"metadata":{},"cell_type":"markdown","source":"Additional Data\nNOTE: additional data is provided for the molecules in Train only!\n\n**dipole_moments.csv** - contains the molecular electric dipole moments. These are three dimensional vectors that indicate the charge distribution in the molecule. The first column (molecule_name) are the names of the molecule, the second to fourth column are the X, Y and Z components respectively of the dipole moment.\n\n**magnetic_shielding_tensors.csv** - contains the magnetic shielding tensors for all atoms in the molecules. The first column (molecule_name) contains the molecule name, the second column (atom_index) contains the index of the atom in the molecule, the third to eleventh columns contain the XX, YX, ZX, XY, YY, ZY, XZ, YZ and ZZ elements of the tensor/matrix respectively.\n\n**mulliken_charges.csv** - contains the mulliken charges for all atoms in the molecules. The first column (molecule_name) contains the name of the molecule, the second column (atom_index) contains the index of the atom in the molecule, the third column (mulliken_charge) contains the mulliken charge of the atom.\n\n**potential_energy.csv** - contains the potential energy of the molecules. The first column (molecule_name) contains the name of the molecule, the second column (potential_energy) contains the potential energy of the molecule.\n\n**scalar_coupling_contributions.csv** - The scalar coupling constants in train.csv (or corresponding files) are a sum of four terms.\n\n**scalar_coupling_contributions.csv** contain all these terms. The first column (molecule_name) are the name of the molecule, the second (atom_index_0) and third column (atom_index_1) are the atom indices of the atom-pair, the fourth column indicates the type of coupling, the fifth column (fc) is the Fermi Contact contribution, the sixth column (sd) is the Spin-dipolar contribution, the seventh column (pso) is the Paramagnetic spin-orbit contribution and the eighth column (dso) is the Diamagnetic spin-orbit contribution."},{"metadata":{"trusted":true},"cell_type":"code","source":"structures = pd.read_csv('../input/structures.csv')\ndipole_moments = pd.read_csv('../input/dipole_moments.csv')\nmagnetic_shielding_tensors = pd.read_csv('../input/magnetic_shielding_tensors.csv')\nmulliken_charges = pd.read_csv('../input/mulliken_charges.csv')\npotential_energy = pd.read_csv('../input/potential_energy.csv')\nscalar_coupling_contributions = pd.read_csv('../input/scalar_coupling_contributions.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dipole_moments.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"magnetic_shielding_tensors.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"mulliken_charges.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"potential_energy.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"scalar_coupling_contributions.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 3. Investigate the train and test sets - EDA"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train['type'].unique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# plot the categorical features - type\ntrain.groupby(\"type\").id.count().sort_values(ascending=False)[:10].plot.bar()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# plot the distribution of the median of target variable by type\nimport matplotlib.pyplot as plt\nplt.style.use('ggplot')\n\nmolecule_type = train.groupby('type',as_index=False)['scalar_coupling_constant'].median()\nfig = plt.figure(figsize=(7,5))\nplt.bar(molecule_type.type,molecule_type.scalar_coupling_constant,width=0.5,alpha=0.8)\n\nplt.xlabel('scalar_coupling_constant')\nplt.ylabel('Median Scalar Constant')\nplt.title('Median of scalar constant across type')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 4. Identify the problem type"},{"metadata":{"trusted":true},"cell_type":"code","source":"import seaborn as sns\nsns.distplot(train['scalar_coupling_constant'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#skewness and kurtosis of the target variable\nprint(\"Skewness: \", train['scalar_coupling_constant'].skew())\nprint(\"Kurtosis: \", train['scalar_coupling_constant'].kurt())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.scalar_coupling_constant.unique()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":" Target is a continuous variable, therefore this is a regression problem. "},{"metadata":{},"cell_type":"markdown","source":"## 5. Feature engineering"},{"metadata":{"trusted":true},"cell_type":"code","source":"# re-load the train and test sets in their raw state\n\n# train = pd.read_csv('../input/train.csv')\n# test = pd.read_csv('../input/test.csv')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5.1. concatenate test and train sets"},{"metadata":{"trusted":true},"cell_type":"code","source":"df_all = pd.concat([train,test],sort=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5.2. create new features using the type variable"},{"metadata":{"trusted":true},"cell_type":"code","source":"# create new features using for joining main data to the structures data\n\ndf_all['jcoupling_type'] = df_all['type'].str.extract(\"(\\d)J\\w\") \ndf_all['jcoupling_atoms'] = df_all['type'].str.extract(\"\\dJ(\\w*)\") \ndf_all['jcoupling_atom_0'] = df_all.type.str[2]\ndf_all['jcoupling_atom_1'] = df_all.type.str[3]\ndf_all['coupling_atom_type'] = df_all['type'].str.extract(\"\\dJ(\\w*)\") \ndf_all['jcoupling_atom_0'] = df_all['atom_index_0'].astype(str) + '_' + df_all['jcoupling_atom_0']\ndf_all['jcoupling_atom_1'] = df_all['atom_index_1'].astype(str) + '_' + df_all['jcoupling_atom_1']\ndf_all['jcoupling_atoms'] = df_all['jcoupling_atom_0'].astype(str) + '_' + df_all['jcoupling_atom_1']\n\nstructures['jcoupling_atoms'] = structures['atom_index'].astype(str) + '_' + structures['atom']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_all.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_all.shape # 7,163,689","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5.3. Left join the concatenated data to the structures dataframe to get xyz values for the atom index_0 "},{"metadata":{"trusted":true},"cell_type":"code","source":"# join the atom structure xyz to the molecule data\n\ndf_new = pd.merge(df_all,structures[['molecule_name','jcoupling_atoms','x','y','z']],left_on=['molecule_name','jcoupling_atom_0'],right_on=['molecule_name','jcoupling_atoms'],how='left')\ndf_new.drop(columns='jcoupling_atoms_y',inplace=True)\ndf_new.rename(columns={\"x\": \"x_atom_0\", \"y\": \"y_atom_0\", \"z\": \"z_atom_0\",\"jcoupling_atoms_x\": \"jcoupling_atoms\"},inplace=True)\ndf_new.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5.4. Left join the concatenated data to the structures dataframe to get xyz values for the atom index_1"},{"metadata":{"trusted":true},"cell_type":"code","source":"df_new = pd.merge(df_new,structures[['molecule_name','jcoupling_atoms','x','y','z']],left_on=['molecule_name','jcoupling_atom_1'],right_on=['molecule_name','jcoupling_atoms'],how='left')\ndf_new.drop(columns='jcoupling_atoms_y',inplace=True)\ndf_new.rename(columns={\"x\": \"x_atom_1\", \"y\": \"y_atom_1\", \"z\": \"z_atom_1\",\"jcoupling_atoms_x\": \"jcoupling_atoms\"},inplace=True)\ndf_new.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5.5. calculate the distance between the coupling atoms "},{"metadata":{"trusted":true},"cell_type":"code","source":"# calculate distance between jcoupling atoms in the molecule structure\n\ndf_new['dist'] = np.sqrt( (df_new.x_atom_0-df_new.x_atom_1)**2 + (df_new.y_atom_0-df_new.y_atom_1)**2 + (df_new.z_atom_0-df_new.z_atom_1)**2)\ndf_new.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_new.shape #7163689","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5.6. select the relevant features for the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"final_features = ['id','molecule_name','atom_index_0','atom_index_1','type','jcoupling_type','coupling_atom_type','dist','scalar_coupling_constant']\nmolecule_atoms = df_new[final_features]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"molecule_atoms.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5.7. encode the type variable"},{"metadata":{"trusted":true},"cell_type":"code","source":"# encode categorical features\n\nfrom sklearn.preprocessing import LabelEncoder\n\nohe = pd.get_dummies(molecule_atoms['type'],prefix='ohe')\n# molecule_atoms.drop('type',axis=1,inplace=True)\nmolecule_atoms = pd.concat([molecule_atoms,ohe],axis=1)\nmolecule_atoms.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5.8. split the dataframe into the train and test sets"},{"metadata":{"trusted":true},"cell_type":"code","source":"train = molecule_atoms[molecule_atoms.id.isin(train.id)]\ntest = molecule_atoms[molecule_atoms.id.isin(test.id)]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# drop the target variable from the test set\ntest = test.drop(columns='scalar_coupling_constant')\ntest.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.columns","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5.9. use the features in the additional files and predicate the correspondance values in the test set"},{"metadata":{"trusted":true},"cell_type":"code","source":"# left join the scalar coupling contribution dataframe to the train set\ntrain_new = pd.merge(train,scalar_coupling_contributions,left_on=['molecule_name','atom_index_0','atom_index_1','type'],right_on=['molecule_name','atom_index_0','atom_index_1','type'],how='left')\ntrain_new.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# left join the mulliken_charges and potential_energy dataframes to the train set\ntrain_new = pd.merge(train_new,potential_energy,left_on=['molecule_name'],right_on=['molecule_name'],how='left')\ntrain_new.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_new.columns # 4,658,147","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**features to be predicted in the test set:**\n\n* fc\n* sd\n* so\n* dso\n* potential_energy"},{"metadata":{},"cell_type":"markdown","source":"### 5.10. Build regression models to predict additional features in the test set"},{"metadata":{"trusted":true},"cell_type":"code","source":"# numerical features except the scalar_coupling_constant,sd, pso, dso, potential_energy - one additional feature at a time to predict\nnum_features = ['atom_index_0', 'atom_index_1', 'jcoupling_type', 'dist',\n       'ohe_1JHC', 'ohe_1JHN', 'ohe_2JHC','ohe_2JHH', 'ohe_2JHN', 'ohe_3JHC', 'ohe_3JHH', 'ohe_3JHN']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# scale the numerical features\nfrom sklearn.preprocessing import StandardScaler\n\ntest_new = test.copy()\nsc = StandardScaler()\ntrain_new[num_features] = sc.fit_transform(train_new[num_features])\ntest_new[num_features] = sc.transform(test_new[num_features])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_new.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# predict fc in the test\nimport xgboost as xgb\nfrom sklearn.model_selection import KFold\nfrom sklearn.metrics import mean_absolute_error\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.ensemble import GradientBoostingRegressor,RandomForestRegressor\nfrom sklearn.tree import DecisionTreeRegressor\n\n\n\n# kf = KFold(n_splits=5,shuffle=True,random_state=123)\n\n# fold = 0\n# fold_metrics = []\n# for train_index, test_index in kf.split(train_new):\n#     cv_train, cv_test = train_new.iloc[train_index], train_new.iloc[test_index]\n#     regressor = GradientBoostingRegressor()\n#     regressor.fit(X=cv_train[num_features],y=cv_train['fc'])\n#     predictions = regressor.predict(cv_test[num_features])\n#     metric = np.log(mean_absolute_error(cv_test['fc'],predictions))\n#     fold_metrics.append(metric)\n#     print('Fold:{}'.format(fold))\n#     print('CV train shape:{}'.format(cv_train.shape))\n#     print('Log mean squared error:{}'.format(metric))\n#     fold+=1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Average and overall metric\n\n# mean_score = np.mean(fold_metrics)\n# overal_score_minimizing = np.mean(fold_metrics)+np.std(fold_metrics)\n# print(mean_score,overal_score_minimizing)\n\n# lr ---> 1.4350236728166157 1.4368374949231812\n# gb ---> 0.978808394321879 0.9800942928921375\n# rf ---> 1.0059670776324137 1.0071424059922574\n# dt ---> 1.1635835592319883 1.1646638382205583\n# xgb --> 0.9794824442750851 0.9803075149244272","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# predict the fc feature\nfrom sklearn.ensemble import GradientBoostingRegressor\n\nregressor = GradientBoostingRegressor()\nregressor.fit(X=train_new[num_features],y=train_new['fc'])\ntest_new['fc'] = regressor.predict(test_new[num_features])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# test_new.head()\ntrain = train_new.copy()\ntest = test_new.copy()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5.11. Hyperparameter tunning"},{"metadata":{"trusted":true},"cell_type":"code","source":"# from sklearn.model_selection import GridSearchCV\n\n# param_grid = {\n#     'max_depth' : [3, 5, 7],\n#     'subsample' : [0.8, 0.9, 1.0]\n# }\n\n# regressor = GradientBoostingRegressor()\n# grid_search = GridSearchCV(estimator = regressor, param_grid = param_grid, \n#                           cv = 3,  verbose = 2)\n# grid_search.fit(train_new[num_features], train_new['fc'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 6. Build models"},{"metadata":{"trusted":true},"cell_type":"code","source":"features = ['atom_index_0', 'atom_index_1', 'dist','fc', 'ohe_1JHC', 'ohe_1JHN', 'ohe_2JHC',\n       'ohe_2JHH', 'ohe_2JHN', 'ohe_3JHC', 'ohe_3JHH', 'ohe_3JHN']\n# , 'jcoupling_type'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Feature Scaling\nfrom sklearn.preprocessing import StandardScaler\n\nsc = StandardScaler()\ntrain[features] = sc.fit_transform(train[features])\ntest[features] = sc.transform(test[features])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train[features].head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 6.1. Linear Regression"},{"metadata":{"trusted":true},"cell_type":"code","source":"# from sklearn.linear_model import LinearRegression\n\n# # fit the model on the train set\n# lr = LinearRegression()\n# lr.fit(X=train[features],y=train['scalar_coupling_constant'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 6.2. Local validation"},{"metadata":{"trusted":true},"cell_type":"code","source":"# kfold cross-validation for evaluating the model\nfrom sklearn.model_selection import KFold\nfrom sklearn.metrics import mean_absolute_error\n\nkf = KFold(n_splits=5,shuffle=True,random_state=123)\n\n# fold = 0\n# fold_metrics = []\n# for train_index, test_index in kf.split(train):\n#     cv_train, cv_test = train.iloc[train_index], train.iloc[test_index]\n#     lr.fit(X=cv_train[features],y=cv_train['scalar_coupling_constant'])\n#     predictions = lr.predict(cv_test[features])\n#     metric = np.log(mean_absolute_error(cv_test['scalar_coupling_constant'],predictions))\n#     fold_metrics.append(metric)\n#     print('Fold:{}'.format(fold))\n#     print('CV train shape:{}'.format(cv_train.shape))\n#     print('Log mean squared error:{}'.format(metric))\n#     fold+=1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Average and overall metric\n\n# mean_score = np.mean(fold_metrics)\n# overal_score_minimizing = np.mean(fold_metrics)+np.std(fold_metrics)\n# print(mean_score,overal_score_minimizing)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# test['scalar_coupling_constant'] = lr.predict(test[features])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# submission_2 = test[['id','scalar_coupling_constant']]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# submission_2.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# submission_2.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# submission_2.to_csv('submission_v2.csv',index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 6.3. RandomForest Regressor"},{"metadata":{"trusted":true},"cell_type":"code","source":"# from sklearn.ensemble import RandomForestRegressor\n\n# rf = RandomForestRegressor()\n\n# fold = 0\n# fold_metrics = []\n# for train_index, test_index in kf.split(train):\n#     cv_train, cv_test = train.iloc[train_index], train.iloc[test_index]\n#     rf.fit(X=cv_train[features],y=cv_train['scalar_coupling_constant'])\n#     predictions = rf.predict(cv_test[features])\n#     metric = np.log(mean_absolute_error(cv_test['scalar_coupling_constant'],predictions))\n#     fold_metrics.append(metric)\n#     print('Fold:{}'.format(fold))\n#     print('CV train shape:{}'.format(cv_train.shape))\n#     print('Log mean squared error:{}'.format(metric))\n#     fold+=1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# mean_score = np.mean(fold_metrics)\n# overal_score_minimizing = np.mean(fold_metrics)+np.std(fold_metrics)\n# print(mean_score,overal_score_minimizing)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.ensemble import GradientBoostingRegressor\n\n# regressor = GradientBoostingRegressor()\n\n# fold = 0\n# fold_metrics = []\n# for train_index, test_index in kf.split(train):\n#     cv_train, cv_test = train.iloc[train_index], train.iloc[test_index]\n#     regressor.fit(X=cv_train[features],y=cv_train['scalar_coupling_constant'])\n#     predictions = regressor.predict(cv_test[features])\n#     metric = np.log(mean_absolute_error(cv_test['scalar_coupling_constant'],predictions))\n#     fold_metrics.append(metric)\n#     print('Fold:{}'.format(fold))\n#     print('CV train shape:{}'.format(cv_train.shape))\n#     print('Log mean squared error:{}'.format(metric))\n#     fold+=1\n\"\"\"\nFold:0\nCV train shape:(3726517, 22)\nLog mean squared error:-1.651942379074779\nFold:1\nCV train shape:(3726517, 22)\nLog mean squared error:-1.6581039302273266\nFold:2\nCV train shape:(3726518, 22)\nLog mean squared error:-1.64427536075409\nFold:3\nCV train shape:(3726518, 22)\nLog mean squared error:-1.6546618867855556\nFold:4\nCV train shape:(3726518, 22)\nLog mean squared error:-1.6543852532228525\n\"\"\"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"mean_score = np.mean(fold_metrics)\noveral_score_minimizing = np.mean(fold_metrics) + np.std(fold_metrics)\nprint(mean_score,overal_score_minimizing) # -1.6526737620129208 -1.648038319556876","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.ensemble import GradientBoostingRegressor\n\ngb = GradientBoostingRegressor()\ngb.fit(X= train[features],y=train['scalar_coupling_constant'])\ntest['scalar_coupling_constant'] = gb.predict(test[features])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_5 = test[['id','scalar_coupling_constant']]\nsubmission_5.to_csv('submission_v5.csv',index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_4.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.head()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}