{"cells":[{"metadata":{"_uuid":"051d70d956493feee0c6d64651c6a088724dca2a","_execution_state":"idle","trusted":false},"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/EdsonAvelar/auc/master/benner.png)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Exploratory Data Analysis, Models and Starter code in R\nThis kernel is aimed to provide some analysis, visualizations and basic models to get started using R. Feel free to vote, fork, copy and edit this kernel for your own work.   \n\nIf you are only interested in the models, you can directly skip to the **Feature Engineering** and **Models** section.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Setup\n* Loading all required libraries\n* Setting a seed. It is important to ensure reproducibility and a good habit too.\n* Listing and reading data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"## loading libraries\nrequire(data.table)\nrequire(ggplot2)\nrequire(gridExtra)\nrequire(Metrics)\nrequire(xgboost)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"## setting seed\nmyfavouritenumber <- 13\nseed <- myfavouritenumber\nset.seed(seed)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"## listing datasets\nlist.files(\"../input\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"## reading data\ntrain <- fread(\"../input/train.csv\")\ntest <- fread(\"../input/test.csv\")\n\ndipole <- fread(\"../input/dipole_moments.csv\")\ntensors <- fread(\"../input/magnetic_shielding_tensors.csv\")\nmulliken <- fread(\"../input/mulliken_charges.csv\")\nenergy <- fread(\"../input/potential_energy.csv\")\ncoupling <- fread(\"../input/scalar_coupling_contributions.csv\")\nstructures <- fread(\"../input/structures.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Train and Test data\nSummarizing the main *train.csv* and *test.csv* data.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# train\ncat(\"Rows in train:\", nrow(train), \"\\nCols in train:\", ncol(train))\nhead(train)\nsummary(train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# test\ncat(\"Rows in test:\", nrow(test), \"\\nCols in test:\", ncol(test))\nhead(test)\nsummary(test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**scalar_coupling_constant** is the target variable to be predicted. Let's quickly look at it's distribution.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"ggplot(train, aes(x = scalar_coupling_constant)) +\n    geom_histogram(color = \"black\", fill = \"lightblue\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It has a huge peak around 0, with a normal-like distribution between -25 and +25. Then it has some values above +75 which gradually tapers down upto 150 and a long tail to 200.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Dipole Moments\nSummarizing the *dipole_moments.csv* data and merging with train. If you merge with test, you will get all NA values since this is provided only for the training data.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# dipole\ncat(\"Rows in dipole:\", nrow(dipole), \"\\nCols in dipole:\", ncol(dipole))\nhead(dipole)\nsummary(dipole)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This is at a molecule level and needs to be merged on *molecule_name*. Adding 'dipole' to the column names so that it is easier to remember.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"setnames(dipole, names(dipole)[-1], paste0(\"dipole_\", tolower(names(dipole)[-1])))\n\ntrain <- merge(train, dipole, all.x = T, by = \"molecule_name\")\n\nhead(train)\nhead(test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Magnetic Shielding Tensors\nSummarizing the *magnetic_shielding_tensors.csv* data and merging with train. If you merge with test, you will get all NA values since this is provided only for the training data.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# tensors\ncat(\"Rows in tensors:\", nrow(tensors), \"\\nCols in tensors:\", ncol(tensors))\nhead(tensors)\nsummary(tensors)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This is at a molecule and atom index level and needs to be merged on *molecule_name* and *atom_index*. Since there are two indices of atom index in the train/test data, creating two datasets for merging convenience. Adding 'tensor' to the column names so that it is easier to remember.\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# preparing two copies of tensors so that it can be merged with each atom index\ntensors_0 <- copy(tensors)\ntensors_1 <- copy(tensors)\n\nsetnames(tensors_0, names(tensors_0)[-1], paste0(tolower(names(tensors_0)[-1]), \"_0\"))\nsetnames(tensors_1, names(tensors_1)[-1], paste0(tolower(names(tensors_1)[-1]), \"_1\"))\n\n# adding 'tensor' to column names to identify feature\nsetnames(tensors_0, names(tensors_0)[-c(1:2)], paste0(\"tensor_\", names(tensors_0)[-c(1:2)]))\nsetnames(tensors_1, names(tensors_1)[-c(1:2)], paste0(\"tensor_\", names(tensors_1)[-c(1:2)]))\n\ntrain <- merge(train, tensors_0, all.x = T, by = c(\"molecule_name\", \"atom_index_0\"))\ntrain <- merge(train, tensors_1, all.x = T, by = c(\"molecule_name\", \"atom_index_1\"))\n\nhead(train)\nhead(test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Mulliken Charges\nSummarizing the *mulliken_charges.csv* data and merging with train. If you merge with test, you will get all NA values since this is provided only for the training data.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# mulliken\ncat(\"Rows in mulliken:\", nrow(mulliken), \"\\nCols in mulliken:\", ncol(mulliken))\nhead(mulliken)\nsummary(mulliken)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This is at a molecule and atom index level and needs to be merged on *molecule_name* and *atom_index*. Since there are two indices of atom index in the train/test data, creating two datasets for merging convenience.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# preparing two copies of mulliken so that it can be merged with each atom index\nmulliken_0 <- copy(mulliken)\nmulliken_1 <- copy(mulliken)\n\nsetnames(mulliken_0, names(mulliken_0)[-1], paste0(names(mulliken_0)[-1], \"_0\"))\nsetnames(mulliken_1, names(mulliken_1)[-1], paste0(names(mulliken_1)[-1], \"_1\"))\n\ntrain <- merge(train, mulliken_0, all.x = T, by = c(\"molecule_name\", \"atom_index_0\"))\ntrain <- merge(train, mulliken_1, all.x = T, by = c(\"molecule_name\", \"atom_index_1\"))\n\nhead(train)\nhead(test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Potential Energy\nSummarizing the *potential_energy.csv* data and merging with train. If you merge with test, you will get all NA values since this is provided only for the training data.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# energy\ncat(\"Rows in energy:\", nrow(energy), \"\\nCols in energy:\", ncol(energy))\nhead(energy)\nsummary(energy)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This is at a molecule level and needs to be merged on *molecule_name*.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train <- merge(train, energy, all.x = T, by = \"molecule_name\")\n\nhead(train)\nhead(test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Scalar Coupling Contributions\nSummarizing the *scalar_coupling_contributions.csv* data and merging with train. If you merge with test, you will get all NA values since this is provided only for the training data.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# coupling\ncat(\"Rows in coupling:\", nrow(coupling), \"\\nCols in coupling:\", ncol(coupling))\nhead(coupling)\nsummary(coupling)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This is at a molecule, atom indexes and type level and needs to be merged on *molecule_name*, *atom_index_0*, *atom_index_1* and *type*.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train <- merge(train, coupling, all.x = T, by = c(\"molecule_name\", \"atom_index_0\", \"atom_index_1\", \"type\"))\n\nhead(train)\nhead(test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Structures\nSummarizing the *structures.csv* data and merging with train/test. This is the only dataset which can be merged directly with test as the data consists of train and test observations, both.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# structures\ncat(\"Rows in structures:\", nrow(structures), \"\\nCols in structures:\", ncol(structures))\nhead(structures)\nsummary(structures)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This is at a molecule and atom index level and needs to be merged on *molecule_name* and *atom_index*. Since there are two indices of atom index in the train/test data, creating two datasets for merging convenience. Adding 'structure' to the column names so that it is easier to remember.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# preparing two copies of structures so that it can be merged with each atom index\nstructures_0 <- copy(structures)\nstructures_1 <- copy(structures)\n\nsetnames(structures_0, names(structures_0)[-1], paste0(names(structures_0)[-1], \"_0\"))\nsetnames(structures_1, names(structures_1)[-1], paste0(names(structures_1)[-1], \"_1\"))\n\n# adding 'structure' to column names to identify feature\nsetnames(structures_0, names(structures_0)[-c(1:2)], paste0(\"structure_\", names(structures_0)[-c(1:2)]))\nsetnames(structures_1, names(structures_1)[-c(1:2)], paste0(\"structure_\", names(structures_1)[-c(1:2)]))\n\ntrain <- merge(train, structures_0, all.x = T, by = c(\"molecule_name\", \"atom_index_0\"))\ntrain <- merge(train, structures_1, all.x = T, by = c(\"molecule_name\", \"atom_index_1\"))\ntest <- merge(test, structures_0, all.x = T, by = c(\"molecule_name\", \"atom_index_0\"))\ntest <- merge(test, structures_1, all.x = T, by = c(\"molecule_name\", \"atom_index_1\"))\n\nhead(train)\nhead(test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Feature Distributions\nComparing distributions of raw features across train and test data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# creating combined data with train and test\ntrain[, data_flag := \"train\"]\ntest[, data_flag := \"test\"]\n\ntrain_test <- rbind(train, test, use.names = T, fill = T)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# atom indexes, structure atom 1 and type\np1 <- ggplot(train_test, aes(x = atom_index_0, fill = factor(data_flag))) +\n          geom_bar(stat = \"count\", position = \"dodge\") +\n          labs(x = \"atom_index_0\") +\n          theme(legend.position = \"top\")\n\np2 <- ggplot(train_test, aes(x = atom_index_1, fill = factor(data_flag))) +\n          geom_bar(stat = \"count\", position = \"dodge\") +\n          labs(x = \"atom_index_1\") +\n          theme(legend.position = \"top\")\n\np3 <- ggplot(train_test, aes(x = structure_atom_1, fill = factor(data_flag))) +\n          geom_bar(stat = \"count\", position = \"dodge\") +\n          labs(x = \"structure_atom_1\") +\n          theme(legend.position = \"top\")\n\np4 <- ggplot(train_test, aes(x = type, fill = factor(data_flag))) +\n          geom_bar(stat = \"count\", position = \"dodge\") +\n          labs(x = \"type\") +\n          theme(legend.position = \"top\")\n\ngrid.arrange(p1, p2, p3, p4, ncol=2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The distributions of atom indexes and type across train and test look very similar. That's good news!","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# structures x, y, z of atom index 0\np1 <- ggplot(train, aes(x = structure_x_0)) +\n        geom_histogram(color = \"black\", fill = \"blue\", alpha = 0.2) +\n        labs(x = \"structure_x_0 (train)\")\n\np2 <- ggplot(test, aes(x = structure_x_0)) +\n        geom_histogram(color = \"black\", fill = \"green\", alpha = 0.2) +\n        labs(x = \"structure_x_0 (test)\")\n\np3 <- ggplot(train, aes(x = structure_y_0)) +\n        geom_histogram(color = \"black\", fill = \"blue\", alpha = 0.2) +\n        labs(x = \"structure_y_0 (train)\")\n\np4 <- ggplot(test, aes(x = structure_y_0)) +\n        geom_histogram(color = \"black\", fill = \"green\", alpha = 0.2) +\n        labs(x = \"structure_y_0 (test)\")\n\np5 <- ggplot(train, aes(x = structure_z_0)) +\n        geom_histogram(color = \"black\", fill = \"blue\", alpha = 0.2) +\n        labs(x = \"structure_z_0 (train)\")\n\np6 <- ggplot(test, aes(x = structure_z_0)) +\n        geom_histogram(color = \"black\", fill = \"green\", alpha = 0.2) +\n        labs(x = \"structure_z_0 (test)\")\n\ngrid.arrange(p1, p2, p3, p4, p5, p6, ncol=2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# structures x, y, z of atom index 1\np1 <- ggplot(train, aes(x = structure_x_1)) +\n        geom_histogram(color = \"black\", fill = \"blue\", alpha = 0.2) +\n        labs(x = \"structure_x_1 (train)\")\n\np2 <- ggplot(test, aes(x = structure_x_1)) +\n        geom_histogram(color = \"black\", fill = \"green\", alpha = 0.2) +\n        labs(x = \"structure_x_1 (test)\")\n\np3 <- ggplot(train, aes(x = structure_y_1)) +\n        geom_histogram(color = \"black\", fill = \"blue\", alpha = 0.2) +\n        labs(x = \"structure_y_1 (train)\")\n\np4 <- ggplot(test, aes(x = structure_y_1)) +\n        geom_histogram(color = \"black\", fill = \"green\", alpha = 0.2) +\n        labs(x = \"structure_y_1 (test)\")\n\np5 <- ggplot(train, aes(x = structure_z_1)) +\n        geom_histogram(color = \"black\", fill = \"blue\", alpha = 0.2) +\n        labs(x = \"structure_z_1 (train)\")\n\np6 <- ggplot(test, aes(x = structure_z_1)) +\n        geom_histogram(color = \"black\", fill = \"green\", alpha = 0.2) +\n        labs(x = \"structure_z_1 (test)\")\n\ngrid.arrange(p1, p2, p3, p4, p5, p6, ncol=2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The distribution of the structures features across train and test look very similar. This should be comforting for generating new features on top of these raw features.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Feature Engineering\nEngineering new features over the existing set of raw features to help improve model performance.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# keeping only features available in test data\nrm(list = ls())\ndetach()\ngc()\n\nrequire(data.table)\nrequire(Metrics)\n\ntrain <- fread(\"../input/train.csv\")\ntest <- fread(\"../input/test.csv\")\n\nstructures_0 <- fread(\"../input/structures.csv\")\nstructures_1 <- copy(structures_0)\n\nsetnames(structures_0, names(structures_0)[-1], paste0(names(structures_0)[-1], \"_0\"))\nsetnames(structures_1, names(structures_1)[-1], paste0(names(structures_1)[-1], \"_1\"))\n\nsetnames(structures_0, names(structures_0)[-c(1:2)], paste0(\"structure_\", names(structures_0)[-c(1:2)]))\nsetnames(structures_1, names(structures_1)[-c(1:2)], paste0(\"structure_\", names(structures_1)[-c(1:2)]))\n\ntrain <- merge(train, structures_0, all.x = T, by = c(\"molecule_name\", \"atom_index_0\"))\ntrain <- merge(train, structures_1, all.x = T, by = c(\"molecule_name\", \"atom_index_1\"))\ntest <- merge(test, structures_0, all.x = T, by = c(\"molecule_name\", \"atom_index_0\"))\ntest <- merge(test, structures_1, all.x = T, by = c(\"molecule_name\", \"atom_index_1\"))\n\ntrain_test <- rbind(train, test, use.names = T, fill = T)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# difference and distance features\ntrain_test[, \":=\"(diff_structure_x = structure_x_0 - structure_x_1,\n                  diff_structure_y = structure_y_0 - structure_y_1,\n                  diff_structure_z = structure_z_0 - structure_z_1,\n                  dist_xyz = sqrt((structure_x_0 - structure_x_1) ^ 2 +\n                                  (structure_y_0 - structure_y_1) ^ 2 +\n                                  (structure_z_0 - structure_z_1) ^ 2)\n                 )]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# grouped features\ntrain_test <- merge(train_test, train_test[, .(count_molecule = .N,\n                                               min_molecule_dist_xyz = min(dist_xyz),\n                                               mean_molecule_dist_xyz = mean(dist_xyz),\n                                               median_molecule_dist_xyz = median(dist_xyz),\n                                               max_molecule_dist_xyz = max(dist_xyz),\n                                               sd_molecule_dist_xyz = sd(dist_xyz)\n                                              ), .(molecule_name)], by = \"molecule_name\")\n\ntrain_test <- merge(train_test, train_test[, .(count_atom_index_0 = .N,\n                                               min_molecule_atom_0_dist_xyz = min(dist_xyz),\n                                               mean_molecule_atom_0_dist_xyz = mean(dist_xyz),\n                                               median_molecule_atom_0_dist_xyz = median(dist_xyz),\n                                               max_molecule_atom_0_dist_xyz = max(dist_xyz),\n                                               sd_molecule_atom_0_dist_xyz = sd(dist_xyz)\n                                              ), .(molecule_name, atom_index_0)], by = c(\"molecule_name\", \"atom_index_0\"))\n\ntrain_test <- merge(train_test, train_test[, .(count_atom_index_1 = .N,\n                                               min_molecule_atom_1_dist_xyz = min(dist_xyz),\n                                               mean_molecule_atom_1_dist_xyz = mean(dist_xyz),\n                                               median_molecule_atom_1_dist_xyz = median(dist_xyz),\n                                               max_molecule_atom_1_dist_xyz = max(dist_xyz),\n                                               sd_molecule_atom_1_dist_xyz = sd(dist_xyz)\n                                              ), .(molecule_name, atom_index_1)], by = c(\"molecule_name\", \"atom_index_1\"))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# difference features\ntrain_test[, \":=\"(diff_mean_molecule_dist_xyz = dist_xyz - mean_molecule_dist_xyz,\n                  diff_median_molecule_dist_xyz = dist_xyz - median_molecule_dist_xyz,\n                  diff_mean_molecule_atom_0_dist_xyz = dist_xyz - mean_molecule_atom_0_dist_xyz,\n                  diff_median_molecule_atom_0_dist_xyz = dist_xyz - median_molecule_atom_0_dist_xyz,\n                  diff_mean_molecule_atom_1_dist_xyz = dist_xyz - mean_molecule_atom_1_dist_xyz,\n                  diff_median_molecule_atom_1_dist_xyz = dist_xyz - median_molecule_atom_1_dist_xyz\n                 )]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# preparing final data for models\nfinal_train <- train_test[!is.na(scalar_coupling_constant)]\nfinal_test <- train_test[is.na(scalar_coupling_constant)]\n\nrm(train_test)\ngc()\n\ntarget <- final_train$scalar_coupling_constant\ntest_ids_full <- final_test$id\n\nfinal_train[, \":=\"(id = NULL, molecule_name = NULL, structure_atom_0 = NULL, structure_atom_1 = NULL, scalar_coupling_constant = NULL)]\nfinal_test[, \":=\"(id = NULL, molecule_name = NULL, structure_atom_0 = NULL, structure_atom_1 = NULL, scalar_coupling_constant = NULL)]\n\ncat(\"Rows in final train:\", nrow(final_train), \"\\nCols in final train:\", ncol(final_train))\nhead(final_train)\nsummary(final_train)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Evaluation Metric\nThe evaluation metric: https://www.kaggle.com/c/champs-scalar-coupling/overview/evaluation is the mean of log of MAE across the coupling types. Creating a function for the same.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"mlmae <- function(actuals, preds, types)\n{\n    sum_lmae <- 0\n    n_types <- length(unique(types))\n    \n    for (type in unique(types))\n    {\n        sum_lmae <- sum_lmae + log(mae(actuals[types == type], preds[types == type]))\n    }\n    \n    return (sum_lmae / n_types)\n}","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Models\nBuilding LightGBM model.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# LGBM CV\nrequire(lightgbm)\n\ntest_ids <- c()\nval_maes <- c()\npreds_test <- c()\nfimp_cv <- data.table()\n\nparams <- list()\nparams[\"objective\"] <- \"regression_l2\"\nparams[\"learning_rate\"] <- 0.25\nparams[\"num_leaves\"] <- 255\nparams[\"min_data_in_leaf\"] <- 50\nparams[\"bagging_fraction\"] <- 0.9\nparams[\"feature_fraction\"] <- 0.9\nparams[\"metric\"] <- \"mae\"\n\nfor (molecule_type in unique(final_train$type))\n{\n    cat(\"Building Model for type:\", molecule_type, \"\\n\")\n    \n    dtrain <- lgb.Dataset(data = as.matrix(final_train[final_train$type == molecule_type, -c(\"type\"), with = F]), label = target[final_train$type == molecule_type])\n    dtest <- as.matrix(final_test[final_test$type == molecule_type, -c(\"type\"), with = F])\n    \n    test_ids <- c(test_ids, test_ids_full[final_test$type == molecule_type])\n\n    model_lgb_cv <- lgb.cv(params = params, data = dtrain, nfold = 4, nrounds = 3001, early_stopping_rounds = 50, eval_freq = 200)\n    val_maes <- c(val_maes, model_lgb_cv$best_score)\n    \n    model_lgb <- lgb.train(params = params, data = dtrain, nrounds = round(1.1 * model_lgb_cv$best_iter))\n    preds_test <- c(preds_test, predict(model_lgb, dtest))\n    \n    fimp <- lgb.importance(model_lgb)\n    fimp_cv <- rbind(fimp_cv, fimp)\n\n    gc()\n}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fimp <- fimp_cv[, .(mean_gain = mean(Gain)), .(Feature)]\nsetorder(fimp, -mean_gain)\nhead(fimp, 50)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# errors by type\ni <- 1\n\nfor (molecule_type in unique(final_train$type))\n{\n    cat(\"MAE of type\", molecule_type, \":\", val_maes[i], \"\\tLogMAE of type\", molecule_type, \":\", log(val_maes[i]), \"\\n\")\n    i <- i + 1\n}\n\ncat(\"\\nMean of Log-MAE CV:\", mean(log(val_maes)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"![](http://)XGBoost Model   \nuncomment code to run (**P.S.** it is slow)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# XGBoost CV (uncomment and run for XGBoost)\n#require(xgboost)\n\n#test_ids <- c()\n#oof_cv <- c()\n#preds_test <- c()\n#target_cv <- c()\n#types_cv <- c()\n#fimp_cv <- data.table()\n\n#params <- list()\n#params[\"eta\"] <- 0.3\n#params[\"max_depth\"] <- 5\n#params[\"subsample\"] <- 0.9\n#params[\"colsample_bytree\"] <- 0.8\n#params[\"min_child_weight\"] <- 50\n#params[\"eval_metric\"] <- \"mae\"\n\n#for (molecule_type in unique(final_train$type))\n#{\n#    cat(\"Building Model for type:\", molecule_type, \"\\n\")\n#    \n#    dtrain <- xgb.DMatrix(data = as.matrix(final_train[final_train$type == molecule_type, -c(\"type\"), with = F]), label = target[final_train$type == molecule_type])\n#    dtest <- xgb.DMatrix(data = as.matrix(final_test[final_test$type == molecule_type, -c(\"type\"), with = F]))\n#    \n#    test_ids <- c(test_ids, test_ids_full[final_test$type == molecule_type])\n#    target_cv <- c(target_cv, target[final_train$type == molecule_type])\n#    \n#    model_xgb_cv <- xgb.cv(params = params, data = dtrain, nfold = 4, nrounds = 700, early_stopping_rounds = 20, print_every_n = 100, prediction = T)\n#    oof_cv <- c(oof_cv, model_xgb_cv$pred)\n#    \n#    model_xgb <- xgb.train(params = params, data = dtrain, nrounds = round(1.1 * model_xgb_cv$best_iteration))\n#    preds_test <- c(preds_test, predict(model_xgb, dtest))\n#    \n#    fimp <- xgb.importance(model = model_xgb, feature_names = names(final_train[, -c(\"type\"), with = F]))\n#    fimp_cv <- rbind(fimp_cv, fimp)\n#    \n#    types_cv <- c(types_cv, rep(molecule_type, length(target[final_train$type == molecule_type])))\n#    \n#    gc()\n#}\n\n# feature importance\n#fimp <- fimp_cv[, .(mean_gain = mean(Gain)), .(Feature)]\n#setorder(fimp, -mean_gain)\n#fimp\n\n# errors by type\n#cv <- data.table(target = target_cv, oof = oof_cv, type = types_cv)\n#setorder(cv, type)\n#\n#for (molecule_type in unique(cv$type))\n#{\n#    cat(\"MAE of type\", molecule_type, \":\", mae(cv$target[cv$type == molecule_type], cv$oof[cv$type == molecule_type]),\n#        \"\\tLogMAE of type\", molecule_type, \":\", log(mae(cv$target[cv$type == molecule_type], cv$oof[cv$type == molecule_type])),\n#        \"\\n\")\n#}\n#\n#cat(\"\\nMAE CV:\", mae(target_cv, oof_cv))\n#cat(\"\\tMean of Log-MAE CV:\", mlmae(target_cv, oof_cv, types_cv))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So, some types are easier to predict with lower scores compared to others. Since the final evaluation metric is a mean of scores across each type, optimizing the scores of each model individually can help in improving final score.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# submission\nsubmit <- data.table(id = test_ids, scalar_coupling_constant = preds_test)\nfwrite(submit, \"submission.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Conclusion\nThere is a lot more of exploration, features, experiments, models that can be done to improve the model performance.\nI hope this was useful for at least some of you. Feel free to share any feedback!   \n\nGood Luck and Happy Modelling!","execution_count":null}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":4}