{"cells":[{"metadata":{"_uuid":"051d70d956493feee0c6d64651c6a088724dca2a","_execution_state":"idle","trusted":true},"cell_type":"code","source":"\n\nlibrary(tidyverse) \nlibrary(recipes)\nlibrary(h2o)\nlibrary(corrplot)\n\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Categorical Features Encoding Prediction II**\n\n![](https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQYrmIPEe--yQ6xtfcYbkn_u3nPXha2XDcjlYi_XLG3U47YsZoC&s)"},{"metadata":{"trusted":true},"cell_type":"code","source":"cat.train <- read.csv('../input/cat-in-the-dat-ii/train.csv')\ncat.test <- read.csv('../input/cat-in-the-dat-ii/test.csv')\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**List of Features with NA **\n"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"\ncat.train  %>% \n        map_chr(~ sum(is.na(.)) )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Combine the Train and Test Data to Preprocess**\n\nDimension of total Data (Train + Test)\n"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"cat.test$target <- NA\ncat <- rbind(cat.train, cat.test)\ndim(cat)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Initial Process The data**\n\n1- Convert bin to 0 and 1\n\n2- Convert ord tor ordered factor\n\n3- Convert Target to factor\n\n\nStructure of the Data"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"\ncat <- cat %>% \n  mutate(\n    bin_3 = ifelse(bin_3 == TRUE, 1, 0),\n    bin_4 = ifelse(bin_4 == 'N', 0,\n                   ifelse(bin_4 == 'Y', 1, NA)),\n    ord_0 = factor(ord_0, ordered = TRUE),\n    ord_1 = factor(ord_1, levels = c('', 'Novice','Contributor','Expert','Master','Grandmaster'), ordered = TRUE),\n    ord_2 = factor(ord_2, levels = c('', 'Freezing','Cold','Warn','Hot','Boiling Hot','Lava Hot'), ordered = TRUE),\n    ord_3 = factor(ord_3, ordered = TRUE),\n    ord_4 = factor(ord_4, ordered = TRUE),\n    ord_5 = factor(ord_5, ordered = TRUE),\n    day = factor(day),\n    month = factor(month),\n    target = factor(target, labels = c('X0','X1'))\n  )\n\nstr(cat)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Using Recipes to Preprocess**\n\n\n![](https://i1.wp.com/blog.datascienceheroes.com/content/images/2019/07/one-hot-recipes-1.png?w=584&ssl=1)\n\n\n**Steps:**\n\n1- Remove id\n\n2- Convert ord in ordinal score\n\n3- Compute Median Impute in ord and bin factors\n\n4- Replace NA to unknown in nom factors\n\n5- Replace non factors with threshold < 0.5% to other\n\n6- Creat dummy variables"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"rec <- recipe(target ~ . , data = cat) %>% \n  step_rm(id) %>%  \n  step_ordinalscore(starts_with('ord_')) %>% \n  step_medianimpute(all_numeric()) %>% \n  step_unknown(all_nominal(), -all_outcomes()) %>% \n  step_other(all_nominal(), -all_outcomes(), threshold = 0.005) %>%\n  step_dummy(all_nominal(), -all_outcomes()) %>% \n  step_zv(all_predictors())\n\npreparo <- prep(rec, cat)\ncat.new <- bake(preparo, cat)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Filter cat Train and Test**\n\nCheck the final dimensions\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"cat.train.new <- cat.new  %>% filter(!is.na(target))\ncat.test.new <- cat.new  %>% filter(is.na(target))\ncat.test.new$target <- NULL\n\ndim(cat.train.new)\ndim(cat.test.new)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Matrix Correlations of ord and bin predictors to outcome**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"rec2 <- rec <- recipe(target ~ . , data = cat) %>% \n  step_rm(id, contains('nom_'), day, month ) %>%  \n  step_ordinalscore(starts_with('ord_')) %>% \n  step_medianimpute(all_numeric()) %>% \n  step_mutate(\n  target = ifelse(target == 'X0', 0, 1))  %>% \n  step_zv(all_predictors())\n\npreparo2 <- prep(rec2, cat)\ncat.corr <- bake(preparo2, cat)\n\ncat.corr <- cat.corr  %>% filter(!is.na(target))\n\ncorr <- cor(cat.corr)\ncorrplot(corr, method = 'number', type = 'upper')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The variable ord_3 is the most correlated with outcome (r = 0.18)"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"ggplot(cat.corr, aes(as.factor(target), y = ..count../sum(..count..), fill = as.factor(target)))+\n      geom_bar()+\n      labs(x = 'Target', y = '%', fill = 'target', title = 'Distribution of the Target')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**The target variable was imbalanced**\n\nBalance the class target may be usefull"},{"metadata":{"trusted":true},"cell_type":"code","source":"ggplot(cat.corr, aes(as.factor(ord_3), fill = as.factor(target)))+\n     geom_bar(position = 'fill')+\n     coord_flip()+\n     labs(y = 'ord_3', fill = 'target', title = 'Distribution ord_3 by target')\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Test with adding a spline in ord factors improve the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"library(splines)\nlibrary(broom)\n\nmodel_01 <- glm(target ~ ord_3 , data = cat.train.new, family = 'binomial')\nmodel_02 <- glm(target ~ ns(ord_3, 2), data = cat.train.new, family = 'binomial')\nmodel_03 <- glm(target ~ ns(ord_3, 3), data = cat.train.new, family = 'binomial')\n\nmodels <- list(model_01, model_02, model_03)\n\nmap(models, glance)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Adding a natural splines improve the model"},{"metadata":{},"cell_type":"markdown","source":"**Model Building with H2o**\n\n![](https://miro.medium.com/max/1200/1*vG03fma-Ee7OJLHJs4F3WA.png)\n\nSteps:\n\n1- h2o Init\n\n2- Split the train data\n\n3- Gerate the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nh2o.init()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"\n#Load the Data\ncat.h2o.train <- as.h2o(cat.train.new)\ncat.h2o.test <- as.h2o(cat.test.new)\n\n## Split the data\nsplit <- h2o.splitFrame(cat.h2o.train, ratios = c(0.75, 0.15))\ncat.train.h2o <- split[[1]]\ncat.valid.h2o <- split[[2]]\ncat.test.h2o <- split[[3]]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"## ---- features -------------------------- ##\n\nx <- setdiff(names(cat.h2o.train), 'target')\ny <- 'target'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Elastic Net**\n\nglm with binomial distribution and regularization with alpha = 0.5 and lamdba search "},{"metadata":{"trusted":true},"cell_type":"code","source":"## --------- Model ------------------------ ##\n\ncat.glm <- h2o.glm(x = x, y = y, training_frame = cat.train.h2o,\n                   validation_frame = cat.valid.h2o, lambda_search = TRUE,\n                   family = 'binomial', balance_classes = TRUE)\n\nh2o.performance(cat.glm, newdata = cat.test.h2o)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The AUC was 0.75 in the test data\n\nBalance the class improve the performance\n\n**Var Impotance Plot**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"h2o.varimp_plot(cat.glm)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Make Prediction on Test Data File**"},{"metadata":{"trusted":true},"cell_type":"code","source":"predito <- h2o.predict(cat.glm, newdata = cat.h2o.test)\ncat.final <- as.data.frame(predito)\ncat.final$id <- cat.test$id\n\ncat.final <- cat.final %>% \n  mutate(target = X1) %>% \n  select(id, target) \n\nhead(cat.final, 20)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"write.csv(cat.final, file = 'submission_02.csv', row.names = FALSE,\n          quote = FALSE)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}