{"cells":[{"metadata":{},"cell_type":"markdown","source":"# LGB starter for R"},{"metadata":{},"cell_type":"markdown","source":"This is a minimal kernel without time series tuning etc. and serves as a starting point for more fancy models."},{"metadata":{},"cell_type":"markdown","source":"## Packages\n\nLet's load the usual suspects first."},{"metadata":{"trusted":true},"cell_type":"code","source":"set.seed(15)\nlibrary(data.table)\nlibrary(tidyverse)\nlibrary(caret)\nlibrary(lightgbm)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Import\n\nNext, we load the data into a list.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"path <- \"../input/ashrae-energy-prediction\"\nall_files <- list.files(path)\nraw_list <- lapply(file.path(path, all_files), fread)\nnames(raw_list) <- gsub(\"(.*)\\\\.csv\", \"\\\\1\", all_files)\nlapply(raw_list, dim)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data preparation \n\nWe join the data and extract some time features. Since we need to repeat the same for the test data, we wrap the steps into a function."},{"metadata":{"_uuid":"051d70d956493feee0c6d64651c6a088724dca2a","_execution_state":"idle","trusted":true},"cell_type":"code","source":"# Public holiday\nholidays <- lubridate::as_datetime(\n  c(\"2016-01-01\", \"2016-01-18\", \"2016-02-15\", \n    \"2016-05-30\", \"2016-07-04\", \"2016-09-05\", \n    \"2016-10-10\", \"2016-11-11\", \"2016-11-24\",\n    \"2016-12-26\", \"2017-01-02\", \"2017-01-16\",\n    \"2017-02-20\", \"2017-05-29\", \"2017-07-04\", \n    \"2017-09-04\", \"2017-10-09\", \"2017-11-10\", \n    \"2017-11-23\", \"2017-12-25\", \"2018-01-01\", \n    \"2018-01-15\", \"2018-02-19\", \"2018-05-28\",\n    \"2018-07-04\", \"2018-09-03\", \"2018-10-08\", \n    \"2018-11-12\", \"2018-11-22\", \"2018-12-25\", \n    \"2019-01-01\"))\n\nprepfun <- function(base, weather, building = raw_list$building_metadata) {\n   out <- base %>% \n      left_join(building, by = \"building_id\") %>% \n      left_join(weather, by = c(\"site_id\", \"timestamp\")) %>% \n      mutate(timestamp = lubridate::as_datetime(timestamp),\n             hour = lubridate::hour(timestamp),\n             month = lubridate::month(timestamp),\n             weekday = lubridate::wday(timestamp),\n             is_holiday = (timestamp %in% holidays) + 0,\n             square_feet = log(square_feet)) %>% \n      select(-timestamp)\n}\n\n# Let's apply it to train\ntrain <- with(raw_list, prepfun(train, weather_train))\nhead(train)\n\n# Output and input featuers\ny <- \"meter_reading\"\nx <- c(\"square_feet\", \"building_id\", \"meter\", \"air_temperature\", \"dew_temperature\", \n       \"primary_use\", \"year_built\", \"hour\", \"site_id\", \"floor_count\", \n       \"weekday\", \"cloud_coverage\", \"precip_depth_1_hr\")\nx_cat <- c(\"building_id\", \"meter\", \"primary_use\", \"site_id\")\n\n\n# Transform response\ntrain[[y]] <- log1p(train[[y]])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data split\n\nThe next step is to split the data set into train and valid."},{"metadata":{"trusted":true},"cell_type":"code","source":"inds <- caret::groupKFold(group = train[[\"month\"]], k = 10)\nvalid <- train[-inds[[1]], c(x, y)]\ntrain <- train[inds[[1]], c(x, y)]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Fit model\n\nNow we are ready to fit a basic tree booster to the data.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# lgb data wrapper\nprep_lgb <- function(data, x) {\n  data %>% \n    select_at(x) %>% \n    mutate_if(Negate(is.numeric), function(z) as.integer(as.factor(z))) %>% \n    data.matrix()\n}\ndtrain <- lgb.Dataset(prep_lgb(train, x), label = train[[y]])\ndvalid <- lgb.Dataset(prep_lgb(valid, x), label = valid[[y]])\n \nparams <- list(learning_rate = 0.2,\n               feature_fraction = 0.85,\n               num_leaves = 40,\n               lambda = 2,\n               alpha = 0.1,\n               nthread = 4,\n               objective = \"regression\",\n               max_bin = 63)\n\nfit_lgb <- lgb.train(params = params,\n                     data = dtrain,\n                     early_stopping_rounds = 100,\n                     eval_freq = 100,\n                     valids = list(train = dtrain, valid = dvalid),\n                     nrounds = 3000)\n\nlgb.save(fit_lgb, \"lgb.csv\")\n# fit_lgb <- lgb.load(\"lgb.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Make submission\n\nThis step is rather painful as the test data is quite large. It will take a couple of minutes."},{"metadata":{"trusted":true},"cell_type":"code","source":"rm(train, valid)\nraw_list$train <- NULL\ngc()\ntest <- with(raw_list, prepfun(test, weather_test))\nraw_list$test <- NULL\ntest <- prep_lgb(test, x = x)\npred <- round(exp(predict(fit_lgb, test)) - 1, 4)\n\nraw_list$sample_submission[[\"meter_reading\"]] <- pmax(0, pred)\nfwrite(raw_list$sample_submission, \"submission.csv\", row.names = FALSE)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}