Summary
\n\nOverall solution was 2nd-level ensembling. We built a lot of models on 1st level (using 9 week as validation set) The training method of most of 1st level models was XGBoost. For second level we used ExtraTrees classifier and linear model from Python scikit-learn. The final result is weighted average of these two models.\nThe most important features are based on the 1-3 weeks lags of target variable grouped by factors and their combinations, aggregated features (min, max, mean, sum) of target variable grouped by factors and their combinations, frequency features of factors variables.\nFor our work, we used R environment with Rstudio IDE, and Python with Jupyter. We combined our work on our personal computers with cloud computing on x1.32xlarge Amazon EC2 instance with web interface for access. Last week of the competition was the continuous calculation of single and 2nd level models on x1.32xlarge Amazon EC2 instance.
\n\nIt was a very interesting competition, our approaches were inspired by many discussions on the competition forum and by many kernels published by other participants.
\n\nFeatures Selection / Engineering
\n\nOne of the main ideas in our approach is that it is important to know what were the previous weeks sales. If previous week too many products were supplied and they were not sold, next week this product amount, supplied to the same store, will be decreased. So, it is very important to include lagged values of target variable Demanda_uni_equil as a feature to predict next sales. \nThe simplified version of the R script “Bimbo XGBoost R script LB:0.457” for such an approach was published on the Kaggle forum of this competition. \n We merged transactions data frame with data frames from files cliente_tabla.csv, producto_tabla.csv using appropriate key fields. We used frequencies features grouped by categorical variables (e.g. Producto_ID, Cliente_ID, Agencia_ID, etc. ) and by different combinations of them. Data frame with new product features received by text parsing of product name variable NombreProducto from file producto_tabla.csv was merged to main data frame via key Producto_ID. As a result of parsing of product names we got a lot of new product features which denote e.g. short names of product, weight, brand, etc. We grouped the products by clusters and used the number of clusters as a new feature. \n We chose the data of previous weeks in respect of training sets weeks for aggregations of target variable Demanda_uni_equil, grouped by factors variables. We calculated mean, min, max, sum of target variable grouped by factors and their combinations. Then these aggregated values was merged into main data sets as the new features using the list of these factors as keys for merging. We also calculated the similar aggregated features grouped by factors for variables Venta_hoy, Venta_uni_hoy, Dev_uni_proxima, Dev_uni_proxima. \nWe chose the set of 8-9 weeks for the validation and 6-7 weeks’ data were considered as a training set for validation. It was necessary to build 2-level models and find appropriate weights for models blending. 8-9th weeks were chosen as a training set for prediction sales of the 10-11th weeks. So, we generated two types of training the data set: 6-7th weeks training data set for the prediction of 8-9th weeks sales and 8-9th weeks for the prediction of 10-11th weeks sales. Since we used the 1st week lag for the target variable as a feature, we needed to calculate this feature additionally for predicting 11th week sales. First we predicted the10th week, then using the predicted target variable for the 10th week, we calculated lagged features with the lag for 1 week and used these calculated features for the prediction of 11th week sales. As the first step for validation, we calculated the prediction for the 8th week, then using predicted target values we calculated 1 week lag for 9th week data set, and then we made the prediction for 9th week sales. It was done to build the validation model similar to the prediction of 10-11 weeks sales. For lag values, we investigated two types of cases with lags for 1-3 weeks and lags for 2-3 weeks. First we used up to 5 weeks lags, on the next study we used maximum 3 weeks lags. In the case of 1-3 weeks lags, for the validation on 8-9th weeks sales, we made the prediction in 2 steps. For building a two-level model, we used only the validation of 9th week sales. Our study shows that using 1-3 weeks lags gives us better scores comparing with 2-3 weeks lags. For the features with 1 week lags we used only the mean value of target variable grouped by the list of factor variables. For 2-3 weeks lags, we also used averaged lags for such features as Dev_proxima, Dev_uni_proxima, Venta_hoy grouped by the list of different factors. Most of the lagged features which were used for classification have 2 weeks lag. We generated more than 300 features. All received results for the validation on 9th week sales and for the prediction of 10-11th weeks, were used for building two-level models. Our study also shows that using only 7th week data for the validation on 8-9th weeks and 9th week for the prediction of sales on 10-11th weeks gives better scores comparing with two weeks training sets, so we built our next single models based on the sets of 7th week for validation and on the sets of 9th week for the prediction. To speed up our calculation we selected 175 top features using function xgb.importance() of ‘xgboost’ R package and then we worked with this set of features. We also calculated the predictions for Venta_hoy", "Venta_uni_hoy", "Dev_uni_proxima", "Dev_proxima" variables which were used on the 2nd level model. On the first level each of us used his own classifier options and features sets which gave results with good scores for single models without high correlation that is important for creating 2 level models.
\n\nTraining Methods
\n\nFor creating single models of the first level, we used xgboost classifier. The options for XGBoost classifiers can be found in our scripts. On the second level, we used validation results of the first level models as training sets for the prediction of target variable for the test set. We created a linear model and model based on ExtraTreesClassifier. On the third level, the results from the second level models were just weighted average.
\n\nReferences
\n\nhttps://www.kaggle.com/lyytinen/grupo-bimbo-inventory-demand/basic-preprocessing-for-products\nhttps://www.kaggle.com/ybabakhin/grupo-bimbo-inventory-demand/products-clustering\nhttps://www.kaggle.com/mlandry/grupo-bimbo-inventory-demand/h2o-gbm/run/263502
\n\nTeam Slippery Appraisals\nYear 2016
", "messages": [ { "id": "136641", "postDate": "09/23/2016 16:47:59", "content": "Summary
\n\nOverall solution was 2nd-level ensembling. We built a lot of models on 1st level (using 9 week as validation set) The training method of most of 1st level models was XGBoost. For second level we used ExtraTrees classifier and linear model from Python scikit-learn. The final result is weighted average of these two models.\nThe most important features are based on the 1-3 weeks lags of target variable grouped by factors and their combinations, aggregated features (min, max, mean, sum) of target variable grouped by factors and their combinations, frequency features of factors variables.\nFor our work, we used R environment with Rstudio IDE, and Python with Jupyter. We combined our work on our personal computers with cloud computing on x1.32xlarge Amazon EC2 instance with web interface for access. Last week of the competition was the continuous calculation of single and 2nd level models on x1.32xlarge Amazon EC2 instance.
\n\nIt was a very interesting competition, our approaches were inspired by many discussions on the competition forum and by many kernels published by other participants.
\n\nFeatures Selection / Engineering
\n\nOne of the main ideas in our approach is that it is important to know what were the previous weeks sales. If previous week too many products were supplied and they were not sold, next week this product amount, supplied to the same store, will be decreased. So, it is very important to include lagged values of target variable Demanda_uni_equil as a feature to predict next sales. \nThe simplified version of the R script “Bimbo XGBoost R script LB:0.457” for such an approach was published on the Kaggle forum of this competition. \n We merged transactions data frame with data frames from files cliente_tabla.csv, producto_tabla.csv using appropriate key fields. We used frequencies features grouped by categorical variables (e.g. Producto_ID, Cliente_ID, Agencia_ID, etc. ) and by different combinations of them. Data frame with new product features received by text parsing of product name variable NombreProducto from file producto_tabla.csv was merged to main data frame via key Producto_ID. As a result of parsing of product names we got a lot of new product features which denote e.g. short names of product, weight, brand, etc. We grouped the products by clusters and used the number of clusters as a new feature. \n We chose the data of previous weeks in respect of training sets weeks for aggregations of target variable Demanda_uni_equil, grouped by factors variables. We calculated mean, min, max, sum of target variable grouped by factors and their combinations. Then these aggregated values was merged into main data sets as the new features using the list of these factors as keys for merging. We also calculated the similar aggregated features grouped by factors for variables Venta_hoy, Venta_uni_hoy, Dev_uni_proxima, Dev_uni_proxima. \nWe chose the set of 8-9 weeks for the validation and 6-7 weeks’ data were considered as a training set for validation. It was necessary to build 2-level models and find appropriate weights for models blending. 8-9th weeks were chosen as a training set for prediction sales of the 10-11th weeks. So, we generated two types of training the data set: 6-7th weeks training data set for the prediction of 8-9th weeks sales and 8-9th weeks for the prediction of 10-11th weeks sales. Since we used the 1st week lag for the target variable as a feature, we needed to calculate this feature additionally for predicting 11th week sales. First we predicted the10th week, then using the predicted target variable for the 10th week, we calculated lagged features with the lag for 1 week and used these calculated features for the prediction of 11th week sales. As the first step for validation, we calculated the prediction for the 8th week, then using predicted target values we calculated 1 week lag for 9th week data set, and then we made the prediction for 9th week sales. It was done to build the validation model similar to the prediction of 10-11 weeks sales. For lag values, we investigated two types of cases with lags for 1-3 weeks and lags for 2-3 weeks. First we used up to 5 weeks lags, on the next study we used maximum 3 weeks lags. In the case of 1-3 weeks lags, for the validation on 8-9th weeks sales, we made the prediction in 2 steps. For building a two-level model, we used only the validation of 9th week sales. Our study shows that using 1-3 weeks lags gives us better scores comparing with 2-3 weeks lags. For the features with 1 week lags we used only the mean value of target variable grouped by the list of factor variables. For 2-3 weeks lags, we also used averaged lags for such features as Dev_proxima, Dev_uni_proxima, Venta_hoy grouped by the list of different factors. Most of the lagged features which were used for classification have 2 weeks lag. We generated more than 300 features. All received results for the validation on 9th week sales and for the prediction of 10-11th weeks, were used for building two-level models. Our study also shows that using only 7th week data for the validation on 8-9th weeks and 9th week for the prediction of sales on 10-11th weeks gives better scores comparing with two weeks training sets, so we built our next single models based on the sets of 7th week for validation and on the sets of 9th week for the prediction. To speed up our calculation we selected 175 top features using function xgb.importance() of ‘xgboost’ R package and then we worked with this set of features. We also calculated the predictions for Venta_hoy", "Venta_uni_hoy", "Dev_uni_proxima", "Dev_proxima" variables which were used on the 2nd level model. On the first level each of us used his own classifier options and features sets which gave results with good scores for single models without high correlation that is important for creating 2 level models.
\n\nTraining Methods
\n\nFor creating single models of the first level, we used xgboost classifier. The options for XGBoost classifiers can be found in our scripts. On the second level, we used validation results of the first level models as training sets for the prediction of target variable for the test set. We created a linear model and model based on ExtraTreesClassifier. On the third level, the results from the second level models were just weighted average.
\n\nReferences
\n\nhttps://www.kaggle.com/lyytinen/grupo-bimbo-inventory-demand/basic-preprocessing-for-products\nhttps://www.kaggle.com/ybabakhin/grupo-bimbo-inventory-demand/products-clustering\nhttps://www.kaggle.com/mlandry/grupo-bimbo-inventory-demand/h2o-gbm/run/263502
\n\nTeam Slippery Appraisals\nYear 2016
", "rawMarkdown": "**Summary**\r\n\r\nOverall solution was 2nd-level ensembling. We built a lot of models on 1st level (using 9 week as validation set) The training method of most of 1st level models was XGBoost. For second level we used ExtraTrees classifier and linear model from Python scikit-learn. The final result is weighted average of these two models.\r\nThe most important features are based on the 1-3 weeks lags of target variable grouped by factors and their combinations, aggregated features (min, max, mean, sum) of target variable grouped by factors and their combinations, frequency features of factors variables.\r\nFor our work, we used R environment with Rstudio IDE, and Python with Jupyter. We combined our work on our personal computers with cloud computing on x1.32xlarge Amazon EC2 instance with web interface for access. Last week of the competition was the continuous calculation of single and 2nd level models on x1.32xlarge Amazon EC2 instance.\r\n\r\nIt was a very interesting competition, our approaches were inspired by many discussions on the competition forum and by many kernels published by other participants.\r\n\r\n**Features Selection / Engineering**\r\n\r\nOne of the main ideas in our approach is that it is important to know what were the previous weeks sales. If previous week too many products were supplied and they were not sold, next week this product amount, supplied to the same store, will be decreased. So, it is very important to include lagged values of target variable Demanda_uni_equil as a feature to predict next sales. \r\nThe simplified version of the R script [“Bimbo XGBoost R script LB:0.457”][1] for such an approach was published on the Kaggle forum of this competition. \r\n We merged transactions data frame with data frames from files cliente_tabla.csv, producto_tabla.csv using appropriate key fields. We used frequencies features grouped by categorical variables (e.g. Producto_ID, Cliente_ID, Agencia_ID, etc. ) and by different combinations of them. Data frame with new product features received by text parsing of product name variable NombreProducto from file producto_tabla.csv was merged to main data frame via key Producto_ID. As a result of parsing of product names we got a lot of new product features which denote e.g. short names of product, weight, brand, etc. We grouped the products by clusters and used the number of clusters as a new feature. \r\n We chose the data of previous weeks in respect of training sets weeks for aggregations of target variable Demanda_uni_equil, grouped by factors variables. We calculated mean, min, max, sum of target variable grouped by factors and their combinations. Then these aggregated values was merged into main data sets as the new features using the list of these factors as keys for merging. We also calculated the similar aggregated features grouped by factors for variables Venta_hoy, Venta_uni_hoy, Dev_uni_proxima, Dev_uni_proxima. \r\nWe chose the set of 8-9 weeks for the validation and 6-7 weeks’ data were considered as a training set for validation. It was necessary to build 2-level models and find appropriate weights for models blending. 8-9th weeks were chosen as a training set for prediction sales of the 10-11th weeks. So, we generated two types of training the data set: 6-7th weeks training data set for the prediction of 8-9th weeks sales and 8-9th weeks for the prediction of 10-11th weeks sales. Since we used the 1st week lag for the target variable as a feature, we needed to calculate this feature additionally for predicting 11th week sales. First we predicted the10th week, then using the predicted target variable for the 10th week, we calculated lagged features with the lag for 1 week and used these calculated features for the prediction of 11th week sales. As the first step for validation, we calculated the prediction for the 8th week, then using predicted target values we calculated 1 week lag for 9th week data set, and then we made the prediction for 9th week sales. It was done to build the validation model similar to the prediction of 10-11 weeks sales. For lag values, we investigated two types of cases with lags for 1-3 weeks and lags for 2-3 weeks. First we used up to 5 weeks lags, on the next study we used maximum 3 weeks lags. In the case of 1-3 weeks lags, for the validation on 8-9th weeks sales, we made the prediction in 2 steps. For building a two-level model, we used only the validation of 9th week sales. Our study shows that using 1-3 weeks lags gives us better scores comparing with 2-3 weeks lags. For the features with 1 week lags we used only the mean value of target variable grouped by the list of factor variables. For 2-3 weeks lags, we also used averaged lags for such features as Dev_proxima, Dev_uni_proxima, Venta_hoy grouped by the list of different factors. Most of the lagged features which were used for classification have 2 weeks lag. We generated more than 300 features. All received results for the validation on 9th week sales and for the prediction of 10-11th weeks, were used for building two-level models. Our study also shows that using only 7th week data for the validation on 8-9th weeks and 9th week for the prediction of sales on 10-11th weeks gives better scores comparing with two weeks training sets, so we built our next single models based on the sets of 7th week for validation and on the sets of 9th week for the prediction. To speed up our calculation we selected 175 top features using function xgb.importance() of ‘xgboost’ R package and then we worked with this set of features. We also calculated the predictions for Venta_hoy\", \"Venta_uni_hoy\", \"Dev_uni_proxima\", \"Dev_proxima\" variables which were used on the 2nd level model. On the first level each of us used his own classifier options and features sets which gave results with good scores for single models without high correlation that is important for creating 2 level models. \r\n\r\n**Training Methods**\r\n\r\nFor creating single models of the first level, we used xgboost classifier. The options for XGBoost classifiers can be found in our scripts. On the second level, we used validation results of the first level models as training sets for the prediction of target variable for the test set. We created a linear model and model based on ExtraTreesClassifier. On the third level, the results from the second level models were just weighted average.\r\n\r\n**References**\r\n\r\nhttps://www.kaggle.com/lyytinen/grupo-bimbo-inventory-demand/basic-preprocessing-for-products\r\nhttps://www.kaggle.com/ybabakhin/grupo-bimbo-inventory-demand/products-clustering\r\nhttps://www.kaggle.com/mlandry/grupo-bimbo-inventory-demand/h2o-gbm/run/263502\r\n\r\n\r\nTeam Slippery Appraisals\r\nYear 2016\r\n\r\n\r\n [1]: https://www.kaggle.com/bpavlyshenko/grupo-bimbo-inventory-demand/bimbo-xgboost-r-script-lb-0-457/discussion", "votes": null }, { "id": "136673", "postDate": "09/23/2016 22:28:31", "content": "This is incredible! Thanks for the all the insights. This is super helpful for learners like myself. =)
", "rawMarkdown": "This is incredible! Thanks for the all the insights. This is super helpful for learners like myself. =)", "votes": null }, { "id": "136701", "postDate": "09/24/2016 04:05:05", "content": "Thanks for sharing Bohdan
", "rawMarkdown": "Thanks for sharing Bohdan", "votes": null }, { "id": "136725", "postDate": "09/24/2016 11:39:28", "content": "Bohdan,
\n\nIs there any chance we can see the code? I pretty much have the exact set of features and validation process and could not get to your score.\nAlso, you said that for the 1st level you use XGBoost Classifier. Do you mean XGBosst Regressor?
\n\nPablo
", "rawMarkdown": "Bohdan,\r\n\r\nIs there any chance we can see the code? I pretty much have the exact set of features and validation process and could not get to your score.\r\nAlso, you said that for the 1st level you use XGBoost Classifier. Do you mean XGBosst Regressor?\r\n\r\nPablo", "votes": null }, { "id": "137661", "postDate": "10/03/2016 19:03:16", "content": "Our teammates used their own codes and features sets to produce first level models, part of the code was written in R, other part - using Python. Then these first level models were used for creating the models on the second level.
\n\n\n\n\nYou said that for the 1st level you use XGBoost Classifier. Do you mean XGBoost Regressor?
\n
For classifier training I used the function xgb.train() of “xgboost” R package.
", "rawMarkdown": "Our teammates used their own codes and features sets to produce first level models, part of the code was written in R, other part - using Python. Then these first level models were used for creating the models on the second level. \r\n\r\n> You said that for the 1st level you use XGBoost Classifier. Do you mean XGBoost Regressor?\r\n\r\nFor classifier training I used the function xgb.train() of “xgboost” R package.", "votes": null }, { "id": "137747", "postDate": "10/04/2016 16:06:15", "content": "Thanks for your sharing, it is very helpful for me!
", "rawMarkdown": "Thanks for your sharing, it is very helpful for me!", "votes": null } ], "comments": [ { "id": 136673, "author_name": "tenaciousterrier", "author_url": "", "post_date": "09/23/2016 22:28:31", "content": "This is incredible! Thanks for the all the insights. This is super helpful for learners like myself. =)
", "votes": null, "replies": [] }, { "id": 136701, "author_name": "", "author_url": "", "post_date": "09/24/2016 04:05:05", "content": "Thanks for sharing Bohdan
", "votes": null, "replies": [] }, { "id": 136725, "author_name": "pablomarin", "author_url": "", "post_date": "09/24/2016 11:39:28", "content": "Bohdan,
\n\nIs there any chance we can see the code? I pretty much have the exact set of features and validation process and could not get to your score.\nAlso, you said that for the 1st level you use XGBoost Classifier. Do you mean XGBosst Regressor?
\n\nPablo
", "votes": null, "replies": [] }, { "id": 137661, "author_name": "bpavlyshenko", "author_url": "", "post_date": "10/03/2016 19:03:16", "content": "Our teammates used their own codes and features sets to produce first level models, part of the code was written in R, other part - using Python. Then these first level models were used for creating the models on the second level.
\n\n\n\n\nYou said that for the 1st level you use XGBoost Classifier. Do you mean XGBoost Regressor?
\n
For classifier training I used the function xgb.train() of “xgboost” R package.
", "votes": null, "replies": [] }, { "id": 137747, "author_name": "jimliu", "author_url": "", "post_date": "10/04/2016 16:06:15", "content": "Thanks for your sharing, it is very helpful for me!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "136641": "**Summary**\r\n\r\nOverall solution was 2nd-level ensembling. We built a lot of models on 1st level (using 9 week as validation set) The training method of most of 1st level models was XGBoost. For second level we used ExtraTrees classifier and linear model from Python scikit-learn. The final result is weighted average of these two models.\r\nThe most important features are based on the 1-3 weeks lags of target variable grouped by factors and their combinations, aggregated features (min, max, mean, sum) of target variable grouped by factors and their combinations, frequency features of factors variables.\r\nFor our work, we used R environment with Rstudio IDE, and Python with Jupyter. We combined our work on our personal computers with cloud computing on x1.32xlarge Amazon EC2 instance with web interface for access. Last week of the competition was the continuous calculation of single and 2nd level models on x1.32xlarge Amazon EC2 instance.\r\n\r\nIt was a very interesting competition, our approaches were inspired by many discussions on the competition forum and by many kernels published by other participants.\r\n\r\n**Features Selection / Engineering**\r\n\r\nOne of the main ideas in our approach is that it is important to know what were the previous weeks sales. If previous week too many products were supplied and they were not sold, next week this product amount, supplied to the same store, will be decreased. So, it is very important to include lagged values of target variable Demanda_uni_equil as a feature to predict next sales. \r\nThe simplified version of the R script [“Bimbo XGBoost R script LB:0.457”][1] for such an approach was published on the Kaggle forum of this competition. \r\n We merged transactions data frame with data frames from files cliente_tabla.csv, producto_tabla.csv using appropriate key fields. We used frequencies features grouped by categorical variables (e.g. Producto_ID, Cliente_ID, Agencia_ID, etc. ) and by different combinations of them. Data frame with new product features received by text parsing of product name variable NombreProducto from file producto_tabla.csv was merged to main data frame via key Producto_ID. As a result of parsing of product names we got a lot of new product features which denote e.g. short names of product, weight, brand, etc. We grouped the products by clusters and used the number of clusters as a new feature. \r\n We chose the data of previous weeks in respect of training sets weeks for aggregations of target variable Demanda_uni_equil, grouped by factors variables. We calculated mean, min, max, sum of target variable grouped by factors and their combinations. Then these aggregated values was merged into main data sets as the new features using the list of these factors as keys for merging. We also calculated the similar aggregated features grouped by factors for variables Venta_hoy, Venta_uni_hoy, Dev_uni_proxima, Dev_uni_proxima. \r\nWe chose the set of 8-9 weeks for the validation and 6-7 weeks’ data were considered as a training set for validation. It was necessary to build 2-level models and find appropriate weights for models blending. 8-9th weeks were chosen as a training set for prediction sales of the 10-11th weeks. So, we generated two types of training the data set: 6-7th weeks training data set for the prediction of 8-9th weeks sales and 8-9th weeks for the prediction of 10-11th weeks sales. Since we used the 1st week lag for the target variable as a feature, we needed to calculate this feature additionally for predicting 11th week sales. First we predicted the10th week, then using the predicted target variable for the 10th week, we calculated lagged features with the lag for 1 week and used these calculated features for the prediction of 11th week sales. As the first step for validation, we calculated the prediction for the 8th week, then using predicted target values we calculated 1 week lag for 9th week data set, and then we made the prediction for 9th week sales. It was done to build the validation model similar to the prediction of 10-11 weeks sales. For lag values, we investigated two types of cases with lags for 1-3 weeks and lags for 2-3 weeks. First we used up to 5 weeks lags, on the next study we used maximum 3 weeks lags. In the case of 1-3 weeks lags, for the validation on 8-9th weeks sales, we made the prediction in 2 steps. For building a two-level model, we used only the validation of 9th week sales. Our study shows that using 1-3 weeks lags gives us better scores comparing with 2-3 weeks lags. For the features with 1 week lags we used only the mean value of target variable grouped by the list of factor variables. For 2-3 weeks lags, we also used averaged lags for such features as Dev_proxima, Dev_uni_proxima, Venta_hoy grouped by the list of different factors. Most of the lagged features which were used for classification have 2 weeks lag. We generated more than 300 features. All received results for the validation on 9th week sales and for the prediction of 10-11th weeks, were used for building two-level models. Our study also shows that using only 7th week data for the validation on 8-9th weeks and 9th week for the prediction of sales on 10-11th weeks gives better scores comparing with two weeks training sets, so we built our next single models based on the sets of 7th week for validation and on the sets of 9th week for the prediction. To speed up our calculation we selected 175 top features using function xgb.importance() of ‘xgboost’ R package and then we worked with this set of features. We also calculated the predictions for Venta_hoy\", \"Venta_uni_hoy\", \"Dev_uni_proxima\", \"Dev_proxima\" variables which were used on the 2nd level model. On the first level each of us used his own classifier options and features sets which gave results with good scores for single models without high correlation that is important for creating 2 level models. \r\n\r\n**Training Methods**\r\n\r\nFor creating single models of the first level, we used xgboost classifier. The options for XGBoost classifiers can be found in our scripts. On the second level, we used validation results of the first level models as training sets for the prediction of target variable for the test set. We created a linear model and model based on ExtraTreesClassifier. On the third level, the results from the second level models were just weighted average.\r\n\r\n**References**\r\n\r\nhttps://www.kaggle.com/lyytinen/grupo-bimbo-inventory-demand/basic-preprocessing-for-products\r\nhttps://www.kaggle.com/ybabakhin/grupo-bimbo-inventory-demand/products-clustering\r\nhttps://www.kaggle.com/mlandry/grupo-bimbo-inventory-demand/h2o-gbm/run/263502\r\n\r\n\r\nTeam Slippery Appraisals\r\nYear 2016\r\n\r\n\r\n [1]: https://www.kaggle.com/bpavlyshenko/grupo-bimbo-inventory-demand/bimbo-xgboost-r-script-lb-0-457/discussion", "136673": "This is incredible! Thanks for the all the insights. This is super helpful for learners like myself. =)", "136701": "Thanks for sharing Bohdan", "136725": "Bohdan,\r\n\r\nIs there any chance we can see the code? I pretty much have the exact set of features and validation process and could not get to your score.\r\nAlso, you said that for the 1st level you use XGBoost Classifier. Do you mean XGBosst Regressor?\r\n\r\nPablo", "137661": "Our teammates used their own codes and features sets to produce first level models, part of the code was written in R, other part - using Python. Then these first level models were used for creating the models on the second level. \r\n\r\n> You said that for the 1st level you use XGBoost Classifier. Do you mean XGBoost Regressor?\r\n\r\nFor classifier training I used the function xgb.train() of “xgboost” R package.", "137747": "Thanks for your sharing, it is very helpful for me!" }, "source": "existing_discussion", "complete": true, "competition_id": 5260, "competition_slug": "grupo-bimbo-inventory-demand", "team_id": 342091, "solution_url": "/competitions/grupo-bimbo-inventory-demand/writeups/the-slippery-appraisals-1-place-solution-of-the-sl" }