![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1779058%2Fc07758d1196e93145a8d581c871e4174%2Favengers-ensemble-8156544.png?generation=1572024079649146&alt=media\")
I'm stopping share things as we formed a team and it will be not fair to my teammates to disclose information that may affect all team.
\n\nPlease upvote and comment original topic and kernels authors if you'll find their work interesting and useful (feedback is very important for them to know they are doing good things). Please also indicate what can be done better (especially for kernles) as it helps authors to improve their work (current and future).
\n\nHere is small summary for current competition:\n- Data minification (https://www.kaggle.com/kyakovlev/ashrae-data-minification)\n- EDA\n- Baseline model (https://www.kaggle.com/kyakovlev/ashrae-baseline-lgbm)\n- CV options (https://www.kaggle.com/kyakovlev/ashrae-cv-options)\n- CV\n- Baseline FE \n(1.22 - https://www.kaggle.com/kyakovlev/ashrae-lgbm-simple-fe -lgbm)\n(1.23 - https://www.kaggle.com/kyakovlev/ashrae-catboost - catboost)\n same kernels with 2 lines change scores 1.13\n- Feature selection\n- Deeper FE\n- Ensembles/Stackings/Blends
\n\nRandom thoughts about competition.
\n\nThis competition is about predicting energy consumption.\nWhen I look at data I see \"not very reach\" features dataset. Seems that we can and have to use a lot of external data.
\n\nWeather - very important (here are some kernels about weather time-series data):\n* https://www.kaggle.com/leandrovrabelo/climate-change-forecast-sarima-model\n* https://www.kaggle.com/selfishgene/filtering-and-auto-correlation-tutorial\n* https://www.kaggle.com/jsphyg/weather-dataset-rattle-package\n* https://www.kaggle.com/c/recruit-restaurant-visitor-forecasting (please see discussions and winning solutions)\nWe don't predict the weather here. But if we do have? What if energy consumption has a strong connection with \"people's feelings\" about the weather tomorrow.
\n\nGeo markers (probably we can supplement data with this information).
\n\nWealth (also with categorical subsets).
\n\nBuilding condition (supplement with some thermal physics data).
\n\netc.
\n\nIt seems to me now that data that we have just a skeleton and we can and have to find interesting correlated data on external resources. \nMy advice here constantly monitor this thread:\nhttps://www.kaggle.com/c/ashrae-energy-prediction/discussion/112841#latest-649772
\n\nMore readings:\nTip: You've already noticed III number in competition name -> search for ASHRAE - Energy Predictor\n* https://www.kaggle.com/c/ashrae-energy-prediction/overview/prior-competitions\n* https://repository.upenn.edu/mlab_papers/75/ (just example of search result)\netc
\n\nOne of the first steps that should be done is data minification (just my opinion).\nYou will need it for:\n* Load data faster -> pickle format (best choice for me)\n* Consume less memory -> Carefull types downcasting\n* Avoid repeated cleaning steps.
\n\nPossible problems:\n* pandas pickle format is \"version dependent\" and you may have problems with loading data generated with a newer version of Pandas\n* Any bad transformation may and will affect your model. Minification should be done very veeeeery careful as you will use it in every single your kernel in the future. If you not sure about downcasting some column -> don't do. If not sure about cleaning -> don't do. All wrong decisions here may and will affect your score. You may spend days searching for error that prevents your model converge well and find that it was you who created an error in the minification step.
\n\nLinks:
\n\n> DON'T copy that kernels blindly. Try to understand the code and write your own that serves you. You may use that links just for reference.
\n\nhttps://www.kaggle.com/kyakovlev/ashrae-data-minification
\n\nThe baseline model is a model without any FE and without any model tuning. It will give you an idea of the very base/ground score that you can have with untransformed data. Conducting experiments you have to compare scores with some stable result.
\n\nLet's think about what ways we have to improve a single model score:\n* FE obviously\n* CV\n* Model tuning \n* Evaluation metric \n* Loss function
\n\nCV\nThink about how you can make it reliable and align your CV with LB.\nWithout good CV all modifications are \"blind shoots\" and can't be analyzed and evaluated.
\n\nModel tuning\nIt's up to you to decide when you want to do tuning, but my advice is to do at least some minimal tuning in the beginning. Normally (in most cases) model performance depends on the data structure and if it will be untouched your initial params can serve you almost until the end of the competition. In the beginning, it is easier to do as you have just a minimal number of features and any GridSearch/Bayesian optimization will run faster.
\n\nEvaluation metric/Loss function\nIn my case, I've used RMSE but the competition metric is RMSLE. If you will use lgbm please consult these examples for custom metric and custom loss function:\n* https://github.com/microsoft/LightGBM/blob/master/examples/python-guide/advanced_example.py\n* https://github.com/microsoft/LightGBM/blob/master/examples/python-guide/sklearn_example.py
\n\nOther baseline models:\n* https://www.kaggle.com/ryches/simple-lgbm-solution (by @ryches / LGBM)\n* https://www.kaggle.com/bejeweled/ashrae-catboost-regressor (by @bejeweled / CatBoost)\n* https://www.kaggle.com/tunguz/simple-linear-regression-benchmark (by @tunguz / Linear regression)\n* https://www.kaggle.com/yukari17/simple-prediction-do-not-use-model-v2 (by @yukari17 / without any Model)
\n\n> Never copy others solutions, use it as a reference and create your own baseline with model you know better. For sure it will take much more time but you will benefit much more from own code. Others approaches could have mistakes, errors, incorrect code.
\n\nBefore reading other's EDAs kernels try to make your own. If you are as lazy as I am - go to read part.
\n\nEDA is one of the most important parts of any competition. This your source for FE/CV/Model. Understanding the data and its structure is a crucial step to move on.
\n\nhttps://www.kaggle.com/kyakovlev/ashrae-cv-options
\n\nWe have several problems here (you may want to check the linked kernel).
\n\nWhat works:\n* Target/mean encoding\n* Outliers imputation\n* Categorisation
\n\nAdversarial validation shows that test set doesn't differ much from train set:\n* https://www.kaggle.com/tunguz/adversarial-ashrae (it's IMPORTANT)
\n\nWe don't have many features so for features selection you can safely use (if you have stable CV):\n* recursive feature elimination (remove 1 by 1)\n* permutation importance\n* recursive feature acceptance (add 1 by 1)
\n\nBlend works - you can have at least 1-2% of RMSLE with different model blending.
\n\nStacking is very tricky as it will overfit hugely for test set. As I see it's almost impossible to use it here. But you may find a way to implement it in your pipeline.
\n\nPrivate/Public Test set split:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113751
\n\nGeo location:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114874\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114447\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113338\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113772\n* https://www.kaggle.com/patrick0302/locate-cities-according-weather-temperature
\n\nMeter split (didn't test by it's interesting):\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113240\n* https://www.kaggle.com/corochann/ashrae-training-lgbm-by-meter-type
\n\nOutliers:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113254
\n\nHolidays:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113286
\n\nMetric:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113064
\n\nFaster submission:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114782
\n\nBetter score understanding (more than 2 decimals):\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113104
\n\nFaster stratified split\n* https://www.kaggle.com/frednavruzov/faster-stratified-cross-validation-upd
", "messages": [ { "id": "649931", "postDate": "10/15/2019 23:03:21", "content": "I'm stopping share things as we formed a team and it will be not fair to my teammates to disclose information that may affect all team.
\n\nPlease upvote and comment original topic and kernels authors if you'll find their work interesting and useful (feedback is very important for them to know they are doing good things). Please also indicate what can be done better (especially for kernles) as it helps authors to improve their work (current and future).
\n\nHere is small summary for current competition:\n- Data minification (https://www.kaggle.com/kyakovlev/ashrae-data-minification)\n- EDA\n- Baseline model (https://www.kaggle.com/kyakovlev/ashrae-baseline-lgbm)\n- CV options (https://www.kaggle.com/kyakovlev/ashrae-cv-options)\n- CV\n- Baseline FE \n(1.22 - https://www.kaggle.com/kyakovlev/ashrae-lgbm-simple-fe -lgbm)\n(1.23 - https://www.kaggle.com/kyakovlev/ashrae-catboost - catboost)\n same kernels with 2 lines change scores 1.13\n- Feature selection\n- Deeper FE\n- Ensembles/Stackings/Blends
\n\nRandom thoughts about competition.
\n\nThis competition is about predicting energy consumption.\nWhen I look at data I see \"not very reach\" features dataset. Seems that we can and have to use a lot of external data.
\n\nWeather - very important (here are some kernels about weather time-series data):\n* https://www.kaggle.com/leandrovrabelo/climate-change-forecast-sarima-model\n* https://www.kaggle.com/selfishgene/filtering-and-auto-correlation-tutorial\n* https://www.kaggle.com/jsphyg/weather-dataset-rattle-package\n* https://www.kaggle.com/c/recruit-restaurant-visitor-forecasting (please see discussions and winning solutions)\nWe don't predict the weather here. But if we do have? What if energy consumption has a strong connection with \"people's feelings\" about the weather tomorrow.
\n\nGeo markers (probably we can supplement data with this information).
\n\nWealth (also with categorical subsets).
\n\nBuilding condition (supplement with some thermal physics data).
\n\netc.
\n\nIt seems to me now that data that we have just a skeleton and we can and have to find interesting correlated data on external resources. \nMy advice here constantly monitor this thread:\nhttps://www.kaggle.com/c/ashrae-energy-prediction/discussion/112841#latest-649772
\n\nMore readings:\nTip: You've already noticed III number in competition name -> search for ASHRAE - Energy Predictor\n* https://www.kaggle.com/c/ashrae-energy-prediction/overview/prior-competitions\n* https://repository.upenn.edu/mlab_papers/75/ (just example of search result)\netc
\n\nOne of the first steps that should be done is data minification (just my opinion).\nYou will need it for:\n* Load data faster -> pickle format (best choice for me)\n* Consume less memory -> Carefull types downcasting\n* Avoid repeated cleaning steps.
\n\nPossible problems:\n* pandas pickle format is \"version dependent\" and you may have problems with loading data generated with a newer version of Pandas\n* Any bad transformation may and will affect your model. Minification should be done very veeeeery careful as you will use it in every single your kernel in the future. If you not sure about downcasting some column -> don't do. If not sure about cleaning -> don't do. All wrong decisions here may and will affect your score. You may spend days searching for error that prevents your model converge well and find that it was you who created an error in the minification step.
\n\nLinks:
\n\n> DON'T copy that kernels blindly. Try to understand the code and write your own that serves you. You may use that links just for reference.
\n\nhttps://www.kaggle.com/kyakovlev/ashrae-data-minification
\n\nThe baseline model is a model without any FE and without any model tuning. It will give you an idea of the very base/ground score that you can have with untransformed data. Conducting experiments you have to compare scores with some stable result.
\n\nLet's think about what ways we have to improve a single model score:\n* FE obviously\n* CV\n* Model tuning \n* Evaluation metric \n* Loss function
\n\nCV\nThink about how you can make it reliable and align your CV with LB.\nWithout good CV all modifications are \"blind shoots\" and can't be analyzed and evaluated.
\n\nModel tuning\nIt's up to you to decide when you want to do tuning, but my advice is to do at least some minimal tuning in the beginning. Normally (in most cases) model performance depends on the data structure and if it will be untouched your initial params can serve you almost until the end of the competition. In the beginning, it is easier to do as you have just a minimal number of features and any GridSearch/Bayesian optimization will run faster.
\n\nEvaluation metric/Loss function\nIn my case, I've used RMSE but the competition metric is RMSLE. If you will use lgbm please consult these examples for custom metric and custom loss function:\n* https://github.com/microsoft/LightGBM/blob/master/examples/python-guide/advanced_example.py\n* https://github.com/microsoft/LightGBM/blob/master/examples/python-guide/sklearn_example.py
\n\nOther baseline models:\n* https://www.kaggle.com/ryches/simple-lgbm-solution (by @ryches / LGBM)\n* https://www.kaggle.com/bejeweled/ashrae-catboost-regressor (by @bejeweled / CatBoost)\n* https://www.kaggle.com/tunguz/simple-linear-regression-benchmark (by @tunguz / Linear regression)\n* https://www.kaggle.com/yukari17/simple-prediction-do-not-use-model-v2 (by @yukari17 / without any Model)
\n\n> Never copy others solutions, use it as a reference and create your own baseline with model you know better. For sure it will take much more time but you will benefit much more from own code. Others approaches could have mistakes, errors, incorrect code.
\n\nBefore reading other's EDAs kernels try to make your own. If you are as lazy as I am - go to read part.
\n\nEDA is one of the most important parts of any competition. This your source for FE/CV/Model. Understanding the data and its structure is a crucial step to move on.
\n\nhttps://www.kaggle.com/kyakovlev/ashrae-cv-options
\n\nWe have several problems here (you may want to check the linked kernel).
\n\nWhat works:\n* Target/mean encoding\n* Outliers imputation\n* Categorisation
\n\nAdversarial validation shows that test set doesn't differ much from train set:\n* https://www.kaggle.com/tunguz/adversarial-ashrae (it's IMPORTANT)
\n\nWe don't have many features so for features selection you can safely use (if you have stable CV):\n* recursive feature elimination (remove 1 by 1)\n* permutation importance\n* recursive feature acceptance (add 1 by 1)
\n\nBlend works - you can have at least 1-2% of RMSLE with different model blending.
\n\nStacking is very tricky as it will overfit hugely for test set. As I see it's almost impossible to use it here. But you may find a way to implement it in your pipeline.
\n\nPrivate/Public Test set split:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113751
\n\nGeo location:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114874\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114447\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113338\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113772\n* https://www.kaggle.com/patrick0302/locate-cities-according-weather-temperature
\n\nMeter split (didn't test by it's interesting):\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113240\n* https://www.kaggle.com/corochann/ashrae-training-lgbm-by-meter-type
\n\nOutliers:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113254
\n\nHolidays:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113286
\n\nMetric:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113064
\n\nFaster submission:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114782
\n\nBetter score understanding (more than 2 decimals):\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113104
\n\nFaster stratified split\n* https://www.kaggle.com/frednavruzov/faster-stratified-cross-validation-upd
", "rawMarkdown": "I'm stopping share things as we formed a team and it will be not fair to my teammates to disclose information that may affect all team.\n\nPlease upvote and comment original topic and kernels authors if you'll find their work interesting and useful (feedback is very important for them to know they are doing good things). Please also indicate what can be done better (especially for kernles) as it helps authors to improve their work (current and future).\n\nHere is small summary for current competition:\n- Data minification (https://www.kaggle.com/kyakovlev/ashrae-data-minification)\n- EDA\n- Baseline model (https://www.kaggle.com/kyakovlev/ashrae-baseline-lgbm)\n- CV options (https://www.kaggle.com/kyakovlev/ashrae-cv-options)\n- CV\n- Baseline FE \n(1.22 - https://www.kaggle.com/kyakovlev/ashrae-lgbm-simple-fe -lgbm)\n(1.23 - https://www.kaggle.com/kyakovlev/ashrae-catboost - catboost)\n same kernels with 2 lines change scores 1.13\n- Feature selection\n- Deeper FE\n- Ensembles/Stackings/Blends\n\n----\n## Part 1\n*Random thoughts about competition.*\n\nThis competition is about predicting energy consumption.\nWhen I look at data I see \"not very reach\" features dataset. Seems that we can and have to use a lot of external data.\n\nWeather - very important (here are some kernels about weather time-series data):\n* https://www.kaggle.com/leandrovrabelo/climate-change-forecast-sarima-model\n* https://www.kaggle.com/selfishgene/filtering-and-auto-correlation-tutorial\n* https://www.kaggle.com/jsphyg/weather-dataset-rattle-package\n* https://www.kaggle.com/c/recruit-restaurant-visitor-forecasting (please see discussions and winning solutions)\nWe don't predict the weather here. But if we do have? What if energy consumption has a strong connection with \"people's feelings\" about the weather tomorrow.\n\nGeo markers (probably we can supplement data with this information).\n\nWealth (also with categorical subsets).\n\nBuilding condition (supplement with some thermal physics data).\n\netc.\n\nIt seems to me now that data that we have just a skeleton and we can and have to find interesting correlated data on external resources. \nMy advice here constantly monitor this thread:\nhttps://www.kaggle.com/c/ashrae-energy-prediction/discussion/112841#latest-649772 \n\nMore readings:\n*Tip: You've already noticed III number in competition name -> search for ASHRAE - Energy Predictor*\n* https://www.kaggle.com/c/ashrae-energy-prediction/overview/prior-competitions\n* https://repository.upenn.edu/mlab_papers/75/ (just example of search result)\netc\n\n----\n\n## Minification.\nOne of the first steps that should be done is data minification (just my opinion).\nYou will need it for:\n* Load data faster -> pickle format (best choice for me)\n* Consume less memory -> Carefull types downcasting\n* Avoid repeated cleaning steps.\n\n**Possible problems:**\n* pandas pickle format is \"version dependent\" and you may have problems with loading data generated with a newer version of Pandas\n* Any bad transformation may and will affect your model. Minification should be done very veeeeery careful as you will use it in every single your kernel in the future. If you not sure about downcasting some column -> don't do. If not sure about cleaning -> don't do. All wrong decisions here may and will affect your score. You may spend days searching for error that prevents your model converge well and find that it was you who created an error in the minification step.\n\n\n**Links:**\n\n* https://www.kaggle.com/kyakovlev/ashrae-data-minification (I do think that my approach is not the best one and not safe to use - just an example)\n* https://www.kaggle.com/hamditarek/reducing-memory-size-for-great-energy-predictor\n\n\n> **DON'T copy that kernels blindly. Try to understand the code and write your own that serves you. You may use that links just for reference.**\n\n----\n\n## Baseline\n\nhttps://www.kaggle.com/kyakovlev/ashrae-data-minification\n\nThe baseline model is a model without any FE and without any model tuning. It will give you an idea of the very base/ground score that you can have with untransformed data. Conducting experiments you have to compare scores with some stable result. \n\nLet's think about what ways we have to improve a single model score:\n* FE obviously\n* CV\n* Model tuning \n* Evaluation metric \n* Loss function\n\n\n**CV**\nThink about how you can make it reliable and align your CV with LB.\nWithout good CV all modifications are \"blind shoots\" and can't be analyzed and evaluated.\n\n\n**Model tuning**\nIt's up to you to decide when you want to do tuning, but my advice is to do at least some minimal tuning in the beginning. Normally (in most cases) model performance depends on the data structure and if it will be untouched your initial params can serve you almost until the end of the competition. In the beginning, it is easier to do as you have just a minimal number of features and any GridSearch/Bayesian optimization will run faster. \n\n\n**Evaluation metric/Loss function**\nIn my case, I've used RMSE but the competition metric is RMSLE. If you will use lgbm please consult these examples for custom metric and custom loss function:\n* https://github.com/microsoft/LightGBM/blob/master/examples/python-guide/advanced_example.py\n* https://github.com/microsoft/LightGBM/blob/master/examples/python-guide/sklearn_example.py\n\n\n**Other baseline models:**\n* https://www.kaggle.com/ryches/simple-lgbm-solution (by @ryches / LGBM)\n* https://www.kaggle.com/bejeweled/ashrae-catboost-regressor (by @bejeweled / CatBoost)\n* https://www.kaggle.com/tunguz/simple-linear-regression-benchmark (by @tunguz / Linear regression)\n* https://www.kaggle.com/yukari17/simple-prediction-do-not-use-model-v2 (by @yukari17 / without any Model)\n\n> **Never copy others solutions, use it as a reference and create your own baseline with model you know better. For sure it will take much more time but you will benefit much more from own code. Others approaches could have mistakes, errors, incorrect code.**\n\n---\n\n## EDA\n\n* https://www.kaggle.com/nroman/eda-for-ashrae\n* https://www.kaggle.com/jtrotman/ashrae-heatmaps\n* https://www.kaggle.com/sudalairajkumar/simple-exploration-notebook-ashrae\n\nBefore reading other's EDAs kernels try to make your own. If you are as lazy as I am - go to read part.\n\nEDA is one of the most important parts of any competition. This your source for FE/CV/Model. Understanding the data and its structure is a crucial step to move on.\n\n---\n\n## CV\n\nhttps://www.kaggle.com/kyakovlev/ashrae-cv-options\n\nWe have several problems here (you may want to check the linked kernel).\n\n* Consumption can't be well generalized (with data we have). For unknown site_id and building_id our predictions are terrible. The same happens with monthly consumption. It means that we should train the final model on all months and all buildings to get the most accurate predictions. You can think of it that consumption is very specific for each building, each zone, and date. Also, it's mean that we don't have all necessary data.\n* The previous point shows us the importance of site_id and building_id which seems to me should be enriched by external data.\n* Not possible to safely use early stoppings. \n* Features validation is becoming very tricky and not very accurate.\n* In my opinion, the best FE validation option is 4/4/4 train/skip/validate - if there is a proved RMSLE boost for the feature it is safe to use it for test predictions.\n* Boosting rounds for the final model should be more than for FE validation but not much.\n\n---\n\n## FE\n\nWhat works:\n* Target/mean encoding\n* Outliers imputation\n* Categorisation \n\nAdversarial validation shows that test set doesn't differ much from train set:\n* https://www.kaggle.com/tunguz/adversarial-ashrae (it's IMPORTANT)\n\nWe don't have many features so for features selection you can safely use (if you have stable CV):\n* recursive feature elimination (remove 1 by 1)\n* permutation importance\n* recursive feature acceptance (add 1 by 1)\n\n---\n\n## Blend/Stacking\n\nBlend works - you can have at least 1-2% of RMSLE with different model blending.\n\nStacking is very tricky as it will overfit hugely for test set. As I see it's almost impossible to use it here. But you may find a way to implement it in your pipeline. \n\n---\n\n## General findings\n\nPrivate/Public Test set split:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113751\n\nGeo location:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114874\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114447\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113338\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113772\n* https://www.kaggle.com/patrick0302/locate-cities-according-weather-temperature\n\nMeter split (didn't test by it's interesting):\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113240\n* https://www.kaggle.com/corochann/ashrae-training-lgbm-by-meter-type\n\nOutliers:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113254\n\nHolidays:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113286\n\nMetric:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113064\n\n---\n\n## Utilities\n\nFaster submission:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114782\n\nBetter score understanding (more than 2 decimals):\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113104\n\nFaster stratified split\n* https://www.kaggle.com/frednavruzov/faster-stratified-cross-validation-upd", "votes": null }, { "id": "650029", "postDate": "10/16/2019 02:53:14", "content": "Look forward to seeing your kernel! \nOne can really learn a lot when taking part in the same competition with you!
", "rawMarkdown": "Look forward to seeing your kernel! \nOne can really learn a lot when taking part in the same competition with you!", "votes": null }, { "id": "650238", "postDate": "10/16/2019 07:58:29", "content": "\n\n\nGold zone / 4 weeks before deadline\n Top 200 / 2 weeks before deadline
\n
IEEE all over again...\nAnything that affects medal zone should be kept private (at least during the last 3-4 weeks of the competition)
", "rawMarkdown": "> Gold zone / 4 weeks before deadline\nTop 200 / 2 weeks before deadline\n\nIEEE all over again...\nAnything that affects medal zone should be kept private (at least during the last 3-4 weeks of the competition)", "votes": null }, { "id": "650245", "postDate": "10/16/2019 08:06:33", "content": "> IEEE all over again…
\n\nIs it good or bad? I do think that 2-4 week is a safe time window to learn and adapt. I also think that kaggle is not about medals but about exchanging ideas and learn (learn during competition when you have \"mojo\" to do experiments and apply new knowledge).
", "rawMarkdown": "> IEEE all over again…\n\nIs it good or bad? I do think that 2-4 week is a safe time window to learn and adapt. I also think that kaggle is not about medals but about exchanging ideas and learn (learn during competition when you have \"mojo\" to do experiments and apply new knowledge).", "votes": null }, { "id": "650287", "postDate": "10/16/2019 08:45:18", "content": "I am looking forward to the notebooks but I do think anything that can get you in medal zone should only be shared as an idea in forums irrespective of time so that you have to work for it rather than enabling freeloaders to inflate the leaderboard. Just a thought, you can still do it as long as it is allowed by rules.
", "rawMarkdown": "I am looking forward to the notebooks but I do think anything that can get you in medal zone should only be shared as an idea in forums irrespective of time so that you have to work for it rather than enabling freeloaders to inflate the leaderboard. Just a thought, you can still do it as long as it is allowed by rules.", "votes": null }, { "id": "650297", "postDate": "10/16/2019 08:48:44", "content": "Good, that means I can drop in late again :P
", "rawMarkdown": "Good, that means I can drop in late again :P", "votes": null }, { "id": "650303", "postDate": "10/16/2019 08:56:34", "content": "\n\n\nIs it good or bad?
\n
It's weird, but it's kinda both.\nI agree with you in general, sharing and learning are good, they're what Kaggle is about.\nBut I'll also try to explain my personal position. I'm so good and fast to adapt as many here. Excluding work and everyday stuff, I have ~1-1.5 hours for kaggling per day, and not every day. Also I'm crazy dude that prefers to develop his algo's and implement new stuff by himself - not just fork-commit-submit or Nforks-combine-submit. Keeping this in mind, I have assburn everytime highscoring public kernel arrives, because I'd like to do things by myself and remain competitive. But that's my personal assburn and I have to live with it - \"if public script beats your model, than your model sucks\" :D
\n\n\n\n\nTop 200 / 2 weeks before deadline
\n
I think this competition will have many team since it's tabular. Hence, there will be, say, 500ppl in medals. If you (not \"you\" personally, just somebody) will post kernel in top200 2 weeks before deadline, it may ruin the competition for participants like me, who don't have much time to spend and makes own models.
\n\nOh, it all looks a bit messy, so, trying to summarize: I think that anything affecting medals shouldn't be posted during the last 3-4 weeks of the competition, for the sake of dudes like me. But that's my personal opinion, and I understand that I'm not alone here :)
", "rawMarkdown": "> Is it good or bad?\n\nIt's weird, but it's kinda both.\nI agree with you in general, sharing and learning are good, they're what Kaggle is about.\nBut I'll also try to explain my personal position. I'm so good and fast to adapt as many here. Excluding work and everyday stuff, I have ~1-1.5 hours for kaggling per day, and not every day. Also I'm crazy dude that prefers to develop his algo's and implement new stuff by himself - not just fork-commit-submit or Nforks-combine-submit. Keeping this in mind, I have assburn everytime highscoring public kernel arrives, because I'd like to do things by myself and remain competitive. But that's my personal assburn and I have to live with it - \"if public script beats your model, than your model sucks\" :D\n> Top 200 / 2 weeks before deadline\n\nI think this competition will have many team since it's tabular. Hence, there will be, say, 500ppl in medals. If you (not \"you\" personally, just somebody) will post kernel in top200 2 weeks before deadline, it may ruin the competition for participants like me, who don't have much time to spend and makes own models.\n\nOh, it all looks a bit messy, so, trying to summarize: I think that anything affecting medals shouldn't be posted during the last 3-4 weeks of the competition, for the sake of dudes like me. But that's my personal opinion, and I understand that I'm not alone here :)", "votes": null }, { "id": "650342", "postDate": "10/16/2019 09:37:24", "content": "Actually, the competition host always benefits from such sharing. Strong straightforward combinations are shared openly and can be further improved or used for blending. This pushes everybody to look up for new ways to tackle the problem.\nIn the end, we get more knowledge and experience. And the host gets better and more variations of solutions.
", "rawMarkdown": "Actually, the competition host always benefits from such sharing. Strong straightforward combinations are shared openly and can be further improved or used for blending. This pushes everybody to look up for new ways to tackle the problem.\nIn the end, we get more knowledge and experience. And the host gets better and more variations of solutions.", "votes": null }, { "id": "650412", "postDate": "10/16/2019 10:48:11", "content": "Great :)
", "rawMarkdown": "Great :)", "votes": null }, { "id": "650474", "postDate": "10/16/2019 12:37:49", "content": "Minification part added:\npickle is not the unique option btw.
\nYou can try use feather DataFrame.to_feather()\nor Google BigQuery table DataFrame.to_gbq() -> it would be amazing if someone can prepare BigQuery integration notebook for with competition\nor parquet DataFrame.to_parquet()
So no need to join before 2 weeks to deadline. Thanks.
", "rawMarkdown": "So no need to join before 2 weeks to deadline. Thanks.", "votes": null }, { "id": "650487", "postDate": "10/16/2019 12:41:52", "content": "@christofhenkel please join before and do same as you did for Jigsaw competition. For such newbie as me (and many other kagglers) in NLP your posts were such a great source of learning.
", "rawMarkdown": "christofhenkel please join before and do same as you did for Jigsaw competition. For such newbie as me (and many other kagglers) in NLP your posts were such a great source of learning.", "votes": null }, { "id": "650809", "postDate": "10/16/2019 17:51:57", "content": "Glad to see you here @kyakovlev This competition has a very large data set. Can you focus on some memory optimization and resampling topics this time?
", "rawMarkdown": "Glad to see you here @kyakovlev This competition has a very large data set. Can you focus on some memory optimization and resampling topics this time?", "votes": null }, { "id": "650818", "postDate": "10/16/2019 18:01:32", "content": "I'll try. Normally (IEEE competition) I'm too lazy to make huge optimisations - Base kernel preparation on kaggle kernels -> then pass all to Google Cloud and run 64cores/64GB machine predictions -> no memory issues and much faster.
\n\nOne thing I can recommend from start - use HDD space for storing \"intermediate\" results and dataframes (kaggle gives us 5GB per kernel).
", "rawMarkdown": "I'll try. Normally (IEEE competition) I'm too lazy to make huge optimisations - Base kernel preparation on kaggle kernels -> then pass all to Google Cloud and run 64cores/64GB machine predictions -> no memory issues and much faster. \n\nOne thing I can recommend from start - use HDD space for storing \"intermediate\" results and dataframes (kaggle gives us 5GB per kernel).", "votes": null }, { "id": "650821", "postDate": "10/16/2019 18:03:53", "content": "M4 Forecasting Competition winner: Hybrid Exponential Smoothing-Recurrent Neural Networks (ES-RNN)
\n\nhttps://eng.uber.com/m4-forecasting-competition/\nhttps://github.com/M4Competition/M4-methods/tree/slaweks_ES-RNN/118%20-%20slaweks17
", "rawMarkdown": "M4 Forecasting Competition winner: Hybrid Exponential Smoothing-Recurrent Neural Networks (ES-RNN)\n\nhttps://eng.uber.com/m4-forecasting-competition/\nhttps://github.com/M4Competition/M4-methods/tree/slaweks_ES-RNN/118%20-%20slaweks17", "votes": null }, { "id": "650824", "postDate": "10/16/2019 18:06:11", "content": "Interesting, but as always -> there is no Silver bullet each dataset is unique and approaches differ a lot.
", "rawMarkdown": "Interesting, but as always -> there is no Silver bullet each dataset is unique and approaches differ a lot.", "votes": null }, { "id": "651309", "postDate": "10/17/2019 10:03:38", "content": "After going through your solution in IEEE Fraud detection, I feel like I can actually do well in this. Thanks for sharing.
", "rawMarkdown": "After going through your solution in IEEE Fraud detection, I feel like I can actually do well in this. Thanks for sharing.", "votes": null }, { "id": "651606", "postDate": "10/17/2019 16:52:02", "content": "Did anyone probed public/private data split?
\n\nIs it timebased splits or random?
\n\nDidn't find information about it in competition description.
", "rawMarkdown": "Did anyone probed public/private data split?\n\nIs it timebased splits or random? \n\nDidn't find information about it in competition description.", "votes": null }, { "id": "651685", "postDate": "10/17/2019 19:45:43", "content": "Seems noone did test)) So I did it.
\n\nhttps://www.kaggle.com/kyakovlev/ashrae-split-test
\n\n> This leaderboard is calculated with approximately 78% of the test data. The final results will be based on the other 22%, so the final standings may be different.
\n\n```\nsubmission = pd.read_csv('../input/ashrae-baseline-lgbm/submission.csv')\npossible_public = int(len(submission)*0.78)
\n\nsubmission.iloc[possible_public:,1] = 0\n```
\n\nLB before - 1.36
\n\nLB after -> 2.60 -> seems it's random split or 22% first lines of data is private (need one more test).
", "rawMarkdown": "Seems noone did test)) So I did it.\n\nhttps://www.kaggle.com/kyakovlev/ashrae-split-test\n\n> This leaderboard is calculated with approximately 78% of the test data. The final results will be based on the other 22%, so the final standings may be different.\n\n```\nsubmission = pd.read_csv('../input/ashrae-baseline-lgbm/submission.csv')\npossible_public = int(len(submission)*0.78)\n\nsubmission.iloc[possible_public:,1] = 0\n```\n\nLB before - 1.36\n\nLB after -> 2.60 -> seems it's random split or 22% first lines of data is private (need one more test).", "votes": null }, { "id": "651710", "postDate": "10/17/2019 20:42:37", "content": "Was going to mention the pandas to_feather format myself. It would be nice to have a comparison between them. I will probably report one tomorrow.
@stanislavblinov, ideally I agree with everything you said and here is my take. Sharing of high scoring kernel is a mixed blessing. On one hand if one is lazy, you can use that kernel and do nothing else which is a waste of time because you may not learn anything. If shared a month before the end of a competition, I personally use it as a motivation to improve my model so that I can beat its score. On the other extreme if there is not much sharing, only hard working people with a lot of time in their hands and those with domain knowledge will do better. For those that are like me and you who is only here to learn and enhance our knowledge whilst challenging our brains, it is 50-50 good and bad :-)
\n\nBy the way I have been in some competitions where the highest scoring kernels have serious design flaws that sent everyone that used it down over 1000 places because they did not read and understand the codes. A good example is Home Credit Default Risk.
", "rawMarkdown": "stanislavblinov, ideally I agree with everything you said and here is my take. Sharing of high scoring kernel is a mixed blessing. On one hand if one is lazy, you can use that kernel and do nothing else which is a waste of time because you may not learn anything. If shared a month before the end of a competition, I personally use it as a motivation to improve my model so that I can beat its score. On the other extreme if there is not much sharing, only hard working people with a lot of time in their hands and those with domain knowledge will do better. For those that are like me and you who is only here to learn and enhance our knowledge whilst challenging our brains, it is 50-50 good and bad :-)\n\nBy the way I have been in some competitions where the highest scoring kernels have serious design flaws that sent everyone that used it down over 1000 places because they did not read and understand the codes. A good example is [Home Credit Default Risk](https://www.kaggle.com/c/home-credit-default-risk).", "votes": null }, { "id": "651768", "postDate": "10/17/2019 22:56:08", "content": "You inspire me ! Congrats for help kagglers ! Great @kyakovlev !✔️
", "rawMarkdown": "You inspire me ! Congrats for help kagglers ! Great @kyakovlev !✔️", "votes": null }, { "id": "651927", "postDate": "10/18/2019 05:11:19", "content": "the competition host always benefits from such sharing. Strong straightforward combinations are shared openly and can be further improved or used for blending. \nIn the end, we get more knowledge and experience. And the host gets better and more variations of solutions.👍 👍
", "rawMarkdown": "the competition host always benefits from such sharing. Strong straightforward combinations are shared openly and can be further improved or used for blending. \nIn the end, we get more knowledge and experience. And the host gets better and more variations of solutions.👍 👍", "votes": null }, { "id": "652239", "postDate": "10/18/2019 14:52:49", "content": "I tried \n\npossible_public = int(len(submission)*0.22)\nsubmission.iloc[:possible_public:,1] = 0\n
LB before - 1.37
\n\nLB after -> 2.53\n -> seems it's random split
", "rawMarkdown": "I tried \n```\npossible_public = int(len(submission)*0.22)\nsubmission.iloc[:possible_public:,1] = 0\n```\n\nLB before - 1.37\n\nLB after -> 2.53\n -> seems it's random split", "votes": null }, { "id": "652295", "postDate": "10/18/2019 16:31:49", "content": "Does data minification only reduces RAM usage, or it improves model training/predicting time too?
", "rawMarkdown": "Does data minification only reduces RAM usage, or it improves model training/predicting time too?", "votes": null }, { "id": "652301", "postDate": "10/18/2019 16:44:14", "content": "For tree-based models no.
\n\nIf I remember well (please correct me) fo lgbm we have such flow:\n* We have dataset with fp8 for example\n* lgbm will reserve memory for fp32\n* lgbm will train on fp8
\n\nThe question here that we need as much free memory as possible for step 2 and minification helps here.
\n\nProbably you have noticed during lgbm training how memory usage jumps in the beginning of training round - this is exactly reservation step.
\n\nNN is a different case. We can use fp16 training which is much faster (in most of cases) than fp32 training.
", "rawMarkdown": "For tree-based models no.\n\nIf I remember well (please correct me) fo lgbm we have such flow:\n* We have dataset with fp8 for example\n* lgbm will reserve memory for fp32\n* lgbm will train on fp8\n\nThe question here that we need as much free memory as possible for step 2 and minification helps here.\n\nProbably you have noticed during lgbm training how memory usage jumps in the beginning of training round - this is exactly reservation step. \n\n---\n\nNN is a different case. We can use fp16 training which is much faster (in most of cases) than fp32 training.", "votes": null }, { "id": "652325", "postDate": "10/18/2019 17:30:57", "content": "Yes, I've noticed that jump in RAM in beginning of training with lgbm, really it's necessary to have free RAM in this step. Thanks for your clarifying answer.
", "rawMarkdown": "Yes, I've noticed that jump in RAM in beginning of training with lgbm, really it's necessary to have free RAM in this step. Thanks for your clarifying answer.", "votes": null }, { "id": "652327", "postDate": "10/18/2019 17:32:35", "content": "@gunesevitan I've added example how to use hdd during kernel training.
\n\nhttps://www.kaggle.com/kyakovlev/ashrae-baseline-lgbm
\n\nThere is also example how to save model and make predictions for multiple models (kfold/seed).
", "rawMarkdown": "gunesevitan I've added example how to use hdd during kernel training.\n\nhttps://www.kaggle.com/kyakovlev/ashrae-baseline-lgbm\n\nThere is also example how to save model and make predictions for multiple models (kfold/seed).", "votes": null }, { "id": "652358", "postDate": "10/18/2019 18:19:54", "content": "Any one succeeded in using one hot encoding with categorical features ? My kernel crushes each time whenever i try
", "rawMarkdown": "Any one succeeded in using one hot encoding with categorical features ? My kernel crushes each time whenever i try", "votes": null }, { "id": "652362", "postDate": "10/18/2019 18:27:49", "content": "if you use lgbm, you can try to set df[column].astype('categorical')
https://lightgbm.readthedocs.io/en/latest/Advanced-Topics.html
\n\nCategorical Feature Support section.
\n\nMore info:\n* https://www.kaggle.com/c/ieee-fraud-detection/discussion/108575#626083\n* https://www.kaggle.com/c/ieee-fraud-detection/discussion/108575#624919
\n\nYour kernel crushes because even 1/0 (bool/int8) encoding 20M row dataset becoming HUGE in memory and there is no any reason (for current dataset) to use pure one-hot encoding when you have Catboost and Lgbm categorical option.
", "rawMarkdown": "if you use lgbm, you can try to set `df[column].astype('categorical')`\n\nhttps://lightgbm.readthedocs.io/en/latest/Advanced-Topics.html\n\nCategorical Feature Support section.\n\nMore info:\n* https://www.kaggle.com/c/ieee-fraud-detection/discussion/108575#626083\n* https://www.kaggle.com/c/ieee-fraud-detection/discussion/108575#624919\n\n---\n\nYour kernel crushes because even 1/0 (bool/int8) encoding 20M row dataset becoming HUGE in memory and there is no any reason (for current dataset) to use pure one-hot encoding when you have Catboost and Lgbm categorical option.", "votes": null }, { "id": "652387", "postDate": "10/18/2019 19:39:45", "content": "Hi @kyakovlev! First, thanks for sharing! I hope you'll do good as IEEE :)
\n\nWhat do you mean when you say:
\n\n\n\n\nFor unknown
\nsite_idandbuilding_idour predictions are terrible
Cause we have the same building and site ids in both train and test as far as my checks are correct :smile:
", "rawMarkdown": "Hi @kyakovlev! First, thanks for sharing! I hope you'll do good as IEEE :)\n\nWhat do you mean when you say:\n> For unknown `site_id` and `building_id` our predictions are terrible\n\nCause we have the same building and site ids in both train and test as far as my checks are correct :smile:", "votes": null }, { "id": "652410", "postDate": "10/18/2019 20:23:19", "content": "Great work
", "rawMarkdown": "Great work", "votes": null }, { "id": "652412", "postDate": "10/18/2019 20:25:38", "content": "You are right.
\n\n> Good thing is all our test buildings and test sites present in train set.
\n\nhttps://www.kaggle.com/kyakovlev/ashrae-cv-options
\n\n\n\n\nWhat do you mean
\n
Be careful with kfold -> if any fold will not have all building_ids and all months data it will make bad predictions for such unknown items.
", "rawMarkdown": "You are right. \n\n> Good thing is all our test buildings and test sites present in train set.\n\nhttps://www.kaggle.com/kyakovlev/ashrae-cv-options\n\n\n----\n\n> What do you mean\n\nBe careful with kfold -> if any fold will not have all building_ids and all months data it will make bad predictions for such unknown items.", "votes": null }, { "id": "652469", "postDate": "10/18/2019 22:20:24", "content": "Thanks, now I see what you meant.\nI'm currently using TimeSeriesSplit but I have to manage to find splits where all building ids are present...
", "rawMarkdown": "Thanks, now I see what you meant.\nI'm currently using TimeSeriesSplit but I have to manage to find splits where all building ids are present...", "votes": null }, { "id": "652525", "postDate": "10/19/2019 01:57:04", "content": "@kyakovlev Thanks for the insights. These are truly some important points to start with!
", "rawMarkdown": "kyakovlev Thanks for the insights. These are truly some important points to start with!", "votes": null }, { "id": "652822", "postDate": "10/19/2019 13:06:57", "content": "Pandas to feather doesn't yet work with float16, so it wont work if you're downcasting data to float16. Be mindful
", "rawMarkdown": "Pandas to feather doesn't yet work with float16, so it wont work if you're downcasting data to float16. Be mindful", "votes": null }, { "id": "652904", "postDate": "10/19/2019 15:52:00", "content": "Feather format based on parquet and it also has no support for float16. But it's fast.
\n\nThe types supported by the file format are intended to be as minimal as possible, with a focus on how the types effect on disk storage. For example, 16-bit ints are not explicitly supported in the storage format since they are covered by 32-bit ints with an efficient encoding. This reduces the complexity of implementing readers and writers for the format. The types are:
\n\nBOOLEAN: 1 bit boolean\nINT32: 32 bit signed ints\nINT64: 64 bit signed ints\nINT96: 96 bit signed ints\nFLOAT: IEEE 32-bit floating point values\nDOUBLE: IEEE 64-bit floating point values\nBYTE_ARRAY: arbitrarily long byte arrays.
\n\nSo you may want to change memory_reducer function to use 32bits minimum formats.
", "rawMarkdown": "Feather format based on parquet and it also has no support for float16. But it's fast.\n\nThe types supported by the file format are intended to be as minimal as possible, with a focus on how the types effect on disk storage. For example, 16-bit ints are not explicitly supported in the storage format since they are covered by 32-bit ints with an efficient encoding. This reduces the complexity of implementing readers and writers for the format. The types are:\n\nBOOLEAN: 1 bit boolean\nINT32: 32 bit signed ints\nINT64: 64 bit signed ints\nINT96: 96 bit signed ints\nFLOAT: IEEE 32-bit floating point values\nDOUBLE: IEEE 64-bit floating point values\nBYTE_ARRAY: arbitrarily long byte arrays.\n\n---\n\nSo you may want to change memory_reducer function to use 32bits minimum formats.", "votes": null }, { "id": "652928", "postDate": "10/19/2019 16:17:29", "content": "> This leaderboard is calculated with approximately 78% of the test data. The final results will be based on the other 22%, so the final standings may be different.
\n\n@d1348k , @kyakovlev, if multiply *0.22 in local validation predictions, help to generalize final results ? LB_before = int(len(submission_LB_before )*0.22), to generalize results for private dataset. ✨
thx a lot!
", "rawMarkdown": "thx a lot!", "votes": null }, { "id": "653187", "postDate": "10/20/2019 03:12:31", "content": "The spike occurs because when you create your lgb datasets, you're likely passing in a pandas dataframe. LGBM's python api converts everything to float64 IIRC. Try passing in a raw numpy array, and you might not see that occur. There's also a flag you can pass where it'll first write to disk rather than creating another copy of the dataframe in ram.
", "rawMarkdown": "The spike occurs because when you create your lgb datasets, you're likely passing in a pandas dataframe. LGBM's python api converts everything to float64 IIRC. Try passing in a raw numpy array, and you might not see that occur. There's also a flag you can pass where it'll first write to disk rather than creating another copy of the dataframe in ram.", "votes": null }, { "id": "653589", "postDate": "10/20/2019 17:26:35", "content": "Great read thanks a lot, I'm pretty sure you'll do similar to IEEE , all the best!
", "rawMarkdown": "Great read thanks a lot, I'm pretty sure you'll do similar to IEEE , all the best!", "votes": null }, { "id": "653625", "postDate": "10/20/2019 18:13:47", "content": "Oh wow. Thanks for putting all of this together. I also very much appreciate your active commenting on all the questions / comments! 👍
\n\nSuper helpful...
", "rawMarkdown": "Oh wow. Thanks for putting all of this together. I also very much appreciate your active commenting on all the questions / comments! 👍 \n\nSuper helpful...", "votes": null }, { "id": "653632", "postDate": "10/20/2019 18:27:48", "content": "A logical split for this would be by building_id - so maybe we looking at only 22% of the buildings.
\n\nHow hard is that assumption to probe?
", "rawMarkdown": "A logical split for this would be by building_id - so maybe we looking at only 22% of the buildings.\n\nHow hard is that assumption to probe?", "votes": null }, { "id": "653990", "postDate": "10/21/2019 09:16:07", "content": "Thanks for sharing!
", "rawMarkdown": "Thanks for sharing!", "votes": null }, { "id": "654282", "postDate": "10/21/2019 16:50:11", "content": "Thanks for sharing. You help me to understand a lot
", "rawMarkdown": "Thanks for sharing. You help me to understand a lot", "votes": null }, { "id": "654337", "postDate": "10/21/2019 18:27:26", "content": "Gilfoyle you should be working on the new internet :D
", "rawMarkdown": "Gilfoyle you should be working on the new internet :D", "votes": null }, { "id": "654353", "postDate": "10/21/2019 18:52:19", "content": "Thanks for the sharing info.\nThat solved some of my annoying.
", "rawMarkdown": "Thanks for the sharing info.\nThat solved some of my annoying.", "votes": null }, { "id": "654406", "postDate": "10/21/2019 20:29:46", "content": "thanks for sharing info.
", "rawMarkdown": "thanks for sharing info.", "votes": null }, { "id": "654597", "postDate": "10/22/2019 04:28:31", "content": "Here is a blog post which compares feather, md5, parquet, pickle etc. And it shows that feather is the most optimized one. I am using feather for sometime (both at work and projects) and found it to be good (Didn't do any comparison though).
\n\nhttps://towardsdatascience.com/the-best-format-to-save-pandas-data-414dca023e0d
", "rawMarkdown": "Here is a blog post which compares feather, md5, parquet, pickle etc. And it shows that feather is the most optimized one. I am using feather for sometime (both at work and projects) and found it to be good (Didn't do any comparison though).\n\nhttps://towardsdatascience.com/the-best-format-to-save-pandas-data-414dca023e0d", "votes": null }, { "id": "654606", "postDate": "10/22/2019 04:47:28", "content": "Thanks about this revelation . I always used to think that everyone else is using kaggle kernel efficiently and I am the only one who find it difficult to use the kaggle kernel because memory/CPU limitation and random failures; May be I process is not efficient enough!
", "rawMarkdown": "Thanks about this revelation . I always used to think that everyone else is using kaggle kernel efficiently and I am the only one who find it difficult to use the kaggle kernel because memory/CPU limitation and random failures; May be I process is not efficient enough!", "votes": null }, { "id": "655037", "postDate": "10/22/2019 16:12:36", "content": "Thanks for sharing! It seems to be very useful!:)
", "rawMarkdown": "Thanks for sharing! It seems to be very useful!:)", "votes": null }, { "id": "655919", "postDate": "10/23/2019 17:09:56", "content": "I don't understand this multiply thing and how you people come to conclusion. Can anyone please explain?
", "rawMarkdown": "I don't understand this multiply thing and how you people come to conclusion. Can anyone please explain?", "votes": null }, { "id": "656074", "postDate": "10/23/2019 21:56:30", "content": "In general case if leaderboard splitting is not randomized (not randomized means that first 78% of data is public and last 22% is private) then turning to zeroes private part (last 22%) of submission would not cause LB decreasing
", "rawMarkdown": "In general case if leaderboard splitting is not randomized (not randomized means that first 78% of data is public and last 22% is private) then turning to zeroes private part (last 22%) of submission would not cause LB decreasing", "votes": null }, { "id": "656081", "postDate": "10/23/2019 22:12:03", "content": "And I was wrong.
\n\nplease see and upvote https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113751
\n\n\n\n\n(test set and row_id column are not strictly time ordered).
\n
It's very important!
", "rawMarkdown": "And I was wrong.\n\nplease see and upvote https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113751\n\n> (test set and row_id column are not strictly time ordered).\n\nIt's very important!", "votes": null }, { "id": "656146", "postDate": "10/24/2019 00:30:23", "content": "Thanks for sharing.
", "rawMarkdown": "Thanks for sharing.", "votes": null }, { "id": "657087", "postDate": "10/24/2019 23:10:51", "content": "Thank you so much! I'm participating in this competition for a project in my data mining course, and this is really helpful for planning it.
", "rawMarkdown": "Thank you so much! I'm participating in this competition for a project in my data mining course, and this is really helpful for planning it.", "votes": null }, { "id": "657585", "postDate": "10/25/2019 09:12:25", "content": "Thanks a lot for sharing!
", "rawMarkdown": "Thanks a lot for sharing!", "votes": null }, { "id": "658104", "postDate": "10/25/2019 17:02:42", "content": "added two models with simple fe:\n(1.22 - https://www.kaggle.com/kyakovlev/ashrae-lgbm-simple-fe -lgbm)\n(1.23 - https://www.kaggle.com/kyakovlev/ashrae-catboost - catboost)
", "rawMarkdown": "added two models with simple fe:\n(1.22 - https://www.kaggle.com/kyakovlev/ashrae-lgbm-simple-fe -lgbm)\n(1.23 - https://www.kaggle.com/kyakovlev/ashrae-catboost - catboost)", "votes": null }, { "id": "658112", "postDate": "10/25/2019 17:10:39", "content": "@kyakovlev considering both of them are doing good , did you try ensembling them ?
", "rawMarkdown": "kyakovlev considering both of them are doing good , did you try ensembling them ?", "votes": null }, { "id": "658114", "postDate": "10/25/2019 17:13:04", "content": "I have xgb gpu ready also))) I'll submit it tomorrow and blend probe.
\n\nThe reason to publish 3 models and blend:\n* Show tiny differences in models train (categoricals/Nans/params/save/load)\n* Show how much score boost blend can give
", "rawMarkdown": "I have xgb gpu ready also))) I'll submit it tomorrow and blend probe.\n\nThe reason to publish 3 models and blend:\n* Show tiny differences in models train (categoricals/Nans/params/save/load)\n* Show how much score boost blend can give", "votes": null }, { "id": "658120", "postDate": "10/25/2019 17:22:16", "content": "I can literally visualize you doing this after ensembling the models tomorrow. \n
Yes, blend works. 1.20 lb for lgbm (1.22) and catboost (1.23) - blend 50/50
", "rawMarkdown": "Yes, blend works. 1.20 lb for lgbm (1.22) and catboost (1.23) - blend 50/50", "votes": null }, { "id": "659018", "postDate": "10/27/2019 00:19:48", "content": "Also same kernels with 2 lines change scores 1.13 and blend should give 1.10 +-
\n\nI just don't think it's fair to share such high score kernels right now.
", "rawMarkdown": "Also same kernels with 2 lines change scores 1.13 and blend should give 1.10 +-\n\nI just don't think it's fair to share such high score kernels right now.", "votes": null }, { "id": "659095", "postDate": "10/27/2019 04:23:36", "content": "Yah right .
\n\n\n\n\nwith 2 lines change
\n
Wow !!
", "rawMarkdown": "Yah right .\n> with 2 lines change\n\nWow !!", "votes": null }, { "id": "661306", "postDate": "10/30/2019 05:37:15", "content": "it is a great share， I have learned a lot from it
", "rawMarkdown": "it is a great share， I have learned a lot from it", "votes": null }, { "id": "661872", "postDate": "10/30/2019 19:54:52", "content": "As allways very impressive !
\n\nThank you so muuch !
", "rawMarkdown": "As allways very impressive !\n\nThank you so muuch !", "votes": null }, { "id": "662439", "postDate": "10/31/2019 14:56:50", "content": "We formed a team. Topic will be frozen for a while. Thank you for all your support.
\n\nGood luck to you.
\n\nMay the Kaggle be with you.
", "rawMarkdown": "We formed a team. Topic will be frozen for a while. Thank you for all your support. \n\nGood luck to you.\n\nMay the Kaggle be with you.", "votes": null }, { "id": "665288", "postDate": "11/04/2019 21:08:53", "content": "Moreover, the fact that building_idis present in both datasets (training and test) and is identical there, means, that models are tuned separately to an each building. Here, if building_idwere not present, and there were only the parameters of different buildings, weather data, etc., that is, a model for an average building, then it really was an universal model. As for me, this is a disadvantage of setting the task of the contest itself. The contest authors will receive not an universal model for their energy modernization tasks, but a one and a half thousand models for individual buildings.
I did an automatic FE using a Featuretools library (see ASHRAE - Automatic FE: Featuretools & Selection FE). The analysis showed that in addition to the usual min/max/sum etc. for features, it is advisable to consider the MODE of some meteorological features. I tried to automate this:
df['data_mode'] = group_df[col].rolling(window=window, min_periods=0).apply(lambda x: mode(x)[0][0], raw=True)
But it doesn't work, unfortunately (both mode(x)[0] and mode(x)[0][0]). Someone else might be able to use it in another way
The format of the most of meteorological features can be optimize by rounding up data to int8 or uint8 without losing precision (see my kernel Very significant safe memory + LightGBM):
\n\ndf['air_temperature'] = np.int8(round(2*df['air_temperature'],0)) # store values with precision 0.5
df['cloud_coverage'] = np.uint8(round(10*df['cloud_coverage'],0)) # store values with precision 0.1
df['dew_temperature'] = np.int8(round(5*df['dew_temperature'],0)) # store values with precision 0.2
df['precip_depth_1_hr'] = np.uint8(np.clip(round(df['precip_depth_1_hr'],0),0,255)) # transform [-1,343] to [0,255]
Very nice !
\n\nThank you very much for all your effort to share on this competition!
\n\nYou have been so helpfull during all past days.
", "rawMarkdown": "Very nice ! \n\nThank you very much for all your effort to share on this competition! \n\nYou have been so helpfull during all past days.", "votes": null }, { "id": "667999", "postDate": "11/07/2019 21:30:12", "content": "@yourssharmaji \nthey are taking 78% (in train case) and 22% (in validation case) of the rows in 'submission.csv' file. Now, they cross-check their position's using 'iloc' in submission.csv to esure if the public/private split is in pattern(time-based) or random...
@kyakovlev looks like it is time to become a master in datasets ;)
", "rawMarkdown": "kyakovlev looks like it is time to become a master in datasets ;)", "votes": null }, { "id": "670800", "postDate": "11/11/2019 21:35:17", "content": "That's the plan. UCF meter readings
", "rawMarkdown": "That's the plan. [UCF meter readings](https://www.kaggle.com/teeyee314/ucf-building-meter-reading)", "votes": null }, { "id": "677905", "postDate": "11/20/2019 18:42:37", "content": "I have a confusion how to predict on test set which does not have the target encoding column that we created in the train set??\nI have created target encoding features based on meter_reading value in the train set with respect to site_id and building_id
Great job ! Thanks for sharing!
", "rawMarkdown": "Great job ! Thanks for sharing!", "votes": null }, { "id": "679958", "postDate": "11/23/2019 18:12:57", "content": "Nice post! Keep it up!
", "rawMarkdown": "Nice post! Keep it up!", "votes": null }, { "id": "690315", "postDate": "12/08/2019 11:24:15", "content": "Very useful!!
", "rawMarkdown": "Very useful!!", "votes": null }, { "id": "695310", "postDate": "12/14/2019 23:44:04", "content": "Thanks Very much, your summaries always provide so much guidance, appreciated.
", "rawMarkdown": "Thanks Very much, your summaries always provide so much guidance, appreciated.", "votes": null }, { "id": "3068939", "postDate": "12/10/2024 20:57:28", "content": "Great post! Thanks!
", "rawMarkdown": "Great post! Thanks!", "votes": null } ], "comments": [ { "id": 3068939, "author_name": "thomaszengerle", "author_url": "", "post_date": "12/10/2024 20:57:28", "content": "Great post! Thanks!
", "votes": null, "replies": [] }, { "id": 650029, "author_name": "alex495240115", "author_url": "", "post_date": "10/16/2019 02:53:14", "content": "Look forward to seeing your kernel! \nOne can really learn a lot when taking part in the same competition with you!
", "votes": null, "replies": [] }, { "id": 650238, "author_name": "stanislavblinov", "author_url": "", "post_date": "10/16/2019 07:58:29", "content": "\n\n\nGold zone / 4 weeks before deadline\n Top 200 / 2 weeks before deadline
\n
IEEE all over again...\nAnything that affects medal zone should be kept private (at least during the last 3-4 weeks of the competition)
", "votes": null, "replies": [ { "id": 650245, "author_name": "kyakovlev", "author_url": "", "post_date": "10/16/2019 08:06:33", "content": "> IEEE all over again…
\n\nIs it good or bad? I do think that 2-4 week is a safe time window to learn and adapt. I also think that kaggle is not about medals but about exchanging ideas and learn (learn during competition when you have \"mojo\" to do experiments and apply new knowledge).
", "votes": null, "replies": [] }, { "id": 650303, "author_name": "stanislavblinov", "author_url": "", "post_date": "10/16/2019 08:56:34", "content": "\n\n\nIs it good or bad?
\n
It's weird, but it's kinda both.\nI agree with you in general, sharing and learning are good, they're what Kaggle is about.\nBut I'll also try to explain my personal position. I'm so good and fast to adapt as many here. Excluding work and everyday stuff, I have ~1-1.5 hours for kaggling per day, and not every day. Also I'm crazy dude that prefers to develop his algo's and implement new stuff by himself - not just fork-commit-submit or Nforks-combine-submit. Keeping this in mind, I have assburn everytime highscoring public kernel arrives, because I'd like to do things by myself and remain competitive. But that's my personal assburn and I have to live with it - \"if public script beats your model, than your model sucks\" :D
\n\n\n\n\nTop 200 / 2 weeks before deadline
\n
I think this competition will have many team since it's tabular. Hence, there will be, say, 500ppl in medals. If you (not \"you\" personally, just somebody) will post kernel in top200 2 weeks before deadline, it may ruin the competition for participants like me, who don't have much time to spend and makes own models.
\n\nOh, it all looks a bit messy, so, trying to summarize: I think that anything affecting medals shouldn't be posted during the last 3-4 weeks of the competition, for the sake of dudes like me. But that's my personal opinion, and I understand that I'm not alone here :)
", "votes": null, "replies": [] }, { "id": 650342, "author_name": "criskiev", "author_url": "", "post_date": "10/16/2019 09:37:24", "content": "Actually, the competition host always benefits from such sharing. Strong straightforward combinations are shared openly and can be further improved or used for blending. This pushes everybody to look up for new ways to tackle the problem.\nIn the end, we get more knowledge and experience. And the host gets better and more variations of solutions.
", "votes": null, "replies": [] }, { "id": 651739, "author_name": "sheriytm", "author_url": "", "post_date": "10/17/2019 21:47:35", "content": "@stanislavblinov, ideally I agree with everything you said and here is my take. Sharing of high scoring kernel is a mixed blessing. On one hand if one is lazy, you can use that kernel and do nothing else which is a waste of time because you may not learn anything. If shared a month before the end of a competition, I personally use it as a motivation to improve my model so that I can beat its score. On the other extreme if there is not much sharing, only hard working people with a lot of time in their hands and those with domain knowledge will do better. For those that are like me and you who is only here to learn and enhance our knowledge whilst challenging our brains, it is 50-50 good and bad :-)
\n\nBy the way I have been in some competitions where the highest scoring kernels have serious design flaws that sent everyone that used it down over 1000 places because they did not read and understand the codes. A good example is Home Credit Default Risk.
", "votes": null, "replies": [] } ] }, { "id": 650287, "author_name": "satian", "author_url": "", "post_date": "10/16/2019 08:45:18", "content": "I am looking forward to the notebooks but I do think anything that can get you in medal zone should only be shared as an idea in forums irrespective of time so that you have to work for it rather than enabling freeloaders to inflate the leaderboard. Just a thought, you can still do it as long as it is allowed by rules.
", "votes": null, "replies": [] }, { "id": 650297, "author_name": "philippsinger", "author_url": "", "post_date": "10/16/2019 08:48:44", "content": "Good, that means I can drop in late again :P
", "votes": null, "replies": [] }, { "id": 650412, "author_name": "basudab", "author_url": "", "post_date": "10/16/2019 10:48:11", "content": "Great :)
", "votes": null, "replies": [] }, { "id": 650474, "author_name": "kyakovlev", "author_url": "", "post_date": "10/16/2019 12:37:49", "content": "Minification part added:\npickle is not the unique option btw.
\nYou can try use feather DataFrame.to_feather()\nor Google BigQuery table DataFrame.to_gbq() -> it would be amazing if someone can prepare BigQuery integration notebook for with competition\nor parquet DataFrame.to_parquet()
Was going to mention the pandas to_feather format myself. It would be nice to have a comparison between them. I will probably report one tomorrow.
Pandas to feather doesn't yet work with float16, so it wont work if you're downcasting data to float16. Be mindful
", "votes": null, "replies": [] }, { "id": 652904, "author_name": "kyakovlev", "author_url": "", "post_date": "10/19/2019 15:52:00", "content": "Feather format based on parquet and it also has no support for float16. But it's fast.
\n\nThe types supported by the file format are intended to be as minimal as possible, with a focus on how the types effect on disk storage. For example, 16-bit ints are not explicitly supported in the storage format since they are covered by 32-bit ints with an efficient encoding. This reduces the complexity of implementing readers and writers for the format. The types are:
\n\nBOOLEAN: 1 bit boolean\nINT32: 32 bit signed ints\nINT64: 64 bit signed ints\nINT96: 96 bit signed ints\nFLOAT: IEEE 32-bit floating point values\nDOUBLE: IEEE 64-bit floating point values\nBYTE_ARRAY: arbitrarily long byte arrays.
\n\nSo you may want to change memory_reducer function to use 32bits minimum formats.
", "votes": null, "replies": [] }, { "id": 654597, "author_name": "arnabbiswas1", "author_url": "", "post_date": "10/22/2019 04:28:31", "content": "Here is a blog post which compares feather, md5, parquet, pickle etc. And it shows that feather is the most optimized one. I am using feather for sometime (both at work and projects) and found it to be good (Didn't do any comparison though).
\n\nhttps://towardsdatascience.com/the-best-format-to-save-pandas-data-414dca023e0d
", "votes": null, "replies": [] } ] }, { "id": 650476, "author_name": "christofhenkel", "author_url": "", "post_date": "10/16/2019 12:38:20", "content": "So no need to join before 2 weeks to deadline. Thanks.
", "votes": null, "replies": [ { "id": 650487, "author_name": "kyakovlev", "author_url": "", "post_date": "10/16/2019 12:41:52", "content": "@christofhenkel please join before and do same as you did for Jigsaw competition. For such newbie as me (and many other kagglers) in NLP your posts were such a great source of learning.
", "votes": null, "replies": [] } ] }, { "id": 650809, "author_name": "gunesevitan", "author_url": "", "post_date": "10/16/2019 17:51:57", "content": "Glad to see you here @kyakovlev This competition has a very large data set. Can you focus on some memory optimization and resampling topics this time?
", "votes": null, "replies": [ { "id": 650818, "author_name": "kyakovlev", "author_url": "", "post_date": "10/16/2019 18:01:32", "content": "I'll try. Normally (IEEE competition) I'm too lazy to make huge optimisations - Base kernel preparation on kaggle kernels -> then pass all to Google Cloud and run 64cores/64GB machine predictions -> no memory issues and much faster.
\n\nOne thing I can recommend from start - use HDD space for storing \"intermediate\" results and dataframes (kaggle gives us 5GB per kernel).
", "votes": null, "replies": [] }, { "id": 652327, "author_name": "kyakovlev", "author_url": "", "post_date": "10/18/2019 17:32:35", "content": "@gunesevitan I've added example how to use hdd during kernel training.
\n\nhttps://www.kaggle.com/kyakovlev/ashrae-baseline-lgbm
\n\nThere is also example how to save model and make predictions for multiple models (kfold/seed).
", "votes": null, "replies": [] }, { "id": 654606, "author_name": "arnabbiswas1", "author_url": "", "post_date": "10/22/2019 04:47:28", "content": "Thanks about this revelation . I always used to think that everyone else is using kaggle kernel efficiently and I am the only one who find it difficult to use the kaggle kernel because memory/CPU limitation and random failures; May be I process is not efficient enough!
", "votes": null, "replies": [] } ] }, { "id": 650821, "author_name": "mileta1976", "author_url": "", "post_date": "10/16/2019 18:03:53", "content": "M4 Forecasting Competition winner: Hybrid Exponential Smoothing-Recurrent Neural Networks (ES-RNN)
\n\nhttps://eng.uber.com/m4-forecasting-competition/\nhttps://github.com/M4Competition/M4-methods/tree/slaweks_ES-RNN/118%20-%20slaweks17
", "votes": null, "replies": [ { "id": 650824, "author_name": "kyakovlev", "author_url": "", "post_date": "10/16/2019 18:06:11", "content": "Interesting, but as always -> there is no Silver bullet each dataset is unique and approaches differ a lot.
", "votes": null, "replies": [] } ] }, { "id": 651309, "author_name": "xceptions", "author_url": "", "post_date": "10/17/2019 10:03:38", "content": "After going through your solution in IEEE Fraud detection, I feel like I can actually do well in this. Thanks for sharing.
", "votes": null, "replies": [] }, { "id": 651606, "author_name": "kyakovlev", "author_url": "", "post_date": "10/17/2019 16:52:02", "content": "Did anyone probed public/private data split?
\n\nIs it timebased splits or random?
\n\nDidn't find information about it in competition description.
", "votes": null, "replies": [ { "id": 651685, "author_name": "kyakovlev", "author_url": "", "post_date": "10/17/2019 19:45:43", "content": "Seems noone did test)) So I did it.
\n\nhttps://www.kaggle.com/kyakovlev/ashrae-split-test
\n\n> This leaderboard is calculated with approximately 78% of the test data. The final results will be based on the other 22%, so the final standings may be different.
\n\n```\nsubmission = pd.read_csv('../input/ashrae-baseline-lgbm/submission.csv')\npossible_public = int(len(submission)*0.78)
\n\nsubmission.iloc[possible_public:,1] = 0\n```
\n\nLB before - 1.36
\n\nLB after -> 2.60 -> seems it's random split or 22% first lines of data is private (need one more test).
", "votes": null, "replies": [] }, { "id": 652239, "author_name": "d1348k", "author_url": "", "post_date": "10/18/2019 14:52:49", "content": "I tried \n\npossible_public = int(len(submission)*0.22)\nsubmission.iloc[:possible_public:,1] = 0\n
LB before - 1.37
\n\nLB after -> 2.53\n -> seems it's random split
", "votes": null, "replies": [] }, { "id": 652928, "author_name": "caesarlupum", "author_url": "", "post_date": "10/19/2019 16:17:29", "content": "> This leaderboard is calculated with approximately 78% of the test data. The final results will be based on the other 22%, so the final standings may be different.
\n\n@d1348k , @kyakovlev, if multiply *0.22 in local validation predictions, help to generalize final results ? LB_before = int(len(submission_LB_before )*0.22), to generalize results for private dataset. ✨
A logical split for this would be by building_id - so maybe we looking at only 22% of the buildings.
\n\nHow hard is that assumption to probe?
", "votes": null, "replies": [] }, { "id": 655919, "author_name": "yourssharmaji", "author_url": "", "post_date": "10/23/2019 17:09:56", "content": "I don't understand this multiply thing and how you people come to conclusion. Can anyone please explain?
", "votes": null, "replies": [] }, { "id": 656074, "author_name": "sand9888", "author_url": "", "post_date": "10/23/2019 21:56:30", "content": "In general case if leaderboard splitting is not randomized (not randomized means that first 78% of data is public and last 22% is private) then turning to zeroes private part (last 22%) of submission would not cause LB decreasing
", "votes": null, "replies": [] }, { "id": 656081, "author_name": "kyakovlev", "author_url": "", "post_date": "10/23/2019 22:12:03", "content": "And I was wrong.
\n\nplease see and upvote https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113751
\n\n\n\n\n(test set and row_id column are not strictly time ordered).
\n
It's very important!
", "votes": null, "replies": [] }, { "id": 667999, "author_name": "atulanandjha", "author_url": "", "post_date": "11/07/2019 21:30:12", "content": "@yourssharmaji \nthey are taking 78% (in train case) and 22% (in validation case) of the rows in 'submission.csv' file. Now, they cross-check their position's using 'iloc' in submission.csv to esure if the public/private split is in pattern(time-based) or random...
You inspire me ! Congrats for help kagglers ! Great @kyakovlev !✔️
", "votes": null, "replies": [] }, { "id": 651927, "author_name": "kuldeepsingharya", "author_url": "", "post_date": "10/18/2019 05:11:19", "content": "the competition host always benefits from such sharing. Strong straightforward combinations are shared openly and can be further improved or used for blending. \nIn the end, we get more knowledge and experience. And the host gets better and more variations of solutions.👍 👍
", "votes": null, "replies": [] }, { "id": 652295, "author_name": "endrewrossin", "author_url": "", "post_date": "10/18/2019 16:31:49", "content": "Does data minification only reduces RAM usage, or it improves model training/predicting time too?
", "votes": null, "replies": [ { "id": 652301, "author_name": "kyakovlev", "author_url": "", "post_date": "10/18/2019 16:44:14", "content": "For tree-based models no.
\n\nIf I remember well (please correct me) fo lgbm we have such flow:\n* We have dataset with fp8 for example\n* lgbm will reserve memory for fp32\n* lgbm will train on fp8
\n\nThe question here that we need as much free memory as possible for step 2 and minification helps here.
\n\nProbably you have noticed during lgbm training how memory usage jumps in the beginning of training round - this is exactly reservation step.
\n\nNN is a different case. We can use fp16 training which is much faster (in most of cases) than fp32 training.
", "votes": null, "replies": [] }, { "id": 652325, "author_name": "endrewrossin", "author_url": "", "post_date": "10/18/2019 17:30:57", "content": "Yes, I've noticed that jump in RAM in beginning of training with lgbm, really it's necessary to have free RAM in this step. Thanks for your clarifying answer.
", "votes": null, "replies": [] }, { "id": 653187, "author_name": "authman", "author_url": "", "post_date": "10/20/2019 03:12:31", "content": "The spike occurs because when you create your lgb datasets, you're likely passing in a pandas dataframe. LGBM's python api converts everything to float64 IIRC. Try passing in a raw numpy array, and you might not see that occur. There's also a flag you can pass where it'll first write to disk rather than creating another copy of the dataframe in ram.
", "votes": null, "replies": [] } ] }, { "id": 652358, "author_name": "khairulislam", "author_url": "", "post_date": "10/18/2019 18:19:54", "content": "Any one succeeded in using one hot encoding with categorical features ? My kernel crushes each time whenever i try
", "votes": null, "replies": [ { "id": 652362, "author_name": "kyakovlev", "author_url": "", "post_date": "10/18/2019 18:27:49", "content": "if you use lgbm, you can try to set df[column].astype('categorical')
https://lightgbm.readthedocs.io/en/latest/Advanced-Topics.html
\n\nCategorical Feature Support section.
\n\nMore info:\n* https://www.kaggle.com/c/ieee-fraud-detection/discussion/108575#626083\n* https://www.kaggle.com/c/ieee-fraud-detection/discussion/108575#624919
\n\nYour kernel crushes because even 1/0 (bool/int8) encoding 20M row dataset becoming HUGE in memory and there is no any reason (for current dataset) to use pure one-hot encoding when you have Catboost and Lgbm categorical option.
", "votes": null, "replies": [] } ] }, { "id": 652387, "author_name": "raimonds1993", "author_url": "", "post_date": "10/18/2019 19:39:45", "content": "Hi @kyakovlev! First, thanks for sharing! I hope you'll do good as IEEE :)
\n\nWhat do you mean when you say:
\n\n\n\n\nFor unknown
\nsite_idandbuilding_idour predictions are terrible
Cause we have the same building and site ids in both train and test as far as my checks are correct :smile:
", "votes": null, "replies": [ { "id": 652412, "author_name": "kyakovlev", "author_url": "", "post_date": "10/18/2019 20:25:38", "content": "You are right.
\n\n> Good thing is all our test buildings and test sites present in train set.
\n\nhttps://www.kaggle.com/kyakovlev/ashrae-cv-options
\n\n\n\n\nWhat do you mean
\n
Be careful with kfold -> if any fold will not have all building_ids and all months data it will make bad predictions for such unknown items.
", "votes": null, "replies": [] }, { "id": 652469, "author_name": "raimonds1993", "author_url": "", "post_date": "10/18/2019 22:20:24", "content": "Thanks, now I see what you meant.\nI'm currently using TimeSeriesSplit but I have to manage to find splits where all building ids are present...
", "votes": null, "replies": [] }, { "id": 665288, "author_name": "vbmokin", "author_url": "", "post_date": "11/04/2019 21:08:53", "content": "Moreover, the fact that building_idis present in both datasets (training and test) and is identical there, means, that models are tuned separately to an each building. Here, if building_idwere not present, and there were only the parameters of different buildings, weather data, etc., that is, a model for an average building, then it really was an universal model. As for me, this is a disadvantage of setting the task of the contest itself. The contest authors will receive not an universal model for their energy modernization tasks, but a one and a half thousand models for individual buildings.
Great work
", "votes": null, "replies": [] }, { "id": 652525, "author_name": "adityajumde", "author_url": "", "post_date": "10/19/2019 01:57:04", "content": "@kyakovlev Thanks for the insights. These are truly some important points to start with!
", "votes": null, "replies": [] }, { "id": 653124, "author_name": "leozhudd", "author_url": "", "post_date": "10/20/2019 01:17:58", "content": "thx a lot!
", "votes": null, "replies": [] }, { "id": 653589, "author_name": "frtgnn", "author_url": "", "post_date": "10/20/2019 17:26:35", "content": "Great read thanks a lot, I'm pretty sure you'll do similar to IEEE , all the best!
", "votes": null, "replies": [] }, { "id": 653625, "author_name": "chmaxx", "author_url": "", "post_date": "10/20/2019 18:13:47", "content": "Oh wow. Thanks for putting all of this together. I also very much appreciate your active commenting on all the questions / comments! 👍
\n\nSuper helpful...
", "votes": null, "replies": [] }, { "id": 653990, "author_name": "mpanfil", "author_url": "", "post_date": "10/21/2019 09:16:07", "content": "Thanks for sharing!
", "votes": null, "replies": [] }, { "id": 654282, "author_name": "", "author_url": "", "post_date": "10/21/2019 16:50:11", "content": "Thanks for sharing. You help me to understand a lot
", "votes": null, "replies": [ { "id": 654337, "author_name": "gunesevitan", "author_url": "", "post_date": "10/21/2019 18:27:26", "content": "Gilfoyle you should be working on the new internet :D
", "votes": null, "replies": [] } ] }, { "id": 654353, "author_name": "takach", "author_url": "", "post_date": "10/21/2019 18:52:19", "content": "Thanks for the sharing info.\nThat solved some of my annoying.
", "votes": null, "replies": [] }, { "id": 654406, "author_name": "narain21", "author_url": "", "post_date": "10/21/2019 20:29:46", "content": "thanks for sharing info.
", "votes": null, "replies": [] }, { "id": 655037, "author_name": "vovanf98", "author_url": "", "post_date": "10/22/2019 16:12:36", "content": "Thanks for sharing! It seems to be very useful!:)
", "votes": null, "replies": [] }, { "id": 656146, "author_name": "narain21", "author_url": "", "post_date": "10/24/2019 00:30:23", "content": "Thanks for sharing.
", "votes": null, "replies": [] }, { "id": 657087, "author_name": "jamesss2006", "author_url": "", "post_date": "10/24/2019 23:10:51", "content": "Thank you so much! I'm participating in this competition for a project in my data mining course, and this is really helpful for planning it.
", "votes": null, "replies": [] }, { "id": 657585, "author_name": "dizzudin2", "author_url": "", "post_date": "10/25/2019 09:12:25", "content": "Thanks a lot for sharing!
", "votes": null, "replies": [] }, { "id": 658104, "author_name": "kyakovlev", "author_url": "", "post_date": "10/25/2019 17:02:42", "content": "added two models with simple fe:\n(1.22 - https://www.kaggle.com/kyakovlev/ashrae-lgbm-simple-fe -lgbm)\n(1.23 - https://www.kaggle.com/kyakovlev/ashrae-catboost - catboost)
", "votes": null, "replies": [ { "id": 658112, "author_name": "khairulislam", "author_url": "", "post_date": "10/25/2019 17:10:39", "content": "@kyakovlev considering both of them are doing good , did you try ensembling them ?
", "votes": null, "replies": [] }, { "id": 658114, "author_name": "kyakovlev", "author_url": "", "post_date": "10/25/2019 17:13:04", "content": "I have xgb gpu ready also))) I'll submit it tomorrow and blend probe.
\n\nThe reason to publish 3 models and blend:\n* Show tiny differences in models train (categoricals/Nans/params/save/load)\n* Show how much score boost blend can give
", "votes": null, "replies": [] }, { "id": 658120, "author_name": "khairulislam", "author_url": "", "post_date": "10/25/2019 17:22:16", "content": "I can literally visualize you doing this after ensembling the models tomorrow. \n
Yes, blend works. 1.20 lb for lgbm (1.22) and catboost (1.23) - blend 50/50
", "votes": null, "replies": [] }, { "id": 659018, "author_name": "kyakovlev", "author_url": "", "post_date": "10/27/2019 00:19:48", "content": "Also same kernels with 2 lines change scores 1.13 and blend should give 1.10 +-
\n\nI just don't think it's fair to share such high score kernels right now.
", "votes": null, "replies": [] }, { "id": 659095, "author_name": "khairulislam", "author_url": "", "post_date": "10/27/2019 04:23:36", "content": "Yah right .
\n\n\n\n\nwith 2 lines change
\n
Wow !!
", "votes": null, "replies": [] } ] }, { "id": 661306, "author_name": "jf0214", "author_url": "", "post_date": "10/30/2019 05:37:15", "content": "it is a great share， I have learned a lot from it
", "votes": null, "replies": [] }, { "id": 661872, "author_name": "pfichou", "author_url": "", "post_date": "10/30/2019 19:54:52", "content": "As allways very impressive !
\n\nThank you so muuch !
", "votes": null, "replies": [] }, { "id": 662439, "author_name": "kyakovlev", "author_url": "", "post_date": "10/31/2019 14:56:50", "content": "We formed a team. Topic will be frozen for a while. Thank you for all your support.
\n\nGood luck to you.
\n\nMay the Kaggle be with you.
", "votes": null, "replies": [] }, { "id": 665315, "author_name": "vbmokin", "author_url": "", "post_date": "11/04/2019 21:38:43", "content": "I did an automatic FE using a Featuretools library (see ASHRAE - Automatic FE: Featuretools & Selection FE). The analysis showed that in addition to the usual min/max/sum etc. for features, it is advisable to consider the MODE of some meteorological features. I tried to automate this:
df['data_mode'] = group_df[col].rolling(window=window, min_periods=0).apply(lambda x: mode(x)[0][0], raw=True)
But it doesn't work, unfortunately (both mode(x)[0] and mode(x)[0][0]). Someone else might be able to use it in another way
The format of the most of meteorological features can be optimize by rounding up data to int8 or uint8 without losing precision (see my kernel Very significant safe memory + LightGBM):
\n\ndf['air_temperature'] = np.int8(round(2*df['air_temperature'],0)) # store values with precision 0.5
df['cloud_coverage'] = np.uint8(round(10*df['cloud_coverage'],0)) # store values with precision 0.1
df['dew_temperature'] = np.int8(round(5*df['dew_temperature'],0)) # store values with precision 0.2
df['precip_depth_1_hr'] = np.uint8(np.clip(round(df['precip_depth_1_hr'],0),0,255)) # transform [-1,343] to [0,255]
Very nice !
\n\nThank you very much for all your effort to share on this competition!
\n\nYou have been so helpfull during all past days.
", "votes": null, "replies": [] }, { "id": 670279, "author_name": "nroman", "author_url": "", "post_date": "11/11/2019 09:10:21", "content": "@kyakovlev looks like it is time to become a master in datasets ;)
", "votes": null, "replies": [ { "id": 670800, "author_name": "teeyee314", "author_url": "", "post_date": "11/11/2019 21:35:17", "content": "That's the plan. UCF meter readings
", "votes": null, "replies": [] } ] }, { "id": 677905, "author_name": "imnitishng", "author_url": "", "post_date": "11/20/2019 18:42:37", "content": "I have a confusion how to predict on test set which does not have the target encoding column that we created in the train set??\nI have created target encoding features based on meter_reading value in the train set with respect to site_id and building_id
Great job ! Thanks for sharing!
", "votes": null, "replies": [] }, { "id": 679958, "author_name": "vladyelisieiev", "author_url": "", "post_date": "11/23/2019 18:12:57", "content": "Nice post! Keep it up!
", "votes": null, "replies": [] }, { "id": 690315, "author_name": "echowaiwai", "author_url": "", "post_date": "12/08/2019 11:24:15", "content": "Very useful!!
", "votes": null, "replies": [] }, { "id": 695310, "author_name": "cv13j0", "author_url": "", "post_date": "12/14/2019 23:44:04", "content": "Thanks Very much, your summaries always provide so much guidance, appreciated.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "649931": "I'm stopping share things as we formed a team and it will be not fair to my teammates to disclose information that may affect all team.\n\nPlease upvote and comment original topic and kernels authors if you'll find their work interesting and useful (feedback is very important for them to know they are doing good things). Please also indicate what can be done better (especially for kernles) as it helps authors to improve their work (current and future).\n\nHere is small summary for current competition:\n- Data minification (https://www.kaggle.com/kyakovlev/ashrae-data-minification)\n- EDA\n- Baseline model (https://www.kaggle.com/kyakovlev/ashrae-baseline-lgbm)\n- CV options (https://www.kaggle.com/kyakovlev/ashrae-cv-options)\n- CV\n- Baseline FE \n(1.22 - https://www.kaggle.com/kyakovlev/ashrae-lgbm-simple-fe -lgbm)\n(1.23 - https://www.kaggle.com/kyakovlev/ashrae-catboost - catboost)\n same kernels with 2 lines change scores 1.13\n- Feature selection\n- Deeper FE\n- Ensembles/Stackings/Blends\n\n----\n## Part 1\n*Random thoughts about competition.*\n\nThis competition is about predicting energy consumption.\nWhen I look at data I see \"not very reach\" features dataset. Seems that we can and have to use a lot of external data.\n\nWeather - very important (here are some kernels about weather time-series data):\n* https://www.kaggle.com/leandrovrabelo/climate-change-forecast-sarima-model\n* https://www.kaggle.com/selfishgene/filtering-and-auto-correlation-tutorial\n* https://www.kaggle.com/jsphyg/weather-dataset-rattle-package\n* https://www.kaggle.com/c/recruit-restaurant-visitor-forecasting (please see discussions and winning solutions)\nWe don't predict the weather here. But if we do have? What if energy consumption has a strong connection with \"people's feelings\" about the weather tomorrow.\n\nGeo markers (probably we can supplement data with this information).\n\nWealth (also with categorical subsets).\n\nBuilding condition (supplement with some thermal physics data).\n\netc.\n\nIt seems to me now that data that we have just a skeleton and we can and have to find interesting correlated data on external resources. \nMy advice here constantly monitor this thread:\nhttps://www.kaggle.com/c/ashrae-energy-prediction/discussion/112841#latest-649772 \n\nMore readings:\n*Tip: You've already noticed III number in competition name -> search for ASHRAE - Energy Predictor*\n* https://www.kaggle.com/c/ashrae-energy-prediction/overview/prior-competitions\n* https://repository.upenn.edu/mlab_papers/75/ (just example of search result)\netc\n\n----\n\n## Minification.\nOne of the first steps that should be done is data minification (just my opinion).\nYou will need it for:\n* Load data faster -> pickle format (best choice for me)\n* Consume less memory -> Carefull types downcasting\n* Avoid repeated cleaning steps.\n\n**Possible problems:**\n* pandas pickle format is \"version dependent\" and you may have problems with loading data generated with a newer version of Pandas\n* Any bad transformation may and will affect your model. Minification should be done very veeeeery careful as you will use it in every single your kernel in the future. If you not sure about downcasting some column -> don't do. If not sure about cleaning -> don't do. All wrong decisions here may and will affect your score. You may spend days searching for error that prevents your model converge well and find that it was you who created an error in the minification step.\n\n\n**Links:**\n\n* https://www.kaggle.com/kyakovlev/ashrae-data-minification (I do think that my approach is not the best one and not safe to use - just an example)\n* https://www.kaggle.com/hamditarek/reducing-memory-size-for-great-energy-predictor\n\n\n> **DON'T copy that kernels blindly. Try to understand the code and write your own that serves you. You may use that links just for reference.**\n\n----\n\n## Baseline\n\nhttps://www.kaggle.com/kyakovlev/ashrae-data-minification\n\nThe baseline model is a model without any FE and without any model tuning. It will give you an idea of the very base/ground score that you can have with untransformed data. Conducting experiments you have to compare scores with some stable result. \n\nLet's think about what ways we have to improve a single model score:\n* FE obviously\n* CV\n* Model tuning \n* Evaluation metric \n* Loss function\n\n\n**CV**\nThink about how you can make it reliable and align your CV with LB.\nWithout good CV all modifications are \"blind shoots\" and can't be analyzed and evaluated.\n\n\n**Model tuning**\nIt's up to you to decide when you want to do tuning, but my advice is to do at least some minimal tuning in the beginning. Normally (in most cases) model performance depends on the data structure and if it will be untouched your initial params can serve you almost until the end of the competition. In the beginning, it is easier to do as you have just a minimal number of features and any GridSearch/Bayesian optimization will run faster. \n\n\n**Evaluation metric/Loss function**\nIn my case, I've used RMSE but the competition metric is RMSLE. If you will use lgbm please consult these examples for custom metric and custom loss function:\n* https://github.com/microsoft/LightGBM/blob/master/examples/python-guide/advanced_example.py\n* https://github.com/microsoft/LightGBM/blob/master/examples/python-guide/sklearn_example.py\n\n\n**Other baseline models:**\n* https://www.kaggle.com/ryches/simple-lgbm-solution (by @ryches / LGBM)\n* https://www.kaggle.com/bejeweled/ashrae-catboost-regressor (by @bejeweled / CatBoost)\n* https://www.kaggle.com/tunguz/simple-linear-regression-benchmark (by @tunguz / Linear regression)\n* https://www.kaggle.com/yukari17/simple-prediction-do-not-use-model-v2 (by @yukari17 / without any Model)\n\n> **Never copy others solutions, use it as a reference and create your own baseline with model you know better. For sure it will take much more time but you will benefit much more from own code. Others approaches could have mistakes, errors, incorrect code.**\n\n---\n\n## EDA\n\n* https://www.kaggle.com/nroman/eda-for-ashrae\n* https://www.kaggle.com/jtrotman/ashrae-heatmaps\n* https://www.kaggle.com/sudalairajkumar/simple-exploration-notebook-ashrae\n\nBefore reading other's EDAs kernels try to make your own. If you are as lazy as I am - go to read part.\n\nEDA is one of the most important parts of any competition. This your source for FE/CV/Model. Understanding the data and its structure is a crucial step to move on.\n\n---\n\n## CV\n\nhttps://www.kaggle.com/kyakovlev/ashrae-cv-options\n\nWe have several problems here (you may want to check the linked kernel).\n\n* Consumption can't be well generalized (with data we have). For unknown site_id and building_id our predictions are terrible. The same happens with monthly consumption. It means that we should train the final model on all months and all buildings to get the most accurate predictions. You can think of it that consumption is very specific for each building, each zone, and date. Also, it's mean that we don't have all necessary data.\n* The previous point shows us the importance of site_id and building_id which seems to me should be enriched by external data.\n* Not possible to safely use early stoppings. \n* Features validation is becoming very tricky and not very accurate.\n* In my opinion, the best FE validation option is 4/4/4 train/skip/validate - if there is a proved RMSLE boost for the feature it is safe to use it for test predictions.\n* Boosting rounds for the final model should be more than for FE validation but not much.\n\n---\n\n## FE\n\nWhat works:\n* Target/mean encoding\n* Outliers imputation\n* Categorisation \n\nAdversarial validation shows that test set doesn't differ much from train set:\n* https://www.kaggle.com/tunguz/adversarial-ashrae (it's IMPORTANT)\n\nWe don't have many features so for features selection you can safely use (if you have stable CV):\n* recursive feature elimination (remove 1 by 1)\n* permutation importance\n* recursive feature acceptance (add 1 by 1)\n\n---\n\n## Blend/Stacking\n\nBlend works - you can have at least 1-2% of RMSLE with different model blending.\n\nStacking is very tricky as it will overfit hugely for test set. As I see it's almost impossible to use it here. But you may find a way to implement it in your pipeline. \n\n---\n\n## General findings\n\nPrivate/Public Test set split:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113751\n\nGeo location:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114874\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114447\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113338\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113772\n* https://www.kaggle.com/patrick0302/locate-cities-according-weather-temperature\n\nMeter split (didn't test by it's interesting):\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113240\n* https://www.kaggle.com/corochann/ashrae-training-lgbm-by-meter-type\n\nOutliers:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113254\n\nHolidays:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113286\n\nMetric:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113064\n\n---\n\n## Utilities\n\nFaster submission:\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114782\n\nBetter score understanding (more than 2 decimals):\n* https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113104\n\nFaster stratified split\n* https://www.kaggle.com/frednavruzov/faster-stratified-cross-validation-upd", "650029": "Look forward to seeing your kernel! \nOne can really learn a lot when taking part in the same competition with you!", "650238": "> Gold zone / 4 weeks before deadline\nTop 200 / 2 weeks before deadline\n\nIEEE all over again...\nAnything that affects medal zone should be kept private (at least during the last 3-4 weeks of the competition)", "650245": "> IEEE all over again…\n\nIs it good or bad? I do think that 2-4 week is a safe time window to learn and adapt. I also think that kaggle is not about medals but about exchanging ideas and learn (learn during competition when you have \"mojo\" to do experiments and apply new knowledge).", "650287": "I am looking forward to the notebooks but I do think anything that can get you in medal zone should only be shared as an idea in forums irrespective of time so that you have to work for it rather than enabling freeloaders to inflate the leaderboard. Just a thought, you can still do it as long as it is allowed by rules.", "650297": "Good, that means I can drop in late again :P", "650303": "> Is it good or bad?\n\nIt's weird, but it's kinda both.\nI agree with you in general, sharing and learning are good, they're what Kaggle is about.\nBut I'll also try to explain my personal position. I'm so good and fast to adapt as many here. Excluding work and everyday stuff, I have ~1-1.5 hours for kaggling per day, and not every day. Also I'm crazy dude that prefers to develop his algo's and implement new stuff by himself - not just fork-commit-submit or Nforks-combine-submit. Keeping this in mind, I have assburn everytime highscoring public kernel arrives, because I'd like to do things by myself and remain competitive. But that's my personal assburn and I have to live with it - \"if public script beats your model, than your model sucks\" :D\n> Top 200 / 2 weeks before deadline\n\nI think this competition will have many team since it's tabular. Hence, there will be, say, 500ppl in medals. If you (not \"you\" personally, just somebody) will post kernel in top200 2 weeks before deadline, it may ruin the competition for participants like me, who don't have much time to spend and makes own models.\n\nOh, it all looks a bit messy, so, trying to summarize: I think that anything affecting medals shouldn't be posted during the last 3-4 weeks of the competition, for the sake of dudes like me. But that's my personal opinion, and I understand that I'm not alone here :)", "650342": "Actually, the competition host always benefits from such sharing. Strong straightforward combinations are shared openly and can be further improved or used for blending. This pushes everybody to look up for new ways to tackle the problem.\nIn the end, we get more knowledge and experience. And the host gets better and more variations of solutions.", "650412": "Great :)", "650474": "Minification part added:\npickle is not the unique option btw. \nYou can try use feather `DataFrame.to_feather()`\nor Google BigQuery table `DataFrame.to_gbq()` -> it would be amazing if someone can prepare BigQuery integration notebook for with competition\nor parquet `DataFrame.to_parquet()`", "650476": "So no need to join before 2 weeks to deadline. Thanks.", "650487": "christofhenkel please join before and do same as you did for Jigsaw competition. For such newbie as me (and many other kagglers) in NLP your posts were such a great source of learning.", "650809": "Glad to see you here @kyakovlev This competition has a very large data set. Can you focus on some memory optimization and resampling topics this time?", "650818": "I'll try. Normally (IEEE competition) I'm too lazy to make huge optimisations - Base kernel preparation on kaggle kernels -> then pass all to Google Cloud and run 64cores/64GB machine predictions -> no memory issues and much faster. \n\nOne thing I can recommend from start - use HDD space for storing \"intermediate\" results and dataframes (kaggle gives us 5GB per kernel).", "650821": "M4 Forecasting Competition winner: Hybrid Exponential Smoothing-Recurrent Neural Networks (ES-RNN)\n\nhttps://eng.uber.com/m4-forecasting-competition/\nhttps://github.com/M4Competition/M4-methods/tree/slaweks_ES-RNN/118%20-%20slaweks17", "650824": "Interesting, but as always -> there is no Silver bullet each dataset is unique and approaches differ a lot.", "651309": "After going through your solution in IEEE Fraud detection, I feel like I can actually do well in this. Thanks for sharing.", "651606": "Did anyone probed public/private data split?\n\nIs it timebased splits or random? \n\nDidn't find information about it in competition description.", "651685": "Seems noone did test)) So I did it.\n\nhttps://www.kaggle.com/kyakovlev/ashrae-split-test\n\n> This leaderboard is calculated with approximately 78% of the test data. The final results will be based on the other 22%, so the final standings may be different.\n\n```\nsubmission = pd.read_csv('../input/ashrae-baseline-lgbm/submission.csv')\npossible_public = int(len(submission)*0.78)\n\nsubmission.iloc[possible_public:,1] = 0\n```\n\nLB before - 1.36\n\nLB after -> 2.60 -> seems it's random split or 22% first lines of data is private (need one more test).", "651710": "Was going to mention the pandas `to_feather` format myself. It would be nice to have a comparison between them. I will probably report one tomorrow.", "651739": "stanislavblinov, ideally I agree with everything you said and here is my take. Sharing of high scoring kernel is a mixed blessing. On one hand if one is lazy, you can use that kernel and do nothing else which is a waste of time because you may not learn anything. If shared a month before the end of a competition, I personally use it as a motivation to improve my model so that I can beat its score. On the other extreme if there is not much sharing, only hard working people with a lot of time in their hands and those with domain knowledge will do better. For those that are like me and you who is only here to learn and enhance our knowledge whilst challenging our brains, it is 50-50 good and bad :-)\n\nBy the way I have been in some competitions where the highest scoring kernels have serious design flaws that sent everyone that used it down over 1000 places because they did not read and understand the codes. A good example is [Home Credit Default Risk](https://www.kaggle.com/c/home-credit-default-risk).", "651768": "You inspire me ! Congrats for help kagglers ! Great @kyakovlev !✔️", "651927": "the competition host always benefits from such sharing. Strong straightforward combinations are shared openly and can be further improved or used for blending. \nIn the end, we get more knowledge and experience. And the host gets better and more variations of solutions.👍 👍", "652239": "I tried \n```\npossible_public = int(len(submission)*0.22)\nsubmission.iloc[:possible_public:,1] = 0\n```\n\nLB before - 1.37\n\nLB after -> 2.53\n -> seems it's random split", "652295": "Does data minification only reduces RAM usage, or it improves model training/predicting time too?", "652301": "For tree-based models no.\n\nIf I remember well (please correct me) fo lgbm we have such flow:\n* We have dataset with fp8 for example\n* lgbm will reserve memory for fp32\n* lgbm will train on fp8\n\nThe question here that we need as much free memory as possible for step 2 and minification helps here.\n\nProbably you have noticed during lgbm training how memory usage jumps in the beginning of training round - this is exactly reservation step. \n\n---\n\nNN is a different case. We can use fp16 training which is much faster (in most of cases) than fp32 training.", "652325": "Yes, I've noticed that jump in RAM in beginning of training with lgbm, really it's necessary to have free RAM in this step. Thanks for your clarifying answer.", "652327": "gunesevitan I've added example how to use hdd during kernel training.\n\nhttps://www.kaggle.com/kyakovlev/ashrae-baseline-lgbm\n\nThere is also example how to save model and make predictions for multiple models (kfold/seed).", "652358": "Any one succeeded in using one hot encoding with categorical features ? My kernel crushes each time whenever i try", "652362": "if you use lgbm, you can try to set `df[column].astype('categorical')`\n\nhttps://lightgbm.readthedocs.io/en/latest/Advanced-Topics.html\n\nCategorical Feature Support section.\n\nMore info:\n* https://www.kaggle.com/c/ieee-fraud-detection/discussion/108575#626083\n* https://www.kaggle.com/c/ieee-fraud-detection/discussion/108575#624919\n\n---\n\nYour kernel crushes because even 1/0 (bool/int8) encoding 20M row dataset becoming HUGE in memory and there is no any reason (for current dataset) to use pure one-hot encoding when you have Catboost and Lgbm categorical option.", "652387": "Hi @kyakovlev! First, thanks for sharing! I hope you'll do good as IEEE :)\n\nWhat do you mean when you say:\n> For unknown `site_id` and `building_id` our predictions are terrible\n\nCause we have the same building and site ids in both train and test as far as my checks are correct :smile:", "652410": "Great work", "652412": "You are right. \n\n> Good thing is all our test buildings and test sites present in train set.\n\nhttps://www.kaggle.com/kyakovlev/ashrae-cv-options\n\n\n----\n\n> What do you mean\n\nBe careful with kfold -> if any fold will not have all building_ids and all months data it will make bad predictions for such unknown items.", "652469": "Thanks, now I see what you meant.\nI'm currently using TimeSeriesSplit but I have to manage to find splits where all building ids are present...", "652525": "kyakovlev Thanks for the insights. These are truly some important points to start with!", "652822": "Pandas to feather doesn't yet work with float16, so it wont work if you're downcasting data to float16. Be mindful", "652904": "Feather format based on parquet and it also has no support for float16. But it's fast.\n\nThe types supported by the file format are intended to be as minimal as possible, with a focus on how the types effect on disk storage. For example, 16-bit ints are not explicitly supported in the storage format since they are covered by 32-bit ints with an efficient encoding. This reduces the complexity of implementing readers and writers for the format. The types are:\n\nBOOLEAN: 1 bit boolean\nINT32: 32 bit signed ints\nINT64: 64 bit signed ints\nINT96: 96 bit signed ints\nFLOAT: IEEE 32-bit floating point values\nDOUBLE: IEEE 64-bit floating point values\nBYTE_ARRAY: arbitrarily long byte arrays.\n\n---\n\nSo you may want to change memory_reducer function to use 32bits minimum formats.", "652928": "> This leaderboard is calculated with approximately 78% of the test data. The final results will be based on the other 22%, so the final standings may be different.\n\n@d1348k , @kyakovlev, if multiply `` *0.22 in local validation predictions``, help to generalize final results ? ``LB_before = int(len(submission_LB_before )*0.22)``, to generalize results for private dataset. ✨", "653124": "thx a lot!", "653187": "The spike occurs because when you create your lgb datasets, you're likely passing in a pandas dataframe. LGBM's python api converts everything to float64 IIRC. Try passing in a raw numpy array, and you might not see that occur. There's also a flag you can pass where it'll first write to disk rather than creating another copy of the dataframe in ram.", "653589": "Great read thanks a lot, I'm pretty sure you'll do similar to IEEE , all the best!", "653625": "Oh wow. Thanks for putting all of this together. I also very much appreciate your active commenting on all the questions / comments! 👍 \n\nSuper helpful...", "653632": "A logical split for this would be by building_id - so maybe we looking at only 22% of the buildings.\n\nHow hard is that assumption to probe?", "653990": "Thanks for sharing!", "654282": "Thanks for sharing. You help me to understand a lot", "654337": "Gilfoyle you should be working on the new internet :D", "654353": "Thanks for the sharing info.\nThat solved some of my annoying.", "654406": "thanks for sharing info.", "654597": "Here is a blog post which compares feather, md5, parquet, pickle etc. And it shows that feather is the most optimized one. I am using feather for sometime (both at work and projects) and found it to be good (Didn't do any comparison though).\n\nhttps://towardsdatascience.com/the-best-format-to-save-pandas-data-414dca023e0d", "654606": "Thanks about this revelation . I always used to think that everyone else is using kaggle kernel efficiently and I am the only one who find it difficult to use the kaggle kernel because memory/CPU limitation and random failures; May be I process is not efficient enough!", "655037": "Thanks for sharing! It seems to be very useful!:)", "655919": "I don't understand this multiply thing and how you people come to conclusion. Can anyone please explain?", "656074": "In general case if leaderboard splitting is not randomized (not randomized means that first 78% of data is public and last 22% is private) then turning to zeroes private part (last 22%) of submission would not cause LB decreasing", "656081": "And I was wrong.\n\nplease see and upvote https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113751\n\n> (test set and row_id column are not strictly time ordered).\n\nIt's very important!", "656146": "Thanks for sharing.", "657087": "Thank you so much! I'm participating in this competition for a project in my data mining course, and this is really helpful for planning it.", "657585": "Thanks a lot for sharing!", "658104": "added two models with simple fe:\n(1.22 - https://www.kaggle.com/kyakovlev/ashrae-lgbm-simple-fe -lgbm)\n(1.23 - https://www.kaggle.com/kyakovlev/ashrae-catboost - catboost)", "658112": "kyakovlev considering both of them are doing good , did you try ensembling them ?", "658114": "I have xgb gpu ready also))) I'll submit it tomorrow and blend probe.\n\nThe reason to publish 3 models and blend:\n* Show tiny differences in models train (categoricals/Nans/params/save/load)\n* Show how much score boost blend can give", "658120": "I can literally visualize you doing this after ensembling the models tomorrow. \n", "658876": "Yes, blend works. 1.20 lb for lgbm (1.22) and catboost (1.23) - blend 50/50", "659018": "Also same kernels with 2 lines change scores 1.13 and blend should give 1.10 +-\n\nI just don't think it's fair to share such high score kernels right now.", "659095": "Yah right .\n> with 2 lines change\n\nWow !!", "661306": "it is a great share， I have learned a lot from it", "661872": "As allways very impressive !\n\nThank you so muuch !", "662439": "We formed a team. Topic will be frozen for a while. Thank you for all your support. \n\nGood luck to you.\n\nMay the Kaggle be with you.", "665288": "Moreover, the fact that `building_id `is present in both datasets (training and test) and is identical there, means, that models are tuned separately to an each building. Here, if `building_id `were not present, and there were only the parameters of different buildings, weather data, etc., that is, a model for an average building, then it really was an universal model. As for me, this is a disadvantage of setting the task of the contest itself. The contest authors will receive not an universal model for their energy modernization tasks, but a one and a half thousand models for individual buildings.", "665315": "I did an automatic FE using a Featuretools library (see [ASHRAE - Automatic FE: Featuretools & Selection FE](https://www.kaggle.com/vbmokin/ashrae-automatic-fe-featuretools-selection-fe)). The analysis showed that in addition to the usual `min/max/sum etc.` for features, it is advisable to consider the MODE of some meteorological features. I tried to automate this: \n\n`df['data_mode'] = group_df[col].rolling(window=window, min_periods=0).apply(lambda x: mode(x)[0][0], raw=True)`\n\nBut it doesn't work, unfortunately (both `mode(x)[0]` and `mode(x)[0][0]`). Someone else might be able to use it in another way", "665320": "The format of the most of meteorological features can be optimize by rounding up data to int8 or uint8 without losing precision (see my kernel [Very significant safe memory + LightGBM](https://www.kaggle.com/vbmokin/very-significant-safe-memory-lightgbm)):\n\n`df['air_temperature'] = np.int8(round(2*df['air_temperature'],0)) # store values with precision 0.5`\n\n`df['cloud_coverage'] = np.uint8(round(10*df['cloud_coverage'],0)) # store values with precision 0.1`\n\n`df['dew_temperature'] = np.int8(round(5*df['dew_temperature'],0)) # store values with precision 0.2`\n\n`df['precip_depth_1_hr'] = np.uint8(np.clip(round(df['precip_depth_1_hr'],0),0,255)) # transform [-1,343] to [0,255]`", "665358": "Very nice ! \n\nThank you very much for all your effort to share on this competition! \n\nYou have been so helpfull during all past days.", "667999": "yourssharmaji \nthey are taking **78%** (in train case) and **22%** (in validation case) of the rows in '`submission.csv`' file. Now, they cross-check their position's using '`iloc`' in *submission.csv* to esure if the public/private split is in pattern(time-based) or random...", "670279": "kyakovlev looks like it is time to become a master in datasets ;)", "670800": "That's the plan. [UCF meter readings](https://www.kaggle.com/teeyee314/ucf-building-meter-reading)", "677905": "I have a confusion how to predict on test set which does not have the target encoding column that we created in the train set??\nI have created target encoding features based on `meter_reading` value in the train set with respect to `site_id` and `building_id`", "678159": "Great job ! Thanks for sharing!", "679958": "Nice post! Keep it up!", "690315": "Very useful!!", "695310": "Thanks Very much, your summaries always provide so much guidance, appreciated.", "3068939": "Great post! Thanks!" }, "source": "meta" }