Just wanted to share a notebook demonstrating the use of H2O4GPU and Datatable.
\n\nFor some reason I find pandas merge and groupies a bit too slow and lightgbm on CPU is not as fast as I’d like.
\n\nSo I decided to install a new virtual env for GPU accelerated packages and Datatable.
\n\nYou can find the notebook on my GitHub repo here :
\n\nhttps://github.com/goldentom42/kaggle_examples/blob/master/champs_scalar_coupling_h2o4gpu_dt.ipynb
\n\nModels optimize almost directly the challenge evaluation metric using special weights.
\n\n5-fold CV is around -0.105 and LB score -0.268 with a small number of features.
\n\nHope you’ll find it useful and please let me know your thoughts :)
\n\nUpdate to release GPU memory:
\n\nXGBoost does not seem to release GPU memory once a model is trained (same issue as with CPU)\nSo you would need to add the following lines of code to release it between 2 folds : \n```\n from sklearn.externals import joblib\n import gc
\n\n# To release GPU memory\n# https://github.com/dmlc/xgboost/issues/3045\n# 1. pickle model\nmodels.append('xgboost_model_fold_%d.gz' % (fold_ + 1))\njoblib.dump(model, models[-1]) \n# del booster\nmodel.get_booster().__del__()\n# del the rest\ndel model, trn_X, val_X\n# Garbage collect\ngc.collect()\n```
", "messages": [ { "id": "566548", "postDate": "07/02/2019 09:44:34", "content": "Just wanted to share a notebook demonstrating the use of H2O4GPU and Datatable.
\n\nFor some reason I find pandas merge and groupies a bit too slow and lightgbm on CPU is not as fast as I’d like.
\n\nSo I decided to install a new virtual env for GPU accelerated packages and Datatable.
\n\nYou can find the notebook on my GitHub repo here :
\n\nhttps://github.com/goldentom42/kaggle_examples/blob/master/champs_scalar_coupling_h2o4gpu_dt.ipynb
\n\nModels optimize almost directly the challenge evaluation metric using special weights.
\n\n5-fold CV is around -0.105 and LB score -0.268 with a small number of features.
\n\nHope you’ll find it useful and please let me know your thoughts :)
\n\nUpdate to release GPU memory:
\n\nXGBoost does not seem to release GPU memory once a model is trained (same issue as with CPU)\nSo you would need to add the following lines of code to release it between 2 folds : \n```\n from sklearn.externals import joblib\n import gc
\n\n# To release GPU memory\n# https://github.com/dmlc/xgboost/issues/3045\n# 1. pickle model\nmodels.append('xgboost_model_fold_%d.gz' % (fold_ + 1))\njoblib.dump(model, models[-1]) \n# del booster\nmodel.get_booster().__del__()\n# del the rest\ndel model, trn_X, val_X\n# Garbage collect\ngc.collect()\n```
", "rawMarkdown": "Just wanted to share a notebook demonstrating the use of [H2O4GPU](https://github.com/h2oai/h2o4gpu) and [Datatable](https://github.com/h2oai/datatable).\n\nFor some reason I find pandas merge and groupies a bit too slow and lightgbm on CPU is not as fast as I’d like.\n\nSo I decided to install a new virtual env for GPU accelerated packages and Datatable.\n\nYou can find the notebook on my GitHub repo here : \n\nhttps://github.com/goldentom42/kaggle_examples/blob/master/champs_scalar_coupling_h2o4gpu_dt.ipynb\n\nModels optimize almost directly the challenge evaluation metric using special weights.\n\n5-fold CV is around -0.105 and LB score -0.268 with a small number of features.\n\nHope you’ll find it useful and please let me know your thoughts :) \n\nUpdate to release GPU memory:\n\nXGBoost does not seem to release GPU memory once a model is trained (same issue as with CPU)\nSo you would need to add the following lines of code to release it between 2 folds : \n```\n from sklearn.externals import joblib\n import gc\n\n # To release GPU memory\n # https://github.com/dmlc/xgboost/issues/3045\n # 1. pickle model\n models.append('xgboost_model_fold_%d.gz' % (fold_ + 1))\n joblib.dump(model, models[-1]) \n # del booster\n model.get_booster().__del__()\n # del the rest\n del model, trn_X, val_X\n # Garbage collect\n gc.collect()\n```", "votes": null }, { "id": "566554", "postDate": "07/02/2019 09:57:16", "content": "Thanks for sharing your notebook, Olivier - there are lots of useful code snippets.
\n\nI've seen lots of people include features such as the electronegativity, atomic mass, atomic radius etc of atom_1. As these quantities are entirely dependent on 'type', I can't understand how they could be useful. They contain the same information as the categorical feature 'type', right? I can understand that they could be useful for feature engineering because they depend on one of the atoms in question but the raw electronegativity and atomic radius of atom_1 seem useless to me.
\n\nPerhaps I'm missing something?
", "rawMarkdown": "Thanks for sharing your notebook, Olivier - there are lots of useful code snippets.\n\nI've seen lots of people include features such as the electronegativity, atomic mass, atomic radius etc of atom_1. As these quantities are entirely dependent on 'type', I can't understand how they could be useful. They contain the same information as the categorical feature 'type', right? I can understand that they could be useful for feature engineering because they depend on one of the atoms in question but the raw electronegativity and atomic radius of atom_1 seem useless to me.\n\nPerhaps I'm missing something?", "votes": null }, { "id": "566569", "postDate": "07/02/2019 10:34:39", "content": "Thanks for your feedback @joshxsarah.
\n\nWhat you're saying sounds reasonable and if that's the case dropping these duplicate features should improve your score.
", "rawMarkdown": "Thanks for your feedback @joshxsarah. \n\nWhat you're saying sounds reasonable and if that's the case dropping these *duplicate* features should improve your score.", "votes": null }, { "id": "566857", "postDate": "07/02/2019 16:59:18", "content": "Merci le Breton!
", "rawMarkdown": "Merci le Breton!", "votes": null }, { "id": "567332", "postDate": "07/03/2019 11:04:19", "content": "I love datatable, just a but immature but would switch from pandas in a sec when it's more robust.
", "rawMarkdown": "I love datatable, just a but immature but would switch from pandas in a sec when it's more robust.", "votes": null }, { "id": "567513", "postDate": "07/03/2019 16:09:33", "content": "Hi @antorsae, what do you find not robust in datatable or not mature ?
\n\nThis far I have not find things I can't really do.
", "rawMarkdown": "Hi @antorsae, what do you find not robust in datatable or not mature ? \n\nThis far I have not find things I can't really do.", "votes": null }, { "id": "567584", "postDate": "07/03/2019 18:17:07", "content": "Less documentation. For example I don't know how to do the equivalent of:
\n\nThe team is very responsive but as of yet there's bugs (this one affected me https://github.com/h2oai/datatable/pull/1645 I think you haver other(s) https://github.com/h2oai/datatable/blob/master/CHANGELOG.md) that are not released as an official version.
", "rawMarkdown": "Less documentation. For example I don't know how to do the equivalent of:\n\n- df.factorize(...)\n- pd.get_dummies(....)\n- for x,y in df.group_by(....)\n\nThe team is very responsive but as of yet there's bugs (this one affected me https://github.com/h2oai/datatable/pull/1645 I think you haver other(s) https://github.com/h2oai/datatable/blob/master/CHANGELOG.md) that are not released as an official version.", "votes": null }, { "id": "567881", "postDate": "07/04/2019 05:43:48", "content": "Oh I see. I don't think there are plans to create equivalents of the above utility functions, at least not that I know.
\n\nI coded my own version of df.factorize in the notebook I released. I don't like the fact it depends on pandas.index but at the same time I dislike LabelEncoder a lot more (would raise an exception when unknown values are encountered, which makes it useless in production, IMHO).
For pd.get_dummies, sklearn OneHotEncoder is good enough, but you have to go through the process of creating the names for the columns.
For the groupby thing, I think I would simply go through the unique group values and make a select in the dt.Frame, which should be quick enough.
What I miss the most from pandas is map, apply and transform but at the same time they are so slow that in many competitions we eventually had to turn all of them into numpy operations !
I'll try to release notebooks regularly to show ways to do all these operations we regularly need.
\n\nThanks for your feedback :)
", "rawMarkdown": "Oh I see. I don't think there are plans to create equivalents of the above utility functions, at least not that I know.\n\nI coded my own version of `df.factorize` in the notebook I released. I don't like the fact it depends on `pandas.index` but at the same time I dislike `LabelEncoder` a lot more (would raise an exception when unknown values are encountered, which makes it useless in production, IMHO).\n\nFor `pd.get_dummies`, sklearn OneHotEncoder is good enough, but you have to go through the process of creating the names for the columns.\n\nFor the `groupby` thing, I think I would simply go through the unique group values and make a select in the dt.Frame, which should be quick enough.\n\nWhat I miss the most from pandas is `map`, `apply` and `transform` but at the same time they are so slow that in many competitions we eventually had to turn all of them into `numpy` operations !\n\nI'll try to release notebooks regularly to show ways to do all these operations we regularly need.\n\nThanks for your feedback :)", "votes": null }, { "id": "569652", "postDate": "07/07/2019 05:23:52", "content": "Updated the topic with GPU memory release trick.
", "rawMarkdown": "Updated the topic with GPU memory release trick.", "votes": null } ], "comments": [ { "id": 566554, "author_name": "joshxsarah", "author_url": "", "post_date": "07/02/2019 09:57:16", "content": "Thanks for sharing your notebook, Olivier - there are lots of useful code snippets.
\n\nI've seen lots of people include features such as the electronegativity, atomic mass, atomic radius etc of atom_1. As these quantities are entirely dependent on 'type', I can't understand how they could be useful. They contain the same information as the categorical feature 'type', right? I can understand that they could be useful for feature engineering because they depend on one of the atoms in question but the raw electronegativity and atomic radius of atom_1 seem useless to me.
\n\nPerhaps I'm missing something?
", "votes": null, "replies": [ { "id": 566569, "author_name": "ogrellier", "author_url": "", "post_date": "07/02/2019 10:34:39", "content": "Thanks for your feedback @joshxsarah.
\n\nWhat you're saying sounds reasonable and if that's the case dropping these duplicate features should improve your score.
", "votes": null, "replies": [] } ] }, { "id": 566857, "author_name": "chabir", "author_url": "", "post_date": "07/02/2019 16:59:18", "content": "Merci le Breton!
", "votes": null, "replies": [] }, { "id": 567332, "author_name": "antorsae", "author_url": "", "post_date": "07/03/2019 11:04:19", "content": "I love datatable, just a but immature but would switch from pandas in a sec when it's more robust.
", "votes": null, "replies": [ { "id": 567513, "author_name": "ogrellier", "author_url": "", "post_date": "07/03/2019 16:09:33", "content": "Hi @antorsae, what do you find not robust in datatable or not mature ?
\n\nThis far I have not find things I can't really do.
", "votes": null, "replies": [] }, { "id": 567584, "author_name": "antorsae", "author_url": "", "post_date": "07/03/2019 18:17:07", "content": "Less documentation. For example I don't know how to do the equivalent of:
\n\nThe team is very responsive but as of yet there's bugs (this one affected me https://github.com/h2oai/datatable/pull/1645 I think you haver other(s) https://github.com/h2oai/datatable/blob/master/CHANGELOG.md) that are not released as an official version.
", "votes": null, "replies": [] }, { "id": 567881, "author_name": "ogrellier", "author_url": "", "post_date": "07/04/2019 05:43:48", "content": "Oh I see. I don't think there are plans to create equivalents of the above utility functions, at least not that I know.
\n\nI coded my own version of df.factorize in the notebook I released. I don't like the fact it depends on pandas.index but at the same time I dislike LabelEncoder a lot more (would raise an exception when unknown values are encountered, which makes it useless in production, IMHO).
For pd.get_dummies, sklearn OneHotEncoder is good enough, but you have to go through the process of creating the names for the columns.
For the groupby thing, I think I would simply go through the unique group values and make a select in the dt.Frame, which should be quick enough.
What I miss the most from pandas is map, apply and transform but at the same time they are so slow that in many competitions we eventually had to turn all of them into numpy operations !
I'll try to release notebooks regularly to show ways to do all these operations we regularly need.
\n\nThanks for your feedback :)
", "votes": null, "replies": [] } ] }, { "id": 569652, "author_name": "ogrellier", "author_url": "", "post_date": "07/07/2019 05:23:52", "content": "Updated the topic with GPU memory release trick.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "566548": "Just wanted to share a notebook demonstrating the use of [H2O4GPU](https://github.com/h2oai/h2o4gpu) and [Datatable](https://github.com/h2oai/datatable).\n\nFor some reason I find pandas merge and groupies a bit too slow and lightgbm on CPU is not as fast as I’d like.\n\nSo I decided to install a new virtual env for GPU accelerated packages and Datatable.\n\nYou can find the notebook on my GitHub repo here : \n\nhttps://github.com/goldentom42/kaggle_examples/blob/master/champs_scalar_coupling_h2o4gpu_dt.ipynb\n\nModels optimize almost directly the challenge evaluation metric using special weights.\n\n5-fold CV is around -0.105 and LB score -0.268 with a small number of features.\n\nHope you’ll find it useful and please let me know your thoughts :) \n\nUpdate to release GPU memory:\n\nXGBoost does not seem to release GPU memory once a model is trained (same issue as with CPU)\nSo you would need to add the following lines of code to release it between 2 folds : \n```\n from sklearn.externals import joblib\n import gc\n\n # To release GPU memory\n # https://github.com/dmlc/xgboost/issues/3045\n # 1. pickle model\n models.append('xgboost_model_fold_%d.gz' % (fold_ + 1))\n joblib.dump(model, models[-1]) \n # del booster\n model.get_booster().__del__()\n # del the rest\n del model, trn_X, val_X\n # Garbage collect\n gc.collect()\n```", "566554": "Thanks for sharing your notebook, Olivier - there are lots of useful code snippets.\n\nI've seen lots of people include features such as the electronegativity, atomic mass, atomic radius etc of atom_1. As these quantities are entirely dependent on 'type', I can't understand how they could be useful. They contain the same information as the categorical feature 'type', right? I can understand that they could be useful for feature engineering because they depend on one of the atoms in question but the raw electronegativity and atomic radius of atom_1 seem useless to me.\n\nPerhaps I'm missing something?", "566569": "Thanks for your feedback @joshxsarah. \n\nWhat you're saying sounds reasonable and if that's the case dropping these *duplicate* features should improve your score.", "566857": "Merci le Breton!", "567332": "I love datatable, just a but immature but would switch from pandas in a sec when it's more robust.", "567513": "Hi @antorsae, what do you find not robust in datatable or not mature ? \n\nThis far I have not find things I can't really do.", "567584": "Less documentation. For example I don't know how to do the equivalent of:\n\n- df.factorize(...)\n- pd.get_dummies(....)\n- for x,y in df.group_by(....)\n\nThe team is very responsive but as of yet there's bugs (this one affected me https://github.com/h2oai/datatable/pull/1645 I think you haver other(s) https://github.com/h2oai/datatable/blob/master/CHANGELOG.md) that are not released as an official version.", "567881": "Oh I see. I don't think there are plans to create equivalents of the above utility functions, at least not that I know.\n\nI coded my own version of `df.factorize` in the notebook I released. I don't like the fact it depends on `pandas.index` but at the same time I dislike `LabelEncoder` a lot more (would raise an exception when unknown values are encountered, which makes it useless in production, IMHO).\n\nFor `pd.get_dummies`, sklearn OneHotEncoder is good enough, but you have to go through the process of creating the names for the columns.\n\nFor the `groupby` thing, I think I would simply go through the unique group values and make a select in the dt.Frame, which should be quick enough.\n\nWhat I miss the most from pandas is `map`, `apply` and `transform` but at the same time they are so slow that in many competitions we eventually had to turn all of them into `numpy` operations !\n\nI'll try to release notebooks regularly to show ways to do all these operations we regularly need.\n\nThanks for your feedback :)", "569652": "Updated the topic with GPU memory release trick." }, "source": "meta" }