I noticed that every loop in submission should be less than certain seconds to ensure it will not raise runtime errors when submitting to the LB. I cannot submit my model to the LB because it always raises a runtime error because of either the model or the production of features. Anyone knows how to reduce the loop runtime to help submit it to the LB. Thanks
", "messages": [ { "id": "1631793", "postDate": "12/28/2021 19:26:57", "content": "I noticed that every loop in submission should be less than certain seconds to ensure it will not raise runtime errors when submitting to the LB. I cannot submit my model to the LB because it always raises a runtime error because of either the model or the production of features. Anyone knows how to reduce the loop runtime to help submit it to the LB. Thanks
", "rawMarkdown": "I noticed that every loop in submission should be less than certain seconds to ensure it will not raise runtime errors when submitting to the LB. I cannot submit my model to the LB because it always raises a runtime error because of either the model or the production of features. Anyone knows how to reduce the loop runtime to help submit it to the LB. Thanks", "votes": null }, { "id": "1631804", "postDate": "12/28/2021 19:39:01", "content": "Try to use tqdm in your loop like:
\nfrom tqdm import tqdm\nfor i, (df_test, df_pred) in tqdm(enumerate(iter_test)):\n # your prediction\nand track the number of iterations per second, from my experience they should be 6-7+ iterations per second. This way it would be easier for you to optimise and know what to expect. Its hard to say concrete recommendations for you, one thing you could do if you use separate model for each asset is to calculate features for all assets at one time, not separately. Also try to use .itertuples() or .apply() to loop through df_test and make predictions. And if it's possible for you you can try to use numpy, but this way you will get quite messy code, personally I don't prefer this way. Anyway you have limit for your feature engineering ideas, so try to carefully select features.
", "rawMarkdown": "Try to use tqdm in your loop like:\n```\nfrom tqdm import tqdm\nfor i, (df_test, df_pred) in tqdm(enumerate(iter_test)):\n # your prediction\n\n```and track the number of iterations per second, from my experience they should be 6-7+ iterations per second. This way it would be easier for you to optimise and know what to expect. Its hard to say concrete recommendations for you, one thing you could do if you use separate model for each asset is to calculate features for all assets at one time, not separately. Also try to use .itertuples() or .apply() to loop through df_test and make predictions. And if it's possible for you you can try to use numpy, but this way you will get quite messy code, personally I don't prefer this way. Anyway you have limit for your feature engineering ideas, so try to carefully select features.", "votes": null }, { "id": "1631808", "postDate": "12/28/2021 19:41:43", "content": "If you use LightGBM and you can fit your notebook in 13GB memory you can try to use GPU inference from cuml ForestInference, it also speeds up things in some way
", "rawMarkdown": "If you use LightGBM and you can fit your notebook in 13GB memory you can try to use GPU inference from cuml ForestInference, it also speeds up things in some way", "votes": null }, { "id": "1632549", "postDate": "12/29/2021 18:38:40", "content": "It is true that feature engineering ideas are limited if we have to restrain our loop time. Anyway, thanks for your advice.
", "rawMarkdown": "It is true that feature engineering ideas are limited if we have to restrain our loop time. Anyway, thanks for your advice.", "votes": null }, { "id": "1632689", "postDate": "12/29/2021 23:47:17", "content": "Pandas is generally rather slow. Base python + numpy is enough to build features in a very fast manner. I've shared a notebook that builds ~300 features in less than 10 minutes of submission time budget: https://www.kaggle.com/lucasmorin/on-line-feature-engineering
", "rawMarkdown": "Pandas is generally rather slow. Base python + numpy is enough to build features in a very fast manner. I've shared a notebook that builds ~300 features in less than 10 minutes of submission time budget: https://www.kaggle.com/lucasmorin/on-line-feature-engineering", "votes": null }, { "id": "1635456", "postDate": "01/01/2022 19:10:43", "content": "I have been able to reduce the loop time and use more features for my models using Polars dataframes: https://www.kaggle.com/c/g-research-crypto-forecasting/discussion/298250
", "rawMarkdown": "I have been able to reduce the loop time and use more features for my models using Polars dataframes: https://www.kaggle.com/c/g-research-crypto-forecasting/discussion/298250", "votes": null }, { "id": "1635551", "postDate": "01/01/2022 21:46:00", "content": "One more way is to use Numba https://numba.readthedocs.io/en/stable/ Depending on your approach to feature engineering, Numba might significantly decrease execution time of your code.
", "rawMarkdown": "One more way is to use Numba [https://numba.readthedocs.io/en/stable/](https://numba.readthedocs.io/en/stable/) Depending on your approach to feature engineering, Numba might significantly decrease execution time of your code.", "votes": null }, { "id": "1647357", "postDate": "01/12/2022 14:24:23", "content": "Have you managed to actually submit a model to the competition using those features?
", "rawMarkdown": "Have you managed to actually submit a model to the competition using those features?", "votes": null }, { "id": "1647463", "postDate": "01/12/2022 16:18:07", "content": "In my feature engineering notebook (https://www.kaggle.com/lucasmorin/on-line-feature-engineering) I estimate the submission time budget at 10 min for 300 features. In this notebook: https://www.kaggle.com/lucasmorin/online-fe-nn-loss-shap I submit a baseline NN with those features. I don't really garantee anything (the NN convergence seems rather poor, there might remain some bugs in the FE loop), but it seems to work in principle. It's totally doable to do some extensive feature engineering as numpy is really fast.
", "rawMarkdown": "In my feature engineering notebook (https://www.kaggle.com/lucasmorin/on-line-feature-engineering) I estimate the submission time budget at 10 min for 300 features. In this notebook: https://www.kaggle.com/lucasmorin/online-fe-nn-loss-shap I submit a baseline NN with those features. I don't really garantee anything (the NN convergence seems rather poor, there might remain some bugs in the FE loop), but it seems to work in principle. It's totally doable to do some extensive feature engineering as numpy is really fast.", "votes": null }, { "id": "1647561", "postDate": "01/12/2022 17:33:33", "content": "Thanks. Your notebooks should get many more votes. This competition is odd in that so many highly rated notebooks are not doing much of anything.
", "rawMarkdown": "Thanks. Your notebooks should get many more votes. This competition is odd in that so many highly rated notebooks are not doing much of anything.", "votes": null } ], "comments": [ { "id": 1631804, "author_name": "alankabisov", "author_url": "", "post_date": "12/28/2021 19:39:01", "content": "Try to use tqdm in your loop like:
\nfrom tqdm import tqdm\nfor i, (df_test, df_pred) in tqdm(enumerate(iter_test)):\n # your prediction\nand track the number of iterations per second, from my experience they should be 6-7+ iterations per second. This way it would be easier for you to optimise and know what to expect. Its hard to say concrete recommendations for you, one thing you could do if you use separate model for each asset is to calculate features for all assets at one time, not separately. Also try to use .itertuples() or .apply() to loop through df_test and make predictions. And if it's possible for you you can try to use numpy, but this way you will get quite messy code, personally I don't prefer this way. Anyway you have limit for your feature engineering ideas, so try to carefully select features.
", "votes": null, "replies": [ { "id": 1631808, "author_name": "alankabisov", "author_url": "", "post_date": "12/28/2021 19:41:43", "content": "If you use LightGBM and you can fit your notebook in 13GB memory you can try to use GPU inference from cuml ForestInference, it also speeds up things in some way
", "votes": null, "replies": [] }, { "id": 1632549, "author_name": "sihead", "author_url": "", "post_date": "12/29/2021 18:38:40", "content": "It is true that feature engineering ideas are limited if we have to restrain our loop time. Anyway, thanks for your advice.
", "votes": null, "replies": [] }, { "id": 1632689, "author_name": "lucasmorin", "author_url": "", "post_date": "12/29/2021 23:47:17", "content": "Pandas is generally rather slow. Base python + numpy is enough to build features in a very fast manner. I've shared a notebook that builds ~300 features in less than 10 minutes of submission time budget: https://www.kaggle.com/lucasmorin/on-line-feature-engineering
", "votes": null, "replies": [] }, { "id": 1647357, "author_name": "marketneutral", "author_url": "", "post_date": "01/12/2022 14:24:23", "content": "Have you managed to actually submit a model to the competition using those features?
", "votes": null, "replies": [] }, { "id": 1647463, "author_name": "lucasmorin", "author_url": "", "post_date": "01/12/2022 16:18:07", "content": "In my feature engineering notebook (https://www.kaggle.com/lucasmorin/on-line-feature-engineering) I estimate the submission time budget at 10 min for 300 features. In this notebook: https://www.kaggle.com/lucasmorin/online-fe-nn-loss-shap I submit a baseline NN with those features. I don't really garantee anything (the NN convergence seems rather poor, there might remain some bugs in the FE loop), but it seems to work in principle. It's totally doable to do some extensive feature engineering as numpy is really fast.
", "votes": null, "replies": [] }, { "id": 1647561, "author_name": "marketneutral", "author_url": "", "post_date": "01/12/2022 17:33:33", "content": "Thanks. Your notebooks should get many more votes. This competition is odd in that so many highly rated notebooks are not doing much of anything.
", "votes": null, "replies": [] } ] }, { "id": 1635456, "author_name": "rluethy", "author_url": "", "post_date": "01/01/2022 19:10:43", "content": "I have been able to reduce the loop time and use more features for my models using Polars dataframes: https://www.kaggle.com/c/g-research-crypto-forecasting/discussion/298250
", "votes": null, "replies": [] }, { "id": 1635551, "author_name": "vlpavlov", "author_url": "", "post_date": "01/01/2022 21:46:00", "content": "One more way is to use Numba https://numba.readthedocs.io/en/stable/ Depending on your approach to feature engineering, Numba might significantly decrease execution time of your code.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1631793": "I noticed that every loop in submission should be less than certain seconds to ensure it will not raise runtime errors when submitting to the LB. I cannot submit my model to the LB because it always raises a runtime error because of either the model or the production of features. Anyone knows how to reduce the loop runtime to help submit it to the LB. Thanks", "1631804": "Try to use tqdm in your loop like:\n```\nfrom tqdm import tqdm\nfor i, (df_test, df_pred) in tqdm(enumerate(iter_test)):\n # your prediction\n\n```and track the number of iterations per second, from my experience they should be 6-7+ iterations per second. This way it would be easier for you to optimise and know what to expect. Its hard to say concrete recommendations for you, one thing you could do if you use separate model for each asset is to calculate features for all assets at one time, not separately. Also try to use .itertuples() or .apply() to loop through df_test and make predictions. And if it's possible for you you can try to use numpy, but this way you will get quite messy code, personally I don't prefer this way. Anyway you have limit for your feature engineering ideas, so try to carefully select features.", "1631808": "If you use LightGBM and you can fit your notebook in 13GB memory you can try to use GPU inference from cuml ForestInference, it also speeds up things in some way", "1632549": "It is true that feature engineering ideas are limited if we have to restrain our loop time. Anyway, thanks for your advice.", "1632689": "Pandas is generally rather slow. Base python + numpy is enough to build features in a very fast manner. I've shared a notebook that builds ~300 features in less than 10 minutes of submission time budget: https://www.kaggle.com/lucasmorin/on-line-feature-engineering", "1635456": "I have been able to reduce the loop time and use more features for my models using Polars dataframes: https://www.kaggle.com/c/g-research-crypto-forecasting/discussion/298250", "1635551": "One more way is to use Numba [https://numba.readthedocs.io/en/stable/](https://numba.readthedocs.io/en/stable/) Depending on your approach to feature engineering, Numba might significantly decrease execution time of your code.", "1647357": "Have you managed to actually submit a model to the competition using those features?", "1647463": "In my feature engineering notebook (https://www.kaggle.com/lucasmorin/on-line-feature-engineering) I estimate the submission time budget at 10 min for 300 features. In this notebook: https://www.kaggle.com/lucasmorin/online-fe-nn-loss-shap I submit a baseline NN with those features. I don't really garantee anything (the NN convergence seems rather poor, there might remain some bugs in the FE loop), but it seems to work in principle. It's totally doable to do some extensive feature engineering as numpy is really fast.", "1647561": "Thanks. Your notebooks should get many more votes. This competition is odd in that so many highly rated notebooks are not doing much of anything." }, "source": "meta" }