At the risk of being accused of self-promotion, I have a kernel here that might interest you:
\n\nhttps://www.kaggle.com/bigironsphere/parameter-tuning-in-one-function-with-hyperopt
", "rawMarkdown": "At the risk of being accused of self-promotion, I have a kernel here that might interest you:\n\nhttps://www.kaggle.com/bigironsphere/parameter-tuning-in-one-function-with-hyperopt", "votes": null }, { "id": "590320", "postDate": "08/02/2019 04:29:00", "content": "I'm trying !
", "rawMarkdown": "I'm trying !", "votes": null }, { "id": "590409", "postDate": "08/02/2019 06:57:00", "content": "Your kernel was great in LANL competition with it's small dataset, but here it's unapplicable, I think. It will take over 9000 years to iterate through lgb parameters and run it on each set of parameters with data for this competition. \nTho it's worth a try to make a subsample, but to run it in kaggle kernel one should use like ~0.2% of all data... Not so representative, IMHO.
\n\nI'm not a hater, it's just a thought :)
", "rawMarkdown": "Your kernel was great in LANL competition with it's small dataset, but here it's unapplicable, I think. It will take over 9000 years to iterate through lgb parameters and run it on each set of parameters with data for this competition. \nTho it's worth a try to make a subsample, but to run it in kaggle kernel one should use like ~0.2% of all data... Not so representative, IMHO.\n\nI'm not a hater, it's just a thought :)", "votes": null }, { "id": "590625", "postDate": "08/02/2019 12:12:24", "content": "manual tuning yields great results.
", "rawMarkdown": "manual tuning yields great results.", "votes": null }, { "id": "590808", "postDate": "08/02/2019 16:16:33", "content": "@stanislavblinov Ideally you would limit the parameter space a lot more than in that kernel! The function I wrote is a blueprint that should be altered for use cases. It can indeed be slow, but if you're making models for each type it can run well until you encounter enormous datasets like 3JHC at which point you'd need sampling. But I agree, you would certainly not be able to run 2000 rounds of Hyperopt search with this data.
", "rawMarkdown": "stanislavblinov Ideally you would limit the parameter space a lot more than in that kernel! The function I wrote is a blueprint that should be altered for use cases. It can indeed be slow, but if you're making models for each type it can run well until you encounter enormous datasets like 3JHC at which point you'd need sampling. But I agree, you would certainly not be able to run 2000 rounds of Hyperopt search with this data.", "votes": null }, { "id": "590821", "postDate": "08/02/2019 16:53:08", "content": "Yes....I'm trying !
", "rawMarkdown": "Yes....I'm trying !", "votes": null }, { "id": "590823", "postDate": "08/02/2019 16:55:53", "content": "focus first on major parameters:\nlearning rate\nmax number of leaves\nsubsample
", "rawMarkdown": "focus first on major parameters:\nlearning rate\nmax number of leaves\nsubsample", "votes": null }, { "id": "590830", "postDate": "08/02/2019 17:07:54", "content": "Thank you very much @cpmpml ^ ^
", "rawMarkdown": "Thank you very much @cpmpml ^ ^", "votes": null }, { "id": "590840", "postDate": "08/02/2019 17:29:02", "content": "I'm trying with more parameter\n ('num_leaves': 255,\n 'min_child_samples': 79,\n 'min_data_in_leaf': 100,\n 'objective': 'regression',\n 'max_depth': 9,\n 'learning_rate': 0.2,\n 'min_child_samples': 20,\n 'min_gain_to_split': 0.02,\n 'min_data_in_bin': 3,\n \"boosting_type\": \"gbdt\",\n \"subsample_freq\": 1,\n \"subsample\": 0.9,\n \"bagging_seed\": 11,\n \"metric\": 'mae',\n \"verbosity\": -1,\n 'max_bin': 127,\n 'reg_alpha': 0.1,\n 'reg_lambda': 0.3,\n 'colsample_bytree': 1.0,\n 'num_iterations': 5000)\nand my score is worse. I'will trying focus first on major parameters: learning rate, max number of leaves subsample, max_depth, min_data_in_leaf,...\nI will also try with bayesian optimization (or GridSearchCV) for find major parameter (learning rate, max number of leaves subsample,...)
", "rawMarkdown": "I'm trying with more parameter\n ('num_leaves': 255,\n 'min_child_samples': 79,\n 'min_data_in_leaf': 100,\n 'objective': 'regression',\n 'max_depth': 9,\n 'learning_rate': 0.2,\n 'min_child_samples': 20,\n 'min_gain_to_split': 0.02,\n 'min_data_in_bin': 3,\n \"boosting_type\": \"gbdt\",\n \"subsample_freq\": 1,\n \"subsample\": 0.9,\n \"bagging_seed\": 11,\n \"metric\": 'mae',\n \"verbosity\": -1,\n 'max_bin': 127,\n 'reg_alpha': 0.1,\n 'reg_lambda': 0.3,\n 'colsample_bytree': 1.0,\n 'num_iterations': 5000)\nand my score is worse. I'will trying focus first on major parameters: learning rate, max number of leaves subsample, max_depth, min_data_in_leaf,...\nI will also try with bayesian optimization (or GridSearchCV) for find major parameter (learning rate, max number of leaves subsample,...)", "votes": null }, { "id": "590861", "postDate": "08/02/2019 18:09:07", "content": "you should not tune the seed !!!!!!!!
", "rawMarkdown": "you should not tune the seed !!!!!!!!", "votes": null }, { "id": "590876", "postDate": "08/02/2019 18:30:23", "content": "The best value is always 42 anyway!
", "rawMarkdown": "The best value is always 42 anyway!", "votes": null } ], "comments": [ { "id": 590231, "author_name": "bigironsphere", "author_url": "", "post_date": "08/01/2019 23:01:27", "content": "At the risk of being accused of self-promotion, I have a kernel here that might interest you:
\n\nhttps://www.kaggle.com/bigironsphere/parameter-tuning-in-one-function-with-hyperopt
", "votes": null, "replies": [ { "id": 590320, "author_name": "duykhanh99", "author_url": "", "post_date": "08/02/2019 04:29:00", "content": "I'm trying !
", "votes": null, "replies": [] }, { "id": 590409, "author_name": "stanislavblinov", "author_url": "", "post_date": "08/02/2019 06:57:00", "content": "Your kernel was great in LANL competition with it's small dataset, but here it's unapplicable, I think. It will take over 9000 years to iterate through lgb parameters and run it on each set of parameters with data for this competition. \nTho it's worth a try to make a subsample, but to run it in kaggle kernel one should use like ~0.2% of all data... Not so representative, IMHO.
\n\nI'm not a hater, it's just a thought :)
", "votes": null, "replies": [] }, { "id": 590808, "author_name": "bigironsphere", "author_url": "", "post_date": "08/02/2019 16:16:33", "content": "@stanislavblinov Ideally you would limit the parameter space a lot more than in that kernel! The function I wrote is a blueprint that should be altered for use cases. It can indeed be slow, but if you're making models for each type it can run well until you encounter enormous datasets like 3JHC at which point you'd need sampling. But I agree, you would certainly not be able to run 2000 rounds of Hyperopt search with this data.
", "votes": null, "replies": [] } ] }, { "id": 590625, "author_name": "cpmpml", "author_url": "", "post_date": "08/02/2019 12:12:24", "content": "manual tuning yields great results.
", "votes": null, "replies": [ { "id": 590821, "author_name": "duykhanh99", "author_url": "", "post_date": "08/02/2019 16:53:08", "content": "Yes....I'm trying !
", "votes": null, "replies": [] }, { "id": 590823, "author_name": "cpmpml", "author_url": "", "post_date": "08/02/2019 16:55:53", "content": "focus first on major parameters:\nlearning rate\nmax number of leaves\nsubsample
", "votes": null, "replies": [] }, { "id": 590830, "author_name": "duykhanh99", "author_url": "", "post_date": "08/02/2019 17:07:54", "content": "Thank you very much @cpmpml ^ ^
", "votes": null, "replies": [] }, { "id": 590840, "author_name": "duykhanh99", "author_url": "", "post_date": "08/02/2019 17:29:02", "content": "I'm trying with more parameter\n ('num_leaves': 255,\n 'min_child_samples': 79,\n 'min_data_in_leaf': 100,\n 'objective': 'regression',\n 'max_depth': 9,\n 'learning_rate': 0.2,\n 'min_child_samples': 20,\n 'min_gain_to_split': 0.02,\n 'min_data_in_bin': 3,\n \"boosting_type\": \"gbdt\",\n \"subsample_freq\": 1,\n \"subsample\": 0.9,\n \"bagging_seed\": 11,\n \"metric\": 'mae',\n \"verbosity\": -1,\n 'max_bin': 127,\n 'reg_alpha': 0.1,\n 'reg_lambda': 0.3,\n 'colsample_bytree': 1.0,\n 'num_iterations': 5000)\nand my score is worse. I'will trying focus first on major parameters: learning rate, max number of leaves subsample, max_depth, min_data_in_leaf,...\nI will also try with bayesian optimization (or GridSearchCV) for find major parameter (learning rate, max number of leaves subsample,...)
", "votes": null, "replies": [] }, { "id": 590861, "author_name": "cpmpml", "author_url": "", "post_date": "08/02/2019 18:09:07", "content": "you should not tune the seed !!!!!!!!
", "votes": null, "replies": [] }, { "id": 590876, "author_name": "bigironsphere", "author_url": "", "post_date": "08/02/2019 18:30:23", "content": "The best value is always 42 anyway!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "588114": "", "590231": "At the risk of being accused of self-promotion, I have a kernel here that might interest you:\n\nhttps://www.kaggle.com/bigironsphere/parameter-tuning-in-one-function-with-hyperopt", "590320": "I'm trying !", "590409": "Your kernel was great in LANL competition with it's small dataset, but here it's unapplicable, I think. It will take over 9000 years to iterate through lgb parameters and run it on each set of parameters with data for this competition. \nTho it's worth a try to make a subsample, but to run it in kaggle kernel one should use like ~0.2% of all data... Not so representative, IMHO.\n\nI'm not a hater, it's just a thought :)", "590625": "manual tuning yields great results.", "590808": "stanislavblinov Ideally you would limit the parameter space a lot more than in that kernel! The function I wrote is a blueprint that should be altered for use cases. It can indeed be slow, but if you're making models for each type it can run well until you encounter enormous datasets like 3JHC at which point you'd need sampling. But I agree, you would certainly not be able to run 2000 rounds of Hyperopt search with this data.", "590821": "Yes....I'm trying !", "590823": "focus first on major parameters:\nlearning rate\nmax number of leaves\nsubsample", "590830": "Thank you very much @cpmpml ^ ^", "590840": "I'm trying with more parameter\n ('num_leaves': 255,\n 'min_child_samples': 79,\n 'min_data_in_leaf': 100,\n 'objective': 'regression',\n 'max_depth': 9,\n 'learning_rate': 0.2,\n 'min_child_samples': 20,\n 'min_gain_to_split': 0.02,\n 'min_data_in_bin': 3,\n \"boosting_type\": \"gbdt\",\n \"subsample_freq\": 1,\n \"subsample\": 0.9,\n \"bagging_seed\": 11,\n \"metric\": 'mae',\n \"verbosity\": -1,\n 'max_bin': 127,\n 'reg_alpha': 0.1,\n 'reg_lambda': 0.3,\n 'colsample_bytree': 1.0,\n 'num_iterations': 5000)\nand my score is worse. I'will trying focus first on major parameters: learning rate, max number of leaves subsample, max_depth, min_data_in_leaf,...\nI will also try with bayesian optimization (or GridSearchCV) for find major parameter (learning rate, max number of leaves subsample,...)", "590861": "you should not tune the seed !!!!!!!!", "590876": "The best value is always 42 anyway!" }, "source": "meta" }