For a while, I was stuck at an RMSE of 3.696 using XGBoost. I ran with nround=1,000 and eta=.01, which took about an hour to train. Then, I tested lightgbm. It gave me an RMSE of 3.687 and it took only a couple of minutes to train. So, if you are using xgboost, it seems like lightgbm returns a lower RMSE at a fraction of the time.\nFor those of you that have used both models, have you had the same experience? Share your thoughts in the comments.\nAlso, if you have gotten better results with a different algorithm, I would love to hear about it.
", "messages": [ { "id": "471049", "postDate": "02/14/2019 00:21:37", "content": "For a while, I was stuck at an RMSE of 3.696 using XGBoost. I ran with nround=1,000 and eta=.01, which took about an hour to train. Then, I tested lightgbm. It gave me an RMSE of 3.687 and it took only a couple of minutes to train. So, if you are using xgboost, it seems like lightgbm returns a lower RMSE at a fraction of the time.\nFor those of you that have used both models, have you had the same experience? Share your thoughts in the comments.\nAlso, if you have gotten better results with a different algorithm, I would love to hear about it.
", "rawMarkdown": "For a while, I was stuck at an RMSE of 3.696 using XGBoost. I ran with nround=1,000 and eta=.01, which took about an hour to train. Then, I tested lightgbm. It gave me an RMSE of 3.687 and it took only a couple of minutes to train. So, if you are using xgboost, it seems like lightgbm returns a lower RMSE at a fraction of the time.\nFor those of you that have used both models, have you had the same experience? Share your thoughts in the comments.\nAlso, if you have gotten better results with a different algorithm, I would love to hear about it.", "votes": null }, { "id": "471055", "postDate": "02/14/2019 00:48:26", "content": "I'm using both of them, but I got RMSE score of LightGBM(3.7xx, in CV) is lower than XGBoost(4.1xx, in CV)... Hyperparameter is tuned by optuna.
", "rawMarkdown": "I'm using both of them, but I got RMSE score of LightGBM(3.7xx, in CV) is lower than XGBoost(4.1xx, in CV)... Hyperparameter is tuned by optuna.", "votes": null }, { "id": "471714", "postDate": "02/14/2019 19:56:24", "content": "Light GBM is known for its fast training and differs from XGB in 2 things. \nEFB - efficient feature bundling and GOSS - Gradient one side sampling. You can read more about it online.
\n\nIn short, XGB would give you better accuracy than Lightgbm. Turn off GOSS and EFB in Lightgbm and it should be very very near to XGB in accuracy.
", "rawMarkdown": "Light GBM is known for its fast training and differs from XGB in 2 things. \nEFB - efficient feature bundling and GOSS - Gradient one side sampling. You can read more about it online.\n\nIn short, XGB would give you better accuracy than Lightgbm. Turn off GOSS and EFB in Lightgbm and it should be very very near to XGB in accuracy.", "votes": null }, { "id": "471944", "postDate": "02/15/2019 06:00:16", "content": "Combining both might also help you.
", "rawMarkdown": "Combining both might also help you.", "votes": null }, { "id": "471949", "postDate": "02/15/2019 06:13:46", "content": "Harshit - u guys did a tremendous jobs and you are in 5th Positio. Congratulation man.\nany tips for me?
", "rawMarkdown": "Harshit - u guys did a tremendous jobs and you are in 5th Positio. Congratulation man.\nany tips for me?", "votes": null }, { "id": "472259", "postDate": "02/15/2019 15:36:54", "content": "We started focusing on outliers/manual predictions now, It may overfit in phase two, but we weren't able to improve our single model more now, so adopting this method.
", "rawMarkdown": "We started focusing on outliers/manual predictions now, It may overfit in phase two, but we weren't able to improve our single model more now, so adopting this method.", "votes": null }, { "id": "472280", "postDate": "02/15/2019 16:19:30", "content": "If you don't mind me asking, what is the best single model that you could achieve before post-processing? And, have you tried stacking yet?
", "rawMarkdown": "If you don't mind me asking, what is the best single model that you could achieve before post-processing? And, have you tried stacking yet?", "votes": null }, { "id": "472298", "postDate": "02/15/2019 16:47:53", "content": "Kyle, Best single is 3.680 on lb with cv 3.64997. After stacking it’s 3.677.
", "rawMarkdown": "Kyle, Best single is 3.680 on lb with cv 3.64997. After stacking it’s 3.677.", "votes": null }, { "id": "472403", "postDate": "02/15/2019 20:42:24", "content": "I am using both and i found LightBGM got good score than XGBoost
", "rawMarkdown": "I am using both and i found LightBGM got good score than XGBoost", "votes": null }, { "id": "472689", "postDate": "02/16/2019 13:25:36", "content": "From my experience, now the lightgbm can have better accuracy than xgboost. When using LightGBM , you need to pay attention to max_depth and num_of_leaves.
", "rawMarkdown": "From my experience, now the lightgbm can have better accuracy than xgboost. When using LightGBM , you need to pay attention to max_depth and num_of_leaves.", "votes": null } ], "comments": [ { "id": 471055, "author_name": "xxbxyae", "author_url": "", "post_date": "02/14/2019 00:48:26", "content": "I'm using both of them, but I got RMSE score of LightGBM(3.7xx, in CV) is lower than XGBoost(4.1xx, in CV)... Hyperparameter is tuned by optuna.
", "votes": null, "replies": [] }, { "id": 471714, "author_name": "mayanksoni", "author_url": "", "post_date": "02/14/2019 19:56:24", "content": "Light GBM is known for its fast training and differs from XGB in 2 things. \nEFB - efficient feature bundling and GOSS - Gradient one side sampling. You can read more about it online.
\n\nIn short, XGB would give you better accuracy than Lightgbm. Turn off GOSS and EFB in Lightgbm and it should be very very near to XGB in accuracy.
", "votes": null, "replies": [ { "id": 472689, "author_name": "longyin2", "author_url": "", "post_date": "02/16/2019 13:25:36", "content": "From my experience, now the lightgbm can have better accuracy than xgboost. When using LightGBM , you need to pay attention to max_depth and num_of_leaves.
", "votes": null, "replies": [] } ] }, { "id": 471944, "author_name": "harshit92", "author_url": "", "post_date": "02/15/2019 06:00:16", "content": "Combining both might also help you.
", "votes": null, "replies": [ { "id": 471949, "author_name": "subikashpal", "author_url": "", "post_date": "02/15/2019 06:13:46", "content": "Harshit - u guys did a tremendous jobs and you are in 5th Positio. Congratulation man.\nany tips for me?
", "votes": null, "replies": [] }, { "id": 472259, "author_name": "harshit92", "author_url": "", "post_date": "02/15/2019 15:36:54", "content": "We started focusing on outliers/manual predictions now, It may overfit in phase two, but we weren't able to improve our single model more now, so adopting this method.
", "votes": null, "replies": [] }, { "id": 472280, "author_name": "petersk20", "author_url": "", "post_date": "02/15/2019 16:19:30", "content": "If you don't mind me asking, what is the best single model that you could achieve before post-processing? And, have you tried stacking yet?
", "votes": null, "replies": [] }, { "id": 472298, "author_name": "harshit92", "author_url": "", "post_date": "02/15/2019 16:47:53", "content": "Kyle, Best single is 3.680 on lb with cv 3.64997. After stacking it’s 3.677.
", "votes": null, "replies": [] } ] }, { "id": 472403, "author_name": "kalyankkr", "author_url": "", "post_date": "02/15/2019 20:42:24", "content": "I am using both and i found LightBGM got good score than XGBoost
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "471049": "For a while, I was stuck at an RMSE of 3.696 using XGBoost. I ran with nround=1,000 and eta=.01, which took about an hour to train. Then, I tested lightgbm. It gave me an RMSE of 3.687 and it took only a couple of minutes to train. So, if you are using xgboost, it seems like lightgbm returns a lower RMSE at a fraction of the time.\nFor those of you that have used both models, have you had the same experience? Share your thoughts in the comments.\nAlso, if you have gotten better results with a different algorithm, I would love to hear about it.", "471055": "I'm using both of them, but I got RMSE score of LightGBM(3.7xx, in CV) is lower than XGBoost(4.1xx, in CV)... Hyperparameter is tuned by optuna.", "471714": "Light GBM is known for its fast training and differs from XGB in 2 things. \nEFB - efficient feature bundling and GOSS - Gradient one side sampling. You can read more about it online.\n\nIn short, XGB would give you better accuracy than Lightgbm. Turn off GOSS and EFB in Lightgbm and it should be very very near to XGB in accuracy.", "471944": "Combining both might also help you.", "471949": "Harshit - u guys did a tremendous jobs and you are in 5th Positio. Congratulation man.\nany tips for me?", "472259": "We started focusing on outliers/manual predictions now, It may overfit in phase two, but we weren't able to improve our single model more now, so adopting this method.", "472280": "If you don't mind me asking, what is the best single model that you could achieve before post-processing? And, have you tried stacking yet?", "472298": "Kyle, Best single is 3.680 on lb with cv 3.64997. After stacking it’s 3.677.", "472403": "I am using both and i found LightBGM got good score than XGBoost", "472689": "From my experience, now the lightgbm can have better accuracy than xgboost. When using LightGBM , you need to pay attention to max_depth and num_of_leaves." }, "source": "meta" }