This competition is full of positions, and I don't know if I can get a good result in the end. But I figured I'd forget about the game if I waited another 3 months, so I decided to share my solution anyway.
\nHi all people diving in this competition,
\nWe had a very memorable racing experience together (at least for me). I learned a lot from this competition, and to be honest, I'm just a newbie in machine learning and time series research. I met a lot of talent here @cdeotte @kaggleqrdl @joshuascj @titericz @batprem @vitalykudelya . Too many friends for me to put them all here.
\nThey kindly shared good baselines, ideas and data, which taught me a lot. I must say that we have reached such score by standing on the shoulders of giants.
Our team used GRU blending with XGBoost as a solution.
\nThe composition of time series data is probably: (this is my understanding, I hope there is no error in expression)
$$timeseries=trend+otherinformantion+noise$$
\nThe general idea is to use GRU to capture the \"trend\" information of the data, and XGBoost to capture \"other information\".
\nExpressed in pseudocode:
\nFit stage
\nGRU. fit(trend_data)
\nXGB.train(X_XGB[other_information_features], residual_data)
Inference stage
\nresidual_data_pred=XGB.predict(X_XGB[other_information_features])
\ntrend_data_pred = GRU. predict(X_GRU)
\nsub_data_pred=trend_data_pred +residual_data_pred
Results: 1st month: 1.333; 3rd month: 2.47; 4th month: 2.95; 5th month: 3.39
\nAlso, I use LB to determine my CV’s performance.
\nI treat January (LB) as \"March\", \"April\" and \"May\" respectively to output the results. (Just slide the GRU forecast starting point backwards from 2022-12-01.)
Only GRU results: 3rd month: 2.82, 4th month: 3.15, 5th month: 3.4
\nGRU blending XGB: 3rd month: 2.66, 4th month:2.8, 5th month: 2.5
\nImprove: 3rd month: 0.16, 4th month:0.35, 5th month: 0.9 (Maybe this is somehow the luck)
\nBoth CV score and LB score all get improvement. XGB can help GRU. Thank you.
\nGood luck to us.
Thank you for sharing your solution! It's great to see how your team combined GRU and XGBoost to capture both trend and other information in the data. Your cross-validation and LB tests demonstrate the improvement achieved by blending the two models. It's always interesting to see different approaches to a problem, and your solution offers valuable insights. Good luck, and I hope your solution performs well on the private leaderboard!
", "votes": 3, "replies": [ { "id": 2184035, "author_name": "Shiner", "author_url": "", "post_date": "2023-03-16T05:13:29.367000", "content": "thank you for your good wishes! @siddharthkumarsah . I hope my solution can get a satisfied result, but there are so many talents in this competition. Anyway, wish us good luck.
", "votes": 0, "replies": [] } ] }, { "id": 2183891, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2023-03-16T02:26:20.697000", "content": "I found this to be the most intriguing solution. However, I think the devil is somewhat in the details. I tried similar things, but couldn't quite figure out how to predict 'trend' or when I did, the improvements were somewhat marginal.
\n@judith007 Is the trend here global or per county?
", "votes": 1, "replies": [ { "id": 2184014, "author_name": "Shiner", "author_url": "", "post_date": "2023-03-16T05:01:57.227000", "content": "Trends do not exist in every county, but most counties with high active/popularity have trends, (up or down). But there are a small number of (very few) towns with very low activity that also have a relatively stable upward trend.
\nMy model tells me that the overall/global trend is all up, but slightly. Somehow it is similar to the Linear Model results, that's why linear model works well in this competition I guess.
\nAs analyzed by many Kagglers, last value is a good choice for small counties, because small county didn't have a stable trend. Instead of guessing whether they will rise or fall in the future, it is better to keep the current value unchanged and you may get more Good SMAPE.
", "votes": 2, "replies": [] } ] }, { "id": 2182445, "author_name": "DongYK", "author_url": "", "post_date": "2023-03-15T05:39:20.787000", "content": "Thank you for sharing your solution.
\nI wanted to use LSTM+XGBoost, but I couldn't come up with a way.
\nNow, I can figure out.
Thank you that you like it. If my solution works well in Private LB, I will share my code.
", "votes": 1, "replies": [] } ] }, { "id": 2182354, "author_name": "tarick.morty", "author_url": "", "post_date": "2023-03-15T04:26:14.953000", "content": "Great job @judith007 congratulations for making the ensemble work in your favour. Interesting to see the 5th month outperforming 3 and 4month sMAPE.
", "votes": 1, "replies": [ { "id": 2182394, "author_name": "Shiner", "author_url": "", "post_date": "2023-03-15T05:02:15.977000", "content": "Thank you. I think your work gets a much better CV score than mine, congrats. @tarique7
", "votes": 1, "replies": [ { "id": 2182444, "author_name": "tarick.morty", "author_url": "", "post_date": "2023-03-15T05:38:46.897000", "content": "Well it's really hard to gauge, CV looks good so we will see 😄
", "votes": 1, "replies": [] } ] } ] }, { "id": 2182327, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2023-03-15T04:02:25.307000", "content": "This is a really great result for 5th month - GRU blending XGB: 3rd month: 2.66, 4th month:2.80, 5th month: 2.50
\nCurious though that it performs better than your 4th and 3rd month. Sort of the opposite to what I saw.
", "votes": 1, "replies": [ { "id": 2182392, "author_name": "Shiner", "author_url": "", "post_date": "2023-03-15T05:01:10.260000", "content": "Yes, it is quite strange. However, my CV result on 5th month is 3.15, which is more reasonable and reliable.
", "votes": 0, "replies": [] } ] }, { "id": 2183021, "author_name": "CPMP", "author_url": "", "post_date": "2023-03-15T12:36:07.673000", "content": "Interesting solution. I used a similar one for web traffic forecasting competition (WTF), i.e. trained a NN, then trained XGB on the residuals. I did not have time to explore this here unfortunately. I tis good to see it worked for you.
\nMy WTF solution is at https://github.com/jfpuget/Kaggle
", "votes": 2, "replies": [ { "id": 2184030, "author_name": "Shiner", "author_url": "", "post_date": "2023-03-16T05:09:54.250000", "content": "Thank you for your comment! @cpmpml Yes, my solution is quite similar to yours, because I did browse your code before I implement my solution. It did work to improve GRU.
", "votes": 2, "replies": [] } ] } ], "comment_count": 13, "messages": [ { "id": 2182292, "postDate": "2023-03-15T03:36:21.003Z", "content": "This competition is full of positions, and I don't know if I can get a good result in the end. But I figured I'd forget about the game if I waited another 3 months, so I decided to share my solution anyway.
\nHi all people diving in this competition,
\nWe had a very memorable racing experience together (at least for me). I learned a lot from this competition, and to be honest, I'm just a newbie in machine learning and time series research. I met a lot of talent here @cdeotte @kaggleqrdl @joshuascj @titericz @batprem @vitalykudelya . Too many friends for me to put them all here.
\nThey kindly shared good baselines, ideas and data, which taught me a lot. I must say that we have reached such score by standing on the shoulders of giants.
Our team used GRU blending with XGBoost as a solution.
\nThe composition of time series data is probably: (this is my understanding, I hope there is no error in expression)
$$timeseries=trend+otherinformantion+noise$$
\nThe general idea is to use GRU to capture the \"trend\" information of the data, and XGBoost to capture \"other information\".
\nExpressed in pseudocode:
\nFit stage
\nGRU. fit(trend_data)
\nXGB.train(X_XGB[other_information_features], residual_data)
Inference stage
\nresidual_data_pred=XGB.predict(X_XGB[other_information_features])
\ntrend_data_pred = GRU. predict(X_GRU)
\nsub_data_pred=trend_data_pred +residual_data_pred
Results: 1st month: 1.333; 3rd month: 2.47; 4th month: 2.95; 5th month: 3.39
\nAlso, I use LB to determine my CV’s performance.
\nI treat January (LB) as \"March\", \"April\" and \"May\" respectively to output the results. (Just slide the GRU forecast starting point backwards from 2022-12-01.)
Only GRU results: 3rd month: 2.82, 4th month: 3.15, 5th month: 3.4
\nGRU blending XGB: 3rd month: 2.66, 4th month:2.8, 5th month: 2.5
\nImprove: 3rd month: 0.16, 4th month:0.35, 5th month: 0.9 (Maybe this is somehow the luck)
\nBoth CV score and LB score all get improvement. XGB can help GRU. Thank you.
\nGood luck to us.
Thank you for sharing your solution! It's great to see how your team combined GRU and XGBoost to capture both trend and other information in the data. Your cross-validation and LB tests demonstrate the improvement achieved by blending the two models. It's always interesting to see different approaches to a problem, and your solution offers valuable insights. Good luck, and I hope your solution performs well on the private leaderboard!
", "rawMarkdown": "Thank you for sharing your solution! It's great to see how your team combined GRU and XGBoost to capture both trend and other information in the data. Your cross-validation and LB tests demonstrate the improvement achieved by blending the two models. It's always interesting to see different approaches to a problem, and your solution offers valuable insights. Good luck, and I hope your solution performs well on the private leaderboard!", "votes": 3, "replies": [ { "id": 2184035, "postDate": "2023-03-16T05:13:29.367Z", "content": "thank you for your good wishes! @siddharthkumarsah . I hope my solution can get a satisfied result, but there are so many talents in this competition. Anyway, wish us good luck.
", "rawMarkdown": "thank you for your good wishes! @siddharthkumarsah . I hope my solution can get a satisfied result, but there are so many talents in this competition. Anyway, wish us good luck." } ] }, { "id": 2183891, "postDate": "2023-03-16T02:26:20.697Z", "content": "I found this to be the most intriguing solution. However, I think the devil is somewhat in the details. I tried similar things, but couldn't quite figure out how to predict 'trend' or when I did, the improvements were somewhat marginal.
\n@judith007 Is the trend here global or per county?
", "rawMarkdown": "I found this to be the most intriguing solution. However, I think the devil is somewhat in the details. I tried similar things, but couldn't quite figure out how to predict 'trend' or when I did, the improvements were somewhat marginal. \n\n@judith007 Is the trend here global or per county? \n", "votes": 1, "replies": [ { "id": 2184014, "postDate": "2023-03-16T05:01:57.227Z", "content": "Trends do not exist in every county, but most counties with high active/popularity have trends, (up or down). But there are a small number of (very few) towns with very low activity that also have a relatively stable upward trend.
\nMy model tells me that the overall/global trend is all up, but slightly. Somehow it is similar to the Linear Model results, that's why linear model works well in this competition I guess.
\nAs analyzed by many Kagglers, last value is a good choice for small counties, because small county didn't have a stable trend. Instead of guessing whether they will rise or fall in the future, it is better to keep the current value unchanged and you may get more Good SMAPE.
", "rawMarkdown": "Trends do not exist in every county, but most counties with high active/popularity have trends, (up or down). But there are a small number of (very few) towns with very low activity that also have a relatively stable upward trend. \n\nMy model tells me that the overall/global trend is all up, but slightly. Somehow it is similar to the Linear Model results, that's why linear model works well in this competition I guess.\n\nAs analyzed by many Kagglers, last value is a good choice for small counties, because small county didn't have a stable trend. Instead of guessing whether they will rise or fall in the future, it is better to keep the current value unchanged and you may get more Good SMAPE.", "votes": 2 } ] }, { "id": 2182445, "postDate": "2023-03-15T05:39:20.787Z", "content": "Thank you for sharing your solution.
\nI wanted to use LSTM+XGBoost, but I couldn't come up with a way.
\nNow, I can figure out.
Thank you that you like it. If my solution works well in Private LB, I will share my code.
", "rawMarkdown": "Thank you that you like it. If my solution works well in Private LB, I will share my code.", "votes": 1 } ] }, { "id": 2182354, "postDate": "2023-03-15T04:26:14.953Z", "content": "Great job @judith007 congratulations for making the ensemble work in your favour. Interesting to see the 5th month outperforming 3 and 4month sMAPE.
", "rawMarkdown": "Great job @judith007 congratulations for making the ensemble work in your favour. Interesting to see the 5th month outperforming 3 and 4month sMAPE. ", "votes": 1, "replies": [ { "id": 2182394, "postDate": "2023-03-15T05:02:15.977Z", "content": "Thank you. I think your work gets a much better CV score than mine, congrats. @tarique7
", "rawMarkdown": "Thank you. I think your work gets a much better CV score than mine, congrats. @tarique7 ", "votes": 1, "replies": [ { "id": 2182444, "postDate": "2023-03-15T05:38:46.897Z", "content": "Well it's really hard to gauge, CV looks good so we will see 😄
", "rawMarkdown": "Well it's really hard to gauge, CV looks good so we will see 😄", "votes": 1 } ] } ] }, { "id": 2182327, "postDate": "2023-03-15T04:02:25.307Z", "content": "This is a really great result for 5th month - GRU blending XGB: 3rd month: 2.66, 4th month:2.80, 5th month: 2.50
\nCurious though that it performs better than your 4th and 3rd month. Sort of the opposite to what I saw.
", "rawMarkdown": "This is a really great result for 5th month - GRU blending XGB: 3rd month: 2.66, 4th month:2.80, 5th month: 2.50\n\nCurious though that it performs better than your 4th and 3rd month. Sort of the opposite to what I saw.", "votes": 1, "replies": [ { "id": 2182392, "postDate": "2023-03-15T05:01:10.260Z", "content": "Yes, it is quite strange. However, my CV result on 5th month is 3.15, which is more reasonable and reliable.
", "rawMarkdown": "Yes, it is quite strange. However, my CV result on 5th month is 3.15, which is more reasonable and reliable." } ] }, { "id": 2183021, "postDate": "2023-03-15T12:36:07.673Z", "content": "Interesting solution. I used a similar one for web traffic forecasting competition (WTF), i.e. trained a NN, then trained XGB on the residuals. I did not have time to explore this here unfortunately. I tis good to see it worked for you.
\nMy WTF solution is at https://github.com/jfpuget/Kaggle
", "rawMarkdown": "Interesting solution. I used a similar one for web traffic forecasting competition (WTF), i.e. trained a NN, then trained XGB on the residuals. I did not have time to explore this here unfortunately. I tis good to see it worked for you.\n\nMy WTF solution is at https://github.com/jfpuget/Kaggle", "votes": 2, "replies": [ { "id": 2184030, "postDate": "2023-03-16T05:09:54.250Z", "content": "Thank you for your comment! @cpmpml Yes, my solution is quite similar to yours, because I did browse your code before I implement my solution. It did work to improve GRU.
", "rawMarkdown": "Thank you for your comment! @cpmpml Yes, my solution is quite similar to yours, because I did browse your code before I implement my solution. It did work to improve GRU.", "votes": 2 } ] } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T19:27:40Z", "competition_id": 41881, "competition_slug": "godaddy-microbusiness-density-forecasting", "team_id": 9832840, "solution_url": "/competitions/godaddy-microbusiness-density-forecasting/writeups/s-g-d-godday-s-solution-that-didn-t-appear-in-publ" }