{
  "id": 47667,
  "title": "12th place solution",
  "url": "/competitions/favorita-grocery-sales-forecasting/writeups/antklen-12th-place-solution",
  "author_name": "",
  "post_date": "2018-01-17T14:54:47.774990Z",
  "votes": 14,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Code with main parts of the solution is here \n<a href=\"https://github.com/antklen/kaggle_favorita-grocery-sales\">https://github.com/antklen/kaggle_favorita-grocery-sales</a>. </p>\n\n<h2>Feature engineering</h2>\n\n<p>I used the same features as in public scripts, namely</p>\n\n<ul>\n<li>mean sales during several last periods for item/store pairs</li>\n<li>mean sales during several last periods for item/store pairs for each day of week</li>\n<li>mean ompromotion during several last periods for item/store pairs\nbut as for time windows, I used more history - up to 1 year</li>\n</ul>\n\n<p>plus some other grouping averages</p>\n\n<ul>\n<li>mean sales during several last periods for item/store pairs for each day of month</li>\n<li>mean number of zero sales days during several last periods for item/store pairs</li>\n<li>mean sales during several last periods for store/item family</li>\n<li>mean sales during several last periods for store/item class</li>\n<li>mean sales during several last periods for item/store city</li>\n<li>mean sales during several last periods for item/store stat</li>\n<li>mean sales during several last periods for item/store type</li>\n<li>mean sales during several last periods for item/store cluster</li>\n<li>mean sales during several last periods group by only item</li>\n<li>mean number of zero sales days during several last periods group by only stores</li>\n</ul>\n\n<p>plus some differences between mean features with different time window (for example, mean_7days - mean_28days)</p>\n\n<p>plus linear regression on each item/store pair time series (for incorporating possible time trend)</p>\n\n<p>plus mean promo for all 16 days (mean of promo_0, promo_1, ... in Ceshine Lee's script).</p>\n\n<p>as for categorical features (store and item info - family, type, etc.)</p>\n\n<ul>\n<li>standard label encoding for lightgbm</li>\n<li>embeddings for neural network</li>\n</ul>\n\n<p>To be honest, base mean, mean by day of week and promo features do almost all the job, my additional features added not much value.</p>\n\n<h2>Onpromotion issue</h2>\n\n<p>As for problem with missing onpromotion data, I didn't come up with any clever solution. But taking as less promo based features as possible helped on public LB and helped to have more stable relation between validation and  LB (but still not very good). Firstly, I made some extra features based on promo and got stuck around public scripts, but when I removed them, I was able to climb up on the leaderboard.</p>\n\n<h2>Training and validation scheme</h2>\n\n<p>For validation nothing fancy, I used just the same scheme as in Ceshine Lee's script - 26.07-10.08 for validation, several previous 16-days periods for train. May be it would be useful to have several periods for validation (Giba and CPMP reported that it helped), but we had a lot of data here and training took a lot of time, so I decided to stay with only one period.</p>\n\n<p>What helped to improve score is just to take more data for training. Switching from 6 16-days periods to 20-25 periods  helped. During last several days I was running the same models but for even longer training history - 52, 75 and 104 periods (1 year, 1.5 year and 2 year for training). Taking more history consistently improved validation score, taking too long (104 periods)  made public score worse, but really surprisingly on private LB even 52 periods was too much - 25 periods was enough, I have the best private LB score on single models with 25 periods.</p>\n\n<h2>Models</h2>\n\n<p>I used LightGBM and neural network with categorical embeddings on the same set of features.</p>\n\n<p>Also I used two different approaches:</p>\n\n<ol>\n<li>Train one general model for all lags (very memory-consuming=)</li>\n<li>Train separate model for each lag. But in a little different way than in public scripts. I didn't use the same set of features for all lags. For each lag I constructed a little different set of features on-the-fly. The motivation for that is that I thought that it's not a good idea to feed into the model all day of week mean features when we train model only for one day of week. For example, if we train model for monday, why we add features for all other weekdays? It probably adds some noise.</li>\n</ol>\n\n<p>Switching to these two schemes from scheme used in public scripts really improved my score and led me to the top.</p>\n\n<h2>Final models</h2>\n\n<p>So in the end I had 4 types of models (lightgbm or neural network + general model for all lags or separate model for each lag),  trained on several training sets with different length.\nMy final models were simple averages of some of these models.</p>\n\n<p>Averaging models helped a lot here. Public and private score of my single models have very bad correlation. The best single public model (0.507) has only 0.519 on private, while best private model (0.514) had 0.510 on public. But my blending submissions are much more consistent. All my last blending submissions (about 10 submissions) are in the interval [0.506, 0.508] on public and in the interval [0.513, 0.516] on private.</p>",
  "messages": [
    {
      "id": "269943",
      "postDate": "01/17/2018 14:54:47",
      "content": "<p>Code with main parts of the solution is here \n<a href=\"https://github.com/antklen/kaggle_favorita-grocery-sales\">https://github.com/antklen/kaggle_favorita-grocery-sales</a>. </p>\n\n<h2>Feature engineering</h2>\n\n<p>I used the same features as in public scripts, namely</p>\n\n<ul>\n<li>mean sales during several last periods for item/store pairs</li>\n<li>mean sales during several last periods for item/store pairs for each day of week</li>\n<li>mean ompromotion during several last periods for item/store pairs\nbut as for time windows, I used more history - up to 1 year</li>\n</ul>\n\n<p>plus some other grouping averages</p>\n\n<ul>\n<li>mean sales during several last periods for item/store pairs for each day of month</li>\n<li>mean number of zero sales days during several last periods for item/store pairs</li>\n<li>mean sales during several last periods for store/item family</li>\n<li>mean sales during several last periods for store/item class</li>\n<li>mean sales during several last periods for item/store city</li>\n<li>mean sales during several last periods for item/store stat</li>\n<li>mean sales during several last periods for item/store type</li>\n<li>mean sales during several last periods for item/store cluster</li>\n<li>mean sales during several last periods group by only item</li>\n<li>mean number of zero sales days during several last periods group by only stores</li>\n</ul>\n\n<p>plus some differences between mean features with different time window (for example, mean_7days - mean_28days)</p>\n\n<p>plus linear regression on each item/store pair time series (for incorporating possible time trend)</p>\n\n<p>plus mean promo for all 16 days (mean of promo_0, promo_1, ... in Ceshine Lee's script).</p>\n\n<p>as for categorical features (store and item info - family, type, etc.)</p>\n\n<ul>\n<li>standard label encoding for lightgbm</li>\n<li>embeddings for neural network</li>\n</ul>\n\n<p>To be honest, base mean, mean by day of week and promo features do almost all the job, my additional features added not much value.</p>\n\n<h2>Onpromotion issue</h2>\n\n<p>As for problem with missing onpromotion data, I didn't come up with any clever solution. But taking as less promo based features as possible helped on public LB and helped to have more stable relation between validation and  LB (but still not very good). Firstly, I made some extra features based on promo and got stuck around public scripts, but when I removed them, I was able to climb up on the leaderboard.</p>\n\n<h2>Training and validation scheme</h2>\n\n<p>For validation nothing fancy, I used just the same scheme as in Ceshine Lee's script - 26.07-10.08 for validation, several previous 16-days periods for train. May be it would be useful to have several periods for validation (Giba and CPMP reported that it helped), but we had a lot of data here and training took a lot of time, so I decided to stay with only one period.</p>\n\n<p>What helped to improve score is just to take more data for training. Switching from 6 16-days periods to 20-25 periods  helped. During last several days I was running the same models but for even longer training history - 52, 75 and 104 periods (1 year, 1.5 year and 2 year for training). Taking more history consistently improved validation score, taking too long (104 periods)  made public score worse, but really surprisingly on private LB even 52 periods was too much - 25 periods was enough, I have the best private LB score on single models with 25 periods.</p>\n\n<h2>Models</h2>\n\n<p>I used LightGBM and neural network with categorical embeddings on the same set of features.</p>\n\n<p>Also I used two different approaches:</p>\n\n<ol>\n<li>Train one general model for all lags (very memory-consuming=)</li>\n<li>Train separate model for each lag. But in a little different way than in public scripts. I didn't use the same set of features for all lags. For each lag I constructed a little different set of features on-the-fly. The motivation for that is that I thought that it's not a good idea to feed into the model all day of week mean features when we train model only for one day of week. For example, if we train model for monday, why we add features for all other weekdays? It probably adds some noise.</li>\n</ol>\n\n<p>Switching to these two schemes from scheme used in public scripts really improved my score and led me to the top.</p>\n\n<h2>Final models</h2>\n\n<p>So in the end I had 4 types of models (lightgbm or neural network + general model for all lags or separate model for each lag),  trained on several training sets with different length.\nMy final models were simple averages of some of these models.</p>\n\n<p>Averaging models helped a lot here. Public and private score of my single models have very bad correlation. The best single public model (0.507) has only 0.519 on private, while best private model (0.514) had 0.510 on public. But my blending submissions are much more consistent. All my last blending submissions (about 10 submissions) are in the interval [0.506, 0.508] on public and in the interval [0.513, 0.516] on private.</p>",
      "rawMarkdown": "Code with main parts of the solution is here \nhttps://github.com/antklen/kaggle_favorita-grocery-sales. \n\n\n## Feature engineering ##\n\n I used the same features as in public scripts, namely\n\n - mean sales during several last periods for item/store pairs\n - mean sales during several last periods for item/store pairs for each day of week\n - mean ompromotion during several last periods for item/store pairs\nbut as for time windows, I used more history - up to 1 year\n\nplus some other grouping averages\n\n - mean sales during several last periods for item/store pairs for each day of month\n - mean number of zero sales days during several last periods for item/store pairs\n - mean sales during several last periods for store/item family\n - mean sales during several last periods for store/item class\n - mean sales during several last periods for item/store city\n - mean sales during several last periods for item/store stat\n - mean sales during several last periods for item/store type\n - mean sales during several last periods for item/store cluster\n - mean sales during several last periods group by only item\n - mean number of zero sales days during several last periods group by only stores\n\nplus some differences between mean features with different time window (for example, mean_7days - mean_28days)\n\nplus linear regression on each item/store pair time series (for incorporating possible time trend)\n\nplus mean promo for all 16 days (mean of promo_0, promo_1, ... in Ceshine Lee's script).\n\nas for categorical features (store and item info - family, type, etc.)\n\n - standard label encoding for lightgbm\n - embeddings for neural network\n\nTo be honest, base mean, mean by day of week and promo features do almost all the job, my additional features added not much value.\n\n## Onpromotion issue ##\n\nAs for problem with missing onpromotion data, I didn't come up with any clever solution. But taking as less promo based features as possible helped on public LB and helped to have more stable relation between validation and  LB (but still not very good). Firstly, I made some extra features based on promo and got stuck around public scripts, but when I removed them, I was able to climb up on the leaderboard.\n\n\n## Training and validation scheme ##\n\nFor validation nothing fancy, I used just the same scheme as in Ceshine Lee's script - 26.07-10.08 for validation, several previous 16-days periods for train. May be it would be useful to have several periods for validation (Giba and CPMP reported that it helped), but we had a lot of data here and training took a lot of time, so I decided to stay with only one period.\n\nWhat helped to improve score is just to take more data for training. Switching from 6 16-days periods to 20-25 periods  helped. During last several days I was running the same models but for even longer training history - 52, 75 and 104 periods (1 year, 1.5 year and 2 year for training). Taking more history consistently improved validation score, taking too long (104 periods)  made public score worse, but really surprisingly on private LB even 52 periods was too much - 25 periods was enough, I have the best private LB score on single models with 25 periods.\n\n## Models ##\n\nI used LightGBM and neural network with categorical embeddings on the same set of features.\n\nAlso I used two different approaches:\n\n 1. Train one general model for all lags (very memory-consuming=)\n 2. Train separate model for each lag. But in a little different way than in public scripts. I didn't use the same set of features for all lags. For each lag I constructed a little different set of features on-the-fly. The motivation for that is that I thought that it's not a good idea to feed into the model all day of week mean features when we train model only for one day of week. For example, if we train model for monday, why we add features for all other weekdays? It probably adds some noise.\n\nSwitching to these two schemes from scheme used in public scripts really improved my score and led me to the top.\n\n## Final models ##\n\nSo in the end I had 4 types of models (lightgbm or neural network + general model for all lags or separate model for each lag),  trained on several training sets with different length.\nMy final models were simple averages of some of these models.\n\nAveraging models helped a lot here. Public and private score of my single models have very bad correlation. The best single public model (0.507) has only 0.519 on private, while best private model (0.514) had 0.510 on public. But my blending submissions are much more consistent. All my last blending submissions (about 10 submissions) are in the interval [0.506, 0.508] on public and in the interval [0.513, 0.516] on private.",
      "votes": null
    },
    {
      "id": "270434",
      "postDate": "01/18/2018 08:03:58",
      "content": "<p>Thanks for sharing and congrats on the gold medal!</p>",
      "rawMarkdown": "Thanks for sharing and congrats on the gold medal!",
      "votes": null
    },
    {
      "id": "270924",
      "postDate": "01/19/2018 08:53:16",
      "content": "<p>Thanks and congrats on one more step towards Grandmaster=)</p>",
      "rawMarkdown": "Thanks and congrats on one more step towards Grandmaster=)",
      "votes": null
    },
    {
      "id": "573964",
      "postDate": "07/13/2019 03:36:49",
      "content": "<p>Thanks for sharing. How's your linear regression features, how much can you gain from it.</p>",
      "rawMarkdown": "Thanks for sharing. How's your linear regression features, how much can you gain from it.",
      "votes": null
    },
    {
      "id": "579790",
      "postDate": "07/19/2019 07:57:53",
      "content": "<p>As far as I remember, there was almost no gain from it.</p>",
      "rawMarkdown": "As far as I remember, there was almost no gain from it.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 270434,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "01/18/2018 08:03:58",
      "content": "<p>Thanks for sharing and congrats on the gold medal!</p>",
      "votes": null,
      "replies": [
        {
          "id": 270924,
          "author_name": "antklen",
          "author_url": "",
          "post_date": "01/19/2018 08:53:16",
          "content": "<p>Thanks and congrats on one more step towards Grandmaster=)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 573964,
      "author_name": "longyin2",
      "author_url": "",
      "post_date": "07/13/2019 03:36:49",
      "content": "<p>Thanks for sharing. How's your linear regression features, how much can you gain from it.</p>",
      "votes": null,
      "replies": [
        {
          "id": 579790,
          "author_name": "antklen",
          "author_url": "",
          "post_date": "07/19/2019 07:57:53",
          "content": "<p>As far as I remember, there was almost no gain from it.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "269943": "Code with main parts of the solution is here \nhttps://github.com/antklen/kaggle_favorita-grocery-sales. \n\n\n## Feature engineering ##\n\n I used the same features as in public scripts, namely\n\n - mean sales during several last periods for item/store pairs\n - mean sales during several last periods for item/store pairs for each day of week\n - mean ompromotion during several last periods for item/store pairs\nbut as for time windows, I used more history - up to 1 year\n\nplus some other grouping averages\n\n - mean sales during several last periods for item/store pairs for each day of month\n - mean number of zero sales days during several last periods for item/store pairs\n - mean sales during several last periods for store/item family\n - mean sales during several last periods for store/item class\n - mean sales during several last periods for item/store city\n - mean sales during several last periods for item/store stat\n - mean sales during several last periods for item/store type\n - mean sales during several last periods for item/store cluster\n - mean sales during several last periods group by only item\n - mean number of zero sales days during several last periods group by only stores\n\nplus some differences between mean features with different time window (for example, mean_7days - mean_28days)\n\nplus linear regression on each item/store pair time series (for incorporating possible time trend)\n\nplus mean promo for all 16 days (mean of promo_0, promo_1, ... in Ceshine Lee's script).\n\nas for categorical features (store and item info - family, type, etc.)\n\n - standard label encoding for lightgbm\n - embeddings for neural network\n\nTo be honest, base mean, mean by day of week and promo features do almost all the job, my additional features added not much value.\n\n## Onpromotion issue ##\n\nAs for problem with missing onpromotion data, I didn't come up with any clever solution. But taking as less promo based features as possible helped on public LB and helped to have more stable relation between validation and  LB (but still not very good). Firstly, I made some extra features based on promo and got stuck around public scripts, but when I removed them, I was able to climb up on the leaderboard.\n\n\n## Training and validation scheme ##\n\nFor validation nothing fancy, I used just the same scheme as in Ceshine Lee's script - 26.07-10.08 for validation, several previous 16-days periods for train. May be it would be useful to have several periods for validation (Giba and CPMP reported that it helped), but we had a lot of data here and training took a lot of time, so I decided to stay with only one period.\n\nWhat helped to improve score is just to take more data for training. Switching from 6 16-days periods to 20-25 periods  helped. During last several days I was running the same models but for even longer training history - 52, 75 and 104 periods (1 year, 1.5 year and 2 year for training). Taking more history consistently improved validation score, taking too long (104 periods)  made public score worse, but really surprisingly on private LB even 52 periods was too much - 25 periods was enough, I have the best private LB score on single models with 25 periods.\n\n## Models ##\n\nI used LightGBM and neural network with categorical embeddings on the same set of features.\n\nAlso I used two different approaches:\n\n 1. Train one general model for all lags (very memory-consuming=)\n 2. Train separate model for each lag. But in a little different way than in public scripts. I didn't use the same set of features for all lags. For each lag I constructed a little different set of features on-the-fly. The motivation for that is that I thought that it's not a good idea to feed into the model all day of week mean features when we train model only for one day of week. For example, if we train model for monday, why we add features for all other weekdays? It probably adds some noise.\n\nSwitching to these two schemes from scheme used in public scripts really improved my score and led me to the top.\n\n## Final models ##\n\nSo in the end I had 4 types of models (lightgbm or neural network + general model for all lags or separate model for each lag),  trained on several training sets with different length.\nMy final models were simple averages of some of these models.\n\nAveraging models helped a lot here. Public and private score of my single models have very bad correlation. The best single public model (0.507) has only 0.519 on private, while best private model (0.514) had 0.510 on public. But my blending submissions are much more consistent. All my last blending submissions (about 10 submissions) are in the interval [0.506, 0.508] on public and in the interval [0.513, 0.516] on private.",
    "270434": "Thanks for sharing and congrats on the gold medal!",
    "270924": "Thanks and congrats on one more step towards Grandmaster=)",
    "573964": "Thanks for sharing. How's your linear regression features, how much can you gain from it.",
    "579790": "As far as I remember, there was almost no gain from it."
  },
  "source": "meta"
}