{
  "id": 77245,
  "title": "Stacking",
  "url": "/competitions/elo-merchant-category-recommendation/discussion/77245",
  "author_name": "",
  "post_date": "2019-01-10T23:19:31.666478Z",
  "votes": null,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I am relatively new to stacking in machine learning competitions and i wonder if i do it wrong or if it is not a good option for this competition. </p>\n\n<p>When i try to stack different models i get the same or worse performance. I tried combinations of LightGBM, Right, ElasticNet, KNN.</p>\n\n<p>Have you tried stacking? How did it go?</p>",
  "messages": [
    {
      "id": "453872",
      "postDate": "01/10/2019 23:19:31",
      "content": "<p>I am relatively new to stacking in machine learning competitions and i wonder if i do it wrong or if it is not a good option for this competition. </p>\n\n<p>When i try to stack different models i get the same or worse performance. I tried combinations of LightGBM, Right, ElasticNet, KNN.</p>\n\n<p>Have you tried stacking? How did it go?</p>",
      "rawMarkdown": "I am relatively new to stacking in machine learning competitions and i wonder if i do it wrong or if it is not a good option for this competition. \n\nWhen i try to stack different models i get the same or worse performance. I tried combinations of LightGBM, Right, ElasticNet, KNN.\n\nHave you tried stacking? How did it go?",
      "votes": null
    },
    {
      "id": "454271",
      "postDate": "01/11/2019 10:38:48",
      "content": "<p>stacking had an improvement of 0.02 on my CV. Concerning Tree based models, you should use max_depth=3 or something small in order to see good results, also the same applies for NNs. Also, in order to stack you must have a decent number of models as input. </p>",
      "rawMarkdown": "stacking had an improvement of 0.02 on my CV. Concerning Tree based models, you should use max_depth=3 or something small in order to see good results, also the same applies for NNs. Also, in order to stack you must have a decent number of models as input.",
      "votes": null
    },
    {
      "id": "454421",
      "postDate": "01/11/2019 16:10:22",
      "content": "<p>I saw in some kernels that they stack only 2 models. I will try to add more than 5 to see what happens. As the meta-classifier i used Ridge. Do you propose tree-based or NN?</p>",
      "rawMarkdown": "I saw in some kernels that they stack only 2 models. I will try to add more than 5 to see what happens. As the meta-classifier i used Ridge. Do you propose tree-based or NN?",
      "votes": null
    },
    {
      "id": "455016",
      "postDate": "01/12/2019 18:49:50",
      "content": "<p>Both shallow NNs and tree-based models work nice. For example, see a a solution writeup\n<a href=\"https://www.kaggle.com/c/otto-group-product-classification-challenge/discussion/14335\">https://www.kaggle.com/c/otto-group-product-classification-challenge/discussion/14335</a></p>",
      "rawMarkdown": "Both shallow NNs and tree-based models work nice. For example, see a a solution writeup\nhttps://www.kaggle.com/c/otto-group-product-classification-challenge/discussion/14335",
      "votes": null
    },
    {
      "id": "455026",
      "postDate": "01/12/2019 19:23:17",
      "content": "<p>Stacking can easily overfit as you leak target information.  Depending on the dataset, it may be safer to be very conservative when stacking for instance using a general linear model before moving to more complex models.</p>",
      "rawMarkdown": "Stacking can easily overfit as you leak target information.  Depending on the dataset, it may be safer to be very conservative when stacking for instance using a general linear model before moving to more complex models.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 454271,
      "author_name": "dkaraflos",
      "author_url": "",
      "post_date": "01/11/2019 10:38:48",
      "content": "<p>stacking had an improvement of 0.02 on my CV. Concerning Tree based models, you should use max_depth=3 or something small in order to see good results, also the same applies for NNs. Also, in order to stack you must have a decent number of models as input. </p>",
      "votes": null,
      "replies": [
        {
          "id": 454421,
          "author_name": "deffro",
          "author_url": "",
          "post_date": "01/11/2019 16:10:22",
          "content": "<p>I saw in some kernels that they stack only 2 models. I will try to add more than 5 to see what happens. As the meta-classifier i used Ridge. Do you propose tree-based or NN?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 455016,
          "author_name": "dkaraflos",
          "author_url": "",
          "post_date": "01/12/2019 18:49:50",
          "content": "<p>Both shallow NNs and tree-based models work nice. For example, see a a solution writeup\n<a href=\"https://www.kaggle.com/c/otto-group-product-classification-challenge/discussion/14335\">https://www.kaggle.com/c/otto-group-product-classification-challenge/discussion/14335</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 455026,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "01/12/2019 19:23:17",
      "content": "<p>Stacking can easily overfit as you leak target information.  Depending on the dataset, it may be safer to be very conservative when stacking for instance using a general linear model before moving to more complex models.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "453872": "I am relatively new to stacking in machine learning competitions and i wonder if i do it wrong or if it is not a good option for this competition. \n\nWhen i try to stack different models i get the same or worse performance. I tried combinations of LightGBM, Right, ElasticNet, KNN.\n\nHave you tried stacking? How did it go?",
    "454271": "stacking had an improvement of 0.02 on my CV. Concerning Tree based models, you should use max_depth=3 or something small in order to see good results, also the same applies for NNs. Also, in order to stack you must have a decent number of models as input.",
    "454421": "I saw in some kernels that they stack only 2 models. I will try to add more than 5 to see what happens. As the meta-classifier i used Ridge. Do you propose tree-based or NN?",
    "455016": "Both shallow NNs and tree-based models work nice. For example, see a a solution writeup\nhttps://www.kaggle.com/c/otto-group-product-classification-challenge/discussion/14335",
    "455026": "Stacking can easily overfit as you leak target information.  Depending on the dataset, it may be safer to be very conservative when stacking for instance using a general linear model before moving to more complex models."
  },
  "source": "meta"
}