{
  "id": 68178,
  "title": "Strategies for Stacking, creation of models, Layers",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/68178",
  "author_name": "",
  "post_date": "2018-10-10T06:27:51.422721500Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hi <strong>Experienced Kagglers</strong>,</p>\n\n<p>We all know more or less about ensemble techniques and how its created. However. when we start, how do we strategies to create multiple models and how do we know how many models or layers should have been created to increase the score.</p>\n\n<p>I have been through the <a href=\"https://www.kaggle.com/yekenot/simple-stacker-lb-0-284\">link</a> to know about stacking. Now to create multiple models, we needs to create different models based on features (Categorical / Numerical) and different algo (Tree base / non tree based / NN etc) and stacked them into layers to see the final CV score. However, how do we work to see, if the CV is improving or not.</p>",
  "messages": [
    {
      "id": "401478",
      "postDate": "10/10/2018 06:27:51",
      "content": "<p>Hi <strong>Experienced Kagglers</strong>,</p>\n\n<p>We all know more or less about ensemble techniques and how its created. However. when we start, how do we strategies to create multiple models and how do we know how many models or layers should have been created to increase the score.</p>\n\n<p>I have been through the <a href=\"https://www.kaggle.com/yekenot/simple-stacker-lb-0-284\">link</a> to know about stacking. Now to create multiple models, we needs to create different models based on features (Categorical / Numerical) and different algo (Tree base / non tree based / NN etc) and stacked them into layers to see the final CV score. However, how do we work to see, if the CV is improving or not.</p>",
      "rawMarkdown": "Hi **Experienced Kagglers**,\n\nWe all know more or less about ensemble techniques and how its created. However. when we start, how do we strategies to create multiple models and how do we know how many models or layers should have been created to increase the score.\n\nI have been through the [link][1] to know about stacking. Now to create multiple models, we needs to create different models based on features (Categorical / Numerical) and different algo (Tree base / non tree based / NN etc) and stacked them into layers to see the final CV score. However, how do we work to see, if the CV is improving or not.\n\n\n  [1]: https://www.kaggle.com/yekenot/simple-stacker-lb-0-284",
      "votes": null
    },
    {
      "id": "401480",
      "postDate": "10/10/2018 06:29:15",
      "content": "<p>Attached is the image from the <a href=\"http://blog.kaggle.com/2016/05/18/home-depot-product-search-relevance-winners-interview-1st-place-alex-andreas-nurlan/\">Link</a>. How these models and layers have been created and based on what strategy ?</p>\n\n<p><strong>OR</strong></p>\n\n<p>will follow the steps</p>\n\n<ol>\n<li>Create a pipeline of models and make a tag it below to which Layers</li>\n<li>Keep on creating features / models and find the correlation with the other models in the same layer and high co-related model should be discarded or feed into the pipeline to see the score ?</li>\n<li>Feed the model into the pipeline to see what's the score</li>\n<li>Will keep on doing this</li>\n</ol>\n\n<p>However, how do we know, we are going in the right direction or is it pure Trial / Error (Based on experience after competing in different competition)</p>\n\n<p><strong>OR</strong></p>\n\n<p>please help to send the Link (or reference) to find more guidance on how to progress bit more of strategies.</p>",
      "rawMarkdown": "Attached is the image from the [Link][1]. How these models and layers have been created and based on what strategy ?\n\n**OR**\n\nwill follow the steps\n\n 1. Create a pipeline of models and make a tag it below to which Layers\n 2. Keep on creating features / models and find the correlation with the other models in the same layer and high co-related model should be discarded or feed into the pipeline to see the score ?\n 3. Feed the model into the pipeline to see what's the score\n 4. Will keep on doing this\n\nHowever, how do we know, we are going in the right direction or is it pure Trial / Error (Based on experience after competing in different competition)\n\n**OR**\n\nplease help to send the Link (or reference) to find more guidance on how to progress bit more of strategies.\n\n\n  [1]: http://blog.kaggle.com/2016/05/18/home-depot-product-search-relevance-winners-interview-1st-place-alex-andreas-nurlan/",
      "votes": null
    },
    {
      "id": "401678",
      "postDate": "10/10/2018 13:45:26",
      "content": "<p>I don't have that much experience in stacking. So take this post with caution( Some points I am suggesting may be wrong )</p>\n\n<p>1) yup, you have to do it like that </p>\n\n<p>2) It's unlikely that highly correlated(&gt;0.99) models will increase the cv score. So if you have time I think you can try to add these predictions and see if they make cv score improve. correlation (&lt;=0.98) helps.</p>\n\n<p>So for stacking also we have a validation (like 5folds) and see the validation score. If the validation score is improving then your base level model predictions are helping.  </p>",
      "rawMarkdown": "I don't have that much experience in stacking. So take this post with caution( Some points I am suggesting may be wrong )\n\n1) yup, you have to do it like that \n\n2) It's unlikely that highly correlated(&gt;0.99) models will increase the cv score. So if you have time I think you can try to add these predictions and see if they make cv score improve. correlation (&lt;=0.98) helps.\n\nSo for stacking also we have a validation (like 5folds) and see the validation score. If the validation score is improving then your base level model predictions are helping.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 401480,
      "author_name": "sushanta",
      "author_url": "",
      "post_date": "10/10/2018 06:29:15",
      "content": "<p>Attached is the image from the <a href=\"http://blog.kaggle.com/2016/05/18/home-depot-product-search-relevance-winners-interview-1st-place-alex-andreas-nurlan/\">Link</a>. How these models and layers have been created and based on what strategy ?</p>\n\n<p><strong>OR</strong></p>\n\n<p>will follow the steps</p>\n\n<ol>\n<li>Create a pipeline of models and make a tag it below to which Layers</li>\n<li>Keep on creating features / models and find the correlation with the other models in the same layer and high co-related model should be discarded or feed into the pipeline to see the score ?</li>\n<li>Feed the model into the pipeline to see what's the score</li>\n<li>Will keep on doing this</li>\n</ol>\n\n<p>However, how do we know, we are going in the right direction or is it pure Trial / Error (Based on experience after competing in different competition)</p>\n\n<p><strong>OR</strong></p>\n\n<p>please help to send the Link (or reference) to find more guidance on how to progress bit more of strategies.</p>",
      "votes": null,
      "replies": [
        {
          "id": 401678,
          "author_name": "suchith0312",
          "author_url": "",
          "post_date": "10/10/2018 13:45:26",
          "content": "<p>I don't have that much experience in stacking. So take this post with caution( Some points I am suggesting may be wrong )</p>\n\n<p>1) yup, you have to do it like that </p>\n\n<p>2) It's unlikely that highly correlated(&gt;0.99) models will increase the cv score. So if you have time I think you can try to add these predictions and see if they make cv score improve. correlation (&lt;=0.98) helps.</p>\n\n<p>So for stacking also we have a validation (like 5folds) and see the validation score. If the validation score is improving then your base level model predictions are helping.  </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "401478": "Hi **Experienced Kagglers**,\n\nWe all know more or less about ensemble techniques and how its created. However. when we start, how do we strategies to create multiple models and how do we know how many models or layers should have been created to increase the score.\n\nI have been through the [link][1] to know about stacking. Now to create multiple models, we needs to create different models based on features (Categorical / Numerical) and different algo (Tree base / non tree based / NN etc) and stacked them into layers to see the final CV score. However, how do we work to see, if the CV is improving or not.\n\n\n  [1]: https://www.kaggle.com/yekenot/simple-stacker-lb-0-284",
    "401480": "Attached is the image from the [Link][1]. How these models and layers have been created and based on what strategy ?\n\n**OR**\n\nwill follow the steps\n\n 1. Create a pipeline of models and make a tag it below to which Layers\n 2. Keep on creating features / models and find the correlation with the other models in the same layer and high co-related model should be discarded or feed into the pipeline to see the score ?\n 3. Feed the model into the pipeline to see what's the score\n 4. Will keep on doing this\n\nHowever, how do we know, we are going in the right direction or is it pure Trial / Error (Based on experience after competing in different competition)\n\n**OR**\n\nplease help to send the Link (or reference) to find more guidance on how to progress bit more of strategies.\n\n\n  [1]: http://blog.kaggle.com/2016/05/18/home-depot-product-search-relevance-winners-interview-1st-place-alex-andreas-nurlan/",
    "401678": "I don't have that much experience in stacking. So take this post with caution( Some points I am suggesting may be wrong )\n\n1) yup, you have to do it like that \n\n2) It's unlikely that highly correlated(&gt;0.99) models will increase the cv score. So if you have time I think you can try to add these predictions and see if they make cv score improve. correlation (&lt;=0.98) helps.\n\nSo for stacking also we have a validation (like 5folds) and see the validation score. If the validation score is improving then your base level model predictions are helping."
  },
  "source": "meta"
}