{
  "id": 82113,
  "title": "First silver zone, Some ideas sharing, Congrats to everyone!",
  "url": "/competitions/elo-merchant-category-recommendation/writeups/c-y-w-first-silver-zone-some-ideas-sharing-congrat",
  "author_name": "",
  "post_date": "2019-02-27T09:33:35.530Z",
  "votes": 3,
  "comment_count": 7,
  "views": 0,
  "content": "<p>Hi Kagglers,</p>\n\n<p>Congrats to everyone participates in this difficult competition! \nThis is my first serious competition and first metal after I joined Kaggle. I indeed learn a lot from everyone's posts and kernels in this competition. To be honest, I am very surprised by my final result because I was expected to probably fall out from the bronze zone according to the public LB score.</p>\n\n<p>Although my final result might not be very outstanding comparing to other talents, I would like to share my solution hope it could help.</p>\n\n<p><strong>FE</strong> </p>\n\n<ul>\n<li>a.   Aggregate the ' hist_transaction data ', choose only ' authorized_flag '  == 1</li>\n<li><p>b.   Aggregate the ' new_transaction data ' using the same features as ' hist_transaction data ' but groupby ' month_lag'  [ 1 , 2],  ' authorized_flag'  == 1 </p></li>\n<li><p>c.  Merge  a, b.  and use log1p to normalize data</p></li>\n<li>e.   I select <strong>the abs(target) &gt;=10 as outlier instead of only choose -33.2</strong></li>\n</ul>\n\n<p><strong>Models</strong>\nwith some feature importance selection done by choosing lgbm results.</p>\n\n<ul>\n<li>a.  1 lgb model and 1 NN model for all data including outliers (5 fold , StratifiedKFold)</li>\n<li>b.  1 lgb model and 1 NN model for all data excluding outliers (5 fold , StratifiedKFold, separate by target values  abs()&gt;5)</li>\n<li>c.  1 lgb model and 1 NN model for outlier detection (5 fold , StratifiedKFold)</li>\n<li>d.  Using Ridge() to stack a, b, c models</li>\n</ul>\n\n<p>I think I am very luck this time because my CV is only 3.63X but the final result is even lower than my CV.  Would like to try more trick sharing from the kagglers to see if the score can improve.</p>\n\n<p>Thanks for reading!</p>",
  "messages": [
    {
      "id": "479600",
      "postDate": "02/27/2019 09:17:10",
      "content": "<p>Hi Kagglers,</p>\n\n<p>Congrats to everyone participates in this difficult competition! \nThis is my first serious competition and first metal after I joined Kaggle. I indeed learn a lot from everyone's posts and kernels in this competition. To be honest, I am very surprised by my final result because I was expected to probably fall out from the bronze zone according to the public LB score.</p>\n\n<p>Although my final result might not be very outstanding comparing to other talents, I would like to share my solution hope it could help.</p>\n\n<p><strong>FE</strong> </p>\n\n<ul>\n<li>a.   Aggregate the ' hist_transaction data ', choose only ' authorized_flag '  == 1</li>\n<li><p>b.   Aggregate the ' new_transaction data ' using the same features as ' hist_transaction data ' but groupby ' month_lag'  [ 1 , 2],  ' authorized_flag'  == 1 </p></li>\n<li><p>c.  Merge  a, b.  and use log1p to normalize data</p></li>\n<li>e.   I select <strong>the abs(target) &gt;=10 as outlier instead of only choose -33.2</strong></li>\n</ul>\n\n<p><strong>Models</strong>\nwith some feature importance selection done by choosing lgbm results.</p>\n\n<ul>\n<li>a.  1 lgb model and 1 NN model for all data including outliers (5 fold , StratifiedKFold)</li>\n<li>b.  1 lgb model and 1 NN model for all data excluding outliers (5 fold , StratifiedKFold, separate by target values  abs()&gt;5)</li>\n<li>c.  1 lgb model and 1 NN model for outlier detection (5 fold , StratifiedKFold)</li>\n<li>d.  Using Ridge() to stack a, b, c models</li>\n</ul>\n\n<p>I think I am very luck this time because my CV is only 3.63X but the final result is even lower than my CV.  Would like to try more trick sharing from the kagglers to see if the score can improve.</p>\n\n<p>Thanks for reading!</p>",
      "rawMarkdown": "Hi Kagglers,\n\nCongrats to everyone participates in this difficult competition! \nThis is my first serious competition and first metal after I joined Kaggle. I indeed learn a lot from everyone's posts and kernels in this competition. To be honest, I am very surprised by my final result because I was expected to probably fall out from the bronze zone according to the public LB score.\n\nAlthough my final result might not be very outstanding comparing to other talents, I would like to share my solution hope it could help.\n\n**FE** \n\n - a.   Aggregate the ' hist_transaction data ', choose only ' authorized_flag '  == 1\n - b.   Aggregate the ' new_transaction data ' using the same features as ' hist_transaction data ' but groupby ' month_lag'  [ 1 , 2],  ' authorized_flag'  == 1 \n\n -  c.  Merge  a, b.  and use log1p to normalize data\n - e.   I select **the abs(target) &gt;=10 as outlier instead of only choose -33.2**\n\n   \n\n**Models**\nwith some feature importance selection done by choosing lgbm results.\n\n - a.  1 lgb model and 1 NN model for all data including outliers (5 fold , StratifiedKFold)\n - b.  1 lgb model and 1 NN model for all data excluding outliers (5 fold , StratifiedKFold, separate by target values  abs()&gt;5)\n - c.  1 lgb model and 1 NN model for outlier detection (5 fold , StratifiedKFold)\n - d.  Using Ridge() to stack a, b, c models\n\n\n\nI think I am very luck this time because my CV is only 3.63X but the final result is even lower than my CV.  Would like to try more trick sharing from the kagglers to see if the score can improve.\n\nThanks for reading!",
      "votes": null
    },
    {
      "id": "479630",
      "postDate": "02/27/2019 09:45:10",
      "content": "<p>Congratulations <a href=\"/cywwayne\">@cywwayne</a></p>",
      "rawMarkdown": "Congratulations @cywwayne",
      "votes": null
    },
    {
      "id": "479659",
      "postDate": "02/27/2019 10:19:16",
      "content": "<p>Thanks and congratulations! IMy question is tha, as the model c is a classifier, how to stacking with regressor model?</p>",
      "rawMarkdown": "Thanks and congratulations! IMy question is tha, as the model c is a classifier, how to stacking with regressor model?",
      "votes": null
    },
    {
      "id": "479701",
      "postDate": "02/27/2019 10:57:02",
      "content": "<p>Thanks! Hope you enjoy the competition as well!</p>",
      "rawMarkdown": "Thanks! Hope you enjoy the competition as well!",
      "votes": null
    },
    {
      "id": "479726",
      "postDate": "02/27/2019 11:14:32",
      "content": "<p>Hi @Manchoi,\nI’m not sure if this makes sense to you or not. As the classifier provides a probability of whether it’s an outlier or not, I use the prob as one feature for the final stacking Ridge regression model. By doing so, the ridge regression would decide the coef of the prob and I won’t need to manually adjust the threshold of outlier.</p>",
      "rawMarkdown": "Hi @Manchoi,\nI’m not sure if this makes sense to you or not. As the classifier provides a probability of whether it’s an outlier or not, I use the prob as one feature for the final stacking Ridge regression model. By doing so, the ridge regression would decide the coef of the prob and I won’t need to manually adjust the threshold of outlier.",
      "votes": null
    },
    {
      "id": "479748",
      "postDate": "02/27/2019 11:45:30",
      "content": "<p><a href=\"/cywwayne\">@cywwayne</a>, Thanks for your reply, and I understand it as take the prob as the meta feature for the secondary estimator. And how many estimators you haved used in your first level stacking?</p>",
      "rawMarkdown": "cywwayne, Thanks for your reply, and I understand it as take the prob as the meta feature for the secondary estimator. And how many estimators you haved used in your first level stacking?",
      "votes": null
    },
    {
      "id": "479850",
      "postDate": "02/27/2019 13:50:33",
      "content": "<p>@Manchoi , 6 in total, as I mentioned in the models part, each section has one lgbm, one NN using the exactly same features! Hope the explanation help!</p>",
      "rawMarkdown": "Manchoi , 6 in total, as I mentioned in the models part, each section has one lgbm, one NN using the exactly same features! Hope the explanation help!",
      "votes": null
    },
    {
      "id": "480393",
      "postDate": "02/28/2019 06:04:45",
      "content": "<p>Great help! Thanks again!</p>",
      "rawMarkdown": "Great help! Thanks again!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 479630,
      "author_name": "karthik7395",
      "author_url": "",
      "post_date": "02/27/2019 09:45:10",
      "content": "<p>Congratulations <a href=\"/cywwayne\">@cywwayne</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 479701,
          "author_name": "cywwayne",
          "author_url": "",
          "post_date": "02/27/2019 10:57:02",
          "content": "<p>Thanks! Hope you enjoy the competition as well!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 479659,
      "author_name": "liewmanchoi",
      "author_url": "",
      "post_date": "02/27/2019 10:19:16",
      "content": "<p>Thanks and congratulations! IMy question is tha, as the model c is a classifier, how to stacking with regressor model?</p>",
      "votes": null,
      "replies": [
        {
          "id": 479726,
          "author_name": "cywwayne",
          "author_url": "",
          "post_date": "02/27/2019 11:14:32",
          "content": "<p>Hi @Manchoi,\nI’m not sure if this makes sense to you or not. As the classifier provides a probability of whether it’s an outlier or not, I use the prob as one feature for the final stacking Ridge regression model. By doing so, the ridge regression would decide the coef of the prob and I won’t need to manually adjust the threshold of outlier.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 479748,
          "author_name": "liewmanchoi",
          "author_url": "",
          "post_date": "02/27/2019 11:45:30",
          "content": "<p><a href=\"/cywwayne\">@cywwayne</a>, Thanks for your reply, and I understand it as take the prob as the meta feature for the secondary estimator. And how many estimators you haved used in your first level stacking?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 479850,
          "author_name": "cywwayne",
          "author_url": "",
          "post_date": "02/27/2019 13:50:33",
          "content": "<p>@Manchoi , 6 in total, as I mentioned in the models part, each section has one lgbm, one NN using the exactly same features! Hope the explanation help!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 480393,
          "author_name": "liewmanchoi",
          "author_url": "",
          "post_date": "02/28/2019 06:04:45",
          "content": "<p>Great help! Thanks again!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "479600": "Hi Kagglers,\n\nCongrats to everyone participates in this difficult competition! \nThis is my first serious competition and first metal after I joined Kaggle. I indeed learn a lot from everyone's posts and kernels in this competition. To be honest, I am very surprised by my final result because I was expected to probably fall out from the bronze zone according to the public LB score.\n\nAlthough my final result might not be very outstanding comparing to other talents, I would like to share my solution hope it could help.\n\n**FE** \n\n - a.   Aggregate the ' hist_transaction data ', choose only ' authorized_flag '  == 1\n - b.   Aggregate the ' new_transaction data ' using the same features as ' hist_transaction data ' but groupby ' month_lag'  [ 1 , 2],  ' authorized_flag'  == 1 \n\n -  c.  Merge  a, b.  and use log1p to normalize data\n - e.   I select **the abs(target) &gt;=10 as outlier instead of only choose -33.2**\n\n   \n\n**Models**\nwith some feature importance selection done by choosing lgbm results.\n\n - a.  1 lgb model and 1 NN model for all data including outliers (5 fold , StratifiedKFold)\n - b.  1 lgb model and 1 NN model for all data excluding outliers (5 fold , StratifiedKFold, separate by target values  abs()&gt;5)\n - c.  1 lgb model and 1 NN model for outlier detection (5 fold , StratifiedKFold)\n - d.  Using Ridge() to stack a, b, c models\n\n\n\nI think I am very luck this time because my CV is only 3.63X but the final result is even lower than my CV.  Would like to try more trick sharing from the kagglers to see if the score can improve.\n\nThanks for reading!",
    "479630": "Congratulations @cywwayne",
    "479659": "Thanks and congratulations! IMy question is tha, as the model c is a classifier, how to stacking with regressor model?",
    "479701": "Thanks! Hope you enjoy the competition as well!",
    "479726": "Hi @Manchoi,\nI’m not sure if this makes sense to you or not. As the classifier provides a probability of whether it’s an outlier or not, I use the prob as one feature for the final stacking Ridge regression model. By doing so, the ridge regression would decide the coef of the prob and I won’t need to manually adjust the threshold of outlier.",
    "479748": "cywwayne, Thanks for your reply, and I understand it as take the prob as the meta feature for the secondary estimator. And how many estimators you haved used in your first level stacking?",
    "479850": "Manchoi , 6 in total, as I mentioned in the models part, each section has one lgbm, one NN using the exactly same features! Hope the explanation help!",
    "480393": "Great help! Thanks again!"
  },
  "source": "meta"
}