{
  "id": 74307,
  "title": "How useful is the resulting model for Elo?",
  "url": "/competitions/elo-merchant-category-recommendation/discussion/74307",
  "author_name": "",
  "post_date": "2018-12-11T05:57:47.073963200Z",
  "votes": 6,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Excluding outliers (-33.219) from target variable, middle 80% of loyalty score lies between -1.90 and 1.84\nRMSE of best models is in the range of 3.6 to 3.7</p>\n\n<p>That is close to 200% error range for any given prediction. How useful is this output anyway? I think something is a major miss with this competition. Unless there is some kind of a breakthrough, any incremental accuracy improvements will be useless in general.</p>",
  "messages": [
    {
      "id": "436938",
      "postDate": "12/11/2018 05:57:47",
      "content": "<p>Excluding outliers (-33.219) from target variable, middle 80% of loyalty score lies between -1.90 and 1.84\nRMSE of best models is in the range of 3.6 to 3.7</p>\n\n<p>That is close to 200% error range for any given prediction. How useful is this output anyway? I think something is a major miss with this competition. Unless there is some kind of a breakthrough, any incremental accuracy improvements will be useless in general.</p>",
      "rawMarkdown": "Excluding outliers (-33.219) from target variable, middle 80% of loyalty score lies between -1.90 and 1.84\nRMSE of best models is in the range of 3.6 to 3.7\n\nThat is close to 200% error range for any given prediction. How useful is this output anyway? I think something is a major miss with this competition. Unless there is some kind of a breakthrough, any incremental accuracy improvements will be useless in general.",
      "votes": null
    },
    {
      "id": "436994",
      "postDate": "12/11/2018 08:06:12",
      "content": "<p>I am trying to classify Card_id in test file  is outlier or not\nI think it is the key to break through,somehow it's not that easy </p>",
      "rawMarkdown": "I am trying to classify Card_id in test file  is outlier or not\nI think it is the key to break through,somehow it's not that easy",
      "votes": null
    },
    {
      "id": "437000",
      "postDate": "12/11/2018 08:10:11",
      "content": "<p>I built a simple random forest classifier for classifying if the card_id is an outlier or not. Got 98.9% accuracy on my validation data. But not sure how useful that model is to Elo in terms of solving the business problem they are trying to solve.</p>",
      "rawMarkdown": "I built a simple random forest classifier for classifying if the card_id is an outlier or not. Got 98.9% accuracy on my validation data. But not sure how useful that model is to Elo in terms of solving the business problem they are trying to solve.",
      "votes": null
    },
    {
      "id": "437004",
      "postDate": "12/11/2018 08:28:21",
      "content": "<p>I think accuracy does not help get a higher place in LB. If Outlier is 1 non_outlier is 0 maybe you should check the precision rate of your validation data. </p>",
      "rawMarkdown": "I think accuracy does not help get a higher place in LB. If Outlier is 1 non_outlier is 0 maybe you should check the precision rate of your validation data.",
      "votes": null
    },
    {
      "id": "437972",
      "postDate": "12/12/2018 21:36:46",
      "content": "<p>Agree with Luque. \nData is quite imbalanced in terms of outliers, so would be better to go for other metrics. I tried lightgbm with 5 CV and got auc score of 0.901 on validation set. I haven't checked precision recall score yet. </p>",
      "rawMarkdown": "Agree with Luque. \nData is quite imbalanced in terms of outliers, so would be better to go for other metrics. I tried lightgbm with 5 CV and got auc score of 0.901 on validation set. I haven't checked precision recall score yet.",
      "votes": null
    },
    {
      "id": "437995",
      "postDate": "12/12/2018 23:07:10",
      "content": "<p>It's too early to say. There are two months to go. I'm sure 3.6X won't last long on LB. </p>",
      "rawMarkdown": "It's too early to say. There are two months to go. I'm sure 3.6X won't last long on LB.",
      "votes": null
    },
    {
      "id": "483179",
      "postDate": "03/04/2019 09:11:52",
      "content": "<p>well I 3.578 is the best it got :). I seriously doubt the usefulness of this competition for everyone involved.</p>",
      "rawMarkdown": "well I 3.578 is the best it got :). I seriously doubt the usefulness of this competition for everyone involved.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 436994,
      "author_name": "luque1008",
      "author_url": "",
      "post_date": "12/11/2018 08:06:12",
      "content": "<p>I am trying to classify Card_id in test file  is outlier or not\nI think it is the key to break through,somehow it's not that easy </p>",
      "votes": null,
      "replies": [
        {
          "id": 437000,
          "author_name": "darshak",
          "author_url": "",
          "post_date": "12/11/2018 08:10:11",
          "content": "<p>I built a simple random forest classifier for classifying if the card_id is an outlier or not. Got 98.9% accuracy on my validation data. But not sure how useful that model is to Elo in terms of solving the business problem they are trying to solve.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 437004,
          "author_name": "luque1008",
          "author_url": "",
          "post_date": "12/11/2018 08:28:21",
          "content": "<p>I think accuracy does not help get a higher place in LB. If Outlier is 1 non_outlier is 0 maybe you should check the precision rate of your validation data. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 437972,
          "author_name": "harshit92",
          "author_url": "",
          "post_date": "12/12/2018 21:36:46",
          "content": "<p>Agree with Luque. \nData is quite imbalanced in terms of outliers, so would be better to go for other metrics. I tried lightgbm with 5 CV and got auc score of 0.901 on validation set. I haven't checked precision recall score yet. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 437995,
      "author_name": "xwan254",
      "author_url": "",
      "post_date": "12/12/2018 23:07:10",
      "content": "<p>It's too early to say. There are two months to go. I'm sure 3.6X won't last long on LB. </p>",
      "votes": null,
      "replies": [
        {
          "id": 483179,
          "author_name": "darshak",
          "author_url": "",
          "post_date": "03/04/2019 09:11:52",
          "content": "<p>well I 3.578 is the best it got :). I seriously doubt the usefulness of this competition for everyone involved.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "436938": "Excluding outliers (-33.219) from target variable, middle 80% of loyalty score lies between -1.90 and 1.84\nRMSE of best models is in the range of 3.6 to 3.7\n\nThat is close to 200% error range for any given prediction. How useful is this output anyway? I think something is a major miss with this competition. Unless there is some kind of a breakthrough, any incremental accuracy improvements will be useless in general.",
    "436994": "I am trying to classify Card_id in test file  is outlier or not\nI think it is the key to break through,somehow it's not that easy",
    "437000": "I built a simple random forest classifier for classifying if the card_id is an outlier or not. Got 98.9% accuracy on my validation data. But not sure how useful that model is to Elo in terms of solving the business problem they are trying to solve.",
    "437004": "I think accuracy does not help get a higher place in LB. If Outlier is 1 non_outlier is 0 maybe you should check the precision rate of your validation data.",
    "437972": "Agree with Luque. \nData is quite imbalanced in terms of outliers, so would be better to go for other metrics. I tried lightgbm with 5 CV and got auc score of 0.901 on validation set. I haven't checked precision recall score yet.",
    "437995": "It's too early to say. There are two months to go. I'm sure 3.6X won't last long on LB.",
    "483179": "well I 3.578 is the best it got :). I seriously doubt the usefulness of this competition for everyone involved."
  },
  "source": "meta"
}