{
  "id": 74519,
  "title": "A large ratio of RMSE is caused by outlier",
  "url": "/competitions/elo-merchant-category-recommendation/discussion/74519",
  "author_name": "Zhuang Jia",
  "post_date": "2018-12-13T03:39:41.676000",
  "votes": 30,
  "comment_count": 8,
  "views": 0,
  "content": "<p><code>train.target</code> has 2207 outliers(<code>-33.219281</code>). Total RMSE of <code>train.target</code> is:</p>\n\n<p><code>python\ntrain = pd.read_csv('input/train.csv')\n</code></p>\n\n<p><code>\ntarget = train.target.values\n</code></p>\n\n<p><code>\nnp.sqrt(mean_squared_error(target, np.zeros(train.shape[0])))    # returns 3.8705589161316296\n</code></p>\n\n<p>RMSE caused by outliers:</p>\n\n<p><code>python\ntarget[target &gt; -33] = 0\n</code></p>\n\n<p><code>\nnp.sqrt(mean_squared_error(target, np.zeros(train.shape[0])))    # returns 3.4730013468168988\n</code></p>\n\n<p>which means 89.7% (3.47/3.87) of the RMSE is caused by outliers? What should we deal with this?</p>",
  "messages": [
    {
      "id": 438064,
      "postDate": "2018-12-13T03:39:41.677Z",
      "content": "<p><code>train.target</code> has 2207 outliers(<code>-33.219281</code>). Total RMSE of <code>train.target</code> is:</p>\n\n<p><code>python\ntrain = pd.read_csv('input/train.csv')\n</code></p>\n\n<p><code>\ntarget = train.target.values\n</code></p>\n\n<p><code>\nnp.sqrt(mean_squared_error(target, np.zeros(train.shape[0])))    # returns 3.8705589161316296\n</code></p>\n\n<p>RMSE caused by outliers:</p>\n\n<p><code>python\ntarget[target &gt; -33] = 0\n</code></p>\n\n<p><code>\nnp.sqrt(mean_squared_error(target, np.zeros(train.shape[0])))    # returns 3.4730013468168988\n</code></p>\n\n<p>which means 89.7% (3.47/3.87) of the RMSE is caused by outliers? What should we deal with this?</p>",
      "rawMarkdown": "`train.target` has 2207 outliers(`-33.219281`). Total RMSE of `train.target` is:\n\n```python\ntrain = pd.read_csv('input/train.csv')\n```\n\n```\ntarget = train.target.values\n```\n\n```\nnp.sqrt(mean_squared_error(target, np.zeros(train.shape[0])))    # returns 3.8705589161316296\n```\n\nRMSE caused by outliers:\n\n```python\ntarget[target &gt; -33] = 0\n```\n\n```\nnp.sqrt(mean_squared_error(target, np.zeros(train.shape[0])))    # returns 3.4730013468168988\n```\n\nwhich means 89.7% (3.47/3.87) of the RMSE is caused by outliers? What should we deal with this?",
      "votes": 29
    },
    {
      "id": 439923,
      "postDate": "2018-12-16T16:46:13.733Z",
      "content": "<p>I experienced the same problem when I eliminate outliers I have a local RMSE of 1.9 but the LB RMSE was 3.9, I think the best approach could be to carry two models (one for values close to -33 and another for the more typical values) and then use a simple linear ensemble.</p>",
      "rawMarkdown": "I experienced the same problem when I eliminate outliers I have a local RMSE of 1.9 but the LB RMSE was 3.9, I think the best approach could be to carry two models (one for values close to -33 and another for the more typical values) and then use a simple linear ensemble.",
      "votes": 6,
      "replies": [
        {
          "id": 442114,
          "postDate": "2018-12-19T13:54:54.100Z",
          "content": "<p>That's great advice!!</p>",
          "rawMarkdown": "That's great advice!!",
          "votes": 1
        }
      ]
    },
    {
      "id": 443756,
      "postDate": "2018-12-22T11:01:51.210Z",
      "content": "<p>he analysis is particularly clear. Thanks!!!</p>",
      "rawMarkdown": "he analysis is particularly clear. Thanks!!!"
    },
    {
      "id": 439896,
      "postDate": "2018-12-16T15:43:06Z",
      "content": "<p>Really. Train as well as Test also contains large set of outliers. That results into high CV but low LB score.</p>",
      "rawMarkdown": "Really. Train as well as Test also contains large set of outliers. That results into high CV but low LB score."
    },
    {
      "id": 438957,
      "postDate": "2018-12-14T13:12:22.093Z",
      "content": "<p>Good question!</p>",
      "rawMarkdown": "Good question!"
    },
    {
      "id": 438293,
      "postDate": "2018-12-13T12:29:50.123Z",
      "content": "<p>Good question..</p>",
      "rawMarkdown": "Good question.."
    },
    {
      "id": 438939,
      "postDate": "2018-12-14T12:54:51.763Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 446079,
      "postDate": "2018-12-27T13:16:32.433Z",
      "content": "<p>Thank you ~  </p>",
      "rawMarkdown": "Thank you ~  "
    }
  ],
  "comments": [
    {
      "id": 439923,
      "author_name": "C4rl05/V",
      "author_url": "",
      "post_date": "2018-12-16T16:46:13.733000",
      "content": "<p>I experienced the same problem when I eliminate outliers I have a local RMSE of 1.9 but the LB RMSE was 3.9, I think the best approach could be to carry two models (one for values close to -33 and another for the more typical values) and then use a simple linear ensemble.</p>",
      "votes": 6,
      "replies": [
        {
          "id": 442114,
          "author_name": "codereason",
          "author_url": "",
          "post_date": "2018-12-19T13:54:54.100000",
          "content": "<p>That's great advice!!</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 443756,
      "author_name": "Xin Guo",
      "author_url": "",
      "post_date": "2018-12-22T11:01:51.210000",
      "content": "<p>he analysis is particularly clear. Thanks!!!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 439896,
      "author_name": "jaiminpatel",
      "author_url": "",
      "post_date": "2018-12-16T15:43:06",
      "content": "<p>Really. Train as well as Test also contains large set of outliers. That results into high CV but low LB score.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 438957,
      "author_name": "Jianfei Wang",
      "author_url": "",
      "post_date": "2018-12-14T13:12:22.093000",
      "content": "<p>Good question!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 438293,
      "author_name": "Jonathan Mallia",
      "author_url": "",
      "post_date": "2018-12-13T12:29:50.123000",
      "content": "<p>Good question..</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 438939,
      "author_name": "",
      "author_url": "",
      "post_date": "2018-12-14T12:54:51.763000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 446079,
      "author_name": "SUNYC",
      "author_url": "",
      "post_date": "2018-12-27T13:16:32.433000",
      "content": "<p>Thank you ~  </p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "438064": "`train.target` has 2207 outliers(`-33.219281`). Total RMSE of `train.target` is:\n\n```python\ntrain = pd.read_csv('input/train.csv')\n```\n\n```\ntarget = train.target.values\n```\n\n```\nnp.sqrt(mean_squared_error(target, np.zeros(train.shape[0])))    # returns 3.8705589161316296\n```\n\nRMSE caused by outliers:\n\n```python\ntarget[target &gt; -33] = 0\n```\n\n```\nnp.sqrt(mean_squared_error(target, np.zeros(train.shape[0])))    # returns 3.4730013468168988\n```\n\nwhich means 89.7% (3.47/3.87) of the RMSE is caused by outliers? What should we deal with this?",
    "439923": "I experienced the same problem when I eliminate outliers I have a local RMSE of 1.9 but the LB RMSE was 3.9, I think the best approach could be to carry two models (one for values close to -33 and another for the more typical values) and then use a simple linear ensemble.",
    "443756": "he analysis is particularly clear. Thanks!!!",
    "439896": "Really. Train as well as Test also contains large set of outliers. That results into high CV but low LB score.",
    "438957": "Good question!",
    "438293": "Good question..",
    "438939": "",
    "446079": "Thank you ~  "
  }
}