{
  "id": 90167,
  "title": "Data leak?",
  "url": "/competitions/dota-2-prediction/discussion/90167",
  "author_name": "",
  "post_date": "2019-04-21T10:36:36.901029600Z",
  "votes": 12,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Hello. We think that there is a data leak in the deaths column of the dataset. It seems that the provided values are the number of deaths during the entire game instead of during the first five minutes. \nThe median number of deaths per team is 31 (which is very high for only 5 minutes of gameplay). The number of kills seems to be good though (with a median of 2). \nMoreover, when predicting the outcome of the match using only the difference in the number of deaths between the two teams (i.e. if one team has less deaths than the other one then we predict it wins) we get a train ROC of 0.87 while with the kills (which are supposed to encode the same information) we get a train ROC of 0.57.</p>",
  "messages": [
    {
      "id": "520578",
      "postDate": "04/21/2019 10:36:36",
      "content": "<p>Hello. We think that there is a data leak in the deaths column of the dataset. It seems that the provided values are the number of deaths during the entire game instead of during the first five minutes. \nThe median number of deaths per team is 31 (which is very high for only 5 minutes of gameplay). The number of kills seems to be good though (with a median of 2). \nMoreover, when predicting the outcome of the match using only the difference in the number of deaths between the two teams (i.e. if one team has less deaths than the other one then we predict it wins) we get a train ROC of 0.87 while with the kills (which are supposed to encode the same information) we get a train ROC of 0.57.</p>",
      "rawMarkdown": "Hello. We think that there is a data leak in the deaths column of the dataset. It seems that the provided values are the number of deaths during the entire game instead of during the first five minutes. \nThe median number of deaths per team is 31 (which is very high for only 5 minutes of gameplay). The number of kills seems to be good though (with a median of 2). \nMoreover, when predicting the outcome of the match using only the difference in the number of deaths between the two teams (i.e. if one team has less deaths than the other one then we predict it wins) we get a train ROC of 0.87 while with the kills (which are supposed to encode the same information) we get a train ROC of 0.57.",
      "votes": null
    },
    {
      "id": "520967",
      "postDate": "04/22/2019 04:51:40",
      "content": "<p>True that, deaths compared to kills is very different. 'Death' median cannot be like high compared to first 5 mins. Valuable find out 💯 </p>",
      "rawMarkdown": "True that, deaths compared to kills is very different. 'Death' median cannot be like high compared to first 5 mins. Valuable find out 💯",
      "votes": null
    },
    {
      "id": "521181",
      "postDate": "04/22/2019 13:58:05",
      "content": "<p>Thanks Jimmy, you and your team are right. There was a bug in the extraction code, which filtered kill/death events incorrectly and counted all the deaths over the course of the full match. The bug was fixed. We're uploading a new train and test set and invalidating current scores. Kudos to you!</p>",
      "rawMarkdown": "Thanks Jimmy, you and your team are right. There was a bug in the extraction code, which filtered kill/death events incorrectly and counted all the deaths over the course of the full match. The bug was fixed. We're uploading a new train and test set and invalidating current scores. Kudos to you!",
      "votes": null
    },
    {
      "id": "521316",
      "postDate": "04/22/2019 19:01:59",
      "content": "<p>Definitely data leak, kills should match deaths.</p>\n\n<p>Just for the record: <strong>eli5 feature importance</strong> with LightGBM I did has values for 'deaths' 0.07-0.10, while next in line is below 0.004!</p>",
      "rawMarkdown": "Definitely data leak, kills should match deaths.\n\nJust for the record: **eli5 feature importance** with LightGBM I did has values for 'deaths' 0.07-0.10, while next in line is below 0.004!",
      "votes": null
    },
    {
      "id": "521325",
      "postDate": "04/22/2019 19:19:38",
      "content": "<p>Kills do not necessarily need to match deaths. You can die to the environment (get killed by a tower, creeps or neutral monster packs). I believe if no enemy player damaged you before you die to an environmental damage source, no kill is awarded.</p>",
      "rawMarkdown": "Kills do not necessarily need to match deaths. You can die to the environment (get killed by a tower, creeps or neutral monster packs). I believe if no enemy player damaged you before you die to an environmental damage source, no kill is awarded.",
      "votes": null
    },
    {
      "id": "521584",
      "postDate": "04/23/2019 04:28:07",
      "content": "<p>Of course, I did not express myself correct. They could be \"approximate\" to each other :)</p>\n\n<p>I doubt that in ProGames such things occur often.</p>",
      "rawMarkdown": "Of course, I did not express myself correct. They could be \"approximate\" to each other :)\n\nI doubt that in ProGames such things occur often.",
      "votes": null
    },
    {
      "id": "524616",
      "postDate": "04/29/2019 07:39:54",
      "content": "<p>：）, thank you, Jimmy.</p>",
      "rawMarkdown": "：）, thank you, Jimmy.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 520967,
      "author_name": "",
      "author_url": "",
      "post_date": "04/22/2019 04:51:40",
      "content": "<p>True that, deaths compared to kills is very different. 'Death' median cannot be like high compared to first 5 mins. Valuable find out 💯 </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 521181,
      "author_name": "vidosits",
      "author_url": "",
      "post_date": "04/22/2019 13:58:05",
      "content": "<p>Thanks Jimmy, you and your team are right. There was a bug in the extraction code, which filtered kill/death events incorrectly and counted all the deaths over the course of the full match. The bug was fixed. We're uploading a new train and test set and invalidating current scores. Kudos to you!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 521316,
      "author_name": "wlakinsson",
      "author_url": "",
      "post_date": "04/22/2019 19:01:59",
      "content": "<p>Definitely data leak, kills should match deaths.</p>\n\n<p>Just for the record: <strong>eli5 feature importance</strong> with LightGBM I did has values for 'deaths' 0.07-0.10, while next in line is below 0.004!</p>",
      "votes": null,
      "replies": [
        {
          "id": 521325,
          "author_name": "hamelg",
          "author_url": "",
          "post_date": "04/22/2019 19:19:38",
          "content": "<p>Kills do not necessarily need to match deaths. You can die to the environment (get killed by a tower, creeps or neutral monster packs). I believe if no enemy player damaged you before you die to an environmental damage source, no kill is awarded.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 521584,
          "author_name": "wlakinsson",
          "author_url": "",
          "post_date": "04/23/2019 04:28:07",
          "content": "<p>Of course, I did not express myself correct. They could be \"approximate\" to each other :)</p>\n\n<p>I doubt that in ProGames such things occur often.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 524616,
      "author_name": "maximumm",
      "author_url": "",
      "post_date": "04/29/2019 07:39:54",
      "content": "<p>：）, thank you, Jimmy.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "520578": "Hello. We think that there is a data leak in the deaths column of the dataset. It seems that the provided values are the number of deaths during the entire game instead of during the first five minutes. \nThe median number of deaths per team is 31 (which is very high for only 5 minutes of gameplay). The number of kills seems to be good though (with a median of 2). \nMoreover, when predicting the outcome of the match using only the difference in the number of deaths between the two teams (i.e. if one team has less deaths than the other one then we predict it wins) we get a train ROC of 0.87 while with the kills (which are supposed to encode the same information) we get a train ROC of 0.57.",
    "520967": "True that, deaths compared to kills is very different. 'Death' median cannot be like high compared to first 5 mins. Valuable find out 💯",
    "521181": "Thanks Jimmy, you and your team are right. There was a bug in the extraction code, which filtered kill/death events incorrectly and counted all the deaths over the course of the full match. The bug was fixed. We're uploading a new train and test set and invalidating current scores. Kudos to you!",
    "521316": "Definitely data leak, kills should match deaths.\n\nJust for the record: **eli5 feature importance** with LightGBM I did has values for 'deaths' 0.07-0.10, while next in line is below 0.004!",
    "521325": "Kills do not necessarily need to match deaths. You can die to the environment (get killed by a tower, creeps or neutral monster packs). I believe if no enemy player damaged you before you die to an environmental damage source, no kill is awarded.",
    "521584": "Of course, I did not express myself correct. They could be \"approximate\" to each other :)\n\nI doubt that in ProGames such things occur often.",
    "524616": "：）, thank you, Jimmy."
  },
  "source": "meta"
}