{
  "id": 16116,
  "title": "To the admins: How can we take seriously the LB?",
  "url": "/competitions/flavours-of-physics/discussion/16116",
  "author_name": "",
  "post_date": "2015-08-25T09:38:56.673Z",
  "votes": 5,
  "comment_count": 8,
  "views": 1511,
  "content": "<p>Can we get a definite commitment from the admins, how can we handle the LB entries like [out of the competition], [anything I want to write here] etc, because I think this kind of vagueness can set back the drive of many of the competitors.\nFurthermore can I get a statement or links to the previously done statements about, how can we use which data, here I mean mainly the wobbliness around the &quot;control channel thing&quot; and so on. Thank you.</p>",
  "messages": [
    {
      "id": "90305",
      "postDate": "08/25/2015 09:38:56",
      "content": "<p>Can we get a definite commitment from the admins, how can we handle the LB entries like [out of the competition], [anything I want to write here] etc, because I think this kind of vagueness can set back the drive of many of the competitors.\nFurthermore can I get a statement or links to the previously done statements about, how can we use which data, here I mean mainly the wobbliness around the &quot;control channel thing&quot; and so on. Thank you.</p>",
      "rawMarkdown": "Can we get a definite commitment from the admins, how can we handle the LB entries like [out of the competition], [anything I want to write here] etc, because I think this kind of vagueness can set back the drive of many of the competitors.\r\nFurthermore can I get a statement or links to the previously done statements about, how can we use which data, here I mean mainly the wobbliness around the \"control channel thing\" and so on. Thank you.",
      "votes": null
    },
    {
      "id": "90445",
      "postDate": "08/26/2015 16:00:35",
      "content": "<p>[quote=George Solymosi;90305]</p>\n\n<p>Can we get a definite commitment from the admins, how can we handle the LB entries like [out of the competition], [anything I want to write here] etc, because I think this kind of vagueness can set back the drive of many of the competitors.\nFurthermore can I get a statement or links to the previously done statements about, how can we use which data, here I mean mainly the wobbliness around the &quot;control channel thing&quot; and so on. Thank you.</p>\n\n<p>[/quote]\nIn my opinion, these 'out of competition' labels discourage people from anticipating this competition. New comers are scared by both the label and the extreme high score on the LB, and people may be confused by the label (why these guys are out of competition) and they also think they have no hope of competing in this game because of the very high score.</p>\n\n<p>Data leakage is not a new secret on Kaggle (the best or worst one I saw was the Facebook Stackoverflow competition where test data overlapped with the training, amazing!) Our dear admins should have found some solutions to mitigate. Maybe the admin can talk to the top rank guys?</p>",
      "rawMarkdown": "[quote=George Solymosi;90305]\r\n\r\nCan we get a definite commitment from the admins, how can we handle the LB entries like [out of the competition], [anything I want to write here] etc, because I think this kind of vagueness can set back the drive of many of the competitors.\r\nFurthermore can I get a statement or links to the previously done statements about, how can we use which data, here I mean mainly the wobbliness around the \"control channel thing\" and so on. Thank you.\r\n\r\n[/quote]\r\nIn my opinion, these 'out of competition' labels discourage people from anticipating this competition. New comers are scared by both the label and the extreme high score on the LB, and people may be confused by the label (why these guys are out of competition) and they also think they have no hope of competing in this game because of the very high score.\r\n\r\nData leakage is not a new secret on Kaggle (the best or worst one I saw was the Facebook Stackoverflow competition where test data overlapped with the training, amazing!) Our dear admins should have found some solutions to mitigate. Maybe the admin can talk to the top rank guys?",
      "votes": null
    },
    {
      "id": "90470",
      "postDate": "08/26/2015 19:18:17",
      "content": "",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "90472",
      "postDate": "08/26/2015 19:43:42",
      "content": "<p>[quote=phunter;90445]</p>\n\n<p>In my opinion, these 'out of competition' labels discourage people from anticipating this competition. New comers are scared by both the label and the extreme high score on the LB, and people may be confused by the label (why these guys are out of competition) and they also think they have no hope of competing in this game because of the very high score.</p>\n\n<p>Data leakage is not a new secret on Kaggle (the best or worst one I saw was the Facebook Stackoverflow competition where test data overlapped with the training, amazing!) Our dear admins should have found some solutions to mitigate. Maybe the admin can talk to the top rank guys?</p>\n\n<p>[/quote]</p>\n\n<p>Same thoughts.</p>",
      "rawMarkdown": "[quote=phunter;90445]\r\n\r\nIn my opinion, these 'out of competition' labels discourage people from anticipating this competition. New comers are scared by both the label and the extreme high score on the LB, and people may be confused by the label (why these guys are out of competition) and they also think they have no hope of competing in this game because of the very high score.\r\n\r\nData leakage is not a new secret on Kaggle (the best or worst one I saw was the Facebook Stackoverflow competition where test data overlapped with the training, amazing!) Our dear admins should have found some solutions to mitigate. Maybe the admin can talk to the top rank guys?\r\n\r\n[/quote]\r\n\r\nSame thoughts.",
      "votes": null
    },
    {
      "id": "91074",
      "postDate": "08/31/2015 18:42:31",
      "content": "<p>Hi @George, @phunter </p>\n\n<p>Thank you for your suggestions. We will take some actions to mitigate. </p>\n\n<p>Every Kaggle competition is different. We do try our best to avoid data leakage, but it's not always perfectly avoidable given the natural constraints of the problem. </p>",
      "rawMarkdown": "Hi @George, @phunter \r\n\r\nThank you for your suggestions. We will take some actions to mitigate. \r\n\r\nEvery Kaggle competition is different. We do try our best to avoid data leakage, but it's not always perfectly avoidable given the natural constraints of the problem.",
      "votes": null
    },
    {
      "id": "91126",
      "postDate": "09/01/2015 00:15:34",
      "content": "<p>[quote=Wendy Kan;91074]</p>\n\n<p>Hi @George, @phunter </p>\n\n<p>Thank you for your suggestions. We will take some actions to mitigate. </p>\n\n<p>Every Kaggle competition is different. We do try our best to avoid data leakage, but it's not always perfectly avoidable given the natural constraints of the problem. </p>\n\n<p>[/quote]\nThanks to Wendy and all admins. The current LB shows the top 7 teams have stopped their submissions, and teams from 8th to lower ranks are still actively submitting, from which we can see who are the real competitors. We also see a giant leap from 8th to 7th score which I don't think a smart feature engineering + a smart model can make. This situation really scares the competitors and people may see no hope for a better rank so they may give up.</p>",
      "rawMarkdown": "[quote=Wendy Kan;91074]\r\n\r\nHi @George, @phunter \r\n\r\nThank you for your suggestions. We will take some actions to mitigate. \r\n\r\nEvery Kaggle competition is different. We do try our best to avoid data leakage, but it's not always perfectly avoidable given the natural constraints of the problem. \r\n\r\n[/quote]\r\nThanks to Wendy and all admins. The current LB shows the top 7 teams have stopped their submissions, and teams from 8th to lower ranks are still actively submitting, from which we can see who are the real competitors. We also see a giant leap from 8th to 7th score which I don't think a smart feature engineering + a smart model can make. This situation really scares the competitors and people may see no hope for a better rank so they may give up.",
      "votes": null
    },
    {
      "id": "91128",
      "postDate": "09/01/2015 00:23:02",
      "content": "<p>Does the fact that the top 7 teams have not submitted recently really mean anything?\nHow do you conclude from that these teams are out of the competition?</p>",
      "rawMarkdown": "Does the fact that the top 7 teams have not submitted recently really mean anything?\r\nHow do you conclude from that these teams are out of the competition?",
      "votes": null
    },
    {
      "id": "91153",
      "postDate": "09/01/2015 03:44:36",
      "content": "<p>Since correlation is not causation, I can't conclude the top 7 teams are out of competition. But these scores are high enough to scare new comers away, plus top 7 teams haven't updated for a long time, plus the top 2 have 'out of competition' labels. All of these look so scary.</p>",
      "rawMarkdown": "Since correlation is not causation, I can't conclude the top 7 teams are out of competition. But these scores are high enough to scare new comers away, plus top 7 teams haven't updated for a long time, plus the top 2 have 'out of competition' labels. All of these look so scary.",
      "votes": null
    },
    {
      "id": "91194",
      "postDate": "09/01/2015 14:19:07",
      "content": "<p>[quote=Wendy Kan;91074]</p>\n\n<p>Hi @George, @phunter </p>\n\n<p>Thank you for your suggestions. We will take some actions to mitigate. </p>\n\n<p>Every Kaggle competition is different. We do try our best to avoid data leakage, but it's not always perfectly avoidable given the natural constraints of the problem. </p>\n\n<p>[/quote]</p>\n\n<p>Thank you Wendy, we are really hoping to see for your real, and prompt  - as soon as possible - solution for this anomaly. By the lack of the urgent action, this competion won't achieve its real score, as @phunter accurately described above. Thanks for your efforts. </p>",
      "rawMarkdown": "[quote=Wendy Kan;91074]\r\n\r\nHi @George, @phunter \r\n\r\nThank you for your suggestions. We will take some actions to mitigate. \r\n\r\nEvery Kaggle competition is different. We do try our best to avoid data leakage, but it's not always perfectly avoidable given the natural constraints of the problem. \r\n\r\n[/quote]\r\n\r\nThank you Wendy, we are really hoping to see for your real, and prompt  - as soon as possible - solution for this anomaly. By the lack of the urgent action, this competion won't achieve its real score, as @phunter accurately described above. Thanks for your efforts.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 90445,
      "author_name": "phunter",
      "author_url": "",
      "post_date": "08/26/2015 16:00:35",
      "content": "<p>[quote=George Solymosi;90305]</p>\n\n<p>Can we get a definite commitment from the admins, how can we handle the LB entries like [out of the competition], [anything I want to write here] etc, because I think this kind of vagueness can set back the drive of many of the competitors.\nFurthermore can I get a statement or links to the previously done statements about, how can we use which data, here I mean mainly the wobbliness around the &quot;control channel thing&quot; and so on. Thank you.</p>\n\n<p>[/quote]\nIn my opinion, these 'out of competition' labels discourage people from anticipating this competition. New comers are scared by both the label and the extreme high score on the LB, and people may be confused by the label (why these guys are out of competition) and they also think they have no hope of competing in this game because of the very high score.</p>\n\n<p>Data leakage is not a new secret on Kaggle (the best or worst one I saw was the Facebook Stackoverflow competition where test data overlapped with the training, amazing!) Our dear admins should have found some solutions to mitigate. Maybe the admin can talk to the top rank guys?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 90470,
      "author_name": "",
      "author_url": "",
      "post_date": "08/26/2015 19:18:17",
      "content": "",
      "votes": null,
      "replies": []
    },
    {
      "id": 90472,
      "author_name": "",
      "author_url": "",
      "post_date": "08/26/2015 19:43:42",
      "content": "<p>[quote=phunter;90445]</p>\n\n<p>In my opinion, these 'out of competition' labels discourage people from anticipating this competition. New comers are scared by both the label and the extreme high score on the LB, and people may be confused by the label (why these guys are out of competition) and they also think they have no hope of competing in this game because of the very high score.</p>\n\n<p>Data leakage is not a new secret on Kaggle (the best or worst one I saw was the Facebook Stackoverflow competition where test data overlapped with the training, amazing!) Our dear admins should have found some solutions to mitigate. Maybe the admin can talk to the top rank guys?</p>\n\n<p>[/quote]</p>\n\n<p>Same thoughts.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 91074,
      "author_name": "wendykan",
      "author_url": "",
      "post_date": "08/31/2015 18:42:31",
      "content": "<p>Hi @George, @phunter </p>\n\n<p>Thank you for your suggestions. We will take some actions to mitigate. </p>\n\n<p>Every Kaggle competition is different. We do try our best to avoid data leakage, but it's not always perfectly avoidable given the natural constraints of the problem. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 91126,
      "author_name": "phunter",
      "author_url": "",
      "post_date": "09/01/2015 00:15:34",
      "content": "<p>[quote=Wendy Kan;91074]</p>\n\n<p>Hi @George, @phunter </p>\n\n<p>Thank you for your suggestions. We will take some actions to mitigate. </p>\n\n<p>Every Kaggle competition is different. We do try our best to avoid data leakage, but it's not always perfectly avoidable given the natural constraints of the problem. </p>\n\n<p>[/quote]\nThanks to Wendy and all admins. The current LB shows the top 7 teams have stopped their submissions, and teams from 8th to lower ranks are still actively submitting, from which we can see who are the real competitors. We also see a giant leap from 8th to 7th score which I don't think a smart feature engineering + a smart model can make. This situation really scares the competitors and people may see no hope for a better rank so they may give up.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 91128,
      "author_name": "dbarge",
      "author_url": "",
      "post_date": "09/01/2015 00:23:02",
      "content": "<p>Does the fact that the top 7 teams have not submitted recently really mean anything?\nHow do you conclude from that these teams are out of the competition?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 91153,
      "author_name": "phunter",
      "author_url": "",
      "post_date": "09/01/2015 03:44:36",
      "content": "<p>Since correlation is not causation, I can't conclude the top 7 teams are out of competition. But these scores are high enough to scare new comers away, plus top 7 teams haven't updated for a long time, plus the top 2 have 'out of competition' labels. All of these look so scary.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 91194,
      "author_name": "",
      "author_url": "",
      "post_date": "09/01/2015 14:19:07",
      "content": "<p>[quote=Wendy Kan;91074]</p>\n\n<p>Hi @George, @phunter </p>\n\n<p>Thank you for your suggestions. We will take some actions to mitigate. </p>\n\n<p>Every Kaggle competition is different. We do try our best to avoid data leakage, but it's not always perfectly avoidable given the natural constraints of the problem. </p>\n\n<p>[/quote]</p>\n\n<p>Thank you Wendy, we are really hoping to see for your real, and prompt  - as soon as possible - solution for this anomaly. By the lack of the urgent action, this competion won't achieve its real score, as @phunter accurately described above. Thanks for your efforts. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "90305": "Can we get a definite commitment from the admins, how can we handle the LB entries like [out of the competition], [anything I want to write here] etc, because I think this kind of vagueness can set back the drive of many of the competitors.\r\nFurthermore can I get a statement or links to the previously done statements about, how can we use which data, here I mean mainly the wobbliness around the \"control channel thing\" and so on. Thank you.",
    "90445": "[quote=George Solymosi;90305]\r\n\r\nCan we get a definite commitment from the admins, how can we handle the LB entries like [out of the competition], [anything I want to write here] etc, because I think this kind of vagueness can set back the drive of many of the competitors.\r\nFurthermore can I get a statement or links to the previously done statements about, how can we use which data, here I mean mainly the wobbliness around the \"control channel thing\" and so on. Thank you.\r\n\r\n[/quote]\r\nIn my opinion, these 'out of competition' labels discourage people from anticipating this competition. New comers are scared by both the label and the extreme high score on the LB, and people may be confused by the label (why these guys are out of competition) and they also think they have no hope of competing in this game because of the very high score.\r\n\r\nData leakage is not a new secret on Kaggle (the best or worst one I saw was the Facebook Stackoverflow competition where test data overlapped with the training, amazing!) Our dear admins should have found some solutions to mitigate. Maybe the admin can talk to the top rank guys?",
    "90470": "",
    "90472": "[quote=phunter;90445]\r\n\r\nIn my opinion, these 'out of competition' labels discourage people from anticipating this competition. New comers are scared by both the label and the extreme high score on the LB, and people may be confused by the label (why these guys are out of competition) and they also think they have no hope of competing in this game because of the very high score.\r\n\r\nData leakage is not a new secret on Kaggle (the best or worst one I saw was the Facebook Stackoverflow competition where test data overlapped with the training, amazing!) Our dear admins should have found some solutions to mitigate. Maybe the admin can talk to the top rank guys?\r\n\r\n[/quote]\r\n\r\nSame thoughts.",
    "91074": "Hi @George, @phunter \r\n\r\nThank you for your suggestions. We will take some actions to mitigate. \r\n\r\nEvery Kaggle competition is different. We do try our best to avoid data leakage, but it's not always perfectly avoidable given the natural constraints of the problem.",
    "91126": "[quote=Wendy Kan;91074]\r\n\r\nHi @George, @phunter \r\n\r\nThank you for your suggestions. We will take some actions to mitigate. \r\n\r\nEvery Kaggle competition is different. We do try our best to avoid data leakage, but it's not always perfectly avoidable given the natural constraints of the problem. \r\n\r\n[/quote]\r\nThanks to Wendy and all admins. The current LB shows the top 7 teams have stopped their submissions, and teams from 8th to lower ranks are still actively submitting, from which we can see who are the real competitors. We also see a giant leap from 8th to 7th score which I don't think a smart feature engineering + a smart model can make. This situation really scares the competitors and people may see no hope for a better rank so they may give up.",
    "91128": "Does the fact that the top 7 teams have not submitted recently really mean anything?\r\nHow do you conclude from that these teams are out of the competition?",
    "91153": "Since correlation is not causation, I can't conclude the top 7 teams are out of competition. But these scores are high enough to scare new comers away, plus top 7 teams haven't updated for a long time, plus the top 2 have 'out of competition' labels. All of these look so scary.",
    "91194": "[quote=Wendy Kan;91074]\r\n\r\nHi @George, @phunter \r\n\r\nThank you for your suggestions. We will take some actions to mitigate. \r\n\r\nEvery Kaggle competition is different. We do try our best to avoid data leakage, but it's not always perfectly avoidable given the natural constraints of the problem. \r\n\r\n[/quote]\r\n\r\nThank you Wendy, we are really hoping to see for your real, and prompt  - as soon as possible - solution for this anomaly. By the lack of the urgent action, this competion won't achieve its real score, as @phunter accurately described above. Thanks for your efforts."
  },
  "source": "meta"
}