{
  "id": 16256,
  "title": "Please publish the true value of the test data and the scoring function",
  "url": "/competitions/grasp-and-lift-eeg-detection/discussion/16256",
  "author_name": "",
  "post_date": "2015-09-01T08:27:19.723Z",
  "votes": null,
  "comment_count": 2,
  "views": 689,
  "content": "<p>Hi!\nGood luck to everyone.</p>\n\n<p>Can the organizers of the competition publish the test value and the scoring function? I want to continue and explore the different methods.</p>\n\n<p>Thanks</p>",
  "messages": [
    {
      "id": "91165",
      "postDate": "09/01/2015 08:27:19",
      "content": "<p>Hi!\nGood luck to everyone.</p>\n\n<p>Can the organizers of the competition publish the test value and the scoring function? I want to continue and explore the different methods.</p>\n\n<p>Thanks</p>",
      "rawMarkdown": "Hi!\r\nGood luck to everyone.\r\n\r\nCan the organizers of the competition publish the test value and the scoring function? I want to continue and explore the different methods.\r\n\r\nThanks",
      "votes": null
    },
    {
      "id": "91169",
      "postDate": "09/01/2015 10:06:37",
      "content": "<p>We can certainly publish the true labels for the public leaderboard. I'll let you know once we've uploaded them somewhere. Regarding the labels for the private leaderboard, I'm not sure we are willing to publish them yet. We might want to host further competitions on this in the future.\nIn the meantime you can still submit to this competition and get the corresponding score. </p>\n\n<p>The evaluation metric is a average Area Under ROC for each column. I don't think Kaggle will publish their evaluation code, but it should be very easy to libraries in your language that can compute that. Also the winners are obliged to make their solution open source. So you could check to see if they also publish their cross-validation code.</p>",
      "rawMarkdown": "We can certainly publish the true labels for the public leaderboard. I'll let you know once we've uploaded them somewhere. Regarding the labels for the private leaderboard, I'm not sure we are willing to publish them yet. We might want to host further competitions on this in the future.\r\nIn the meantime you can still submit to this competition and get the corresponding score. \r\n\r\nThe evaluation metric is a average Area Under ROC for each column. I don't think Kaggle will publish their evaluation code, but it should be very easy to libraries in your language that can compute that. Also the winners are obliged to make their solution open source. So you could check to see if they also publish their cross-validation code.",
      "votes": null
    },
    {
      "id": "91171",
      "postDate": "09/01/2015 10:28:56",
      "content": "<p>Thank!</p>",
      "rawMarkdown": "Thank!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 91169,
      "author_name": "qwlouse",
      "author_url": "",
      "post_date": "09/01/2015 10:06:37",
      "content": "<p>We can certainly publish the true labels for the public leaderboard. I'll let you know once we've uploaded them somewhere. Regarding the labels for the private leaderboard, I'm not sure we are willing to publish them yet. We might want to host further competitions on this in the future.\nIn the meantime you can still submit to this competition and get the corresponding score. </p>\n\n<p>The evaluation metric is a average Area Under ROC for each column. I don't think Kaggle will publish their evaluation code, but it should be very easy to libraries in your language that can compute that. Also the winners are obliged to make their solution open source. So you could check to see if they also publish their cross-validation code.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 91171,
      "author_name": "ori226",
      "author_url": "",
      "post_date": "09/01/2015 10:28:56",
      "content": "<p>Thank!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "91165": "Hi!\r\nGood luck to everyone.\r\n\r\nCan the organizers of the competition publish the test value and the scoring function? I want to continue and explore the different methods.\r\n\r\nThanks",
    "91169": "We can certainly publish the true labels for the public leaderboard. I'll let you know once we've uploaded them somewhere. Regarding the labels for the private leaderboard, I'm not sure we are willing to publish them yet. We might want to host further competitions on this in the future.\r\nIn the meantime you can still submit to this competition and get the corresponding score. \r\n\r\nThe evaluation metric is a average Area Under ROC for each column. I don't think Kaggle will publish their evaluation code, but it should be very easy to libraries in your language that can compute that. Also the winners are obliged to make their solution open source. So you could check to see if they also publish their cross-validation code.",
    "91171": "Thank!"
  },
  "source": "meta"
}