{
  "id": 14648,
  "title": "Results are now final",
  "url": "/competitions/facebook-recruiting-iv-human-or-bot/discussion/14648",
  "author_name": "",
  "post_date": "2015-06-09T21:31:49.867Z",
  "votes": 1,
  "comment_count": 9,
  "views": 2544,
  "content": "<p>Cheaters removed.&nbsp;Congrats to the winners!</p>\n<p>If you haven't done so, please upload your resume/CV to the &quot;upload model&quot; section before this Friday, 6/12, so Facebook can start contacting you.&nbsp;</p>\n<p>Kaggle would like to thank everyone for participating in our most popular recruiting competition. We've been very impressed in the work that was done, given that the dataset is very noisy.&nbsp;</p>\n<p>See you in another competition!</p>",
  "messages": [
    {
      "id": "81408",
      "postDate": "06/09/2015 21:31:49",
      "content": "<p>Cheaters removed.&nbsp;Congrats to the winners!</p>\n<p>If you haven't done so, please upload your resume/CV to the &quot;upload model&quot; section before this Friday, 6/12, so Facebook can start contacting you.&nbsp;</p>\n<p>Kaggle would like to thank everyone for participating in our most popular recruiting competition. We've been very impressed in the work that was done, given that the dataset is very noisy.&nbsp;</p>\n<p>See you in another competition!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "81411",
      "postDate": "06/09/2015 21:57:41",
      "content": "<p>Aaaaand this is the thread where we come to cry.</p>\n<p>I had a 0.93738 score in the private leaderboard in one of my submissions when I was using ensembling.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "81413",
      "postDate": "06/09/2015 22:04:33",
      "content": "<p>[quote=Jos&#233; Mar&#237;a Mateos;81411]<br>Aaaaand this is the thread where we come to cry.<br>[/quote]<br>No, it is not. You are looking for <a href=\"https://www.kaggle.com/c/facebook-recruiting-iv-human-or-bot/forums/t/14639/poor-result-due-to-overfitting-how-to-prevent-next-time\">this</a>&nbsp;one :)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "81416",
      "postDate": "06/09/2015 22:13:49",
      "content": "<p>Oh, I can cry in two different places now. Thanks! :D</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "81462",
      "postDate": "06/10/2015 08:27:17",
      "content": "<p>[quote=Wendy Kan;81408]</p>\n<p>Cheaters removed.&nbsp;Congrats to the winners!</p>\n<p>[/quote]</p>\n\n<p>Just curious. How are cheaters identified?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "81464",
      "postDate": "06/10/2015 08:33:32",
      "content": "<p>[quote=Adarsh Chavakula;81462]</p>\n<p>[quote=Wendy Kan;81408]</p>\n<p>Cheaters removed.&nbsp;Congrats to the winners!</p>\n<p>[/quote]</p>\n<p>Just curious. How are cheaters identified?</p>\n<p>[/quote]</p>\n<p>TOP SECRET!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "81485",
      "postDate": "06/10/2015 15:02:06",
      "content": "<p>[quote=Adarsh Chavakula;81462]</p>\n<p>[quote=Wendy Kan;81408]</p>\n<p>Cheaters removed.&nbsp;Congrats to the winners!</p>\n<p>[/quote]</p>\n<p>Just curious. How are cheaters identified?</p>\n<p>[/quote]</p>\n\n<p>That is a whole another competition. :)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "81504",
      "postDate": "06/10/2015 17:49:03",
      "content": "<p>Does everyone's 5-fold, 10-fold cv score close to their private LB score?</p>\n<p>Even though the final models I have which score 0.914 on public and 0.932 on private LB have 0.93 cv score, I do have an earlier model which score 0.905 on public and 0.934 on private LB, and the cv score is only 0.915. &nbsp;</p>\n<p>It makes me wonder if we will need the right set of features to make the cv score close to the private LB score, and the earlier model I build is actually overfitting to the test data...</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "81527",
      "postDate": "06/10/2015 20:07:10",
      "content": "<p>Personally I think that given the fact that we actually have very few bot examples in the training set, it is kind of hard to have a reliable estimate of the AUC of a model thanks to the training set. I'm not really surprised that there could be noticeable variation of score between CV score on training set and final score on the&nbsp;private LB&nbsp;set, even if you do your estimation properly.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "81543",
      "postDate": "06/10/2015 22:33:31",
      "content": "<p>Yes, but cv score is all we got, I guess we can make lots and lots of features and maybe cv score and final score will&nbsp;eventually be very close no matter what.</p>\n<p>Sometimes&nbsp;the power of good classification algorithms and perfect model is not comparable to a larger datasets.&nbsp;</p>\n<p>[quote=Thomas ROBERT;81527]</p>\n<p>Personally I think that given the fact that we actually have very few bot examples in the training set, it is kind of hard to have a reliable estimate of the AUC of a model thanks to the training set. I'm not really surprised that there could be noticeable variation of score between CV score on training set and final score on the&nbsp;private LB&nbsp;set, even if you do your estimation properly.</p>\n<p>[/quote]</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 81411,
      "author_name": "rinzewind",
      "author_url": "",
      "post_date": "06/09/2015 21:57:41",
      "content": "<p>Aaaaand this is the thread where we come to cry.</p>\n<p>I had a 0.93738 score in the private leaderboard in one of my submissions when I was using ensembling.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 81413,
      "author_name": "carloshuertas",
      "author_url": "",
      "post_date": "06/09/2015 22:04:33",
      "content": "<p>[quote=Jos&#233; Mar&#237;a Mateos;81411]<br>Aaaaand this is the thread where we come to cry.<br>[/quote]<br>No, it is not. You are looking for <a href=\"https://www.kaggle.com/c/facebook-recruiting-iv-human-or-bot/forums/t/14639/poor-result-due-to-overfitting-how-to-prevent-next-time\">this</a>&nbsp;one :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 81416,
      "author_name": "rinzewind",
      "author_url": "",
      "post_date": "06/09/2015 22:13:49",
      "content": "<p>Oh, I can cry in two different places now. Thanks! :D</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 81462,
      "author_name": "adarshchavakula",
      "author_url": "",
      "post_date": "06/10/2015 08:27:17",
      "content": "<p>[quote=Wendy Kan;81408]</p>\n<p>Cheaters removed.&nbsp;Congrats to the winners!</p>\n<p>[/quote]</p>\n\n<p>Just curious. How are cheaters identified?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 81464,
      "author_name": "abhishek",
      "author_url": "",
      "post_date": "06/10/2015 08:33:32",
      "content": "<p>[quote=Adarsh Chavakula;81462]</p>\n<p>[quote=Wendy Kan;81408]</p>\n<p>Cheaters removed.&nbsp;Congrats to the winners!</p>\n<p>[/quote]</p>\n<p>Just curious. How are cheaters identified?</p>\n<p>[/quote]</p>\n<p>TOP SECRET!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 81485,
      "author_name": "",
      "author_url": "",
      "post_date": "06/10/2015 15:02:06",
      "content": "<p>[quote=Adarsh Chavakula;81462]</p>\n<p>[quote=Wendy Kan;81408]</p>\n<p>Cheaters removed.&nbsp;Congrats to the winners!</p>\n<p>[/quote]</p>\n<p>Just curious. How are cheaters identified?</p>\n<p>[/quote]</p>\n\n<p>That is a whole another competition. :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 81504,
      "author_name": "skylibrary",
      "author_url": "",
      "post_date": "06/10/2015 17:49:03",
      "content": "<p>Does everyone's 5-fold, 10-fold cv score close to their private LB score?</p>\n<p>Even though the final models I have which score 0.914 on public and 0.932 on private LB have 0.93 cv score, I do have an earlier model which score 0.905 on public and 0.934 on private LB, and the cv score is only 0.915. &nbsp;</p>\n<p>It makes me wonder if we will need the right set of features to make the cv score close to the private LB score, and the earlier model I build is actually overfitting to the test data...</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 81527,
      "author_name": "thomasrobertfr",
      "author_url": "",
      "post_date": "06/10/2015 20:07:10",
      "content": "<p>Personally I think that given the fact that we actually have very few bot examples in the training set, it is kind of hard to have a reliable estimate of the AUC of a model thanks to the training set. I'm not really surprised that there could be noticeable variation of score between CV score on training set and final score on the&nbsp;private LB&nbsp;set, even if you do your estimation properly.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 81543,
      "author_name": "skylibrary",
      "author_url": "",
      "post_date": "06/10/2015 22:33:31",
      "content": "<p>Yes, but cv score is all we got, I guess we can make lots and lots of features and maybe cv score and final score will&nbsp;eventually be very close no matter what.</p>\n<p>Sometimes&nbsp;the power of good classification algorithms and perfect model is not comparable to a larger datasets.&nbsp;</p>\n<p>[quote=Thomas ROBERT;81527]</p>\n<p>Personally I think that given the fact that we actually have very few bot examples in the training set, it is kind of hard to have a reliable estimate of the AUC of a model thanks to the training set. I'm not really surprised that there could be noticeable variation of score between CV score on training set and final score on the&nbsp;private LB&nbsp;set, even if you do your estimation properly.</p>\n<p>[/quote]</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "81408": "",
    "81411": "",
    "81413": "",
    "81416": "",
    "81462": "",
    "81464": "",
    "81485": "",
    "81504": "",
    "81527": "",
    "81543": ""
  },
  "source": "meta"
}