![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/477085/11395/gel_gel.jpg\")
It is not a secret that all the teams have been using \"outlier\" overrides to improve their standings in the public LB. I have tested out, that having a single override can make a 0.003-0.004 score difference. So imagine the situation at the top... There is no way of knowing where you actually stand! - current public LB gold position (except #1 guy) are within 3 overrides range!
\n\nSo for all who feels bad about their current leaderboard rank - if you haven't done public LB override overfitting (like tuning the \"perfect\" threshold for you outlier classifier) - I am quite sure you are going to improve in the final standings!
\n\nTherefore - this is going to be a crazy!
\n\np.s. we haven't focused on overrides for a long time now
\n\nThere is couple overrides I was able to get during my testing phase early in the competition:\nC_ID_b789a1772c, \nC_ID_944c62886f
Again, these are going to be useless in the end!
", "messages": [ { "id": 477051, "postDate": "2019-02-23T18:54:48.660Z", "content": "It is not a secret that all the teams have been using \"outlier\" overrides to improve their standings in the public LB. I have tested out, that having a single override can make a 0.003-0.004 score difference. So imagine the situation at the top... There is no way of knowing where you actually stand! - current public LB gold position (except #1 guy) are within 3 overrides range!
\n\nSo for all who feels bad about their current leaderboard rank - if you haven't done public LB override overfitting (like tuning the \"perfect\" threshold for you outlier classifier) - I am quite sure you are going to improve in the final standings!
\n\nTherefore - this is going to be a crazy!
\n\np.s. we haven't focused on overrides for a long time now
\n\nThere is couple overrides I was able to get during my testing phase early in the competition:\nC_ID_b789a1772c, \nC_ID_944c62886f
Again, these are going to be useless in the end!
", "rawMarkdown": "It is not a secret that all the teams have been using \"outlier\" overrides to improve their standings in the public LB. I have tested out, that having a single override can make a 0.003-0.004 score difference. So imagine the situation at the top... There is no way of knowing where you actually stand! - current public LB gold position (except #1 guy) are within 3 overrides range!\n\nSo for all who feels bad about their current leaderboard rank - if you haven't done public LB override overfitting (like tuning the \"perfect\" threshold for you outlier classifier) - I am quite sure you are going to improve in the final standings!\n\nTherefore - this is going to be a crazy!\n\np.s. we haven't focused on overrides for a long time now\n\nThere is couple overrides I was able to get during my testing phase early in the competition:\n```C_ID_b789a1772c, \nC_ID_944c62886f```\n\nAgain, these are going to be useless in the end!\n", "votes": 60 }, { "id": 477085, "postDate": "2019-02-23T20:47:28.323Z", "content": "Waiting for it.
\n\n
\n\n\nThere is couple overrides I was able to get during my testing phase early in the competition:\n C_ID_b789a1772c, C_ID_944c62886f
\n \nAgain, these are going to be useless in the end!
\n
I like it, there are bound to be some 0.003+ leaps up the public leaderboard in the last days, thanks for doing your bit to increase the shake-up even more ;)
\n\nFor any n00bs wondering about shake-up, it can be measured with a metric introduced (I think) by @BreakfastPirate , who sadly seems to have deleted his account recently. I hope he's ok (anyone know if/why he left?) For anyone not familiar with the name I suggest you use the search box to seek out his great discussion posts before they disappear (he had over 60 golds!)
\n\nTo fill in, I've created a kernel on Meta Kaggle to compute shake-up for all competitions, and render HTML and PNG tables, like this (filtered view of only 800+ team competitions):
\n\nhttps://www.kaggle.com/jtrotman/meta-kaggle-competition-shake-up
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/477110/11396/shakeup_teams_over_800.png\")
A few days after each competition ends it's details go in to Meta-Kaggle, so anyone can fork my kernel at any time & get a new up to date list :)
\n\nMy bet for Elo is at least 0.1, so probably a new top 5 entry in that table...
", "rawMarkdown": "> There is couple overrides I was able to get during my testing phase early in the competition:\n> C_ID_b789a1772c, C_ID_944c62886f\n> \n> Again, these are going to be useless in the end!\n\nI like it, there are bound to be some 0.003+ leaps up the public leaderboard in the last days, thanks for doing your bit to *increase* the shake-up even more ;)\n\nFor any n00bs wondering about shake-up, it can be measured with a metric introduced (I think) by @BreakfastPirate , who sadly seems to have deleted his account recently. I hope he's ok (anyone know if/why he left?) For anyone not familiar with the name I suggest you use the search box to seek out his great discussion posts before they disappear (he had **over 60** golds!)\n\nTo fill in, I've created a [kernel on Meta Kaggle][1] to compute shake-up for all competitions, and render HTML and PNG tables, like this (filtered view of only 800+ team competitions):\n\nhttps://www.kaggle.com/jtrotman/meta-kaggle-competition-shake-up\n\n![enter image description here][2]\n\nA few days after each competition ends it's details go in to Meta-Kaggle, so anyone can fork my kernel at any time & get a new up to date list :)\n\nMy bet for Elo is at least 0.1, so probably a new top 5 entry in that table...\n\n\n [1]: https://www.kaggle.com/jtrotman/meta-kaggle-competition-shake-up\n [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/477110/11396/shakeup_teams_over_800.png \"Competition Shake-up\"\n", "votes": 16, "replies": [ { "id": 477147, "postDate": "2019-02-24T01:50:28.290Z", "content": "@jtrotman, thanks for this. Like you I also miss @BreakfastPirate and his LB shakeup posts. I hope he is okay too.
", "rawMarkdown": "@jtrotman, thanks for this. Like you I also miss @BreakfastPirate and his LB shakeup posts. I hope he is okay too.", "votes": 1 }, { "id": 477187, "postDate": "2019-02-24T04:27:51.677Z", "content": "Nice charts!
", "rawMarkdown": "Nice charts!", "votes": 1 } ] }, { "id": 479109, "postDate": "2019-02-27T01:10:52.950Z", "content": "I hear the folks competing in the Earthquake Prediction competition are good at predicting when a shake will occur...
", "rawMarkdown": "I hear the folks competing in the [Earthquake Prediction](https://www.kaggle.com/c/LANL-Earthquake-Prediction) competition are good at predicting when a shake will occur...", "votes": 6, "replies": [ { "id": 479186, "postDate": "2019-02-27T02:20:40.247Z", "content": "Good one @addisonhoward. I am in that one too. Whatever that says about my choices:-)
\n\nHere is my post on feature selection techniques https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/82052
", "rawMarkdown": "Good one @addisonhoward. I am in that one too. Whatever that says about my choices:-)\n\nHere is my post on feature selection techniques https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/82052", "votes": 1 } ] }, { "id": 477521, "postDate": "2019-02-24T19:21:23.480Z", "content": "Thanks ,
\n\nQuote by W. Edwards Deming:\n \"In God we trust, all others must bring data.\"\nAfter this competition:\n \"In God we trust, all others must bring data. .. Without Outliers !\"
", "rawMarkdown": "Thanks , \n\nQuote by W. Edwards Deming:\n \"In God we trust, all others must bring data.\"\nAfter this competition:\n \"In God we trust, all others must bring data. .. Without Outliers !\"\n", "votes": 3 }, { "id": 477866, "postDate": "2019-02-25T11:53:41.467Z", "content": "In overfitting we trust
", "rawMarkdown": "In overfitting we trust", "votes": 4 }, { "id": 479135, "postDate": "2019-02-27T01:32:18.740Z", "content": "The shakeup was so huge that the 1st place PL finished just behind me, down 1412 places
", "rawMarkdown": "The shakeup was so huge that the 1st place PL finished just behind me, down 1412 places", "votes": 1, "replies": [ { "id": 479188, "postDate": "2019-02-27T02:23:25.060Z", "content": "And the 2nd place team ended down at 2460 @bernnik.
", "rawMarkdown": "And the 2nd place team ended down at 2460 @bernnik.", "votes": 2 }, { "id": 479250, "postDate": "2019-02-27T03:12:33.653Z", "content": "yes, that is just unbelievable.\nCongrads your place, it is really good. What kind of outlier prediction did you use? (I suppose without that prediction it was impossible to be in first 100)
", "rawMarkdown": "yes, that is just unbelievable.\nCongrads your place, it is really good. What kind of outlier prediction did you use? (I suppose without that prediction it was impossible to be in first 100) ", "votes": 1 }, { "id": 479261, "postDate": "2019-02-27T03:18:42.313Z", "content": "Thanks @bernnik. I really never found the right way to deal with the outliers. So I focused on proper feature engineer and feature reduction techniques. That helped a lot in improving my CV as well as LB in the last 3 days.
", "rawMarkdown": "Thanks @bernnik. I really never found the right way to deal with the outliers. So I focused on proper feature engineer and [feature reduction techniques][1]. That helped a lot in improving my CV as well as LB in the last 3 days.\n\n\n [1]: https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/82052" } ] }, { "id": 477364, "postDate": "2019-02-24T12:12:34.067Z", "content": "totally right, many would fall !!! excited to see what's private looks like
", "rawMarkdown": "totally right, many would fall !!! excited to see what's private looks like", "votes": 1 }, { "id": 477060, "postDate": "2019-02-23T19:19:20.887Z", "content": "Yeah exactly, there will be really big shakeup. At first, I didn't use outlier detection effectively, so the improvements in lb score was really small then I start to use it effectively and my lb score rose significantly. I wish I could stop myself from doing it.
", "rawMarkdown": "Yeah exactly, there will be really big shakeup. At first, I didn't use outlier detection effectively, so the improvements in lb score was really small then I start to use it effectively and my lb score rose significantly. I wish I could stop myself from doing it.", "votes": 1 }, { "id": 477293, "postDate": "2019-02-24T09:57:23.877Z", "content": "Any comparison with Mercedes competition is totally wrong ... LOL.
", "rawMarkdown": "Any comparison with Mercedes competition is totally wrong ... LOL.", "votes": 2, "replies": [ { "id": 477303, "postDate": "2019-02-24T10:09:33.933Z", "content": "indeed - LOL.
", "rawMarkdown": "indeed - LOL.", "votes": 1 } ] }, { "id": 477145, "postDate": "2019-02-24T01:47:34.667Z", "content": "@raddar, your observations are right on and thanks for all your contributions.
\n\nI believe it may all come down to the best well thought out feature engineering. There was so much talk about whether we can even trust our local CV because of the aforementioned outlier issue. It is frustrating but I discovered once I significantly reduced my features through various methods, my CV is pretty consistent moving in the same direction with the LB. I know where I am right now is not where my best model of LB=0.680 will be ranked. But the unknowns are aplenty when it comes to where we would all be in the private LB. I really am curiously waiting to see where all this end. Whatever happens, I thoroughy enjoyed playing with this data.
\n\nGood luck everyone and happy kaggling.
", "rawMarkdown": "@raddar, your observations are right on and thanks for all your contributions.\n\nI believe it may all come down to the best well thought out feature engineering. There was so much talk about whether we can even trust our local CV because of the aforementioned outlier issue. It is frustrating but I discovered once I significantly reduced my features through various methods, my CV is pretty consistent moving in the same direction with the LB. I know where I am right now is not where my best model of LB=0.680 will be ranked. But the unknowns are aplenty when it comes to where we would all be in the private LB. I really am curiously waiting to see where all this end. Whatever happens, I thoroughy enjoyed playing with this data.\n\nGood luck everyone and happy kaggling.", "votes": 2 }, { "id": 479032, "postDate": "2019-02-27T00:00:08.473Z", "content": "Such a huge shake-up😳😱
", "rawMarkdown": "Such a huge shake-up😳😱" }, { "id": 478599, "postDate": "2019-02-26T11:35:41.727Z", "content": "3q
", "rawMarkdown": "3q" }, { "id": 478525, "postDate": "2019-02-26T09:12:28.940Z", "content": "Trust your CV.
", "rawMarkdown": "Trust your CV." }, { "id": 478501, "postDate": "2019-02-26T08:31:34.743Z", "content": "Okay buddy
", "rawMarkdown": "Okay buddy" }, { "id": 478443, "postDate": "2019-02-26T06:48:36.433Z", "content": "Absolutely! I tried so many times, but still for best LB Score my threshold is down to 42 % ( Oops!). Can't say anything about Private LB. It would be interesting to see kingdoms fall and rise.
", "rawMarkdown": "Absolutely! I tried so many times, but still for best LB Score my threshold is down to 42 % ( Oops!). Can't say anything about Private LB. It would be interesting to see kingdoms fall and rise." }, { "id": 477470, "postDate": "2019-02-24T17:01:44.573Z", "content": "awesome....bring it on
", "rawMarkdown": "awesome....bring it on" }, { "id": 477421, "postDate": "2019-02-24T14:10:27.027Z", "content": ".
", "rawMarkdown": ".\n" }, { "id": 477132, "postDate": "2019-02-24T00:49:52.083Z", "content": "I modified the public outliers detection script concept to make sure all parts are cross-validated. I just hope I can keep in the medal zone in this game lol.
", "rawMarkdown": "I modified the public outliers detection script concept to make sure all parts are cross-validated. I just hope I can keep in the medal zone in this game lol." }, { "id": 477121, "postDate": "2019-02-23T23:37:40.197Z", "content": "Actually, i gave up the competition a few days ago, when my best CV model scored 3.74 on the LB and manually changing 5 cards values, the same submission scored 3.681....., so i concluded that we may see up to +- 1000 positions shakeups on this competition
", "rawMarkdown": "Actually, i gave up the competition a few days ago, when my best CV model scored 3.74 on the LB and manually changing 5 cards values, the same submission scored 3.681....., so i concluded that we may see up to +- 1000 positions shakeups on this competition", "replies": [ { "id": 477127, "postDate": "2019-02-24T00:24:24.763Z", "content": "@dkaraflos, then you got a safe medal already.......
", "rawMarkdown": "@dkaraflos, then you got a safe medal already......." }, { "id": 477129, "postDate": "2019-02-24T00:31:49.833Z", "content": "The truth is that all depend on the outliers predictions. If your submission scores a target of 0 for example, and the ground truth is -33, then due to one observation you may get big shakeup on the private LB. Thus, nothing is safe here :)
\n\nThe thing is that we are blind on this one. I may get way out of the Bronze area, i may get gold medal.... who knows
", "rawMarkdown": "The truth is that all depend on the outliers predictions. If your submission scores a target of 0 for example, and the ground truth is -33, then due to one observation you may get big shakeup on the private LB. Thus, nothing is safe here :)\n\nThe thing is that we are blind on this one. I may get way out of the Bronze area, i may get gold medal.... who knows", "votes": 2 }, { "id": 477131, "postDate": "2019-02-24T00:37:06.957Z", "content": "I learnt a big lesson in this competition that I shouldn't use much post-processing. A strong single model is king.
", "rawMarkdown": "I learnt a big lesson in this competition that I shouldn't use much post-processing. A strong single model is king. ", "votes": 4 }, { "id": 477150, "postDate": "2019-02-24T01:53:52.403Z", "content": "Yep! @pukkinming. Hold on tight to your best single model based on CV.
", "rawMarkdown": "Yep! @pukkinming. Hold on tight to your best single model based on CV.", "votes": 1 } ] }, { "id": 477074, "postDate": "2019-02-23T20:13:14.037Z", "content": "I think many of us here will thank u later for this message.
", "rawMarkdown": "I think many of us here will thank u later for this message. " }, { "id": 480836, "postDate": "2019-02-28T18:25:22.390Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 480861, "postDate": "2019-02-28T19:31:10.457Z", "content": "be example, if score < -20 set score to -33.x
", "rawMarkdown": "be example, if score < -20 set score to -33.x", "votes": 1 } ] }, { "id": 479810, "postDate": "2019-02-27T12:52:20.303Z", "rawMarkdown": "", "votes": 2, "isDeleted": true }, { "id": 479033, "postDate": "2019-02-27T00:06:21.197Z", "rawMarkdown": "", "isDeleted": true }, { "id": 478014, "postDate": "2019-02-25T16:02:52.363Z", "rawMarkdown": "", "votes": 1, "isDeleted": true, "replies": [ { "id": 478021, "postDate": "2019-02-25T16:09:00.373Z", "content": "Just try again next time. Local CV 3.65 shouldn't be that difficult to achieve after adding some new features.
", "rawMarkdown": "Just try again next time. Local CV 3.65 shouldn't be that difficult to achieve after adding some new features." } ] }, { "id": 479658, "postDate": "2019-02-27T10:18:03.510Z", "content": "Thanks!
", "rawMarkdown": "Thanks!", "votes": 1 }, { "id": 478512, "postDate": "2019-02-26T08:52:34.020Z", "content": "thanks
", "rawMarkdown": "thanks" }, { "id": 477332, "postDate": "2019-02-24T10:54:22.727Z", "content": "Thanks.
", "rawMarkdown": "Thanks. " }, { "id": 477157, "postDate": "2019-02-24T02:09:34.323Z", "content": "thank you !
", "rawMarkdown": "thank you !" }, { "id": 477143, "postDate": "2019-02-24T01:43:28.287Z", "content": "Thank you very much, @raddar
", "rawMarkdown": "Thank you very much, @raddar" } ], "comments": [ { "id": 477085, "author_name": "silverstone", "author_url": "", "post_date": "2019-02-23T20:47:28.323000", "content": "Waiting for it.
\n\n
\n\n\nThere is couple overrides I was able to get during my testing phase early in the competition:\n C_ID_b789a1772c, C_ID_944c62886f
\n \nAgain, these are going to be useless in the end!
\n
I like it, there are bound to be some 0.003+ leaps up the public leaderboard in the last days, thanks for doing your bit to increase the shake-up even more ;)
\n\nFor any n00bs wondering about shake-up, it can be measured with a metric introduced (I think) by @BreakfastPirate , who sadly seems to have deleted his account recently. I hope he's ok (anyone know if/why he left?) For anyone not familiar with the name I suggest you use the search box to seek out his great discussion posts before they disappear (he had over 60 golds!)
\n\nTo fill in, I've created a kernel on Meta Kaggle to compute shake-up for all competitions, and render HTML and PNG tables, like this (filtered view of only 800+ team competitions):
\n\nhttps://www.kaggle.com/jtrotman/meta-kaggle-competition-shake-up
\n\n
A few days after each competition ends it's details go in to Meta-Kaggle, so anyone can fork my kernel at any time & get a new up to date list :)
\n\nMy bet for Elo is at least 0.1, so probably a new top 5 entry in that table...
", "votes": 16, "replies": [ { "id": 477147, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-02-24T01:50:28.290000", "content": "@jtrotman, thanks for this. Like you I also miss @BreakfastPirate and his LB shakeup posts. I hope he is okay too.
", "votes": 1, "replies": [] }, { "id": 477187, "author_name": "khyeh", "author_url": "", "post_date": "2019-02-24T04:27:51.677000", "content": "Nice charts!
", "votes": 1, "replies": [] } ] }, { "id": 479109, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-02-27T01:10:52.950000", "content": "I hear the folks competing in the Earthquake Prediction competition are good at predicting when a shake will occur...
", "votes": 6, "replies": [ { "id": 479186, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-02-27T02:20:40.247000", "content": "Good one @addisonhoward. I am in that one too. Whatever that says about my choices:-)
\n\nHere is my post on feature selection techniques https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/82052
", "votes": 1, "replies": [] } ] }, { "id": 477521, "author_name": "Möbius", "author_url": "", "post_date": "2019-02-24T19:21:23.480000", "content": "Thanks ,
\n\nQuote by W. Edwards Deming:\n \"In God we trust, all others must bring data.\"\nAfter this competition:\n \"In God we trust, all others must bring data. .. Without Outliers !\"
", "votes": 3, "replies": [] }, { "id": 477866, "author_name": "kss", "author_url": "", "post_date": "2019-02-25T11:53:41.467000", "content": "In overfitting we trust
", "votes": 4, "replies": [] }, { "id": 479135, "author_name": "Mr Nik", "author_url": "", "post_date": "2019-02-27T01:32:18.740000", "content": "The shakeup was so huge that the 1st place PL finished just behind me, down 1412 places
", "votes": 1, "replies": [ { "id": 479188, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-02-27T02:23:25.060000", "content": "And the 2nd place team ended down at 2460 @bernnik.
", "votes": 2, "replies": [] }, { "id": 479250, "author_name": "Mr Nik", "author_url": "", "post_date": "2019-02-27T03:12:33.653000", "content": "yes, that is just unbelievable.\nCongrads your place, it is really good. What kind of outlier prediction did you use? (I suppose without that prediction it was impossible to be in first 100)
", "votes": 1, "replies": [] }, { "id": 479261, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-02-27T03:18:42.313000", "content": "Thanks @bernnik. I really never found the right way to deal with the outliers. So I focused on proper feature engineer and feature reduction techniques. That helped a lot in improving my CV as well as LB in the last 3 days.
", "votes": 0, "replies": [] } ] }, { "id": 477364, "author_name": "Orange", "author_url": "", "post_date": "2019-02-24T12:12:34.067000", "content": "totally right, many would fall !!! excited to see what's private looks like
", "votes": 1, "replies": [] }, { "id": 477060, "author_name": "Murat Korkmaz", "author_url": "", "post_date": "2019-02-23T19:19:20.887000", "content": "Yeah exactly, there will be really big shakeup. At first, I didn't use outlier detection effectively, so the improvements in lb score was really small then I start to use it effectively and my lb score rose significantly. I wish I could stop myself from doing it.
", "votes": 1, "replies": [] }, { "id": 477293, "author_name": "CPMP", "author_url": "", "post_date": "2019-02-24T09:57:23.877000", "content": "Any comparison with Mercedes competition is totally wrong ... LOL.
", "votes": 2, "replies": [ { "id": 477303, "author_name": "raddar", "author_url": "", "post_date": "2019-02-24T10:09:33.933000", "content": "indeed - LOL.
", "votes": 1, "replies": [] } ] }, { "id": 477145, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-02-24T01:47:34.667000", "content": "@raddar, your observations are right on and thanks for all your contributions.
\n\nI believe it may all come down to the best well thought out feature engineering. There was so much talk about whether we can even trust our local CV because of the aforementioned outlier issue. It is frustrating but I discovered once I significantly reduced my features through various methods, my CV is pretty consistent moving in the same direction with the LB. I know where I am right now is not where my best model of LB=0.680 will be ranked. But the unknowns are aplenty when it comes to where we would all be in the private LB. I really am curiously waiting to see where all this end. Whatever happens, I thoroughy enjoyed playing with this data.
\n\nGood luck everyone and happy kaggling.
", "votes": 2, "replies": [] }, { "id": 479032, "author_name": "Amit Kumar Jaiswal", "author_url": "", "post_date": "2019-02-27T00:00:08.473000", "content": "Such a huge shake-up😳😱
", "votes": 0, "replies": [] }, { "id": 478599, "author_name": "neuronblack", "author_url": "", "post_date": "2019-02-26T11:35:41.727000", "content": "3q
", "votes": 0, "replies": [] }, { "id": 478525, "author_name": "wakame", "author_url": "", "post_date": "2019-02-26T09:12:28.940000", "content": "Trust your CV.
", "votes": 0, "replies": [] }, { "id": 478501, "author_name": "Younes IDSOUGUOU", "author_url": "", "post_date": "2019-02-26T08:31:34.743000", "content": "Okay buddy
", "votes": 0, "replies": [] }, { "id": 478443, "author_name": "Rraj", "author_url": "", "post_date": "2019-02-26T06:48:36.433000", "content": "Absolutely! I tried so many times, but still for best LB Score my threshold is down to 42 % ( Oops!). Can't say anything about Private LB. It would be interesting to see kingdoms fall and rise.
", "votes": 0, "replies": [] }, { "id": 477470, "author_name": "MadRCoder", "author_url": "", "post_date": "2019-02-24T17:01:44.573000", "content": "awesome....bring it on
", "votes": 0, "replies": [] }, { "id": 477421, "author_name": "amitm", "author_url": "", "post_date": "2019-02-24T14:10:27.027000", "content": ".
", "votes": 0, "replies": [] }, { "id": 477132, "author_name": "Anthony Chiu", "author_url": "", "post_date": "2019-02-24T00:49:52.083000", "content": "I modified the public outliers detection script concept to make sure all parts are cross-validated. I just hope I can keep in the medal zone in this game lol.
", "votes": 0, "replies": [] }, { "id": 477121, "author_name": "Dimosthenis Karaflos", "author_url": "", "post_date": "2019-02-23T23:37:40.197000", "content": "Actually, i gave up the competition a few days ago, when my best CV model scored 3.74 on the LB and manually changing 5 cards values, the same submission scored 3.681....., so i concluded that we may see up to +- 1000 positions shakeups on this competition
", "votes": 0, "replies": [ { "id": 477127, "author_name": "FP", "author_url": "", "post_date": "2019-02-24T00:24:24.763000", "content": "@dkaraflos, then you got a safe medal already.......
", "votes": 0, "replies": [] }, { "id": 477129, "author_name": "Dimosthenis Karaflos", "author_url": "", "post_date": "2019-02-24T00:31:49.833000", "content": "The truth is that all depend on the outliers predictions. If your submission scores a target of 0 for example, and the ground truth is -33, then due to one observation you may get big shakeup on the private LB. Thus, nothing is safe here :)
\n\nThe thing is that we are blind on this one. I may get way out of the Bronze area, i may get gold medal.... who knows
", "votes": 2, "replies": [] }, { "id": 477131, "author_name": "FP", "author_url": "", "post_date": "2019-02-24T00:37:06.957000", "content": "I learnt a big lesson in this competition that I shouldn't use much post-processing. A strong single model is king.
", "votes": 4, "replies": [] }, { "id": 477150, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-02-24T01:53:52.403000", "content": "Yep! @pukkinming. Hold on tight to your best single model based on CV.
", "votes": 1, "replies": [] } ] }, { "id": 477074, "author_name": "FP", "author_url": "", "post_date": "2019-02-23T20:13:14.037000", "content": "I think many of us here will thank u later for this message.
", "votes": 0, "replies": [] }, { "id": 480836, "author_name": "", "author_url": "", "post_date": "2019-02-28T18:25:22.390000", "content": "", "votes": 0, "replies": [ { "id": 480861, "author_name": "raddar", "author_url": "", "post_date": "2019-02-28T19:31:10.457000", "content": "be example, if score < -20 set score to -33.x
", "votes": 1, "replies": [] } ] }, { "id": 479810, "author_name": "", "author_url": "", "post_date": "2019-02-27T12:52:20.303000", "content": "", "votes": 2, "replies": [] }, { "id": 479033, "author_name": "", "author_url": "", "post_date": "2019-02-27T00:06:21.197000", "content": "", "votes": 0, "replies": [] }, { "id": 478014, "author_name": "", "author_url": "", "post_date": "2019-02-25T16:02:52.363000", "content": "", "votes": 1, "replies": [ { "id": 478021, "author_name": "FP", "author_url": "", "post_date": "2019-02-25T16:09:00.373000", "content": "Just try again next time. Local CV 3.65 shouldn't be that difficult to achieve after adding some new features.
", "votes": 0, "replies": [] } ] }, { "id": 479658, "author_name": "marshi", "author_url": "", "post_date": "2019-02-27T10:18:03.510000", "content": "Thanks!
", "votes": 1, "replies": [] }, { "id": 478512, "author_name": "k_kjm", "author_url": "", "post_date": "2019-02-26T08:52:34.020000", "content": "thanks
", "votes": 0, "replies": [] }, { "id": 477332, "author_name": "LongYin/杰少", "author_url": "", "post_date": "2019-02-24T10:54:22.727000", "content": "Thanks.
", "votes": 0, "replies": [] }, { "id": 477157, "author_name": "dangdang", "author_url": "", "post_date": "2019-02-24T02:09:34.323000", "content": "thank you !
", "votes": 0, "replies": [] }, { "id": 477143, "author_name": "Manchoi", "author_url": "", "post_date": "2019-02-24T01:43:28.287000", "content": "Thank you very much, @raddar
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "477051": "It is not a secret that all the teams have been using \"outlier\" overrides to improve their standings in the public LB. I have tested out, that having a single override can make a 0.003-0.004 score difference. So imagine the situation at the top... There is no way of knowing where you actually stand! - current public LB gold position (except #1 guy) are within 3 overrides range!\n\nSo for all who feels bad about their current leaderboard rank - if you haven't done public LB override overfitting (like tuning the \"perfect\" threshold for you outlier classifier) - I am quite sure you are going to improve in the final standings!\n\nTherefore - this is going to be a crazy!\n\np.s. we haven't focused on overrides for a long time now\n\nThere is couple overrides I was able to get during my testing phase early in the competition:\n```C_ID_b789a1772c, \nC_ID_944c62886f```\n\nAgain, these are going to be useless in the end!\n", "477085": "Waiting for it.\n\n![enter image description here][1]\n\n\n [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/477085/11395/gel_gel.jpg", "477110": "> There is couple overrides I was able to get during my testing phase early in the competition:\n> C_ID_b789a1772c, C_ID_944c62886f\n> \n> Again, these are going to be useless in the end!\n\nI like it, there are bound to be some 0.003+ leaps up the public leaderboard in the last days, thanks for doing your bit to *increase* the shake-up even more ;)\n\nFor any n00bs wondering about shake-up, it can be measured with a metric introduced (I think) by @BreakfastPirate , who sadly seems to have deleted his account recently. I hope he's ok (anyone know if/why he left?) For anyone not familiar with the name I suggest you use the search box to seek out his great discussion posts before they disappear (he had **over 60** golds!)\n\nTo fill in, I've created a [kernel on Meta Kaggle][1] to compute shake-up for all competitions, and render HTML and PNG tables, like this (filtered view of only 800+ team competitions):\n\nhttps://www.kaggle.com/jtrotman/meta-kaggle-competition-shake-up\n\n![enter image description here][2]\n\nA few days after each competition ends it's details go in to Meta-Kaggle, so anyone can fork my kernel at any time & get a new up to date list :)\n\nMy bet for Elo is at least 0.1, so probably a new top 5 entry in that table...\n\n\n [1]: https://www.kaggle.com/jtrotman/meta-kaggle-competition-shake-up\n [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/477110/11396/shakeup_teams_over_800.png \"Competition Shake-up\"\n", "479109": "I hear the folks competing in the [Earthquake Prediction](https://www.kaggle.com/c/LANL-Earthquake-Prediction) competition are good at predicting when a shake will occur...", "477521": "Thanks , \n\nQuote by W. Edwards Deming:\n \"In God we trust, all others must bring data.\"\nAfter this competition:\n \"In God we trust, all others must bring data. .. Without Outliers !\"\n", "477866": "In overfitting we trust", "479135": "The shakeup was so huge that the 1st place PL finished just behind me, down 1412 places", "477364": "totally right, many would fall !!! excited to see what's private looks like", "477060": "Yeah exactly, there will be really big shakeup. At first, I didn't use outlier detection effectively, so the improvements in lb score was really small then I start to use it effectively and my lb score rose significantly. I wish I could stop myself from doing it.", "477293": "Any comparison with Mercedes competition is totally wrong ... LOL.", "477145": "@raddar, your observations are right on and thanks for all your contributions.\n\nI believe it may all come down to the best well thought out feature engineering. There was so much talk about whether we can even trust our local CV because of the aforementioned outlier issue. It is frustrating but I discovered once I significantly reduced my features through various methods, my CV is pretty consistent moving in the same direction with the LB. I know where I am right now is not where my best model of LB=0.680 will be ranked. But the unknowns are aplenty when it comes to where we would all be in the private LB. I really am curiously waiting to see where all this end. Whatever happens, I thoroughy enjoyed playing with this data.\n\nGood luck everyone and happy kaggling.", "479032": "Such a huge shake-up😳😱", "478599": "3q", "478525": "Trust your CV.", "478501": "Okay buddy", "478443": "Absolutely! I tried so many times, but still for best LB Score my threshold is down to 42 % ( Oops!). Can't say anything about Private LB. It would be interesting to see kingdoms fall and rise.", "477470": "awesome....bring it on", "477421": ".\n", "477132": "I modified the public outliers detection script concept to make sure all parts are cross-validated. I just hope I can keep in the medal zone in this game lol.", "477121": "Actually, i gave up the competition a few days ago, when my best CV model scored 3.74 on the LB and manually changing 5 cards values, the same submission scored 3.681....., so i concluded that we may see up to +- 1000 positions shakeups on this competition", "477074": "I think many of us here will thank u later for this message. ", "480836": "", "479810": "", "479033": "", "478014": "", "479658": "Thanks!", "478512": "thanks", "477332": "Thanks. ", "477157": "thank you !", "477143": "Thank you very much, @raddar" } }