{ "id": 105237, "title": "Official - Don't use the -2.163 submission", "url": "/competitions/champs-scalar-coupling/discussion/105237", "author_name": "inversion", "post_date": "2019-08-21T23:09:12.893000", "votes": 33, "comment_count": 63, "views": 0, "content": "

I think everyone is aware of the unfortunate \"public for a few minutes\" incident. It appears that 4 teams were able to access this submission.

\n\n

We have two choices:

\n\n
    \n
  1. Ask the teams that were able to download the submission to no longer make use of it or share it further, and to not select that submission as their final choice. (It's easy for us determine.) Worst case scenario, a few teams get an unfair bump.
    2. \n
  2. Allow that submission to be available to everyone. Worst case scenario, all positions below 45 become a \"who clicked fastest\" contest, which is unfair to potentially thousands of competitors.
    3. \n
\n\n

With that said, we are choosing option #1, and asking the teams that have that information not to use it.

", "messages": [ { "id": 604966, "postDate": "2019-08-21T23:09:12.893Z", "content": "

I think everyone is aware of the unfortunate \"public for a few minutes\" incident. It appears that 4 teams were able to access this submission.

\n\n

We have two choices:

\n\n
    \n
  1. Ask the teams that were able to download the submission to no longer make use of it or share it further, and to not select that submission as their final choice. (It's easy for us determine.) Worst case scenario, a few teams get an unfair bump.
    2. \n
  2. Allow that submission to be available to everyone. Worst case scenario, all positions below 45 become a \"who clicked fastest\" contest, which is unfair to potentially thousands of competitors.
    3. \n
\n\n

With that said, we are choosing option #1, and asking the teams that have that information not to use it.

", "rawMarkdown": "I think everyone is aware of the unfortunate \"public for a few minutes\" incident. It appears that 4 teams were able to access this submission.\n\nWe have two choices:\n\n1. Ask the teams that were able to download the submission to no longer make use of it or share it further, and to not select that submission as their final choice. (It's easy for us determine.) Worst case scenario, a few teams get an unfair bump.\n2. Allow that submission to be available to everyone. Worst case scenario, all positions below 45 become a \"who clicked fastest\" contest, which is unfair to potentially thousands of competitors.\n\nWith that said, **we are choosing option #1**, and asking the teams that have that information not to use it.", "votes": 33 }, { "id": 605308, "postDate": "2019-08-22T08:39:19.497Z", "content": "

inversion - i just emailed you to make some things clear. our score increased to -2.329 at Wed Aug 21 2019 14:38:06 GMT-0500 (Central Daylight Time) [as far as i can tell, this is before the -2.16 submission was made public]

\n\n

after the -2.165 submission was released, we submitted a 30% 70% blend on Wed Aug 21 2019 15:24:55 GMT -0500 (Central Daylight Time) and received a tiny bit worse score. We submitted this before your announcement, so we did not know it was wrong.

\n\n

In the future we will not be using this file. I just want to clear things up in case you think my team was suspicious.

", "rawMarkdown": "inversion - i just emailed you to make some things clear. our score increased to -2.329 at Wed Aug 21 2019 14:38:06 GMT-0500 (Central Daylight Time) [as far as i can tell, this is before the -2.16 submission was made public]\n\nafter the -2.165 submission was released, we submitted a 30% 70% blend on Wed Aug 21 2019 15:24:55 GMT -0500 (Central Daylight Time) and received a tiny bit worse score. We submitted this before your announcement, so we did not know it was wrong.\n\nIn the future we will not be using this file. I just want to clear things up in case you think my team was suspicious. ", "votes": 9, "replies": [ { "id": 605341, "postDate": "2019-08-22T09:20:11.777Z", "content": "
\n

in case you think my team was suspicious.

\n
\n\n

Nice try ;)

", "rawMarkdown": "> in case you think my team was suspicious. \n\nNice try ;)", "votes": 4 }, { "id": 605352, "postDate": "2019-08-22T09:31:28.667Z", "content": "

thanks @returnofsputnik :)

", "rawMarkdown": "thanks @returnofsputnik :)" } ] }, { "id": 604971, "postDate": "2019-08-21T23:12:50.410Z", "content": "

Wow. I wonder how it could be detected if someone is blending it secretly.

", "rawMarkdown": "Wow. I wonder how it could be detected if someone is blending it secretly.", "votes": 9 }, { "id": 605278, "postDate": "2019-08-22T08:04:02.123Z", "content": "

Although it may be a bit unfair, I still agree with the option #1.\nThat should be better than struggling with 2000 teams of -2.163.\nNow all my hope lies in Kaggle's detection algorithm, otherwise, my three months of hard work will be equal to 0.

", "rawMarkdown": "Although it may be a bit unfair, I still agree with the option #1.\nThat should be better than struggling with 2000 teams of -2.163.\nNow all my hope lies in Kaggle's detection algorithm, otherwise, my three months of hard work will be equal to 0.", "votes": 8 }, { "id": 604981, "postDate": "2019-08-21T23:31:14.547Z", "content": "

I would propose option 3.

\n\n

I kindly ask the meme team open source the solution of -2.16 so ALL people can learn from the code and decide how to use it. Based on my shallow ML understanding, there is no hard evidence Kaggle can give against any team who submitted a blending with that file. If any evidence, it will be statistical and nondeterministic.

\n\n

I don't like the idea at all that a few teams get an unfair bump no matter how small that number is.

", "rawMarkdown": "I would propose option 3.\n\nI kindly ask the meme team open source the solution of -2.16 so ALL people can learn from the code and decide how to use it. Based on my shallow ML understanding, there is no hard evidence Kaggle can give against any team who submitted a blending with that file. If any evidence, it will be statistical and nondeterministic. \n\nI don't like the idea at all that a few teams get an unfair bump no matter how small that number is.", "votes": 7, "replies": [ { "id": 604983, "postDate": "2019-08-21T23:37:09.517Z", "content": "

Jiwei - if you do a linear combination blend, they can detect if by looking at one of your old submissions (if you blend it with your old submission). Besides that, they cant detect if you blend nonlinearly, or blend with something you haven't submitted before...

\n\n

Also, they can probably detect which users have downloaded the data by looking at the GET requests on that kernel page. But that wont reveal if you use it in your final 2 subs

\n\n

I support option 3

", "rawMarkdown": "Jiwei - if you do a linear combination blend, they can detect if by looking at one of your old submissions (if you blend it with your old submission). Besides that, they cant detect if you blend nonlinearly, or blend with something you haven't submitted before...\n\nAlso, they can probably detect which users have downloaded the data by looking at the GET requests on that kernel page. But that wont reveal if you use it in your final 2 subs\n\nI support option 3", "votes": 1 }, { "id": 604984, "postDate": "2019-08-21T23:38:24.733Z", "content": "

yeah, actually I can use it as pseudo label and train a neural network and then blend. How can you detect that?

", "rawMarkdown": "yeah, actually I can use it as pseudo label and train a neural network and then blend. How can you detect that?", "votes": 4 }, { "id": 604986, "postDate": "2019-08-21T23:54:59.073Z", "content": "

Thanks Jiwei your suggestion is reasonable.\nif the solution is made public people who are most impacted and will be devastated are the ones b/w roughly lets say 45 to 300.\nI feel its good idea to take a vote from the people who will be most impacted.\nI personally feel option 1 is a good choice at least 200 deserving people will get a medal vs handful of people got lucked out

\n\n

Type I errors are generally considered more serious than Type II errors.

", "rawMarkdown": "Thanks Jiwei your suggestion is reasonable.\nif the solution is made public people who are most impacted and will be devastated are the ones b/w roughly lets say 45 to 300.\nI feel its good idea to take a vote from the people who will be most impacted.\nI personally feel option 1 is a good choice at least 200 deserving people will get a medal vs handful of people got lucked out\n\nType I errors are generally considered more serious than Type II errors.\n\n", "votes": 6 }, { "id": 605233, "postDate": "2019-08-22T07:22:45.970Z", "content": "
\n

I don't like the idea at all that a few teams get an unfair bump no matter how small that number is.

\n
\n\n

And I don't like the idea at all that all teams below 2.16 will get this unfair bump tho they don't deserve it. Some worked for 3 moths almost every day, and some will just click 2 buttons. It's 1 week to deadine, so it's already not the \"if-public-kernel-beats-your-model-than-your-model-sucks\" situation.\nSo in summary, I'm totally with @mks2192 on this.

", "rawMarkdown": "> I don't like the idea at all that a few teams get an unfair bump no matter how small that number is.\n\nAnd I don't like the idea at all that all teams below 2.16 will get this unfair bump tho they don't deserve it. Some worked for 3 moths almost every day, and some will just click 2 buttons. It's 1 week to deadine, so it's already not the \"*if-public-kernel-beats-your-model-than-your-model-sucks*\" situation.\nSo in summary, I'm totally with @mks2192 on this.", "votes": 4 }, { "id": 605384, "postDate": "2019-08-22T10:22:32.050Z", "content": "

That is really a breathless operation...

", "rawMarkdown": "That is really a breathless operation...", "votes": 1 }, { "id": 605986, "postDate": "2019-08-23T03:40:54.177Z", "content": "

@jiweiliu - Are you sure you would support your proposed option 3 if the submission had come in 0.01 ahead of your current position? We are sitting a few places below the submission. I don't care that 4 people/teams jumped ahead of our hard work, but if 1000 people did, that would not be fair. On the other hand, we could simply blend our working solution with the submission and move past all 1000!

", "rawMarkdown": "@jiweiliu - Are you sure you would support your proposed option 3 if the submission had come in 0.01 ahead of your current position? We are sitting a few places below the submission. I don't care that 4 people/teams jumped ahead of our hard work, but if 1000 people did, that would not be fair. On the other hand, we could simply blend our working solution with the submission and move past all 1000!", "votes": 1 }, { "id": 607270, "postDate": "2019-08-24T23:39:44.130Z", "content": "

It is no different. Our rank will be improved if we can blend that file. But it is not the point here. There is nothing in kaggle rules that forbids sharing high score kernel towards the end of competition. It is discouraged but not forbidden. Can we just follow the rule and leave the impact aside? I feel sad even admin bends the rule.

", "rawMarkdown": "It is no different. Our rank will be improved if we can blend that file. But it is not the point here. There is nothing in kaggle rules that forbids sharing high score kernel towards the end of competition. It is discouraged but not forbidden. Can we just follow the rule and leave the impact aside? I feel sad even admin bends the rule.", "votes": 1 }, { "id": 607282, "postDate": "2019-08-25T00:38:45.857Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 605312, "postDate": "2019-08-22T08:41:38.573Z", "content": "

I got a kernel gold medal - yay I like kernels\nMy position just tanked - boo I hate kernels\nMy brain is so confused right now.

", "rawMarkdown": "I got a kernel gold medal - yay I like kernels\nMy position just tanked - boo I hate kernels\nMy brain is so confused right now.", "votes": 5 }, { "id": 608950, "postDate": "2019-08-27T09:34:57.213Z", "content": "

You're funny.

", "rawMarkdown": "You're funny.", "votes": 6 }, { "id": 606042, "postDate": "2019-08-23T06:02:26.757Z", "content": "

What if somebody wanted to share a high scoring kernel intentionally?\nThis has happened before and will happen again.

\n\n

@inversion what's the motivation of Kaggle not being proactive and implement a kernel freeze (no sharing) close to competition end?

", "rawMarkdown": "What if somebody wanted to share a high scoring kernel intentionally?\nThis has happened before and will happen again. \n\n@inversion what's the motivation of Kaggle not being proactive and implement a kernel freeze (no sharing) close to competition end? ", "votes": 6, "replies": [ { "id": 606051, "postDate": "2019-08-23T06:12:40.157Z", "content": "

why would somebody share a very high scoring submission intentionally? there are penalties after all. as far as I've surmised it was a naked submission file.

\n\n

a couple of things come to mind:\n- a publicity stunt\n- when the submission is an overfit it may throw the competitors off the trail

", "rawMarkdown": "why would somebody share a very high scoring submission **intentionally**? there are penalties after all. as far as I've surmised it was a naked submission file.\n\na couple of things come to mind:\n- a publicity stunt\n- when the submission is an overfit it may throw the competitors off the trail" }, { "id": 606065, "postDate": "2019-08-23T06:30:06.843Z", "content": "

@stocks It happened in Talking Data, it is not a remote hypothetical possibility.

", "rawMarkdown": "@stocks It happened in Talking Data, it is not a remote hypothetical possibility.", "votes": 1 }, { "id": 606159, "postDate": "2019-08-23T09:24:35.780Z", "content": "

Kernel freezing will make things worse, they will share in a private forum more. \nOnly prize money matters truly in life, then, prize zone competitors never share for sure. \nMedal fighters are different. Anyone can share, in their real life, in the private page... how do we know that?\nThen, just let it be. And be the strongest in a competition so ruiners can’t do anything to you.

", "rawMarkdown": "Kernel freezing will make things worse, they will share in a private forum more. \nOnly prize money matters truly in life, then, prize zone competitors never share for sure. \nMedal fighters are different. Anyone can share, in their real life, in the private page... how do we know that?\nThen, just let it be. And be the strongest in a competition so ruiners can’t do anything to you. " }, { "id": 606164, "postDate": "2019-08-23T09:29:06.620Z", "content": "

I remember, in Instant Gratification, someone posted a top (20-30) solution, but with intentionally manipulated private test rows. If used that, private LB will drop. Then Chris Deotte analysed the submission and warned everybody not to use that file.

", "rawMarkdown": "I remember, in Instant Gratification, someone posted a top (20-30) solution, but with intentionally manipulated private test rows. If used that, private LB will drop. Then Chris Deotte analysed the submission and warned everybody not to use that file. ", "votes": 2 }, { "id": 606216, "postDate": "2019-08-23T10:28:40.827Z", "content": "

Really? :D\nIt is like Da Vinci solution, in some of his works there were intentional errors, so in the end, it worked only when you thought about it and corrected it. No one could steal the solution and built it without an effort.\nSometimes I wonder, whether it is similar in public kernels. Like if people post a neural network, which they use with an amazing score, but in the public version they delete a layer, so it scores much worse. No one can get the best score for free, your original solution is not threatened, but you still get a medal for helping others.\nWell, maybe not that similar if it was posted intentionally with the goal to damage others..

", "rawMarkdown": "Really? :D\nIt is like Da Vinci solution, in some of his works there were intentional errors, so in the end, it worked only when you thought about it and corrected it. No one could steal the solution and built it without an effort.\nSometimes I wonder, whether it is similar in public kernels. Like if people post a neural network, which they use with an amazing score, but in the public version they delete a layer, so it scores much worse. No one can get the best score for free, your original solution is not threatened, but you still get a medal for helping others.\nWell, maybe not that similar if it was posted intentionally with the goal to damage others..", "votes": 1 }, { "id": 607347, "postDate": "2019-08-25T04:17:57.173Z", "content": "

I thought about what Kha Vo said about kernel freezing and I disagree with him. I still think that it would be helpful. If a medal from a competition is slipping under someone's fingers, but his score is still good enough, then he may have a motivation to post hist kernel in last week to get at least a medal for the kernel. If kernels were freezed, they wouldn't have such motivation.

", "rawMarkdown": "I thought about what Kha Vo said about kernel freezing and I disagree with him. I still think that it would be helpful. If a medal from a competition is slipping under someone's fingers, but his score is still good enough, then he may have a motivation to post hist kernel in last week to get at least a medal for the kernel. If kernels were freezed, they wouldn't have such motivation." }, { "id": 607392, "postDate": "2019-08-25T06:47:09.650Z", "content": "
\n

If a medal from a competition is slipping under someone's fingers, but his score is still good enough, then he may have a motivation to post hist kernel in last week to get at least a medal for the kernel. If kernels were freezed, they wouldn't have such motivation.

\n
\n\n

Apply this reasoning to gold medal and you get what happened in Talking Data. A guys saw gold was out of reach last day and he posted his silver medal kernel.

", "rawMarkdown": "> If a medal from a competition is slipping under someone's fingers, but his score is still good enough, then he may have a motivation to post hist kernel in last week to get at least a medal for the kernel. If kernels were freezed, they wouldn't have such motivation.\n\nApply this reasoning to gold medal and you get what happened in Talking Data. A guys saw gold was out of reach last day and he posted his silver medal kernel. \n" }, { "id": 607410, "postDate": "2019-08-25T07:26:46.800Z", "content": "

I was unaware that Kaggle has already tried the kernel freeze idea in a slightly different form (freezing submissions only, not the sharing of kernels). Interesting discussion here: https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/56182#325316

", "rawMarkdown": "I was unaware that Kaggle has already tried the kernel freeze idea in a slightly different form (freezing submissions only, not the sharing of kernels). Interesting discussion here: https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/56182#325316", "votes": 2 }, { "id": 609393, "postDate": "2019-08-27T16:53:56.227Z", "content": "

One of major reasons, why i changed one of my recent public kernels to private.

", "rawMarkdown": "One of major reasons, why i changed one of my recent public kernels to private. ", "votes": 1 } ] }, { "id": 605047, "postDate": "2019-08-22T02:22:49.053Z", "content": "

Ok, I don't like this solution but I will accept it if that is final. Just in case anyone doesn't know, I copied that kernel and downloaded the file and emailed Inversion immediately. My plan was to share it but it is not approved by admin.

", "rawMarkdown": "Ok, I don't like this solution but I will accept it if that is final. Just in case anyone doesn't know, I copied that kernel and downloaded the file and emailed Inversion immediately. My plan was to share it but it is not approved by admin.", "votes": 3 }, { "id": 605766, "postDate": "2019-08-22T18:09:56.180Z", "content": "

letting a team or two to get the submit on the private sharing and some places on the LB is way more fair than opening it publicly and therefore wasting 3 months of work of lots of people. those who argue for option 3 should just wait another week to get the solution which ideally should be replicable and provided as code. But not before the competition ends.

", "rawMarkdown": "letting a team or two to get the submit on the private sharing and some places on the LB is way more fair than opening it publicly and therefore wasting 3 months of work of lots of people. those who argue for option 3 should just wait another week to get the solution which ideally should be replicable and provided as code. But not before the competition ends.", "votes": 3 }, { "id": 604970, "postDate": "2019-08-21T23:11:33.757Z", "content": "
\n

It's easy for us determine

\n
\n\n

What if a team submitted a blending with it? Can kaggle detect it?

", "rawMarkdown": "> It's easy for us determine\n\nWhat if a team submitted a blending with it? Can kaggle detect it?", "votes": 3, "replies": [ { "id": 604972, "postDate": "2019-08-21T23:13:21.450Z", "content": "

You'd be surprised at what signals are in the data when you look.

", "rawMarkdown": "You'd be surprised at what signals are in the data when you look.", "votes": 5 }, { "id": 604973, "postDate": "2019-08-21T23:14:32.357Z", "content": "

Ok, I'm going to take your word for it 😅

", "rawMarkdown": "Ok, I'm going to take your word for it 😅 " }, { "id": 604975, "postDate": "2019-08-21T23:15:36.653Z", "content": "

Actually, could you point me to a literature if any? I'm interested from a theoretical perspective.

", "rawMarkdown": "Actually, could you point me to a literature if any? I'm interested from a theoretical perspective.", "votes": 3 }, { "id": 605018, "postDate": "2019-08-22T01:06:30.490Z", "content": "

I'm guessing just correlation would tell you quite a bit.

", "rawMarkdown": "I'm guessing just correlation would tell you quite a bit." }, { "id": 605138, "postDate": "2019-08-22T05:20:43.950Z", "content": "
\n

You'd be surprised at what signals are in the data when you look.

\n
\n\n

I'm surprised everyday when I see how Kagglers can get signal out of training data.

", "rawMarkdown": "> You'd be surprised at what signals are in the data when you look.\n\nI'm surprised everyday when I see how Kagglers can get signal out of training data." }, { "id": 605152, "postDate": "2019-08-22T05:30:54.880Z", "content": "
\n

Actually, could you point me to a literature if any? I'm interested from a theoretical perspective.

\n
\n\n

You don't expect Kaggle staff to explain how they detect privtate sharing, do you? ;)

", "rawMarkdown": "> Actually, could you point me to a literature if any? I'm interested from a theoretical perspective.\n\nYou don't expect Kaggle staff to explain how they detect privtate sharing, do you? ;)", "votes": 6 }, { "id": 605582, "postDate": "2019-08-22T14:09:22.740Z", "content": "

why not? I bet they have room to improve given the x5 situation. ;)

", "rawMarkdown": "why not? I bet they have room to improve given the x5 situation. ;)" }, { "id": 605591, "postDate": "2019-08-22T14:17:25.697Z", "content": "

X5 has teamed in open image competition :) Problem is solved...

", "rawMarkdown": "X5 has teamed in open image competition :) Problem is solved...", "votes": 1 }, { "id": 605593, "postDate": "2019-08-22T14:18:35.173Z", "content": "

haha, that's fair. Actually I asked about literature, not what they do in house. Isn't it something interesting to read ? ;)

", "rawMarkdown": "haha, that's fair. Actually I asked about literature, not what they do in house. Isn't it something interesting to read ? ;)", "votes": 1 }, { "id": 606235, "postDate": "2019-08-23T11:08:20.840Z", "rawMarkdown": "", "votes": -1 }, { "id": 606264, "postDate": "2019-08-23T11:31:50.757Z", "content": "

I think there's another commotion in private sharing which involve X5 again (Open Images Instance Segmentation). X5 issue is still unresolved.

", "rawMarkdown": "I think there's another commotion in private sharing which involve X5 again (Open Images Instance Segmentation). X5 issue is still unresolved.", "votes": 1 } ] }, { "id": 609298, "postDate": "2019-08-27T15:12:40.880Z", "content": "

I believe that the correct answer is 'yes'.

", "rawMarkdown": "I believe that the correct answer is 'yes'.", "votes": 4 }, { "id": 605014, "postDate": "2019-08-22T01:01:04.400Z", "content": "

Thank you @inversion for clearing it up\nI agree with the choice made and am glad we are not removed from the competition\nI think @jesucristo made it very clear in the discussions that it was an accident he regrets

", "rawMarkdown": "Thank you @inversion for clearing it up\nI agree with the choice made and am glad we are not removed from the competition\nI think @jesucristo made it very clear in the discussions that it was an accident he regrets", "votes": 4 }, { "id": 605992, "postDate": "2019-08-23T03:56:41.747Z", "content": "

Solution 1 is too soft. The leak file is shared in a different forum, according to someone saying that, then it’s unfair for those who cannot access that forum.

\n\n

What if in the future, a leak like that presents for 15 mins, rather than 5 mins like this one?

\n\n

However, middle table is now an unfair competition. I vote for option 2.

\n\n

All teams can use it as a blend with their own model.

", "rawMarkdown": "Solution 1 is too soft. The leak file is shared in a different forum, according to someone saying that, then it’s unfair for those who cannot access that forum. \n\nWhat if in the future, a leak like that presents for 15 mins, rather than 5 mins like this one?\n\nHowever, middle table is now an unfair competition. I vote for option 2. \n\nAll teams can use it as a blend with their own model. ", "votes": 3, "replies": [ { "id": 605994, "postDate": "2019-08-23T04:14:58.193Z", "content": "

I think then people who are sharing and promoting should also be penalised

", "rawMarkdown": "I think then people who are sharing and promoting should also be penalised ", "votes": 1 }, { "id": 606039, "postDate": "2019-08-23T05:53:42.797Z", "content": "

I agree with you saying that the solution is too soft if the only consequence is getting kicked out of the competition. This way if a team is not in the top 10% then they can try to use the leak since they have nothing to lose.

", "rawMarkdown": "I agree with you saying that the solution is too soft if the only consequence is getting kicked out of the competition. This way if a team is not in the top 10% then they can try to use the leak since they have nothing to lose.", "votes": 2 }, { "id": 607274, "postDate": "2019-08-24T23:51:06.177Z", "content": "

I don't care about soft. The point is option 1 is not fair, in the sense that not everyone has access to that file. We are willing to follow the option 1 but I have no doubts that some teams will use that file and not be punished at all for private sharing. As confirmed by admin, that's the worst case and a price kaggle is willing to pay. It is really disappointing of kaggle's attitudes toward investigation and punishing private sharing. X5 and the like always get away.

", "rawMarkdown": "I don't care about soft. The point is option 1 is not fair, in the sense that not everyone has access to that file. We are willing to follow the option 1 but I have no doubts that some teams will use that file and not be punished at all for private sharing. As confirmed by admin, that's the worst case and a price kaggle is willing to pay. It is really disappointing of kaggle's attitudes toward investigation and punishing private sharing. X5 and the like always get away." } ] }, { "id": 604989, "postDate": "2019-08-22T00:00:16.553Z", "content": "

Option #1 is also \"unfair to potentially thousands of competitors\". It means option #1 is much worse than option #2. Another truth is those \"a few teams\" just kicked our team out of the silver line. I could not see any fairness.

", "rawMarkdown": "Option #1 is also \"unfair to potentially thousands of competitors\". It means option #1 is much worse than option #2. Another truth is those \"a few teams\" just kicked our team out of the silver line. I could not see any fairness.", "votes": 1 }, { "id": 609284, "postDate": "2019-08-27T15:02:12.090Z", "content": "

Can you give me the answers? LOL

", "rawMarkdown": "Can you give me the answers? LOL", "votes": 2 }, { "id": 605526, "postDate": "2019-08-22T13:04:28.627Z", "content": "

It may take some time to finalize Private LB - just a hunch. I am guessing that out of ~300 medals, 100 will blend with the rogue submission and manage to bypass Kaggle's efforts. According to this scenario ~200 medals are still for grabs for those who prefer not to 'cheat' (the caveat is that the kernel was indeed public). Basically, we have to deal with an extra level of challenge in the last week.

", "rawMarkdown": "It may take some time to finalize Private LB - just a hunch. I am guessing that out of ~300 medals, 100 will blend with the rogue submission and manage to bypass Kaggle's efforts. According to this scenario ~200 medals are still for grabs for those who prefer not to 'cheat' (the caveat is that the kernel was indeed public). Basically, we have to deal with an extra level of challenge in the last week." }, { "id": 605306, "postDate": "2019-08-22T08:37:43.553Z", "content": "

I'm so sorry and I feel so bad... sorry everyone who was affected and this morning a friend told me the file is a ods.ai ...\nPlease don't use it.\n@inversion I'm sorry to cause trouble, thank you so much for your response, it really helps :)\nGood luck, and be careful when you create kernels, I don't know if it was when invited my mates or how... anyway I'll run everything on local now.

", "rawMarkdown": "I'm so sorry and I feel so bad... sorry everyone who was affected and this morning a friend told me the file is a ods.ai ...\nPlease don't use it.\n@inversion I'm sorry to cause trouble, thank you so much for your response, it really helps :)\nGood luck, and be careful when you create kernels, I don't know if it was when invited my mates or how... anyway I'll run everything on local now. ", "votes": -2 }, { "id": 606008, "postDate": "2019-08-23T04:56:22.660Z", "content": "

I am assuming that Kaggle is monitoring new submissions for evidence of blending with the rogue. If there is enough evidence that the rogue file is in the hands of more than 10-20 teams then they could always change the rules to plan B.

", "rawMarkdown": "I am assuming that Kaggle is monitoring new submissions for evidence of blending with the rogue. If there is enough evidence that the rogue file is in the hands of more than 10-20 teams then they could always change the rules to plan B.", "replies": [ { "id": 606011, "postDate": "2019-08-23T05:02:13.617Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 606018, "postDate": "2019-08-23T05:10:01.867Z", "content": "

no need to accuse anybody. if they see enough evidence of covert blending then the file will be allowed for blending to everyone. \nfor now I do hope that the leak got plugged in successfully. I am probably very naive. LOL

", "rawMarkdown": "no need to accuse anybody. if they see enough evidence of covert blending then the file will be allowed for blending to everyone. \nfor now I do hope that the leak got plugged in successfully. I am probably very naive. LOL" }, { "id": 606021, "postDate": "2019-08-23T05:16:28.283Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 606457, "postDate": "2019-08-23T15:47:15.877Z", "rawMarkdown": "", "isDeleted": true }, { "id": 606201, "postDate": "2019-08-23T10:07:28.333Z", "rawMarkdown": "", "isDeleted": true }, { "id": 605111, "postDate": "2019-08-22T04:39:59.053Z", "rawMarkdown": "", "votes": -1, "isDeleted": true, "replies": [ { "id": 605403, "postDate": "2019-08-22T10:53:50.730Z", "content": "

Did you see inversion's post? They changed the rules.

", "rawMarkdown": "Did you see inversion's post? They changed the rules. ", "votes": 2 }, { "id": 605497, "postDate": "2019-08-22T12:27:39.163Z", "rawMarkdown": "", "votes": -2, "isDeleted": true }, { "id": 605506, "postDate": "2019-08-22T12:42:21.677Z", "content": "

Back then it was fair, now it is not fair, because we have a new rule.

\n\n

I very disagree with your point of view, on the contrary, I think, they should introduce a new rule every time when it helps the competition to be fair to more people. No rules are perfect and there is not a rule for everything. Where would we be if we have not been changing them and adding new ones..

", "rawMarkdown": "Back then it was fair, now it is not fair, because we have a new rule.\n\nI very disagree with your point of view, on the contrary, I think, they should introduce a new rule every time when it helps the competition to be fair to more people. No rules are perfect and there is not a rule for everything. Where would we be if we have not been changing them and adding new ones.." }, { "id": 605541, "postDate": "2019-08-22T13:22:00.700Z", "rawMarkdown": "", "isDeleted": true }, { "id": 605553, "postDate": "2019-08-22T13:38:51.613Z", "content": "

this leak qualifies as force majeure. in other words, the rule change is totally warranted. \n*disclaimer: I am not a lawyer.

", "rawMarkdown": "this leak qualifies as ***[force majeure](https://en.wikipedia.org/wiki/Force_majeure)***. in other words, the rule change is totally warranted. \n*disclaimer: I am not a lawyer.", "votes": 1 }, { "id": 605572, "postDate": "2019-08-22T14:01:09.987Z", "rawMarkdown": "", "votes": -4, "isDeleted": true } ] } ], "comments": [ { "id": 605308, "author_name": "CoreyJamesLevinson", "author_url": "", "post_date": "2019-08-22T08:39:19.497000", "content": "

inversion - i just emailed you to make some things clear. our score increased to -2.329 at Wed Aug 21 2019 14:38:06 GMT-0500 (Central Daylight Time) [as far as i can tell, this is before the -2.16 submission was made public]

\n\n

after the -2.165 submission was released, we submitted a 30% 70% blend on Wed Aug 21 2019 15:24:55 GMT -0500 (Central Daylight Time) and received a tiny bit worse score. We submitted this before your announcement, so we did not know it was wrong.

\n\n

In the future we will not be using this file. I just want to clear things up in case you think my team was suspicious.

", "votes": 9, "replies": [ { "id": 605341, "author_name": "CPMP", "author_url": "", "post_date": "2019-08-22T09:20:11.777000", "content": "
\n

in case you think my team was suspicious.

\n
\n\n

Nice try ;)

", "votes": 4, "replies": [] }, { "id": 605352, "author_name": "Nanashi", "author_url": "", "post_date": "2019-08-22T09:31:28.667000", "content": "

thanks @returnofsputnik :)

", "votes": 0, "replies": [] } ] }, { "id": 604971, "author_name": "Yirun Zhang", "author_url": "", "post_date": "2019-08-21T23:12:50.410000", "content": "

Wow. I wonder how it could be detected if someone is blending it secretly.

", "votes": 9, "replies": [] }, { "id": 605278, "author_name": "heng", "author_url": "", "post_date": "2019-08-22T08:04:02.123000", "content": "

Although it may be a bit unfair, I still agree with the option #1.\nThat should be better than struggling with 2000 teams of -2.163.\nNow all my hope lies in Kaggle's detection algorithm, otherwise, my three months of hard work will be equal to 0.

", "votes": 8, "replies": [] }, { "id": 604981, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-08-21T23:31:14.547000", "content": "

I would propose option 3.

\n\n

I kindly ask the meme team open source the solution of -2.16 so ALL people can learn from the code and decide how to use it. Based on my shallow ML understanding, there is no hard evidence Kaggle can give against any team who submitted a blending with that file. If any evidence, it will be statistical and nondeterministic.

\n\n

I don't like the idea at all that a few teams get an unfair bump no matter how small that number is.

", "votes": 7, "replies": [ { "id": 604983, "author_name": "CoreyJamesLevinson", "author_url": "", "post_date": "2019-08-21T23:37:09.517000", "content": "

Jiwei - if you do a linear combination blend, they can detect if by looking at one of your old submissions (if you blend it with your old submission). Besides that, they cant detect if you blend nonlinearly, or blend with something you haven't submitted before...

\n\n

Also, they can probably detect which users have downloaded the data by looking at the GET requests on that kernel page. But that wont reveal if you use it in your final 2 subs

\n\n

I support option 3

", "votes": 1, "replies": [] }, { "id": 604984, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-08-21T23:38:24.733000", "content": "

yeah, actually I can use it as pseudo label and train a neural network and then blend. How can you detect that?

", "votes": 4, "replies": [] }, { "id": 604986, "author_name": "Manoj", "author_url": "", "post_date": "2019-08-21T23:54:59.073000", "content": "

Thanks Jiwei your suggestion is reasonable.\nif the solution is made public people who are most impacted and will be devastated are the ones b/w roughly lets say 45 to 300.\nI feel its good idea to take a vote from the people who will be most impacted.\nI personally feel option 1 is a good choice at least 200 deserving people will get a medal vs handful of people got lucked out

\n\n

Type I errors are generally considered more serious than Type II errors.

", "votes": 6, "replies": [] }, { "id": 605233, "author_name": "Stanislav Blinov", "author_url": "", "post_date": "2019-08-22T07:22:45.970000", "content": "
\n

I don't like the idea at all that a few teams get an unfair bump no matter how small that number is.

\n
\n\n

And I don't like the idea at all that all teams below 2.16 will get this unfair bump tho they don't deserve it. Some worked for 3 moths almost every day, and some will just click 2 buttons. It's 1 week to deadine, so it's already not the \"if-public-kernel-beats-your-model-than-your-model-sucks\" situation.\nSo in summary, I'm totally with @mks2192 on this.

", "votes": 4, "replies": [] }, { "id": 605384, "author_name": "Yirun Zhang", "author_url": "", "post_date": "2019-08-22T10:22:32.050000", "content": "

That is really a breathless operation...

", "votes": 1, "replies": [] }, { "id": 605986, "author_name": "Bill Holst", "author_url": "", "post_date": "2019-08-23T03:40:54.177000", "content": "

@jiweiliu - Are you sure you would support your proposed option 3 if the submission had come in 0.01 ahead of your current position? We are sitting a few places below the submission. I don't care that 4 people/teams jumped ahead of our hard work, but if 1000 people did, that would not be fair. On the other hand, we could simply blend our working solution with the submission and move past all 1000!

", "votes": 1, "replies": [] }, { "id": 607270, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-08-24T23:39:44.130000", "content": "

It is no different. Our rank will be improved if we can blend that file. But it is not the point here. There is nothing in kaggle rules that forbids sharing high score kernel towards the end of competition. It is discouraged but not forbidden. Can we just follow the rule and leave the impact aside? I feel sad even admin bends the rule.

", "votes": 1, "replies": [] }, { "id": 607282, "author_name": "", "author_url": "", "post_date": "2019-08-25T00:38:45.857000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 605312, "author_name": "Scirpus", "author_url": "", "post_date": "2019-08-22T08:41:38.573000", "content": "

I got a kernel gold medal - yay I like kernels\nMy position just tanked - boo I hate kernels\nMy brain is so confused right now.

", "votes": 5, "replies": [] }, { "id": 608950, "author_name": "CPMP", "author_url": "", "post_date": "2019-08-27T09:34:57.213000", "content": "

You're funny.

", "votes": 6, "replies": [] }, { "id": 606042, "author_name": "Andrés Miguel Torrubia Sáez", "author_url": "", "post_date": "2019-08-23T06:02:26.757000", "content": "

What if somebody wanted to share a high scoring kernel intentionally?\nThis has happened before and will happen again.

\n\n

@inversion what's the motivation of Kaggle not being proactive and implement a kernel freeze (no sharing) close to competition end?

", "votes": 6, "replies": [ { "id": 606051, "author_name": "Mycroft Holmes", "author_url": "", "post_date": "2019-08-23T06:12:40.157000", "content": "

why would somebody share a very high scoring submission intentionally? there are penalties after all. as far as I've surmised it was a naked submission file.

\n\n

a couple of things come to mind:\n- a publicity stunt\n- when the submission is an overfit it may throw the competitors off the trail

", "votes": 0, "replies": [] }, { "id": 606065, "author_name": "CPMP", "author_url": "", "post_date": "2019-08-23T06:30:06.843000", "content": "

@stocks It happened in Talking Data, it is not a remote hypothetical possibility.

", "votes": 1, "replies": [] }, { "id": 606159, "author_name": "Kha Vo", "author_url": "", "post_date": "2019-08-23T09:24:35.780000", "content": "

Kernel freezing will make things worse, they will share in a private forum more. \nOnly prize money matters truly in life, then, prize zone competitors never share for sure. \nMedal fighters are different. Anyone can share, in their real life, in the private page... how do we know that?\nThen, just let it be. And be the strongest in a competition so ruiners can’t do anything to you.

", "votes": 0, "replies": [] }, { "id": 606164, "author_name": "Kha Vo", "author_url": "", "post_date": "2019-08-23T09:29:06.620000", "content": "

I remember, in Instant Gratification, someone posted a top (20-30) solution, but with intentionally manipulated private test rows. If used that, private LB will drop. Then Chris Deotte analysed the submission and warned everybody not to use that file.

", "votes": 2, "replies": [] }, { "id": 606216, "author_name": "somnambWl", "author_url": "", "post_date": "2019-08-23T10:28:40.827000", "content": "

Really? :D\nIt is like Da Vinci solution, in some of his works there were intentional errors, so in the end, it worked only when you thought about it and corrected it. No one could steal the solution and built it without an effort.\nSometimes I wonder, whether it is similar in public kernels. Like if people post a neural network, which they use with an amazing score, but in the public version they delete a layer, so it scores much worse. No one can get the best score for free, your original solution is not threatened, but you still get a medal for helping others.\nWell, maybe not that similar if it was posted intentionally with the goal to damage others..

", "votes": 1, "replies": [] }, { "id": 607347, "author_name": "somnambWl", "author_url": "", "post_date": "2019-08-25T04:17:57.173000", "content": "

I thought about what Kha Vo said about kernel freezing and I disagree with him. I still think that it would be helpful. If a medal from a competition is slipping under someone's fingers, but his score is still good enough, then he may have a motivation to post hist kernel in last week to get at least a medal for the kernel. If kernels were freezed, they wouldn't have such motivation.

", "votes": 0, "replies": [] }, { "id": 607392, "author_name": "CPMP", "author_url": "", "post_date": "2019-08-25T06:47:09.650000", "content": "
\n

If a medal from a competition is slipping under someone's fingers, but his score is still good enough, then he may have a motivation to post hist kernel in last week to get at least a medal for the kernel. If kernels were freezed, they wouldn't have such motivation.

\n
\n\n

Apply this reasoning to gold medal and you get what happened in Talking Data. A guys saw gold was out of reach last day and he posted his silver medal kernel.

", "votes": 0, "replies": [] }, { "id": 607410, "author_name": "datasaurus", "author_url": "", "post_date": "2019-08-25T07:26:46.800000", "content": "

I was unaware that Kaggle has already tried the kernel freeze idea in a slightly different form (freezing submissions only, not the sharing of kernels). Interesting discussion here: https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/56182#325316

", "votes": 2, "replies": [] }, { "id": 609393, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2019-08-27T16:53:56.227000", "content": "

One of major reasons, why i changed one of my recent public kernels to private.

", "votes": 1, "replies": [] } ] }, { "id": 605047, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-08-22T02:22:49.053000", "content": "

Ok, I don't like this solution but I will accept it if that is final. Just in case anyone doesn't know, I copied that kernel and downloaded the file and emailed Inversion immediately. My plan was to share it but it is not approved by admin.

", "votes": 3, "replies": [] }, { "id": 605766, "author_name": "snovik", "author_url": "", "post_date": "2019-08-22T18:09:56.180000", "content": "

letting a team or two to get the submit on the private sharing and some places on the LB is way more fair than opening it publicly and therefore wasting 3 months of work of lots of people. those who argue for option 3 should just wait another week to get the solution which ideally should be replicable and provided as code. But not before the competition ends.

", "votes": 3, "replies": [] }, { "id": 604970, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-08-21T23:11:33.757000", "content": "
\n

It's easy for us determine

\n
\n\n

What if a team submitted a blending with it? Can kaggle detect it?

", "votes": 3, "replies": [ { "id": 604972, "author_name": "inversion", "author_url": "", "post_date": "2019-08-21T23:13:21.450000", "content": "

You'd be surprised at what signals are in the data when you look.

", "votes": 5, "replies": [] }, { "id": 604973, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-08-21T23:14:32.357000", "content": "

Ok, I'm going to take your word for it 😅

", "votes": 0, "replies": [] }, { "id": 604975, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-08-21T23:15:36.653000", "content": "

Actually, could you point me to a literature if any? I'm interested from a theoretical perspective.

", "votes": 3, "replies": [] }, { "id": 605018, "author_name": "Andy Atkinson", "author_url": "", "post_date": "2019-08-22T01:06:30.490000", "content": "

I'm guessing just correlation would tell you quite a bit.

", "votes": 0, "replies": [] }, { "id": 605138, "author_name": "CPMP", "author_url": "", "post_date": "2019-08-22T05:20:43.950000", "content": "
\n

You'd be surprised at what signals are in the data when you look.

\n
\n\n

I'm surprised everyday when I see how Kagglers can get signal out of training data.

", "votes": 0, "replies": [] }, { "id": 605152, "author_name": "CPMP", "author_url": "", "post_date": "2019-08-22T05:30:54.880000", "content": "
\n

Actually, could you point me to a literature if any? I'm interested from a theoretical perspective.

\n
\n\n

You don't expect Kaggle staff to explain how they detect privtate sharing, do you? ;)

", "votes": 6, "replies": [] }, { "id": 605582, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-08-22T14:09:22.740000", "content": "

why not? I bet they have room to improve given the x5 situation. ;)

", "votes": 0, "replies": [] }, { "id": 605591, "author_name": "CPMP", "author_url": "", "post_date": "2019-08-22T14:17:25.697000", "content": "

X5 has teamed in open image competition :) Problem is solved...

", "votes": 1, "replies": [] }, { "id": 605593, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-08-22T14:18:35.173000", "content": "

haha, that's fair. Actually I asked about literature, not what they do in house. Isn't it something interesting to read ? ;)

", "votes": 1, "replies": [] }, { "id": 606235, "author_name": "Shubhinder", "author_url": "", "post_date": "2019-08-23T11:08:20.840000", "content": "", "votes": -1, "replies": [] }, { "id": 606264, "author_name": "FGPC", "author_url": "", "post_date": "2019-08-23T11:31:50.757000", "content": "

I think there's another commotion in private sharing which involve X5 again (Open Images Instance Segmentation). X5 issue is still unresolved.

", "votes": 1, "replies": [] } ] }, { "id": 609298, "author_name": "Mycroft Holmes", "author_url": "", "post_date": "2019-08-27T15:12:40.880000", "content": "

I believe that the correct answer is 'yes'.

", "votes": 4, "replies": [] }, { "id": 605014, "author_name": "Cozy Doomer", "author_url": "", "post_date": "2019-08-22T01:01:04.400000", "content": "

Thank you @inversion for clearing it up\nI agree with the choice made and am glad we are not removed from the competition\nI think @jesucristo made it very clear in the discussions that it was an accident he regrets

", "votes": 4, "replies": [] }, { "id": 605992, "author_name": "Kha Vo", "author_url": "", "post_date": "2019-08-23T03:56:41.747000", "content": "

Solution 1 is too soft. The leak file is shared in a different forum, according to someone saying that, then it’s unfair for those who cannot access that forum.

\n\n

What if in the future, a leak like that presents for 15 mins, rather than 5 mins like this one?

\n\n

However, middle table is now an unfair competition. I vote for option 2.

\n\n

All teams can use it as a blend with their own model.

", "votes": 3, "replies": [ { "id": 605994, "author_name": "Manoj", "author_url": "", "post_date": "2019-08-23T04:14:58.193000", "content": "

I think then people who are sharing and promoting should also be penalised

", "votes": 1, "replies": [] }, { "id": 606039, "author_name": "Turbo", "author_url": "", "post_date": "2019-08-23T05:53:42.797000", "content": "

I agree with you saying that the solution is too soft if the only consequence is getting kicked out of the competition. This way if a team is not in the top 10% then they can try to use the leak since they have nothing to lose.

", "votes": 2, "replies": [] }, { "id": 607274, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-08-24T23:51:06.177000", "content": "

I don't care about soft. The point is option 1 is not fair, in the sense that not everyone has access to that file. We are willing to follow the option 1 but I have no doubts that some teams will use that file and not be punished at all for private sharing. As confirmed by admin, that's the worst case and a price kaggle is willing to pay. It is really disappointing of kaggle's attitudes toward investigation and punishing private sharing. X5 and the like always get away.

", "votes": 0, "replies": [] } ] }, { "id": 604989, "author_name": "Daniel", "author_url": "", "post_date": "2019-08-22T00:00:16.553000", "content": "

Option #1 is also \"unfair to potentially thousands of competitors\". It means option #1 is much worse than option #2. Another truth is those \"a few teams\" just kicked our team out of the silver line. I could not see any fairness.

", "votes": 1, "replies": [] }, { "id": 609284, "author_name": "Scirpus", "author_url": "", "post_date": "2019-08-27T15:02:12.090000", "content": "

Can you give me the answers? LOL

", "votes": 2, "replies": [] }, { "id": 605526, "author_name": "Mycroft Holmes", "author_url": "", "post_date": "2019-08-22T13:04:28.627000", "content": "

It may take some time to finalize Private LB - just a hunch. I am guessing that out of ~300 medals, 100 will blend with the rogue submission and manage to bypass Kaggle's efforts. According to this scenario ~200 medals are still for grabs for those who prefer not to 'cheat' (the caveat is that the kernel was indeed public). Basically, we have to deal with an extra level of challenge in the last week.

", "votes": 0, "replies": [] }, { "id": 605306, "author_name": "Nanashi", "author_url": "", "post_date": "2019-08-22T08:37:43.553000", "content": "

I'm so sorry and I feel so bad... sorry everyone who was affected and this morning a friend told me the file is a ods.ai ...\nPlease don't use it.\n@inversion I'm sorry to cause trouble, thank you so much for your response, it really helps :)\nGood luck, and be careful when you create kernels, I don't know if it was when invited my mates or how... anyway I'll run everything on local now.

", "votes": -2, "replies": [] }, { "id": 606008, "author_name": "Mycroft Holmes", "author_url": "", "post_date": "2019-08-23T04:56:22.660000", "content": "

I am assuming that Kaggle is monitoring new submissions for evidence of blending with the rogue. If there is enough evidence that the rogue file is in the hands of more than 10-20 teams then they could always change the rules to plan B.

", "votes": 0, "replies": [ { "id": 606011, "author_name": "", "author_url": "", "post_date": "2019-08-23T05:02:13.617000", "content": "", "votes": 1, "replies": [] }, { "id": 606018, "author_name": "Mycroft Holmes", "author_url": "", "post_date": "2019-08-23T05:10:01.867000", "content": "

no need to accuse anybody. if they see enough evidence of covert blending then the file will be allowed for blending to everyone. \nfor now I do hope that the leak got plugged in successfully. I am probably very naive. LOL

", "votes": 0, "replies": [] }, { "id": 606021, "author_name": "", "author_url": "", "post_date": "2019-08-23T05:16:28.283000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 606457, "author_name": "", "author_url": "", "post_date": "2019-08-23T15:47:15.877000", "content": "", "votes": 0, "replies": [] }, { "id": 606201, "author_name": "", "author_url": "", "post_date": "2019-08-23T10:07:28.333000", "content": "", "votes": 0, "replies": [] }, { "id": 605111, "author_name": "", "author_url": "", "post_date": "2019-08-22T04:39:59.053000", "content": "", "votes": -1, "replies": [ { "id": 605403, "author_name": "fafafa fafafafa", "author_url": "", "post_date": "2019-08-22T10:53:50.730000", "content": "

Did you see inversion's post? They changed the rules.

", "votes": 2, "replies": [] }, { "id": 605497, "author_name": "", "author_url": "", "post_date": "2019-08-22T12:27:39.163000", "content": "", "votes": -2, "replies": [] }, { "id": 605506, "author_name": "somnambWl", "author_url": "", "post_date": "2019-08-22T12:42:21.677000", "content": "

Back then it was fair, now it is not fair, because we have a new rule.

\n\n

I very disagree with your point of view, on the contrary, I think, they should introduce a new rule every time when it helps the competition to be fair to more people. No rules are perfect and there is not a rule for everything. Where would we be if we have not been changing them and adding new ones..

", "votes": 0, "replies": [] }, { "id": 605541, "author_name": "", "author_url": "", "post_date": "2019-08-22T13:22:00.700000", "content": "", "votes": 0, "replies": [] }, { "id": 605553, "author_name": "Mycroft Holmes", "author_url": "", "post_date": "2019-08-22T13:38:51.613000", "content": "

this leak qualifies as force majeure. in other words, the rule change is totally warranted. \n*disclaimer: I am not a lawyer.

", "votes": 1, "replies": [] }, { "id": 605572, "author_name": "", "author_url": "", "post_date": "2019-08-22T14:01:09.987000", "content": "", "votes": -4, "replies": [] } ] } ], "raw_markdown_by_id": { "604966": "I think everyone is aware of the unfortunate \"public for a few minutes\" incident. It appears that 4 teams were able to access this submission.\n\nWe have two choices:\n\n1. Ask the teams that were able to download the submission to no longer make use of it or share it further, and to not select that submission as their final choice. (It's easy for us determine.) Worst case scenario, a few teams get an unfair bump.\n2. Allow that submission to be available to everyone. Worst case scenario, all positions below 45 become a \"who clicked fastest\" contest, which is unfair to potentially thousands of competitors.\n\nWith that said, **we are choosing option #1**, and asking the teams that have that information not to use it.", "605308": "inversion - i just emailed you to make some things clear. our score increased to -2.329 at Wed Aug 21 2019 14:38:06 GMT-0500 (Central Daylight Time) [as far as i can tell, this is before the -2.16 submission was made public]\n\nafter the -2.165 submission was released, we submitted a 30% 70% blend on Wed Aug 21 2019 15:24:55 GMT -0500 (Central Daylight Time) and received a tiny bit worse score. We submitted this before your announcement, so we did not know it was wrong.\n\nIn the future we will not be using this file. I just want to clear things up in case you think my team was suspicious. ", "604971": "Wow. I wonder how it could be detected if someone is blending it secretly.", "605278": "Although it may be a bit unfair, I still agree with the option #1.\nThat should be better than struggling with 2000 teams of -2.163.\nNow all my hope lies in Kaggle's detection algorithm, otherwise, my three months of hard work will be equal to 0.", "604981": "I would propose option 3.\n\nI kindly ask the meme team open source the solution of -2.16 so ALL people can learn from the code and decide how to use it. Based on my shallow ML understanding, there is no hard evidence Kaggle can give against any team who submitted a blending with that file. If any evidence, it will be statistical and nondeterministic. \n\nI don't like the idea at all that a few teams get an unfair bump no matter how small that number is.", "605312": "I got a kernel gold medal - yay I like kernels\nMy position just tanked - boo I hate kernels\nMy brain is so confused right now.", "608950": "You're funny.", "606042": "What if somebody wanted to share a high scoring kernel intentionally?\nThis has happened before and will happen again. \n\n@inversion what's the motivation of Kaggle not being proactive and implement a kernel freeze (no sharing) close to competition end? ", "605047": "Ok, I don't like this solution but I will accept it if that is final. Just in case anyone doesn't know, I copied that kernel and downloaded the file and emailed Inversion immediately. My plan was to share it but it is not approved by admin.", "605766": "letting a team or two to get the submit on the private sharing and some places on the LB is way more fair than opening it publicly and therefore wasting 3 months of work of lots of people. those who argue for option 3 should just wait another week to get the solution which ideally should be replicable and provided as code. But not before the competition ends.", "604970": "> It's easy for us determine\n\nWhat if a team submitted a blending with it? Can kaggle detect it?", "609298": "I believe that the correct answer is 'yes'.", "605014": "Thank you @inversion for clearing it up\nI agree with the choice made and am glad we are not removed from the competition\nI think @jesucristo made it very clear in the discussions that it was an accident he regrets", "605992": "Solution 1 is too soft. The leak file is shared in a different forum, according to someone saying that, then it’s unfair for those who cannot access that forum. \n\nWhat if in the future, a leak like that presents for 15 mins, rather than 5 mins like this one?\n\nHowever, middle table is now an unfair competition. I vote for option 2. \n\nAll teams can use it as a blend with their own model. ", "604989": "Option #1 is also \"unfair to potentially thousands of competitors\". It means option #1 is much worse than option #2. Another truth is those \"a few teams\" just kicked our team out of the silver line. I could not see any fairness.", "609284": "Can you give me the answers? LOL", "605526": "It may take some time to finalize Private LB - just a hunch. I am guessing that out of ~300 medals, 100 will blend with the rogue submission and manage to bypass Kaggle's efforts. According to this scenario ~200 medals are still for grabs for those who prefer not to 'cheat' (the caveat is that the kernel was indeed public). Basically, we have to deal with an extra level of challenge in the last week.", "605306": "I'm so sorry and I feel so bad... sorry everyone who was affected and this morning a friend told me the file is a ods.ai ...\nPlease don't use it.\n@inversion I'm sorry to cause trouble, thank you so much for your response, it really helps :)\nGood luck, and be careful when you create kernels, I don't know if it was when invited my mates or how... anyway I'll run everything on local now. ", "606008": "I am assuming that Kaggle is monitoring new submissions for evidence of blending with the rogue. If there is enough evidence that the rogue file is in the hands of more than 10-20 teams then they could always change the rules to plan B.", "606457": "", "606201": "", "605111": "" } }