Yesteday suddenly our team lost 11 positions, from 8th to 17th, in just a few hours.
\n\nWell, it may happen in a competition that your team is overcome by some smarter or luckier teams, but I've never seen such a imponent moving all in a sudden (and resulting in almost the same score for many of the teams involved) in all my 8 year Kaggle experience.
\n\nSomething should be going on and we are completely unaware of it. Yet, we checked the discussion board and the notebooks for any hint, but we found nothing, zero. Sincerely we suspect someone did private sharing of ideas, code or submissions, I do not fear of hinting openly at that.
\n\nThis has been a misfortunated competition, but as a team we held on until the end and working hard up so far because we were sincerely interested in the topic. But now the competition is ending in an even worse way than we expected.
\n\n@inversion @addisonhoward @sohier I hope you will enforce strict controls on the leaderboard, after this competition has ended. Thank you a lot.
", "messages": [ { "id": 697824, "postDate": "2019-12-18T13:15:50.247Z", "content": "Yesteday suddenly our team lost 11 positions, from 8th to 17th, in just a few hours.
\n\nWell, it may happen in a competition that your team is overcome by some smarter or luckier teams, but I've never seen such a imponent moving all in a sudden (and resulting in almost the same score for many of the teams involved) in all my 8 year Kaggle experience.
\n\nSomething should be going on and we are completely unaware of it. Yet, we checked the discussion board and the notebooks for any hint, but we found nothing, zero. Sincerely we suspect someone did private sharing of ideas, code or submissions, I do not fear of hinting openly at that.
\n\nThis has been a misfortunated competition, but as a team we held on until the end and working hard up so far because we were sincerely interested in the topic. But now the competition is ending in an even worse way than we expected.
\n\n@inversion @addisonhoward @sohier I hope you will enforce strict controls on the leaderboard, after this competition has ended. Thank you a lot.
", "rawMarkdown": "Yesteday suddenly our team lost 11 positions, from 8th to 17th, in just a few hours. \n\nWell, it may happen in a competition that your team is overcome by some smarter or luckier teams, but I've never seen such a imponent moving all in a sudden (and resulting in almost the same score for many of the teams involved) in all my 8 year Kaggle experience.\n\nSomething should be going on and we are completely unaware of it. Yet, we checked the discussion board and the notebooks for any hint, but we found nothing, zero. Sincerely we suspect someone did private sharing of ideas, code or submissions, I do not fear of hinting openly at that. \n\nThis has been a misfortunated competition, but as a team we held on until the end and working hard up so far because we were sincerely interested in the topic. But now the competition is ending in an even worse way than we expected.\n\n@inversion @addisonhoward @sohier I hope you will enforce strict controls on the leaderboard, after this competition has ended. Thank you a lot.", "votes": 37 }, { "id": 698035, "postDate": "2019-12-18T17:58:22.547Z", "content": "Another team jumped to the 37th place with 4 submissions. Check this thread and you will understand what is going on right now.\nhttps://www.kaggle.com/c/ieee-fraud-detection/discussion/110778
", "rawMarkdown": "Another team jumped to the 37th place with 4 submissions. Check this thread and you will understand what is going on right now.\nhttps://www.kaggle.com/c/ieee-fraud-detection/discussion/110778", "votes": 12, "replies": [ { "id": 698054, "postDate": "2019-12-18T18:36:33.370Z", "content": "By checking the top 100 team (not single participant) on the LB which have few (less than 10) submits, I think you might be right.
", "rawMarkdown": "By checking the top 100 team (not single participant) on the LB which have few (less than 10) submits, I think you might be right.", "votes": 3 }, { "id": 698513, "postDate": "2019-12-19T10:33:42.863Z", "content": "I see another one in top 100 who used only 4 submissions and that account was created 9 days ago.\nLegendary.
", "rawMarkdown": "I see another one in top 100 who used only 4 submissions and that account was created 9 days ago.\nLegendary.", "votes": 3 } ] }, { "id": 698098, "postDate": "2019-12-18T19:57:56.880Z", "content": "We lost close to 150 positions in the last week.
\nI'm going out of the medal......
Haha, I feel you...Just went out of the medals the last day...Have dropped more than 100 positions in the last 3 days without any1 posting any better-scoring kernels. C'est la vie! (btw I m not french, I hope I got it right)
", "rawMarkdown": "Haha, I feel you...Just went out of the medals the last day...Have dropped more than 100 positions in the last 3 days without any1 posting any better-scoring kernels. C'est la vie! (btw I m not french, I hope I got it right)", "votes": 1 }, { "id": 699087, "postDate": "2019-12-20T03:26:28.197Z", "content": "\bYes. Faisons de notre mieux!! (btw I m not french too, I hope I got it right)
", "rawMarkdown": "\bYes. Faisons de notre mieux!! (btw I m not french too, I hope I got it right)", "votes": 1 }, { "id": 699264, "postDate": "2019-12-20T08:48:55.850Z", "content": "\n\n\nFaisons de notre mieux!
\n
Lol, yes we are actually exactly at the same place! Let's see where we both end up!
", "rawMarkdown": "> Faisons de notre mieux! \n\nLol, yes we are actually exactly at the same place! Let's see where we both end up!", "votes": 1 } ] }, { "id": 698224, "postDate": "2019-12-19T00:45:47.453Z", "content": "We lost close to 70 position in last few days. I am certain that we missed the trick.
", "rawMarkdown": "We lost close to 70 position in last few days. I am certain that we missed the trick.", "votes": 3, "replies": [ { "id": 698639, "postDate": "2019-12-19T13:55:09.990Z", "content": "Yesterday night i made improvements by 7 positions and this morning we slipped again by 16 positions. There is something very strange
", "rawMarkdown": "Yesterday night i made improvements by 7 positions and this morning we slipped again by 16 positions. There is something very strange", "votes": 3 }, { "id": 698655, "postDate": "2019-12-19T14:17:34.757Z", "content": "I agree. Thanks to your work I was between 60 and 70 yesterday but now I am out of 100. The only high score published recently is the R code \"ASHRAE 2*lightGBM without Leak Data\" but I got no improvement blending with that...
", "rawMarkdown": "I agree. Thanks to your work I was between 60 and 70 yesterday but now I am out of 100. The only high score published recently is the R code \"ASHRAE 2*lightGBM without Leak Data\" but I got no improvement blending with that...", "votes": 1 }, { "id": 698794, "postDate": "2019-12-19T17:39:43.467Z", "content": "Slipped by another 20 in 3 hrs. ☹️
", "rawMarkdown": "Slipped by another 20 in 3 hrs. ☹️ ", "votes": 1 }, { "id": 698893, "postDate": "2019-12-19T20:44:27.007Z", "content": "ASHRAE 2*lightGBM without Leak Data in R is a last minute submission. I guess people are blending a lot with that model to improve their scores.
", "rawMarkdown": "ASHRAE 2*lightGBM without Leak Data in R is a last minute submission. I guess people are blending a lot with that model to improve their scores. ", "votes": 1 } ] }, { "id": 697838, "postDate": "2019-12-18T13:33:33.827Z", "content": "Yesterday was 3 days before deadline. It could be a coincedence - if many teams at once started to blend final submissions. But also it could be not.\nWhat I noticed - @reneiw jumped in the green zone with just 11 submissions. Might be that he's very talented and just didn't made any submissions. A lot of scientists work hard for a couple final results and don't use every submission for checking along the way. Just no history at all on the 3-years-old account is a bit suspicious. But I do not call names and give labels.\nAnyway, let's not start witch hunting and hope that Kaggle algorithms will do the work.
", "rawMarkdown": "Yesterday was 3 days before deadline. It could be a coincedence - if many teams at once started to blend final submissions. But also it could be not.\nWhat I noticed - @reneiw jumped in the green zone with just 11 submissions. Might be that he's very talented and just didn't made any submissions. A lot of scientists work hard for a couple final results and don't use every submission for checking along the way. Just no history at all on the 3-years-old account is a bit suspicious. But I do not call names and give labels.\nAnyway, let's not start witch hunting and hope that Kaggle algorithms will do the work.", "votes": 3, "replies": [ { "id": 697849, "postDate": "2019-12-18T13:41:03.530Z", "content": "I don't think it is possible to place in such high rank with that many submissions in this competition. My experiments costed me at least 60 submissions.
", "rawMarkdown": "I don't think it is possible to place in such high rank with that many submissions in this competition. My experiments costed me at least 60 submissions. ", "votes": 7 }, { "id": 697855, "postDate": "2019-12-18T13:45:39.330Z", "content": "yes ... I mean it can happen but in this competition you cannot rely only on local validation you do need to submit to understand better what is the best strategy etc ....
", "rawMarkdown": "yes ... I mean it can happen but in this competition you cannot rely only on local validation you do need to submit to understand better what is the best strategy etc ....", "votes": 6 }, { "id": 697859, "postDate": "2019-12-18T13:47:34.357Z", "content": "@gunesevitan it's your experience, and mine too. But maybe this guy has a great experience, how could we know?\nIt's been known Uncle @cpmpml jumped in the gold zone with the first submission, for example.
", "rawMarkdown": "@gunesevitan it's your experience, and mine too. But maybe this guy has a great experience, how could we know?\nIt's been known Uncle @cpmpml jumped in the gold zone with the first submission, for example.", "votes": 1 }, { "id": 698092, "postDate": "2019-12-18T19:43:07.183Z", "content": "There are several users of the wiener family (reneiw upside down) also 3 years old without any activity. Will they all be a cluster of magic accounts?
", "rawMarkdown": "There are several users of the wiener family (reneiw upside down) also 3 years old without any activity. Will they all be a cluster of magic accounts?", "votes": 5 }, { "id": 698094, "postDate": "2019-12-18T19:47:11.357Z", "content": "Ok now I agree that it's fishy.
", "rawMarkdown": "Ok now I agree that it's fishy.", "votes": 2 }, { "id": 698764, "postDate": "2019-12-19T17:11:21.827Z", "content": "interesting, reneiw's name become evidence of conspiracy, i started using reneiw ten years ago when starting my study in control engineering, difficult to imagine what Wiener would think. btw, curious about the glorious deeds of the wiener family...
\n\n3 years old without any activity is can't be more normal for me, i dislike pure competition at all, even join in this competition is due to highly related to my main work but not for competition itself, evaluation of building energy saving efficiency is a real problem and simultaneously a hard problem, in my experience even the AB test split by time is sometimes difficult to correctly evaluate the energy saving effect due to many non-control non-constant factors, evaluation based on the environment variables is an important direction, the huge amount of data in the competition can be beneficial to my work research.
\n\nif just to explain the two questions above, i think I have no reason to make this explanation, the main reason for typing so many words to reply is, the reason your guys to doubt is reach high score with few commits, this can't stand, isn't this really reasonable performance? Do you think we should get high scores through a large number of submissions to be healthy? seems most people think so, that's also the reason I don't like competitions. No one has the energy to try a hundred models or methods, it is nothing more than taking a small number of methods to test online and then constantly using small tricks to correct and then test again, full of guess guess guess collision collision collision ensemble ensemble ensemble routine, take so much time and effort and commits, no real industrial value was created in the end, just to cater to this batch of test sets. Obtaining relatively good and stable results through a concise method originally does not need to submit tests so much times, if get the right method, 20% time can reach the front row, and the remain 80% time is spent from the front row to the top, which industrial added value is relatively small and unstable, i was lucky to find a concise way to complete the first step. With all due respect, except for pure learning purposes, if your 50 submissions are not as good as 20 of others, give up early, if you still want to win, find other competitions to try, there are always competitions where you can come up with better solutions.
\n\nToo much pessimistic talk about the phenomenon of competition, turn back to this competition now, actually the current score doesn't explain anything, i used leak data in depth, and the rapid improvement in results is also due to leak data, this very likely overfitting to the leak data or better than other people who have no use for leaking data Is an illusion, i mainly want to share the interesting thing that why suddenly decided to use the leak data in depth(for row level train), because i realized that this leak data is not a true leak, it has practical useable industry value. It is difficult to correctly evaluate the energy impact of a building renovation based on historical data alone, because energy use may change by time even if the building is without any renovation, but if there are buildings that definitely known without any renovation, and their energy data can help to predict other buildings energy use in the same period if they have no renovation also, then we can more accurately to evaluate the energy effect of building renovation, and the leak data of this competition act as the role of buildings data definitely known without renovation for evaluation period.
", "rawMarkdown": "interesting, reneiw's name become evidence of conspiracy, i started using reneiw ten years ago when starting my study in control engineering, difficult to imagine what Wiener would think. btw, curious about the glorious deeds of the wiener family...\n\n3 years old without any activity is can't be more normal for me, i dislike pure competition at all, even join in this competition is due to highly related to my main work but not for competition itself, evaluation of building energy saving efficiency is a real problem and simultaneously a hard problem, in my experience even the AB test split by time is sometimes difficult to correctly evaluate the energy saving effect due to many non-control non-constant factors, evaluation based on the environment variables is an important direction, the huge amount of data in the competition can be beneficial to my work research.\n\nif just to explain the two questions above, i think I have no reason to make this explanation, the main reason for typing so many words to reply is, the reason your guys to doubt is reach high score with few commits, this can't stand, isn't this really reasonable performance? Do you think we should get high scores through a large number of submissions to be healthy? seems most people think so, that's also the reason I don't like competitions. No one has the energy to try a hundred models or methods, it is nothing more than taking a small number of methods to test online and then constantly using small tricks to correct and then test again, full of guess guess guess collision collision collision ensemble ensemble ensemble routine, take so much time and effort and commits, no real industrial value was created in the end, just to cater to this batch of test sets. Obtaining relatively good and stable results through a concise method originally does not need to submit tests so much times, if get the right method, 20% time can reach the front row, and the remain 80% time is spent from the front row to the top, which industrial added value is relatively small and unstable, i was lucky to find a concise way to complete the first step. With all due respect, except for pure learning purposes, if your 50 submissions are not as good as 20 of others, give up early, if you still want to win, find other competitions to try, there are always competitions where you can come up with better solutions.\n\nToo much pessimistic talk about the phenomenon of competition, turn back to this competition now, actually the current score doesn't explain anything, i used leak data in depth, and the rapid improvement in results is also due to leak data, this very likely overfitting to the leak data or better than other people who have no use for leaking data Is an illusion, i mainly want to share the interesting thing that why suddenly decided to use the leak data in depth(for row level train), because i realized that this leak data is not a true leak, it has practical useable industry value. It is difficult to correctly evaluate the energy impact of a building renovation based on historical data alone, because energy use may change by time even if the building is without any renovation, but if there are buildings that definitely known without any renovation, and their energy data can help to predict other buildings energy use in the same period if they have no renovation also, then we can more accurately to evaluate the energy effect of building renovation, and the leak data of this competition act as the role of buildings data definitely known without renovation for evaluation period.\n", "votes": 3 }, { "id": 698781, "postDate": "2019-12-19T17:26:54.867Z", "rawMarkdown": "", "votes": -4, "isDeleted": true }, { "id": 698815, "postDate": "2019-12-19T18:07:33.563Z", "content": "Pay attention to your words, i have no responsibility to \"explain\" for 100 Chinese novices, I can directly tell you that there are two accounts associated with me, one of my own, and one of my colleagues, the remaining 98 don't look for me. my colleague started early then me and i actually started modeling this week begin with use some organized data from my colleague about his 10th submission, then we model separately from this point, yes, you read it right, i use colleague‘s initial data but did not merge team with him, if foul delete all my data and grades, i have no interest in studying kaggle data usage rules, hopefully deleting my account will allow you to enter money region.
", "rawMarkdown": "Pay attention to your words, i have no responsibility to \"explain\" for 100 Chinese novices, I can directly tell you that there are two accounts associated with me, one of my own, and one of my colleagues, the remaining 98 don't look for me. my colleague started early then me and i actually started modeling this week begin with use some organized data from my colleague about his 10th submission, then we model separately from this point, yes, you read it right, i use colleague‘s initial data but did not merge team with him, if foul delete all my data and grades, i have no interest in studying kaggle data usage rules, hopefully deleting my account will allow you to enter money region.", "votes": 2 }, { "id": 698821, "postDate": "2019-12-19T18:17:13.570Z", "content": "Disclaimer: I have not entered this competition, not even looked at data. I just want to react to what looks like a witch hunt.
\n\n\n\n> Do you think we should get high scores through a large number of submissions to be healthy?
\n\nYou're right questioning this. There are famous examples of people jumping high in LB with 1 submission, including the famous @speculation win with 1 sub only. Small number of subs plus good LB rank aren't suspicious in principle.
\n\n> i use colleague‘s initial data but did not merge team with him
\n\nWell, that's against competition rules. But you know it given you write:
\n\n> i have no interest in studying kaggle data usage rules, hopefully deleting my account will allow you to enter money region.
\n\nIf you know how it will end then why did you enter? If it is only to see how your model(s) will fare on private test data, issue is that you won't see your submissions at all if you're removed. I know because this happened to me after my team was removed because a team member broke some rules and was caught.
", "rawMarkdown": "Disclaimer: I have not entered this competition, not even looked at data. I just want to react to what looks like a witch hunt.\n\n@reneiw \n\n> Do you think we should get high scores through a large number of submissions to be healthy?\n\nYou're right questioning this. There are famous examples of people jumping high in LB with 1 submission, including the famous @speculation win with 1 sub only. Small number of subs plus good LB rank aren't suspicious in principle.\n\n> i use colleague‘s initial data but did not merge team with him\n\nWell, that's against competition rules. But you know it given you write:\n\n> i have no interest in studying kaggle data usage rules, hopefully deleting my account will allow you to enter money region.\n\nIf you know how it will end then why did you enter? If it is only to see how your model(s) will fare on private test data, issue is that you won't see your submissions at all if you're removed. I know because this happened to me after my team was removed because a team member broke some rules and was caught.\n", "votes": 9 }, { "id": 698826, "postDate": "2019-12-19T18:30:23.433Z", "content": "Because I only have time to realy started modeling in this last 3 days, if merged team will take up the submission opportunities of my colleague for the last few days, so I decide to keep my own account \nwithout in-depth investigation of kaggle rules last week. can't see the private score is really a problem.
", "rawMarkdown": "Because I only have time to realy started modeling in this last 3 days, if merged team will take up the submission opportunities of my colleague for the last few days, so I decide to keep my own account \nwithout in-depth investigation of kaggle rules last week. can't see the private score is really a problem.", "votes": -2 }, { "id": 698827, "postDate": "2019-12-19T18:31:46.773Z", "rawMarkdown": "", "votes": 1 }, { "id": 698832, "postDate": "2019-12-19T18:40:23.160Z", "content": "I don't care about the money. I wasn't accusing you with anything at all. I just asked about your opinion about this situation, but you misinterpreted my question. Actually, I wasn't expecting you to do private sharing since you are at the very top, but you just said it yourself.
", "rawMarkdown": "I don't care about the money. I wasn't accusing you with anything at all. I just asked about your opinion about this situation, but you misinterpreted my question. Actually, I wasn't expecting you to do private sharing since you are at the very top, but you just said it yourself.", "votes": 1 }, { "id": 698840, "postDate": "2019-12-19T18:58:28.067Z", "content": "Beside the main topic, I hope you post your solution and how you used leak data for predictions. would be interesting and good to study ;-) Thanks for your long explanation btw.
", "rawMarkdown": "Beside the main topic, I hope you post your solution and how you used leak data for predictions. would be interesting and good to study ;-) Thanks for your long explanation btw.", "votes": 1 }, { "id": 698853, "postDate": "2019-12-19T19:18:38.433Z", "content": "emmm, looks like there are differences in language habits... It is a pity that I have no opinion on this, i have not followed user details for leaderboard.
\n\nI think private sharing is fine since the competition is almost over so does not substantially affect others, hope to see the private list first and then deal with foul, help me to verify whether it is stable to use some buildings data to help predict other buildings. Unfortunately it seems even the highest online score is too big so not applicable at all in real world, in actual projects, even if more precise data is used to make more accurate prediction models, customers still choose AB test
", "rawMarkdown": "emmm, looks like there are differences in language habits... It is a pity that I have no opinion on this, i have not followed user details for leaderboard.\n\nI think private sharing is fine since the competition is almost over so does not substantially affect others, hope to see the private list first and then deal with foul, help me to verify whether it is stable to use some buildings data to help predict other buildings. Unfortunately it seems even the highest online score is too big so not applicable at all in real world, in actual projects, even if more precise data is used to make more accurate prediction models, customers still choose AB test\n", "votes": -5 }, { "id": 698877, "postDate": "2019-12-19T19:56:29.350Z", "content": "\n\n\nBecause I only have time to realy started modeling in this last 3 days, if merged team will take up the submission opportunities of my colleague for the last few days, so I decide to keep my own account without in-depth investigation of kaggle rules last week
\n
You realize that doing this will also result in your colleague removal as he shared privately with you?
", "rawMarkdown": "> Because I only have time to realy started modeling in this last 3 days, if merged team will take up the submission opportunities of my colleague for the last few days, so I decide to keep my own account without in-depth investigation of kaggle rules last week\n\nYou realize that doing this will also result in your colleague removal as he shared privately with you?", "votes": 4 }, { "id": 698879, "postDate": "2019-12-19T19:58:27.707Z", "content": "\n\n\nI think private sharing is fine
\n
No, private sharing is never fine, it is unfair and it is against the rules.
", "rawMarkdown": "> I think private sharing is fine\n\nNo, private sharing is never fine, it is unfair and it is against the rules.", "votes": 11 }, { "id": 698888, "postDate": "2019-12-19T20:32:24.073Z", "content": "\n\n\nI think private sharing is fine
\n \ni have no interest in studying kaggle data usage rules
\n
You come to the community @ you follow the rules, mr. Toxic.
", "rawMarkdown": "> I think private sharing is fine\n\n> i have no interest in studying kaggle data usage rules\n\n\nYou come to the community @ you follow the rules, mr. Toxic." }, { "id": 699041, "postDate": "2019-12-20T02:10:01.973Z", "content": "you misunderstand what I mean, rules are of course important for platform operations, \"I think private sharing is fine\" means i think this is fine when i write this comment when all our submissions are over, not in the competition progress, i misunderstood the comment above means that in my repost body said the use of leak data to improve performance belongs to private sharing... I did n’t share any method or data with anyone during the competition procedure include my colleague, otherwise he can get good grades too.
\n\n\n\n\nYou realize that doing this will also result in your colleague removal as he shared privately with you?
\n
this really don't know, and he didn't actually do anything to shair data for me, we are in different city branches and no daily sharing, just when i start modeling i don’t have time to rearrange large amounts of competition data especially leak data so i download colleague's collated data set directly from company server which he use, he didn't get any benefit from my actions or purposely to private help me, it's my fault if it makes him foul too, fortunately his grades are not very good thus don't hurt by me too much.
", "rawMarkdown": "you misunderstand what I mean, rules are of course important for platform operations, \"I think private sharing is fine\" means i think this is fine when i write this comment when all our submissions are over, not in the competition progress, i misunderstood the comment above means that in my repost body said the use of leak data to improve performance belongs to private sharing... I did n’t share any method or data with anyone during the competition procedure include my colleague, otherwise he can get good grades too.\n\n> You realize that doing this will also result in your colleague removal as he shared privately with you?\n\nthis really don't know, and he didn't actually do anything to shair data for me, we are in different city branches and no daily sharing, just when i start modeling i don’t have time to rearrange large amounts of competition data especially leak data so i download colleague's collated data set directly from company server which he use, he didn't get any benefit from my actions or purposely to private help me, it's my fault if it makes him foul too, fortunately his grades are not very good thus don't hurt by me too much.\n\n\n\n", "votes": 1 }, { "id": 699058, "postDate": "2019-12-20T02:30:17.963Z", "content": "Interesting. You can set anywhere as your location in your profile, it doesn't mean anything. Cheaters should be removed, that's definitely true, but others (from the same Country) are innocent. One (or a group of) people cannot represent the whole Country, correct your words please.
", "rawMarkdown": "Interesting. You can set anywhere as your location in your profile, it doesn't mean anything. Cheaters should be removed, that's definitely true, but others (from the same Country) are innocent. One (or a group of) people cannot represent the whole Country, correct your words please. ", "votes": 6 }, { "id": 699077, "postDate": "2019-12-20T03:07:59.563Z", "content": "@gunesevitan I don't think words that promote outlandish discrimination are welcomed in this community. Certainly made me feel uncomfortable.
", "rawMarkdown": "@gunesevitan I don't think words that promote outlandish discrimination are welcomed in this community. Certainly made me feel uncomfortable.", "votes": 6 }, { "id": 699128, "postDate": "2019-12-20T04:27:36.647Z", "content": "@yiheng @alexanderliao \nI wasn't trying to make any discrimination. Sorry for making you feel uncomfortable. I know there are innocent people among those top teams. I'm not accusing everyone with cheating. \nI thought having too much novices with very few submissions and all of them being from the same country is a little bit weird. I have no intentions of discriminating anyone.
", "rawMarkdown": "@yiheng @alexanderliao \nI wasn't trying to make any discrimination. Sorry for making you feel uncomfortable. I know there are innocent people among those top teams. I'm not accusing everyone with cheating. \nI thought having too much novices with very few submissions and all of them being from the same country is a little bit weird. I have no intentions of discriminating anyone.", "votes": 2 }, { "id": 699375, "postDate": "2019-12-20T11:14:46.257Z", "content": "some unexpected things happened this week cause some trouble to others including myself, since i am a passer of kaggle and maybe disappear another three years from next week so will not see the discussion later, i think i should make some notes lastly,
\n\nI did use data from colleague which can't access by other participate, and it ’s very important for me to get a high score in few days, he spent lot of effort collecting, cleaning, and calibrating competition-related data on forums, this allows me to skip a lot of preprocessing work and go straight to the modeling process. This violates the rules and i am happy to accept punishment to clear all my results. This is my unilateral mistake, hope this will not affect others.
Although using other's initial data can greatly reduce the time spent, it's different from submission counts, this is the key reason I decided to reply, it ’s healthy to get high scores with a small number of submissions, relying on a large number of submissions to get high scores is usually because did not find a suitable method or did not use the same level data (such as this leak data).
Back to the technical problem itself, @hanjoonchoe @chabir , the key issue to using leak data to improve prediction performance is transfer information from some buildings to some others, so learning of leak data cannot rely on building ids in categorical state, i have tried several methods, such as learning training data using neural networks(thus learn the building id embedding simultaneously), and then fine-tune on leak data by freeze the embedding to force learn the new years characteristic by other environmental features; or extract the embedding of building id in nn to use as features substitute categorical buliding id in normal lgbm models; or use lgbm to learning on training data and then ignore the building id feature to learn another model to fit the gap between lgbm prediction value and the leak value. sorry for i can't tell you which one performs better in the end, because for each method i didn't do it fine enough, and I don't think I have solved this technical problem, i don't want to mislead you, believe other excellent groups will give more interesting methods and conclusions.
\n\n\ni download colleague's collated data set directly from company server
\n
This is private sharing of data.
", "rawMarkdown": "> i download colleague's collated data set directly from company server\n\nThis is private sharing of data. ", "votes": 2 } ] }, { "id": 698437, "postDate": "2019-12-19T08:20:31.230Z", "content": "I also notice that someone just join kaggle for few days and their submissions is lower than 10,but their ranks is in 100,this makes me too confuse
", "rawMarkdown": "I also notice that someone just join kaggle for few days and their submissions is lower than 10,but their ranks is in 100,this makes me too confuse\n", "votes": 4 }, { "id": 697921, "postDate": "2019-12-18T14:54:01.630Z", "content": "We lost close to 67 positions in the last 3 days. We were 54 when this week started, currently at 121st.. Not even a single discussion thread on the improvement..
", "rawMarkdown": "We lost close to 67 positions in the last 3 days. We were 54 when this week started, currently at 121st.. Not even a single discussion thread on the improvement.. ", "votes": 4, "replies": [ { "id": 697924, "postDate": "2019-12-18T14:57:13.400Z", "content": "wow, this is quite strange as well ... don't you think so?
", "rawMarkdown": "wow, this is quite strange as well ... don't you think so?", "votes": 2 }, { "id": 697926, "postDate": "2019-12-18T14:57:59.900Z", "content": "how many position on average did you loose during the competition .. I mean given a position how long would it last without submitting? at least one or two days ?
", "rawMarkdown": "how many position on average did you loose during the competition .. I mean given a position how long would it last without submitting? at least one or two days ?", "votes": 1 }, { "id": 698210, "postDate": "2019-12-19T00:00:05.497Z", "content": "Hardly one in 2 days even if we don't submit anything. But this time around, everyday on an average 15 to 25 spots down. There are so many particpants with very less submissions in top 100... That seems to be more strange as well..
", "rawMarkdown": "Hardly one in 2 days even if we don't submit anything. But this time around, everyday on an average 15 to 25 spots down. There are so many particpants with very less submissions in top 100... That seems to be more strange as well.. ", "votes": 2 } ] }, { "id": 697887, "postDate": "2019-12-18T14:19:21.510Z", "content": "maybe some teams found an additional last minute leak?? not sure...will wait and see what happens when Pvt LB is up.
", "rawMarkdown": "maybe some teams found an additional last minute leak?? not sure...will wait and see what happens when Pvt LB is up. ", "votes": 4, "replies": [ { "id": 697933, "postDate": "2019-12-18T15:07:03.430Z", "content": "It's either that or private sharing.
", "rawMarkdown": "It's either that or private sharing.", "votes": 5 }, { "id": 697936, "postDate": "2019-12-18T15:10:00.290Z", "content": "they should share it don't they?
", "rawMarkdown": "they should share it don't they?", "votes": 1 }, { "id": 697955, "postDate": "2019-12-18T15:27:10.017Z", "content": "@pietromarinelli I would not bet on everyone sharing a last minute leak. @gunesevitan - agreed it could be private sharing, but seems like a risky move on the last two days of competition.
", "rawMarkdown": "@pietromarinelli I would not bet on everyone sharing a last minute leak. @gunesevitan - agreed it could be private sharing, but seems like a risky move on the last two days of competition.", "votes": 1 } ] }, { "id": 697876, "postDate": "2019-12-18T14:02:05.570Z", "content": "Please do not submit high scoring predictions :)
", "rawMarkdown": "Please do not submit high scoring predictions :)", "votes": 1 }, { "id": 698912, "postDate": "2019-12-19T21:55:47.727Z", "content": "@reneiw : \nbeside the fact that you could have dragged your colleague in the money, can you explain more about your overall method. I understand that you build a method using the leak data to calculate the behaviour of individual buildings over 3 years: may I ask how you did that ? did you calculate trends, find particular patterns, set up particular feature engineering ?
", "rawMarkdown": "@reneiw : \nbeside the fact that you could have dragged your colleague in the money, can you explain more about your overall method. I understand that you build a method using the leak data to calculate the behaviour of individual buildings over 3 years: may I ask how you did that ? did you calculate trends, find particular patterns, set up particular feature engineering ?\n", "votes": 1 }, { "id": 698229, "postDate": "2019-12-19T01:06:26.543Z", "content": "Lost close to 50 position while score improved by 0.004 in a week. It's really impressive to see the guys who solve the problem within 20 commits. Tired of overfitting to keep rankings anyway.
", "rawMarkdown": "Lost close to 50 position while score improved by 0.004 in a week. It's really impressive to see the guys who solve the problem within 20 commits. Tired of overfitting to keep rankings anyway.", "votes": 1 }, { "id": 697833, "postDate": "2019-12-18T13:29:02.667Z", "content": "One idea might be that people that were previously not using any leak data at all, are now submitting their models using leak data. Thus jumping a lot of places at one time. However, you are right 9 positions in the top 20 looks a bit suspicious.
", "rawMarkdown": "One idea might be that people that were previously not using any leak data at all, are now submitting their models using leak data. Thus jumping a lot of places at one time. However, you are right 9 positions in the top 20 looks a bit suspicious.", "votes": 1, "replies": [ { "id": 697839, "postDate": "2019-12-18T13:34:25.177Z", "content": "well this makes sense! Still to start using it at the same time sounds strange ...
", "rawMarkdown": "well this makes sense! Still to start using it at the same time sounds strange ... ", "votes": 4 }, { "id": 698482, "postDate": "2019-12-19T09:58:57.517Z", "content": "It might because of the leak data usage, I was also thinking should I use leakage or not in the last 1-2 days.
", "rawMarkdown": "It might because of the leak data usage, I was also thinking should I use leakage or not in the last 1-2 days." }, { "id": 698486, "postDate": "2019-12-19T10:03:33.270Z", "content": "yes this is a good point ... will see what happens in few days .. ;)
", "rawMarkdown": "yes this is a good point ... will see what happens in few days .. ;)" } ] }, { "id": 697863, "postDate": "2019-12-18T13:51:34.360Z", "content": "Yesterday, we were 9th with 0.938 score. Are you sure that it happened yesterday or it happened in several hours? If it is within 1-2 days, 8th to 17th is normal considering that it is the time which people start to blend.
\n\nOn the other hand, there are some suspicious jumps but Kaggle admins usually don't consider private sharing as cheating unfortunately.
", "rawMarkdown": "Yesterday, we were 9th with 0.938 score. Are you sure that it happened yesterday or it happened in several hours? If it is within 1-2 days, 8th to 17th is normal considering that it is the time which people start to blend.\n\nOn the other hand, there are some suspicious jumps but Kaggle admins usually don't consider private sharing as cheating unfortunately.", "votes": 2, "replies": [ { "id": 697869, "postDate": "2019-12-18T13:54:53.590Z", "content": "it's happened if few hours and now it is stable again ... this is a 2 submission per day challenge it takes more time to lose positions ... normally you keep your position for at least one day ... while yesterday was like two positions per hour ...
", "rawMarkdown": "it's happened if few hours and now it is stable again ... this is a 2 submission per day challenge it takes more time to lose positions ... normally you keep your position for at least one day ... while yesterday was like two positions per hour ... ", "votes": 3 }, { "id": 697891, "postDate": "2019-12-18T14:23:00.460Z", "content": "\n\n\nAhmet Erdem wrote:\n On the other hand, there are some suspicious jumps but Kaggle admins usually don't consider private sharing as cheating unfortunately.
\n
Yeah I saw many topics raising about private sharing solutions during competitions (like for instance this one and of course the famous X5 ) but it seems kaggle do nothing to punish it.
\n\nThey also need to check solutions for more than only those on prize zone.
", "rawMarkdown": "> **Ahmet Erdem wrote:**\n> On the other hand, there are some suspicious jumps but Kaggle admins usually don't consider private sharing as cheating unfortunately.\n\nYeah I saw many topics raising about private sharing solutions during competitions (like for instance [this one](https://www.kaggle.com/c/ieee-fraud-detection/discussion/110778) and of course the famous X5 ) but it seems kaggle do nothing to punish it.\n\nThey also need to check solutions for more than only those on prize zone. \n", "votes": 6 }, { "id": 697919, "postDate": "2019-12-18T14:51:41.593Z", "content": "for example today we lost 1 position ... for now ... but not 5,6 or 9 whatever it was yesterday ....
", "rawMarkdown": "for example today we lost 1 position ... for now ... but not 5,6 or 9 whatever it was yesterday ....", "votes": 2 }, { "id": 697928, "postDate": "2019-12-18T14:58:50.700Z", "content": "There is another pattern that I observe on Kaggle: Monday effect. People usually focus on their models during whole weekend and run over night and submit on Monday. But this doesn't apply to this case since the day you talk about was Tuesday.
\n\nI guess there will be even more jumps tomorrow. But the question is if our current rankings are even similar to the future private LB rankings.
", "rawMarkdown": "There is another pattern that I observe on Kaggle: Monday effect. People usually focus on their models during whole weekend and run over night and submit on Monday. But this doesn't apply to this case since the day you talk about was Tuesday.\n\nI guess there will be even more jumps tomorrow. But the question is if our current rankings are even similar to the future private LB rankings.", "votes": 1 }, { "id": 697932, "postDate": "2019-12-18T15:03:16.070Z", "content": "yes, that makes sense ... in general we have to consider that this is a 2 submission per day challenge ... if you have 5 submission per day you can simply try stuff and be lucky ... but with 2 submission per day is different ... anyway will see what happens ... ;)
", "rawMarkdown": "yes, that makes sense ... in general we have to consider that this is a 2 submission per day challenge ... if you have 5 submission per day you can simply try stuff and be lucky ... but with 2 submission per day is different ... anyway will see what happens ... ;)", "votes": 1 } ] }, { "id": 697951, "postDate": "2019-12-18T15:25:02.950Z", "content": "I must have been some change to the public leaderboard.. I observed this around a week ago. I suspect Kaggle moved some of the private LB dataset to the public LB.
\n\nIt now says:\nThis leaderboard is calculated with approximately 78% of the test data.\nThe final results will be based on the other 22%, so the final standings may be different.
\n\nNot sure what was before but 22% remaining set seems low.
", "rawMarkdown": "I must have been some change to the public leaderboard.. I observed this around a week ago. I suspect Kaggle moved some of the private LB dataset to the public LB.\n\nIt now says:\nThis leaderboard is calculated with approximately 78% of the test data.\nThe final results will be based on the other 22%, so the final standings may be different.\n\nNot sure what was before but 22% remaining set seems low.", "replies": [ { "id": 697970, "postDate": "2019-12-18T15:57:23.170Z", "content": "Interesting, maybe they moved all shared external data (leak) into public test data. If so then the shakeup can be huge.
", "rawMarkdown": "Interesting, maybe they moved all shared external data (leak) into public test data. If so then the shakeup can be huge.", "votes": 5 }, { "id": 697994, "postDate": "2019-12-18T16:36:10Z", "content": "@cpmpml - that definitely sounds like a way to tackle partial leaks in the data (like in this competition). If they did move all the leaked data to Public test data and for the teams that did gain positions because of this - would they be more susceptible to suffer from the shakeup (since their submissions have more overlap with the leaked data)?
", "rawMarkdown": "@cpmpml - that definitely sounds like a way to tackle partial leaks in the data (like in this competition). If they did move all the leaked data to Public test data and for the teams that did gain positions because of this - would they be more susceptible to suffer from the shakeup (since their submissions have more overlap with the leaked data)? ", "votes": 1 }, { "id": 698006, "postDate": "2019-12-18T16:55:35.063Z", "content": "This is definitely not the case. It was 22% before and no submission is rescored.
", "rawMarkdown": "This is definitely not the case. It was 22% before and no submission is rescored.", "votes": 7 }, { "id": 698062, "postDate": "2019-12-18T18:59:24.503Z", "content": "They did say that they will remove all public leaks from private test data. This should show at some point.
", "rawMarkdown": "They did say that they will remove all public leaks from private test data. This should show at some point.", "votes": 2 }, { "id": 699380, "postDate": "2019-12-20T11:23:09.020Z", "content": "It shows, with private test data at 11% now instead of 22% initially ;)
", "rawMarkdown": "It shows, with private test data at 11% now instead of 22% initially ;)", "votes": -1 } ] }, { "id": 698748, "postDate": "2019-12-19T16:53:31.947Z", "content": "Some teams have achieved a high score with few submissions, in some cases this competitors have new Kaggle accounts.
", "rawMarkdown": "Some teams have achieved a high score with few submissions, in some cases this competitors have new Kaggle accounts." }, { "id": 699218, "postDate": "2019-12-20T07:22:05.117Z", "rawMarkdown": "", "isDeleted": true }, { "id": 698424, "postDate": "2019-12-19T08:01:44.360Z", "rawMarkdown": "", "isDeleted": true }, { "id": 698016, "postDate": "2019-12-18T17:13:56.307Z", "rawMarkdown": "", "votes": 5, "isDeleted": true } ], "comments": [ { "id": 698035, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2019-12-18T17:58:22.547000", "content": "Another team jumped to the 37th place with 4 submissions. Check this thread and you will understand what is going on right now.\nhttps://www.kaggle.com/c/ieee-fraud-detection/discussion/110778
", "votes": 12, "replies": [ { "id": 698054, "author_name": "Changyi", "author_url": "", "post_date": "2019-12-18T18:36:33.370000", "content": "By checking the top 100 team (not single participant) on the LB which have few (less than 10) submits, I think you might be right.
", "votes": 3, "replies": [] }, { "id": 698513, "author_name": "Jie Lu", "author_url": "", "post_date": "2019-12-19T10:33:42.863000", "content": "I see another one in top 100 who used only 4 submissions and that account was created 9 days ago.\nLegendary.
", "votes": 3, "replies": [] } ] }, { "id": 698098, "author_name": "ysuzuki", "author_url": "", "post_date": "2019-12-18T19:57:56.880000", "content": "We lost close to 150 positions in the last week.
\nI'm going out of the medal......
Haha, I feel you...Just went out of the medals the last day...Have dropped more than 100 positions in the last 3 days without any1 posting any better-scoring kernels. C'est la vie! (btw I m not french, I hope I got it right)
", "votes": 1, "replies": [] }, { "id": 699087, "author_name": "ysuzuki", "author_url": "", "post_date": "2019-12-20T03:26:28.197000", "content": "\bYes. Faisons de notre mieux!! (btw I m not french too, I hope I got it right)
", "votes": 1, "replies": [] }, { "id": 699264, "author_name": "Georgios Sarantitis", "author_url": "", "post_date": "2019-12-20T08:48:55.850000", "content": "\n\n\nFaisons de notre mieux!
\n
Lol, yes we are actually exactly at the same place! Let's see where we both end up!
", "votes": 1, "replies": [] } ] }, { "id": 698224, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2019-12-19T00:45:47.453000", "content": "We lost close to 70 position in last few days. I am certain that we missed the trick.
", "votes": 3, "replies": [ { "id": 698639, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2019-12-19T13:55:09.990000", "content": "Yesterday night i made improvements by 7 positions and this morning we slipped again by 16 positions. There is something very strange
", "votes": 3, "replies": [] }, { "id": 698655, "author_name": "Changyi", "author_url": "", "post_date": "2019-12-19T14:17:34.757000", "content": "I agree. Thanks to your work I was between 60 and 70 yesterday but now I am out of 100. The only high score published recently is the R code \"ASHRAE 2*lightGBM without Leak Data\" but I got no improvement blending with that...
", "votes": 1, "replies": [] }, { "id": 698794, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2019-12-19T17:39:43.467000", "content": "Slipped by another 20 in 3 hrs. ☹️
", "votes": 1, "replies": [] }, { "id": 698893, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2019-12-19T20:44:27.007000", "content": "ASHRAE 2*lightGBM without Leak Data in R is a last minute submission. I guess people are blending a lot with that model to improve their scores.
", "votes": 1, "replies": [] } ] }, { "id": 697838, "author_name": "Stanislav Blinov", "author_url": "", "post_date": "2019-12-18T13:33:33.827000", "content": "Yesterday was 3 days before deadline. It could be a coincedence - if many teams at once started to blend final submissions. But also it could be not.\nWhat I noticed - @reneiw jumped in the green zone with just 11 submissions. Might be that he's very talented and just didn't made any submissions. A lot of scientists work hard for a couple final results and don't use every submission for checking along the way. Just no history at all on the 3-years-old account is a bit suspicious. But I do not call names and give labels.\nAnyway, let's not start witch hunting and hope that Kaggle algorithms will do the work.
", "votes": 3, "replies": [ { "id": 697849, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2019-12-18T13:41:03.530000", "content": "I don't think it is possible to place in such high rank with that many submissions in this competition. My experiments costed me at least 60 submissions.
", "votes": 7, "replies": [] }, { "id": 697855, "author_name": "Pietro Marinelli", "author_url": "", "post_date": "2019-12-18T13:45:39.330000", "content": "yes ... I mean it can happen but in this competition you cannot rely only on local validation you do need to submit to understand better what is the best strategy etc ....
", "votes": 6, "replies": [] }, { "id": 697859, "author_name": "Stanislav Blinov", "author_url": "", "post_date": "2019-12-18T13:47:34.357000", "content": "@gunesevitan it's your experience, and mine too. But maybe this guy has a great experience, how could we know?\nIt's been known Uncle @cpmpml jumped in the gold zone with the first submission, for example.
", "votes": 1, "replies": [] }, { "id": 698092, "author_name": "Manuel Campos", "author_url": "", "post_date": "2019-12-18T19:43:07.183000", "content": "There are several users of the wiener family (reneiw upside down) also 3 years old without any activity. Will they all be a cluster of magic accounts?
", "votes": 5, "replies": [] }, { "id": 698094, "author_name": "Stanislav Blinov", "author_url": "", "post_date": "2019-12-18T19:47:11.357000", "content": "Ok now I agree that it's fishy.
", "votes": 2, "replies": [] }, { "id": 698764, "author_name": "reneiw", "author_url": "", "post_date": "2019-12-19T17:11:21.827000", "content": "interesting, reneiw's name become evidence of conspiracy, i started using reneiw ten years ago when starting my study in control engineering, difficult to imagine what Wiener would think. btw, curious about the glorious deeds of the wiener family...
\n\n3 years old without any activity is can't be more normal for me, i dislike pure competition at all, even join in this competition is due to highly related to my main work but not for competition itself, evaluation of building energy saving efficiency is a real problem and simultaneously a hard problem, in my experience even the AB test split by time is sometimes difficult to correctly evaluate the energy saving effect due to many non-control non-constant factors, evaluation based on the environment variables is an important direction, the huge amount of data in the competition can be beneficial to my work research.
\n\nif just to explain the two questions above, i think I have no reason to make this explanation, the main reason for typing so many words to reply is, the reason your guys to doubt is reach high score with few commits, this can't stand, isn't this really reasonable performance? Do you think we should get high scores through a large number of submissions to be healthy? seems most people think so, that's also the reason I don't like competitions. No one has the energy to try a hundred models or methods, it is nothing more than taking a small number of methods to test online and then constantly using small tricks to correct and then test again, full of guess guess guess collision collision collision ensemble ensemble ensemble routine, take so much time and effort and commits, no real industrial value was created in the end, just to cater to this batch of test sets. Obtaining relatively good and stable results through a concise method originally does not need to submit tests so much times, if get the right method, 20% time can reach the front row, and the remain 80% time is spent from the front row to the top, which industrial added value is relatively small and unstable, i was lucky to find a concise way to complete the first step. With all due respect, except for pure learning purposes, if your 50 submissions are not as good as 20 of others, give up early, if you still want to win, find other competitions to try, there are always competitions where you can come up with better solutions.
\n\nToo much pessimistic talk about the phenomenon of competition, turn back to this competition now, actually the current score doesn't explain anything, i used leak data in depth, and the rapid improvement in results is also due to leak data, this very likely overfitting to the leak data or better than other people who have no use for leaking data Is an illusion, i mainly want to share the interesting thing that why suddenly decided to use the leak data in depth(for row level train), because i realized that this leak data is not a true leak, it has practical useable industry value. It is difficult to correctly evaluate the energy impact of a building renovation based on historical data alone, because energy use may change by time even if the building is without any renovation, but if there are buildings that definitely known without any renovation, and their energy data can help to predict other buildings energy use in the same period if they have no renovation also, then we can more accurately to evaluate the energy effect of building renovation, and the leak data of this competition act as the role of buildings data definitely known without renovation for evaluation period.
", "votes": 3, "replies": [] }, { "id": 698781, "author_name": "", "author_url": "", "post_date": "2019-12-19T17:26:54.867000", "content": "", "votes": -4, "replies": [] }, { "id": 698815, "author_name": "reneiw", "author_url": "", "post_date": "2019-12-19T18:07:33.563000", "content": "Pay attention to your words, i have no responsibility to \"explain\" for 100 Chinese novices, I can directly tell you that there are two accounts associated with me, one of my own, and one of my colleagues, the remaining 98 don't look for me. my colleague started early then me and i actually started modeling this week begin with use some organized data from my colleague about his 10th submission, then we model separately from this point, yes, you read it right, i use colleague‘s initial data but did not merge team with him, if foul delete all my data and grades, i have no interest in studying kaggle data usage rules, hopefully deleting my account will allow you to enter money region.
", "votes": 2, "replies": [] }, { "id": 698821, "author_name": "CPMP", "author_url": "", "post_date": "2019-12-19T18:17:13.570000", "content": "Disclaimer: I have not entered this competition, not even looked at data. I just want to react to what looks like a witch hunt.
\n\n\n\n> Do you think we should get high scores through a large number of submissions to be healthy?
\n\nYou're right questioning this. There are famous examples of people jumping high in LB with 1 submission, including the famous @speculation win with 1 sub only. Small number of subs plus good LB rank aren't suspicious in principle.
\n\n> i use colleague‘s initial data but did not merge team with him
\n\nWell, that's against competition rules. But you know it given you write:
\n\n> i have no interest in studying kaggle data usage rules, hopefully deleting my account will allow you to enter money region.
\n\nIf you know how it will end then why did you enter? If it is only to see how your model(s) will fare on private test data, issue is that you won't see your submissions at all if you're removed. I know because this happened to me after my team was removed because a team member broke some rules and was caught.
", "votes": 9, "replies": [] }, { "id": 698826, "author_name": "reneiw", "author_url": "", "post_date": "2019-12-19T18:30:23.433000", "content": "Because I only have time to realy started modeling in this last 3 days, if merged team will take up the submission opportunities of my colleague for the last few days, so I decide to keep my own account \nwithout in-depth investigation of kaggle rules last week. can't see the private score is really a problem.
", "votes": -2, "replies": [] }, { "id": 698827, "author_name": "tz", "author_url": "", "post_date": "2019-12-19T18:31:46.773000", "content": "", "votes": 1, "replies": [] }, { "id": 698832, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2019-12-19T18:40:23.160000", "content": "I don't care about the money. I wasn't accusing you with anything at all. I just asked about your opinion about this situation, but you misinterpreted my question. Actually, I wasn't expecting you to do private sharing since you are at the very top, but you just said it yourself.
", "votes": 1, "replies": [] }, { "id": 698840, "author_name": "Hanjoon Choe", "author_url": "", "post_date": "2019-12-19T18:58:28.067000", "content": "Beside the main topic, I hope you post your solution and how you used leak data for predictions. would be interesting and good to study ;-) Thanks for your long explanation btw.
", "votes": 1, "replies": [] }, { "id": 698853, "author_name": "reneiw", "author_url": "", "post_date": "2019-12-19T19:18:38.433000", "content": "emmm, looks like there are differences in language habits... It is a pity that I have no opinion on this, i have not followed user details for leaderboard.
\n\nI think private sharing is fine since the competition is almost over so does not substantially affect others, hope to see the private list first and then deal with foul, help me to verify whether it is stable to use some buildings data to help predict other buildings. Unfortunately it seems even the highest online score is too big so not applicable at all in real world, in actual projects, even if more precise data is used to make more accurate prediction models, customers still choose AB test
", "votes": -5, "replies": [] }, { "id": 698877, "author_name": "CPMP", "author_url": "", "post_date": "2019-12-19T19:56:29.350000", "content": "\n\n\nBecause I only have time to realy started modeling in this last 3 days, if merged team will take up the submission opportunities of my colleague for the last few days, so I decide to keep my own account without in-depth investigation of kaggle rules last week
\n
You realize that doing this will also result in your colleague removal as he shared privately with you?
", "votes": 4, "replies": [] }, { "id": 698879, "author_name": "CPMP", "author_url": "", "post_date": "2019-12-19T19:58:27.707000", "content": "\n\n\nI think private sharing is fine
\n
No, private sharing is never fine, it is unfair and it is against the rules.
", "votes": 11, "replies": [] }, { "id": 698888, "author_name": "Stanislav Blinov", "author_url": "", "post_date": "2019-12-19T20:32:24.073000", "content": "\n\n\nI think private sharing is fine
\n \ni have no interest in studying kaggle data usage rules
\n
You come to the community @ you follow the rules, mr. Toxic.
", "votes": 0, "replies": [] }, { "id": 699041, "author_name": "reneiw", "author_url": "", "post_date": "2019-12-20T02:10:01.973000", "content": "you misunderstand what I mean, rules are of course important for platform operations, \"I think private sharing is fine\" means i think this is fine when i write this comment when all our submissions are over, not in the competition progress, i misunderstood the comment above means that in my repost body said the use of leak data to improve performance belongs to private sharing... I did n’t share any method or data with anyone during the competition procedure include my colleague, otherwise he can get good grades too.
\n\n\n\n\nYou realize that doing this will also result in your colleague removal as he shared privately with you?
\n
this really don't know, and he didn't actually do anything to shair data for me, we are in different city branches and no daily sharing, just when i start modeling i don’t have time to rearrange large amounts of competition data especially leak data so i download colleague's collated data set directly from company server which he use, he didn't get any benefit from my actions or purposely to private help me, it's my fault if it makes him foul too, fortunately his grades are not very good thus don't hurt by me too much.
", "votes": 1, "replies": [] }, { "id": 699058, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2019-12-20T02:30:17.963000", "content": "Interesting. You can set anywhere as your location in your profile, it doesn't mean anything. Cheaters should be removed, that's definitely true, but others (from the same Country) are innocent. One (or a group of) people cannot represent the whole Country, correct your words please.
", "votes": 6, "replies": [] }, { "id": 699077, "author_name": "Peiyuan Liao", "author_url": "", "post_date": "2019-12-20T03:07:59.563000", "content": "@gunesevitan I don't think words that promote outlandish discrimination are welcomed in this community. Certainly made me feel uncomfortable.
", "votes": 6, "replies": [] }, { "id": 699128, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2019-12-20T04:27:36.647000", "content": "@yiheng @alexanderliao \nI wasn't trying to make any discrimination. Sorry for making you feel uncomfortable. I know there are innocent people among those top teams. I'm not accusing everyone with cheating. \nI thought having too much novices with very few submissions and all of them being from the same country is a little bit weird. I have no intentions of discriminating anyone.
", "votes": 2, "replies": [] }, { "id": 699375, "author_name": "reneiw", "author_url": "", "post_date": "2019-12-20T11:14:46.257000", "content": "some unexpected things happened this week cause some trouble to others including myself, since i am a passer of kaggle and maybe disappear another three years from next week so will not see the discussion later, i think i should make some notes lastly,
\n\nI did use data from colleague which can't access by other participate, and it ’s very important for me to get a high score in few days, he spent lot of effort collecting, cleaning, and calibrating competition-related data on forums, this allows me to skip a lot of preprocessing work and go straight to the modeling process. This violates the rules and i am happy to accept punishment to clear all my results. This is my unilateral mistake, hope this will not affect others.
Although using other's initial data can greatly reduce the time spent, it's different from submission counts, this is the key reason I decided to reply, it ’s healthy to get high scores with a small number of submissions, relying on a large number of submissions to get high scores is usually because did not find a suitable method or did not use the same level data (such as this leak data).
Back to the technical problem itself, @hanjoonchoe @chabir , the key issue to using leak data to improve prediction performance is transfer information from some buildings to some others, so learning of leak data cannot rely on building ids in categorical state, i have tried several methods, such as learning training data using neural networks(thus learn the building id embedding simultaneously), and then fine-tune on leak data by freeze the embedding to force learn the new years characteristic by other environmental features; or extract the embedding of building id in nn to use as features substitute categorical buliding id in normal lgbm models; or use lgbm to learning on training data and then ignore the building id feature to learn another model to fit the gap between lgbm prediction value and the leak value. sorry for i can't tell you which one performs better in the end, because for each method i didn't do it fine enough, and I don't think I have solved this technical problem, i don't want to mislead you, believe other excellent groups will give more interesting methods and conclusions.
\n\n\ni download colleague's collated data set directly from company server
\n
This is private sharing of data.
", "votes": 2, "replies": [] } ] }, { "id": 698437, "author_name": "Linyan", "author_url": "", "post_date": "2019-12-19T08:20:31.230000", "content": "I also notice that someone just join kaggle for few days and their submissions is lower than 10,but their ranks is in 100,this makes me too confuse
", "votes": 4, "replies": [] }, { "id": 697921, "author_name": "Manoj Prabhakar", "author_url": "", "post_date": "2019-12-18T14:54:01.630000", "content": "We lost close to 67 positions in the last 3 days. We were 54 when this week started, currently at 121st.. Not even a single discussion thread on the improvement..
", "votes": 4, "replies": [ { "id": 697924, "author_name": "Pietro Marinelli", "author_url": "", "post_date": "2019-12-18T14:57:13.400000", "content": "wow, this is quite strange as well ... don't you think so?
", "votes": 2, "replies": [] }, { "id": 697926, "author_name": "Pietro Marinelli", "author_url": "", "post_date": "2019-12-18T14:57:59.900000", "content": "how many position on average did you loose during the competition .. I mean given a position how long would it last without submitting? at least one or two days ?
", "votes": 1, "replies": [] }, { "id": 698210, "author_name": "Manoj Prabhakar", "author_url": "", "post_date": "2019-12-19T00:00:05.497000", "content": "Hardly one in 2 days even if we don't submit anything. But this time around, everyday on an average 15 to 25 spots down. There are so many particpants with very less submissions in top 100... That seems to be more strange as well..
", "votes": 2, "replies": [] } ] }, { "id": 697887, "author_name": "sandy1112", "author_url": "", "post_date": "2019-12-18T14:19:21.510000", "content": "maybe some teams found an additional last minute leak?? not sure...will wait and see what happens when Pvt LB is up.
", "votes": 4, "replies": [ { "id": 697933, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2019-12-18T15:07:03.430000", "content": "It's either that or private sharing.
", "votes": 5, "replies": [] }, { "id": 697936, "author_name": "Pietro Marinelli", "author_url": "", "post_date": "2019-12-18T15:10:00.290000", "content": "they should share it don't they?
", "votes": 1, "replies": [] }, { "id": 697955, "author_name": "sandy1112", "author_url": "", "post_date": "2019-12-18T15:27:10.017000", "content": "@pietromarinelli I would not bet on everyone sharing a last minute leak. @gunesevitan - agreed it could be private sharing, but seems like a risky move on the last two days of competition.
", "votes": 1, "replies": [] } ] }, { "id": 697876, "author_name": "Oleg Knaub", "author_url": "", "post_date": "2019-12-18T14:02:05.570000", "content": "Please do not submit high scoring predictions :)
", "votes": 1, "replies": [] }, { "id": 698912, "author_name": "eagle4", "author_url": "", "post_date": "2019-12-19T21:55:47.727000", "content": "@reneiw : \nbeside the fact that you could have dragged your colleague in the money, can you explain more about your overall method. I understand that you build a method using the leak data to calculate the behaviour of individual buildings over 3 years: may I ask how you did that ? did you calculate trends, find particular patterns, set up particular feature engineering ?
", "votes": 1, "replies": [] }, { "id": 698229, "author_name": "Ado Dream", "author_url": "", "post_date": "2019-12-19T01:06:26.543000", "content": "Lost close to 50 position while score improved by 0.004 in a week. It's really impressive to see the guys who solve the problem within 20 commits. Tired of overfitting to keep rankings anyway.
", "votes": 1, "replies": [] }, { "id": 697833, "author_name": "tz", "author_url": "", "post_date": "2019-12-18T13:29:02.667000", "content": "One idea might be that people that were previously not using any leak data at all, are now submitting their models using leak data. Thus jumping a lot of places at one time. However, you are right 9 positions in the top 20 looks a bit suspicious.
", "votes": 1, "replies": [ { "id": 697839, "author_name": "Pietro Marinelli", "author_url": "", "post_date": "2019-12-18T13:34:25.177000", "content": "well this makes sense! Still to start using it at the same time sounds strange ...
", "votes": 4, "replies": [] }, { "id": 698482, "author_name": "Jie Wu", "author_url": "", "post_date": "2019-12-19T09:58:57.517000", "content": "It might because of the leak data usage, I was also thinking should I use leakage or not in the last 1-2 days.
", "votes": 0, "replies": [] }, { "id": 698486, "author_name": "Pietro Marinelli", "author_url": "", "post_date": "2019-12-19T10:03:33.270000", "content": "yes this is a good point ... will see what happens in few days .. ;)
", "votes": 0, "replies": [] } ] }, { "id": 697863, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2019-12-18T13:51:34.360000", "content": "Yesterday, we were 9th with 0.938 score. Are you sure that it happened yesterday or it happened in several hours? If it is within 1-2 days, 8th to 17th is normal considering that it is the time which people start to blend.
\n\nOn the other hand, there are some suspicious jumps but Kaggle admins usually don't consider private sharing as cheating unfortunately.
", "votes": 2, "replies": [ { "id": 697869, "author_name": "Pietro Marinelli", "author_url": "", "post_date": "2019-12-18T13:54:53.590000", "content": "it's happened if few hours and now it is stable again ... this is a 2 submission per day challenge it takes more time to lose positions ... normally you keep your position for at least one day ... while yesterday was like two positions per hour ...
", "votes": 3, "replies": [] }, { "id": 697891, "author_name": "Serigne ", "author_url": "", "post_date": "2019-12-18T14:23:00.460000", "content": "\n\n\nAhmet Erdem wrote:\n On the other hand, there are some suspicious jumps but Kaggle admins usually don't consider private sharing as cheating unfortunately.
\n
Yeah I saw many topics raising about private sharing solutions during competitions (like for instance this one and of course the famous X5 ) but it seems kaggle do nothing to punish it.
\n\nThey also need to check solutions for more than only those on prize zone.
", "votes": 6, "replies": [] }, { "id": 697919, "author_name": "Pietro Marinelli", "author_url": "", "post_date": "2019-12-18T14:51:41.593000", "content": "for example today we lost 1 position ... for now ... but not 5,6 or 9 whatever it was yesterday ....
", "votes": 2, "replies": [] }, { "id": 697928, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2019-12-18T14:58:50.700000", "content": "There is another pattern that I observe on Kaggle: Monday effect. People usually focus on their models during whole weekend and run over night and submit on Monday. But this doesn't apply to this case since the day you talk about was Tuesday.
\n\nI guess there will be even more jumps tomorrow. But the question is if our current rankings are even similar to the future private LB rankings.
", "votes": 1, "replies": [] }, { "id": 697932, "author_name": "Pietro Marinelli", "author_url": "", "post_date": "2019-12-18T15:03:16.070000", "content": "yes, that makes sense ... in general we have to consider that this is a 2 submission per day challenge ... if you have 5 submission per day you can simply try stuff and be lucky ... but with 2 submission per day is different ... anyway will see what happens ... ;)
", "votes": 1, "replies": [] } ] }, { "id": 697951, "author_name": "Jonathan Mallia", "author_url": "", "post_date": "2019-12-18T15:25:02.950000", "content": "I must have been some change to the public leaderboard.. I observed this around a week ago. I suspect Kaggle moved some of the private LB dataset to the public LB.
\n\nIt now says:\nThis leaderboard is calculated with approximately 78% of the test data.\nThe final results will be based on the other 22%, so the final standings may be different.
\n\nNot sure what was before but 22% remaining set seems low.
", "votes": 0, "replies": [ { "id": 697970, "author_name": "CPMP", "author_url": "", "post_date": "2019-12-18T15:57:23.170000", "content": "Interesting, maybe they moved all shared external data (leak) into public test data. If so then the shakeup can be huge.
", "votes": 5, "replies": [] }, { "id": 697994, "author_name": "sandy1112", "author_url": "", "post_date": "2019-12-18T16:36:10", "content": "@cpmpml - that definitely sounds like a way to tackle partial leaks in the data (like in this competition). If they did move all the leaked data to Public test data and for the teams that did gain positions because of this - would they be more susceptible to suffer from the shakeup (since their submissions have more overlap with the leaked data)?
", "votes": 1, "replies": [] }, { "id": 698006, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2019-12-18T16:55:35.063000", "content": "This is definitely not the case. It was 22% before and no submission is rescored.
", "votes": 7, "replies": [] }, { "id": 698062, "author_name": "CPMP", "author_url": "", "post_date": "2019-12-18T18:59:24.503000", "content": "They did say that they will remove all public leaks from private test data. This should show at some point.
", "votes": 2, "replies": [] }, { "id": 699380, "author_name": "CPMP", "author_url": "", "post_date": "2019-12-20T11:23:09.020000", "content": "It shows, with private test data at 11% now instead of 22% initially ;)
", "votes": -1, "replies": [] } ] }, { "id": 698748, "author_name": "Alan Clappis", "author_url": "", "post_date": "2019-12-19T16:53:31.947000", "content": "Some teams have achieved a high score with few submissions, in some cases this competitors have new Kaggle accounts.
", "votes": 0, "replies": [] }, { "id": 699218, "author_name": "", "author_url": "", "post_date": "2019-12-20T07:22:05.117000", "content": "", "votes": 0, "replies": [] }, { "id": 698424, "author_name": "", "author_url": "", "post_date": "2019-12-19T08:01:44.360000", "content": "", "votes": 0, "replies": [] }, { "id": 698016, "author_name": "", "author_url": "", "post_date": "2019-12-18T17:13:56.307000", "content": "", "votes": 5, "replies": [] } ], "raw_markdown_by_id": { "697824": "Yesteday suddenly our team lost 11 positions, from 8th to 17th, in just a few hours. \n\nWell, it may happen in a competition that your team is overcome by some smarter or luckier teams, but I've never seen such a imponent moving all in a sudden (and resulting in almost the same score for many of the teams involved) in all my 8 year Kaggle experience.\n\nSomething should be going on and we are completely unaware of it. Yet, we checked the discussion board and the notebooks for any hint, but we found nothing, zero. Sincerely we suspect someone did private sharing of ideas, code or submissions, I do not fear of hinting openly at that. \n\nThis has been a misfortunated competition, but as a team we held on until the end and working hard up so far because we were sincerely interested in the topic. But now the competition is ending in an even worse way than we expected.\n\n@inversion @addisonhoward @sohier I hope you will enforce strict controls on the leaderboard, after this competition has ended. Thank you a lot.", "698035": "Another team jumped to the 37th place with 4 submissions. Check this thread and you will understand what is going on right now.\nhttps://www.kaggle.com/c/ieee-fraud-detection/discussion/110778", "698098": "We lost close to 150 positions in the last week. \nI'm going out of the medal......", "698224": "We lost close to 70 position in last few days. I am certain that we missed the trick.", "697838": "Yesterday was 3 days before deadline. It could be a coincedence - if many teams at once started to blend final submissions. But also it could be not.\nWhat I noticed - @reneiw jumped in the green zone with just 11 submissions. Might be that he's very talented and just didn't made any submissions. A lot of scientists work hard for a couple final results and don't use every submission for checking along the way. Just no history at all on the 3-years-old account is a bit suspicious. But I do not call names and give labels.\nAnyway, let's not start witch hunting and hope that Kaggle algorithms will do the work.", "698437": "I also notice that someone just join kaggle for few days and their submissions is lower than 10,but their ranks is in 100,this makes me too confuse\n", "697921": "We lost close to 67 positions in the last 3 days. We were 54 when this week started, currently at 121st.. Not even a single discussion thread on the improvement.. ", "697887": "maybe some teams found an additional last minute leak?? not sure...will wait and see what happens when Pvt LB is up. ", "697876": "Please do not submit high scoring predictions :)", "698912": "@reneiw : \nbeside the fact that you could have dragged your colleague in the money, can you explain more about your overall method. I understand that you build a method using the leak data to calculate the behaviour of individual buildings over 3 years: may I ask how you did that ? did you calculate trends, find particular patterns, set up particular feature engineering ?\n", "698229": "Lost close to 50 position while score improved by 0.004 in a week. It's really impressive to see the guys who solve the problem within 20 commits. Tired of overfitting to keep rankings anyway.", "697833": "One idea might be that people that were previously not using any leak data at all, are now submitting their models using leak data. Thus jumping a lot of places at one time. However, you are right 9 positions in the top 20 looks a bit suspicious.", "697863": "Yesterday, we were 9th with 0.938 score. Are you sure that it happened yesterday or it happened in several hours? If it is within 1-2 days, 8th to 17th is normal considering that it is the time which people start to blend.\n\nOn the other hand, there are some suspicious jumps but Kaggle admins usually don't consider private sharing as cheating unfortunately.", "697951": "I must have been some change to the public leaderboard.. I observed this around a week ago. I suspect Kaggle moved some of the private LB dataset to the public LB.\n\nIt now says:\nThis leaderboard is calculated with approximately 78% of the test data.\nThe final results will be based on the other 22%, so the final standings may be different.\n\nNot sure what was before but 22% remaining set seems low.", "698748": "Some teams have achieved a high score with few submissions, in some cases this competitors have new Kaggle accounts.", "699218": "", "698424": "", "698016": "" } }