Can we stop making high-scoring \"inference\" kernels public? (not starters, they are great and help beginners like me)\nIt just feels unfair that some people submit other people's model weights and place higher than people who are trying their best. It's quite depressing.\nCan we work around this anyhow?
\n\nThere are currently ~38 people on top of the leaderboard having the same score as highest scoring public inference kernel (in some cases this might be a coincidence)
\n\nNo offense to anyone.
\n\nThanks!
\n\nEDIT:\nI have not downvoted anyone's comments and also this post is not to target any individuals. It is just a humble request. I felt this in the last two competitions (this one and Bengali.AI and that's why shared it). If this section is not for such discussions, please let me know, I'll delete the post.
", "messages": [ { "id": "856409", "postDate": "05/21/2020 17:30:07", "content": "Can we stop making high-scoring \"inference\" kernels public? (not starters, they are great and help beginners like me)\nIt just feels unfair that some people submit other people's model weights and place higher than people who are trying their best. It's quite depressing.\nCan we work around this anyhow?
\n\nThere are currently ~38 people on top of the leaderboard having the same score as highest scoring public inference kernel (in some cases this might be a coincidence)
\n\nNo offense to anyone.
\n\nThanks!
\n\nEDIT:\nI have not downvoted anyone's comments and also this post is not to target any individuals. It is just a humble request. I felt this in the last two competitions (this one and Bengali.AI and that's why shared it). If this section is not for such discussions, please let me know, I'll delete the post.
", "rawMarkdown": "Can we stop making high-scoring \"inference\" kernels public? (not starters, they are great and help beginners like me)\nIt just feels unfair that some people submit other people's model weights and place higher than people who are trying their best. It's quite depressing.\nCan we work around this anyhow?\n\nThere are currently ~38 people on top of the leaderboard having the same score as highest scoring public inference kernel (in some cases this might be a coincidence)\n\nNo offense to anyone.\n\nThanks!\n\n**EDIT:**\nI have not downvoted anyone's comments and also this post is not to target any individuals. It is just a humble request. I felt this in the last two competitions (this one and Bengali.AI and that's why shared it). If this section is not for such discussions, please let me know, I'll delete the post.", "votes": null }, { "id": "856432", "postDate": "05/21/2020 17:48:00", "content": "You are right we should avoid posting high scoring inference kernels, but still the competition is in its early stage, so it doesn't matter. you should focus more on improving your model, all these copycats will eventually leave behind.
", "rawMarkdown": "You are right we should avoid posting high scoring inference kernels, but still the competition is in its early stage, so it doesn't matter. you should focus more on improving your model, all these copycats will eventually leave behind.", "votes": null }, { "id": "856494", "postDate": "05/21/2020 18:44:22", "content": "Now, they on top, but this does not have matter. They don't win medal without own ideas.
", "rawMarkdown": "Now, they on top, but this does not have matter. They don't win medal without own ideas.", "votes": null }, { "id": "856632", "postDate": "05/21/2020 23:33:49", "content": "Publishing any kernel is useful for any competitor! It is your choice: make sharing or not your ideas! High score shows really good idea or not. You should enjoy the gift of fate, that you can learn new techniques, doesnt it?
\n\nEveryone has time to improve own solutions. You have 2 months.
\n\nAfter DeepFake competition I don't see any point in continuing to do any competitions. Organizers made big fault with running notebooks and I proved it! But kaggle team closed eyes on this precedent. Huge number of ideas and techniques could help anyone competitor, but I lost without a mark of participation even. I didn't break the rules.
\n\nKaggle Team needs to pay attention on quality of competitions and organizers of big contests.
\n\nI am going to continue publishing kernels, more more and more.
\n\nThank you for your patience.
", "rawMarkdown": "Publishing any kernel is useful for any competitor! It is your choice: make sharing or not your ideas! High score shows really good idea or not. You should enjoy the gift of fate, that you can learn new techniques, doesnt it?\n\nEveryone has time to improve own solutions. You have 2 months.\n\nAfter [DeepFake competition](https://www.kaggle.com/c/deepfake-detection-challenge) I don't see any point in continuing to do any competitions. Organizers made big fault with running notebooks and [I proved it](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/145982)! But kaggle team closed eyes on this precedent. Huge number of ideas and techniques could help anyone competitor, but I lost without a mark of participation even. I didn't break the rules.\n\nKaggle Team needs to pay attention on quality of competitions and organizers of big contests. \n\nI am going to continue publishing kernels, more more and more.\n\nThank you for your patience.", "votes": null }, { "id": "856641", "postDate": "05/22/2020 00:12:08", "content": "I think we should share good inference kernel. just put our models in private dataset.
", "rawMarkdown": "I think we should share good inference kernel. just put our models in private dataset.", "votes": null }, { "id": "856755", "postDate": "05/22/2020 03:40:57", "content": "Yes this is a good idea!
", "rawMarkdown": "Yes this is a good idea!", "votes": null }, { "id": "856760", "postDate": "05/22/2020 03:47:05", "content": "\"You should enjoy the gift of fate, that you can learn new techniques\": I'm just talking about inference kernels and I don't think you learn new techniques from inference kernels. Other than this, I don't understand what you are talking about as I'm bit new to this challenges. Anyways this was just a request. Thanks for the opinion. @shonenkov
", "rawMarkdown": "\"You should enjoy the gift of fate, that you can learn new techniques\": I'm just talking about inference kernels and I don't think you learn new techniques from inference kernels. Other than this, I don't understand what you are talking about as I'm bit new to this challenges. Anyways this was just a request. Thanks for the opinion. @shonenkov", "votes": null }, { "id": "856858", "postDate": "05/22/2020 05:18:18", "content": "I'm confused what happened with your notebook in the deepfake competition. You scored very high on the public LB but got 113th on the private LB?
", "rawMarkdown": "I'm confused what happened with your notebook in the deepfake competition. You scored very high on the public LB but got 113th on the private LB?", "votes": null }, { "id": "856871", "postDate": "05/22/2020 05:40:35", "content": "I 100% understand your frustration, and I agree with you, but publishing gold-level, high-scoring kernels won't solve anything. By doing this, you will only ruin the competition for a lot of people. Please do not do that. Any great idea, trick, or solution is welcome, but please do not share the trained weights.
", "rawMarkdown": "I 100% understand your frustration, and I agree with you, but publishing gold-level, high-scoring kernels won't solve anything. By doing this, you will only ruin the competition for a lot of people. Please do not do that. Any great idea, trick, or solution is welcome, but please do not share the trained weights.", "votes": null }, { "id": "856895", "postDate": "05/22/2020 06:03:23", "content": "Yes exactly! That's what I wanted to communicate with this thread!
", "rawMarkdown": "Yes exactly! That's what I wanted to communicate with this thread!", "votes": null }, { "id": "856932", "postDate": "05/22/2020 06:51:02", "content": "@matthewmasters of course NO! Facebook crushed working kernel 113 public in the private stage and kicked many participants from competition without explanation!
", "rawMarkdown": "matthewmasters of course NO! Facebook crushed working kernel 113 public in the private stage and kicked many participants from competition without explanation!", "votes": null }, { "id": "856957", "postDate": "05/22/2020 07:13:46", "content": "I don't understand why my message was downvoted!? WHY? Kaggle community, competition run less month! You should think about team spirit and use jump start for solving this competition. But you whine and think only about yourself and your positions on the leaderboard. It is disheartening, really!
\n\nMoreover, last kernel that I published, I don't share weights - I am able to listen to your requests. But you gave downvotes without reason. 👎
", "rawMarkdown": "I don't understand why [my message](https://www.kaggle.com/c/global-wheat-detection/discussion/152846#856632) was downvoted!? WHY? Kaggle community, competition run less month! You should think about team spirit and use jump start for solving this competition. But you whine and think only about yourself and your positions on the leaderboard. It is disheartening, really!\n\nMoreover, [last kernel](https://www.kaggle.com/shonenkov/oof-evaluation-efficientdet) that I published, I don't share weights - I am able to listen to your requests. But you gave downvotes without reason. 👎", "votes": null }, { "id": "856973", "postDate": "05/22/2020 07:33:14", "content": "Yeah, I don't understand why they're downvoting your reply to me either. I just looked into what happened in the FB DeepFake detection challenge and that's terrible and definitely discouraging for future competitions :(
\n\nAnyways, I appreciate all your selfless contributions to this competition! Hopefully you will have better luck here
", "rawMarkdown": "Yeah, I don't understand why they're downvoting your reply to me either. I just looked into what happened in the FB DeepFake detection challenge and that's terrible and definitely discouraging for future competitions :(\n\nAnyways, I appreciate all your selfless contributions to this competition! Hopefully you will have better luck here", "votes": null }, { "id": "857044", "postDate": "05/22/2020 08:52:23", "content": "Alex, I think your message was downvoted because of your tone (not because of what you shared): \"I am going to continue publishing kernels, more more and more.\" It feels like you don't give a shit about what other competitors asked (nicely).
I saw your latest kernel (without weights). I think that is a good way to share.
\n\nI think a Kaggle competition is like running a marathon. I ran once, and I had to train almost a year before. Before a Kaggle competition, I learned lots of things, studying hours, days, taking courses, etc. I prepared myself to compete, just like before the marathon.
\n\nI felt ready, I felt motivated. After the first few miles, my position started to decrease. That was ok. I knew there were lots of runners better than I was. Same as here, I don't care about my position, there are dozens of Kagglers better than I am.
\n\nIf a world-class Kenyan runner shares anything, I will listen 100%. Same as here, If you (or any other Kaggler) share anything useful, I will thank, and I will use that information.
\n\nBut imagine this: After the third of that marathon, that Kenyan runner started to Uber dozens of fat, lazy, untrained \"competitors\". And he said, don't worry, it is only the first third, there is plenty of time to beat them.\n(I can not say how fu**ing annoying that would be.)
\n\nMany Kagglers put lots of time, work, and effort into whatever they've achieved. By sharing high-scoring weights, you only destroy all of their motivation. The problem is the high scores (more precisely, the forks and resubmits those high scores without ANY effort), not your sharings.
\n\nSo, keep up the good work and share your thoughts, tricks, ideas, and code (without any weights). I am sure you will get hundreds of upvotes.
", "rawMarkdown": "Alex, I think your message was downvoted because of your tone (not because of what you shared): `\"I am going to continue publishing kernels, more more and more.\"` It feels like you don't give a shit about what other competitors asked (nicely).\n\nI saw your latest kernel (without weights). I think that is a good way to share. \n\nI think a Kaggle competition is like running a marathon. I ran once, and I had to train almost a year before. Before a Kaggle competition, I learned lots of things, studying hours, days, taking courses, etc. I prepared myself to compete, just like before the marathon.\n\nI felt ready, I felt motivated. After the first few miles, my position started to decrease. That was ok. I knew there were lots of runners better than I was. Same as here, I don't care about my position, there are dozens of Kagglers better than I am. \n\nIf a world-class Kenyan runner shares anything, I will listen 100%. Same as here, If you (or any other Kaggler) share anything useful, I will thank, and I will use that information.\n\nBut imagine this: After the third of that marathon, that Kenyan runner started to Uber dozens of fat, lazy, untrained \"competitors\". And he said, don't worry, it is only the first third, there is plenty of time to beat them.\n(I can not say how fu\\*\\*ing annoying that would be.)\n\nMany Kagglers put lots of time, work, and effort into whatever they've achieved. By sharing high-scoring weights, you only destroy all of their motivation. The problem is the high scores (more precisely, the forks and resubmits those high scores without ANY effort), **not your sharings**.\n\nSo, keep up the good work and share your thoughts, tricks, ideas, and code (without any weights). I am sure you will get hundreds of upvotes.", "votes": null }, { "id": "857077", "postDate": "05/22/2020 09:30:02", "content": "Thank you for explanation @pestipeti !
\n\nMy words mean that nobody's opinion will not be able stop publications of my ideas and techniques.
\n\nI am ready sharing weights only by request in comments. If need I will share dataset! It is really important for economy money and save the environment (nature), because using my weights you can tune your model ~5*8h=40h faster! Also I see perspective use my weights for increasing image_size by original size. I believe it gives boost.
", "rawMarkdown": "Thank you for explanation @pestipeti ! \n\nMy words mean that nobody's opinion will not be able stop publications of my ideas and techniques.\n\nI am ready sharing weights only by request in comments. If need I will share dataset! It is really important for economy money and save the environment (nature), because using my weights you can tune your model ~5*8h=40h faster! Also I see perspective use my weights for increasing image_size by original size. I believe it gives boost.", "votes": null }, { "id": "857098", "postDate": "05/22/2020 09:50:10", "content": "This is not a new problem on Kaggle. One or two years ago, Kamil and Kuba from the Neptune.ai team came up with an acceptable solution: only share a solution with a score below medal level.
\n\nTake a look at this thread Lots of respected GMs (Anokas, Dieter, CPMP, and many more) supported the idea.
", "rawMarkdown": "This is not a new problem on Kaggle. One or two years ago, Kamil and Kuba from the Neptune.ai team came up with an acceptable solution: only share a solution with a score below medal level.\n\n[Take a look at this thread](https://www.kaggle.com/c/tgs-salt-identification-challenge/discussion/62162#365436) Lots of respected GMs (Anokas, Dieter, CPMP, and many more) supported the idea.", "votes": null }, { "id": "857114", "postDate": "05/22/2020 10:11:12", "content": "I don't think that only started competition needs in this correction. I am agree It is really good rule for middle and last stage of competition (2-3 weeks before merge deadline). But here is absolutely another situation.
\n\nI repeat, stop downvoting my messages! I don't share my weights without need
", "rawMarkdown": "I don't think that only started competition needs in this correction. I am agree It is really good rule for middle and last stage of competition (2-3 weeks before merge deadline). But here is absolutely another situation. \n\nI repeat, stop downvoting my messages! I don't share my weights without need", "votes": null }, { "id": "857124", "postDate": "05/22/2020 10:23:58", "content": "@pestipeti also, first of all kaggle team has to support the competition spirit. Without correcting precedents such as DeepFake from facebook, any rules for keeping competition spirit are artificial.
", "rawMarkdown": "pestipeti also, first of all kaggle team has to support the competition spirit. Without correcting precedents such as DeepFake from facebook, any rules for keeping competition spirit are artificial.", "votes": null }, { "id": "857125", "postDate": "05/22/2020 10:25:33", "content": "Today you are afraid of my kernel, and tomorrow you will be disqualified for no any reason after hardworking 2 months! Think about it, community
", "rawMarkdown": "Today you are afraid of my kernel, and tomorrow you will be disqualified for no any reason after hardworking 2 months! Think about it, community", "votes": null }, { "id": "857222", "postDate": "05/22/2020 12:29:11", "content": "Nice interesting thread here. Upvote for that!\nSince I'm so invested in this comp here are my 2 cents.\nA) The number one goal of the comp organizers is to get the best score possible.\nB) My number zero goal is to learn.\nC) My number one goal is to get the best score of all competitors.
\n\nSharing great solutions helps A) for sure because it instantly raises the bar. Anyone who was already doing well can incorporate the extra knowledge into their solution and maybe do even better.
\n\nSharing solutions helps B) for sure. No questions about that.
\n\nSharing solutions has little effect on C). It just raises the floor. If you were standing tall before, you will probably be standing tall after (as long as you decide to make use of the shared knowledge). The only situation when sharing solutions screws you over is when that solution renders your own tricks moot. That can hurt... but I would take that hit any day in exchange for the benefits.
\n\nAnd lastly, I don't care about the people up the leaderboard who copied the solution. You think I'm sitting here in 248th place because I don't know how to fork a kernel? It's actually easier to experiment without using bigger better models, or wasting GPU time doing pseudo labelling. I have to innovate if I want to win. I can apply the big guns later.
", "rawMarkdown": "Nice interesting thread here. Upvote for that!\nSince I'm so invested in this comp here are my 2 cents.\nA) The number one goal of the comp organizers is to get the best score possible.\nB) My number zero goal is to learn.\nC) My number one goal is to get the best score of all competitors.\n\nSharing great solutions helps A) for sure because it instantly raises the bar. Anyone who was already doing well can incorporate the extra knowledge into their solution and maybe do even better.\n\nSharing solutions helps B) for sure. No questions about that.\n\nSharing solutions has little effect on C). It just raises the floor. If you were standing tall before, you will probably be standing tall after (as long as you decide to make use of the shared knowledge). The only situation when sharing solutions screws you over is when that solution renders your own tricks moot. That can hurt... but I would take that hit any day in exchange for the benefits.\n\nAnd lastly, I don't care about the people up the leaderboard who copied the solution. You think I'm sitting here in 248th place because I don't know how to fork a kernel? It's actually easier to experiment without using bigger better models, or wasting GPU time doing pseudo labelling. I have to innovate if I want to win. I can apply the big guns later.", "votes": null }, { "id": "857256", "postDate": "05/22/2020 13:04:55", "content": "@alexandersoare you are right. I shared what I felt. No offense to anyone. Sharing absolutely helps to learn new things.
", "rawMarkdown": "alexandersoare you are right. I shared what I felt. No offense to anyone. Sharing absolutely helps to learn new things.", "votes": null }, { "id": "857329", "postDate": "05/22/2020 14:23:55", "content": "Let's look at one extreme. There is no leaderboard, and no way to share. Teams compete privately, and a winner is announced at the end of the competition. No one shares code or ideas. Does that seem like it would be a better format?
\n\nMost don't think so. (BTW, there are challenges on other platforms that run this way, for people would rather compete in such a format. They're just not nearly as popular.)
\n\nThe reason people don't prefer that format is that there is a tremendous personal benefit from seeing what others are doing and learning from from their approach. Does it make it hard? Yes. Can it be more stressful and frustrating for people trying to rank highly? Of course. (I've been there, many times.) Bet it is much better than the alternative.
\n\nSo, we've ruled out that extreme. Once we've ruled out that extreme, it's just a matter of degree. Everyone is going to have a different opinion about how much should be shared and when it's appropriate for it to be shared. There's no way to satisfy all of these conditions.
\n\nSo, we continue to emphasize that people shouldn't post high-scoring public notebooks the last week of the competition. But, other than that, the benefit from people openly sharing their work seems to vastly outweigh the downsides.
", "rawMarkdown": "Let's look at one extreme. There is no leaderboard, and no way to share. Teams compete privately, and a winner is announced at the end of the competition. No one shares code or ideas. Does that seem like it would be a better format?\n\nMost don't think so. (BTW, there are challenges on other platforms that run this way, for people would rather compete in such a format. They're just not nearly as popular.) \n\nThe reason people don't prefer that format is that there is a _tremendous_ personal benefit from seeing what others are doing and learning from from their approach. Does it make it hard? Yes. Can it be more stressful and frustrating for people trying to rank highly? Of course. (I've been there, many times.) Bet it is much better than the alternative.\n\nSo, we've ruled out that extreme. Once we've ruled out that extreme, it's just a matter of degree. Everyone is going to have a different opinion about how much should be shared and when it's appropriate for it to be shared. There's no way to satisfy all of these conditions.\n\nSo, we continue to emphasize that people shouldn't post high-scoring public notebooks the last week of the competition. But, other than that, the benefit from people openly sharing their work seems to vastly outweigh the downsides.", "votes": null }, { "id": "857336", "postDate": "05/22/2020 14:27:59", "content": "With that said, I also find it annoying when people are merely copy/pasting and not putting their own effort into the competition. But that's a small downside compared to the incredible amount of learning I've personally gained from Kaggle. (And @kaushal2896, I similarly hope you find tremendous benefit here.)
", "rawMarkdown": "With that said, I also find it annoying when people are merely copy/pasting and not putting their own effort into the competition. But that's a small downside compared to the incredible amount of learning I've personally gained from Kaggle. (And @kaushal2896, I similarly hope you find tremendous benefit here.)", "votes": null }, { "id": "857339", "postDate": "05/22/2020 14:32:02", "content": "Yes, you are right we should stick to not sharing high scoring kernels in last couple of weeks of the competition. before that it doesn't matter as everyone has equal opportunity to improve their models.
", "rawMarkdown": "Yes, you are right we should stick to not sharing high scoring kernels in last couple of weeks of the competition. before that it doesn't matter as everyone has equal opportunity to improve their models.", "votes": null }, { "id": "857345", "postDate": "05/22/2020 14:39:18", "content": "@rohitsingh9990 Before I joined Kaggle, and when I was still relatively new in machine learning, I entered an image competition. I was doing pretty well. But 3 days before the competition ended, someone posted code in the forums that was an end-to-end pipeline for the problem, using their company's open-source platform. (This was before notebooks even, so it was a post of the raw code.)
\n\nPeople started using it and I dropped on the leaderboard quickly. But, I couldn't get the dependencies to work, and was never able to make use of the code. I ended up dropping significantly, and was very frustrated.
\n\nSo, I'm a big fan of sharing, but also for respecting the competitive spirit as well. :-)
", "rawMarkdown": "rohitsingh9990 Before I joined Kaggle, and when I was still relatively new in machine learning, I entered an image competition. I was doing pretty well. But 3 days before the competition ended, someone posted code in the forums that was an end-to-end pipeline for the problem, using their company's open-source platform. (This was before notebooks even, so it was a post of the raw code.)\n\nPeople started using it and I dropped on the leaderboard quickly. But, I couldn't get the dependencies to work, and was never able to make use of the code. I ended up dropping significantly, and was very frustrated.\n\nSo, I'm a big fan of sharing, but also for respecting the competitive spirit as well. :-)", "votes": null }, { "id": "857349", "postDate": "05/22/2020 14:46:21", "content": "Same here i also like to share a lot, but i think there is another reason for sharing high scoring kernel as it is the easiest way to gain attention and quick upvotes, which i think is fine. if they worked hard they deserve that.
", "rawMarkdown": "Same here i also like to share a lot, but i think there is another reason for sharing high scoring kernel as it is the easiest way to gain attention and quick upvotes, which i think is fine. if they worked hard they deserve that.", "votes": null }, { "id": "857350", "postDate": "05/22/2020 14:46:23", "content": "This Comp is in its beginning phase. Sharing, in the beginning, makes it possible for problem to get solved to its true potential solution. But one should refrain releasing kernals towards the end of the comptetions.
", "rawMarkdown": "This Comp is in its beginning phase. Sharing, in the beginning, makes it possible for problem to get solved to its true potential solution. But one should refrain releasing kernals towards the end of the comptetions.", "votes": null }, { "id": "857358", "postDate": "05/22/2020 14:50:12", "content": "Yes, that's what we should follow.
", "rawMarkdown": "Yes, that's what we should follow.", "votes": null }, { "id": "857362", "postDate": "05/22/2020 14:51:14", "content": "@inversion you are absolutely right. Public kernels are the source of learning. If no one shares any ideas and just compete privately, beginners like me won't be able to learn. Because of sharing, I can look into what others are trying and most importantly why. I'm neither requesting to stop sharing kernels publicly nor I'm questioning this notebook sharing feature of Kaggle. I just felt that we should understand and not share just the inference kernels which scores very high. Sharing baselines kernels, ideas of approaching the problems, data preprocessing, etc. are super helpful. Thanks for your reply BTW.
", "rawMarkdown": "inversion you are absolutely right. Public kernels are the source of learning. If no one shares any ideas and just compete privately, beginners like me won't be able to learn. Because of sharing, I can look into what others are trying and most importantly why. I'm neither requesting to stop sharing kernels publicly nor I'm questioning this notebook sharing feature of Kaggle. I just felt that we should understand and not share just the inference kernels which scores very high. Sharing baselines kernels, ideas of approaching the problems, data preprocessing, etc. are super helpful. Thanks for your reply BTW.", "votes": null }, { "id": "857366", "postDate": "05/22/2020 14:53:57", "content": "@kaushal2896 I definitely see your point. Sharing an inference kernel is effectively the same as just sharing a submission file. No one learns from that.
", "rawMarkdown": "kaushal2896 I definitely see your point. Sharing an inference kernel is effectively the same as just sharing a submission file. No one learns from that.", "votes": null }, { "id": "857382", "postDate": "05/22/2020 15:06:08", "content": "@inversion \nI think disabling/removing the \"Best score\" sorting option during a competition would solve most of the issues. These copy-paste-fork gurus won't bother to look through dozens of pages searching for high scores.
\n\nHigh-scoring notebooks 99% of the time is all about getting lots of votes and a gold kernel medal. There are millions of ensemble of ensemble notebook with zero additional value.
\n\nThey are annoying for those who want to compete, and they are ruining the reputation of the kernel tier.
\n\nPlease consider these:\n- remove the best score sorting option (during a competition)\n- do not advertise the public scores (during a competition)
\n\nHere is some related topic (probably you know all about these):\n- It's all about upvotes and medals\n- (Open solution journal) friendly request for team \"Neptune ML\"\n- The leaderboard has been destroyed
\n\nAll of the frustration is because of the advertised public scores during a competition.
", "rawMarkdown": "inversion \nI think disabling/removing the \"Best score\" sorting option during a competition would solve most of the issues. These copy-paste-fork gurus won't bother to look through dozens of pages searching for high scores. \n\nHigh-scoring notebooks 99% of the time is all about getting lots of votes and a gold kernel medal. There are millions of ensemble of ensemble notebook with zero additional value. \n\nThey are annoying for those who want to compete, and they are ruining the reputation of the kernel tier.\n\nPlease consider these:\n- remove the best score sorting option (during a competition)\n- do not advertise the public scores (during a competition)\n\nHere is some related topic (probably you know all about these):\n- [It's all about upvotes and medals](https://www.kaggle.com/c/m5-forecasting-accuracy/discussion/150568)\n- [(Open solution journal) friendly request for team \"Neptune ML\"](https://www.kaggle.com/c/tgs-salt-identification-challenge/discussion/62162)\n- [The leaderboard has been destroyed](https://www.kaggle.com/c/severstal-steel-defect-detection/discussion/110670)\n\nAll of the frustration is because of the advertised public scores during a competition.", "votes": null }, { "id": "857529", "postDate": "05/22/2020 17:25:51", "content": "I think making the kernels public during the initial stage of a competition is useful for the beginners as they get an idea about how to appraoch towards a good solution.In the end the kernels should be kept private so that the kernel with the best ideas wins.
", "rawMarkdown": "I think making the kernels public during the initial stage of a competition is useful for the beginners as they get an idea about how to appraoch towards a good solution.In the end the kernels should be kept private so that the kernel with the best ideas wins.", "votes": null }, { "id": "857903", "postDate": "05/23/2020 03:37:04", "content": "Actually this a custom in Kaggle . In any competitions , guys will keep publishing kernels (include inference kernel) in the early stage , but during the last month , they will cease to do it .
", "rawMarkdown": "Actually this a custom in Kaggle . In any competitions , guys will keep publishing kernels (include inference kernel) in the early stage , but during the last month , they will cease to do it .", "votes": null }, { "id": "858209", "postDate": "05/23/2020 09:45:59", "content": "Since the competition just gets started, why do we worry so much on high-score kernels. I think everyone have time to learn from them.
", "rawMarkdown": "Since the competition just gets started, why do we worry so much on high-score kernels. I think everyone have time to learn from them.", "votes": null }, { "id": "858272", "postDate": "05/23/2020 10:49:02", "content": "I am doing an academic study on this issue. Whether or not the outcome of a contest improves when contestants share knowledge. The argument against it in the creativty literature is that it creates a form of cognitive fixation - once you saw an idea your mind is focusing on that idea, and you are less likely to come up with your own original idea. Especially when you see others ideas before you have come up with an own. \nI am wondering whether sharing behavior also depends on the prize, the contest owner, or the type of problem (commercial vs. social).
", "rawMarkdown": "I am doing an academic study on this issue. Whether or not the outcome of a contest improves when contestants share knowledge. The argument against it in the creativty literature is that it creates a form of cognitive fixation - once you saw an idea your mind is focusing on that idea, and you are less likely to come up with your own original idea. Especially when you see others ideas before you have come up with an own. \nI am wondering whether sharing behavior also depends on the prize, the contest owner, or the type of problem (commercial vs. social).", "votes": null }, { "id": "858362", "postDate": "05/23/2020 12:06:07", "content": "@kaushal2896 I think I get your point exactly, and I agree on that. In Bengali.Ai competition when @iafoss shared his wonderful kernel that eventually brought some competitor in the medal zone at the end, and funny things were some of the competitors weren't too active to that competition. And in this competition @shonenkov has some wonderful high scored kernel and maybe the same things can repeat here too.
\n\nIMHO, sharing a high score kernel is OK as long as the weights are in private. In addition, sharing ideas that can reach the gold zone is always controversial, but it may be ok since we have 2 months till now. @shonenkov notebooks are just awesome and we need more and more high-quality notebooks from him.
", "rawMarkdown": "kaushal2896 I think I get your point exactly, and I agree on that. In Bengali.Ai competition when @iafoss shared his wonderful kernel that eventually brought some competitor in the medal zone at the end, and funny things were some of the competitors weren't too active to that competition. And in this competition @shonenkov has some wonderful high scored kernel and maybe the same things can repeat here too. \n\nIMHO, sharing a high score kernel is OK as long as the weights are in private. In addition, sharing ideas that can reach the gold zone is always controversial, but it may be ok since we have 2 months till now. @shonenkov notebooks are just awesome and we need more and more high-quality notebooks from him.", "votes": null }, { "id": "858659", "postDate": "05/23/2020 17:04:42", "content": "Actually high scoring kernels being public is a good thing! You can learn many things. Also I have tried test submitting those high scored kernels, It turns out that people already have work on this, this does not make so much difference on the leaderboard. Moreover, you can learn stuff
", "rawMarkdown": "Actually high scoring kernels being public is a good thing! You can learn many things. Also I have tried test submitting those high scored kernels, It turns out that people already have work on this, this does not make so much difference on the leaderboard. Moreover, you can learn stuff", "votes": null }, { "id": "858747", "postDate": "05/23/2020 19:10:09", "content": "As a beginner, Notebook reference is fantastic to learn but at the same time, I think if some user is referring to notebook related to a particular competition then code matching must be done to check the plagarism.
\n\nGreat God, all this time I thought it was Lafoss.
", "rawMarkdown": "Great God, all this time I thought it was Lafoss.", "votes": null }, { "id": "880417", "postDate": "06/10/2020 09:24:37", "content": "Thanks to everyone who shared. I can't bear to give you downvotes. Sharing is free, and at the same time, the greater the ability, the greater the responsibility, and more things to consider, such as fairness. I think you should maintain fairness while sharing, and don't let those lazy people beat those who work hard.
\n\nThanks to everyone who shared again.
", "rawMarkdown": "Thanks to everyone who shared. I can't bear to give you downvotes. Sharing is free, and at the same time, the greater the ability, the greater the responsibility, and more things to consider, such as fairness. I think you should maintain fairness while sharing, and don't let those lazy people beat those who work hard. \n\nThanks to everyone who shared again.", "votes": null } ], "comments": [ { "id": 856432, "author_name": "rohitsingh9990", "author_url": "", "post_date": "05/21/2020 17:48:00", "content": "You are right we should avoid posting high scoring inference kernels, but still the competition is in its early stage, so it doesn't matter. you should focus more on improving your model, all these copycats will eventually leave behind.
", "votes": null, "replies": [] }, { "id": 856494, "author_name": "miklgr500", "author_url": "", "post_date": "05/21/2020 18:44:22", "content": "Now, they on top, but this does not have matter. They don't win medal without own ideas.
", "votes": null, "replies": [] }, { "id": 856632, "author_name": "shonenkov", "author_url": "", "post_date": "05/21/2020 23:33:49", "content": "Publishing any kernel is useful for any competitor! It is your choice: make sharing or not your ideas! High score shows really good idea or not. You should enjoy the gift of fate, that you can learn new techniques, doesnt it?
\n\nEveryone has time to improve own solutions. You have 2 months.
\n\nAfter DeepFake competition I don't see any point in continuing to do any competitions. Organizers made big fault with running notebooks and I proved it! But kaggle team closed eyes on this precedent. Huge number of ideas and techniques could help anyone competitor, but I lost without a mark of participation even. I didn't break the rules.
\n\nKaggle Team needs to pay attention on quality of competitions and organizers of big contests.
\n\nI am going to continue publishing kernels, more more and more.
\n\nThank you for your patience.
", "votes": null, "replies": [ { "id": 856760, "author_name": "kaushal2896", "author_url": "", "post_date": "05/22/2020 03:47:05", "content": "\"You should enjoy the gift of fate, that you can learn new techniques\": I'm just talking about inference kernels and I don't think you learn new techniques from inference kernels. Other than this, I don't understand what you are talking about as I'm bit new to this challenges. Anyways this was just a request. Thanks for the opinion. @shonenkov
", "votes": null, "replies": [] }, { "id": 856858, "author_name": "matthewmasters", "author_url": "", "post_date": "05/22/2020 05:18:18", "content": "I'm confused what happened with your notebook in the deepfake competition. You scored very high on the public LB but got 113th on the private LB?
", "votes": null, "replies": [] }, { "id": 856871, "author_name": "pestipeti", "author_url": "", "post_date": "05/22/2020 05:40:35", "content": "I 100% understand your frustration, and I agree with you, but publishing gold-level, high-scoring kernels won't solve anything. By doing this, you will only ruin the competition for a lot of people. Please do not do that. Any great idea, trick, or solution is welcome, but please do not share the trained weights.
", "votes": null, "replies": [] }, { "id": 856895, "author_name": "kaushal2896", "author_url": "", "post_date": "05/22/2020 06:03:23", "content": "Yes exactly! That's what I wanted to communicate with this thread!
", "votes": null, "replies": [] }, { "id": 856932, "author_name": "shonenkov", "author_url": "", "post_date": "05/22/2020 06:51:02", "content": "@matthewmasters of course NO! Facebook crushed working kernel 113 public in the private stage and kicked many participants from competition without explanation!
", "votes": null, "replies": [] } ] }, { "id": 856641, "author_name": "nvnnghia", "author_url": "", "post_date": "05/22/2020 00:12:08", "content": "I think we should share good inference kernel. just put our models in private dataset.
", "votes": null, "replies": [ { "id": 856755, "author_name": "kaushal2896", "author_url": "", "post_date": "05/22/2020 03:40:57", "content": "Yes this is a good idea!
", "votes": null, "replies": [] } ] }, { "id": 856957, "author_name": "shonenkov", "author_url": "", "post_date": "05/22/2020 07:13:46", "content": "I don't understand why my message was downvoted!? WHY? Kaggle community, competition run less month! You should think about team spirit and use jump start for solving this competition. But you whine and think only about yourself and your positions on the leaderboard. It is disheartening, really!
\n\nMoreover, last kernel that I published, I don't share weights - I am able to listen to your requests. But you gave downvotes without reason. 👎
", "votes": null, "replies": [ { "id": 856973, "author_name": "matthewmasters", "author_url": "", "post_date": "05/22/2020 07:33:14", "content": "Yeah, I don't understand why they're downvoting your reply to me either. I just looked into what happened in the FB DeepFake detection challenge and that's terrible and definitely discouraging for future competitions :(
\n\nAnyways, I appreciate all your selfless contributions to this competition! Hopefully you will have better luck here
", "votes": null, "replies": [] }, { "id": 857044, "author_name": "pestipeti", "author_url": "", "post_date": "05/22/2020 08:52:23", "content": "Alex, I think your message was downvoted because of your tone (not because of what you shared): \"I am going to continue publishing kernels, more more and more.\" It feels like you don't give a shit about what other competitors asked (nicely).
I saw your latest kernel (without weights). I think that is a good way to share.
\n\nI think a Kaggle competition is like running a marathon. I ran once, and I had to train almost a year before. Before a Kaggle competition, I learned lots of things, studying hours, days, taking courses, etc. I prepared myself to compete, just like before the marathon.
\n\nI felt ready, I felt motivated. After the first few miles, my position started to decrease. That was ok. I knew there were lots of runners better than I was. Same as here, I don't care about my position, there are dozens of Kagglers better than I am.
\n\nIf a world-class Kenyan runner shares anything, I will listen 100%. Same as here, If you (or any other Kaggler) share anything useful, I will thank, and I will use that information.
\n\nBut imagine this: After the third of that marathon, that Kenyan runner started to Uber dozens of fat, lazy, untrained \"competitors\". And he said, don't worry, it is only the first third, there is plenty of time to beat them.\n(I can not say how fu**ing annoying that would be.)
\n\nMany Kagglers put lots of time, work, and effort into whatever they've achieved. By sharing high-scoring weights, you only destroy all of their motivation. The problem is the high scores (more precisely, the forks and resubmits those high scores without ANY effort), not your sharings.
\n\nSo, keep up the good work and share your thoughts, tricks, ideas, and code (without any weights). I am sure you will get hundreds of upvotes.
", "votes": null, "replies": [] }, { "id": 857077, "author_name": "shonenkov", "author_url": "", "post_date": "05/22/2020 09:30:02", "content": "Thank you for explanation @pestipeti !
\n\nMy words mean that nobody's opinion will not be able stop publications of my ideas and techniques.
\n\nI am ready sharing weights only by request in comments. If need I will share dataset! It is really important for economy money and save the environment (nature), because using my weights you can tune your model ~5*8h=40h faster! Also I see perspective use my weights for increasing image_size by original size. I believe it gives boost.
", "votes": null, "replies": [] }, { "id": 857098, "author_name": "pestipeti", "author_url": "", "post_date": "05/22/2020 09:50:10", "content": "This is not a new problem on Kaggle. One or two years ago, Kamil and Kuba from the Neptune.ai team came up with an acceptable solution: only share a solution with a score below medal level.
\n\nTake a look at this thread Lots of respected GMs (Anokas, Dieter, CPMP, and many more) supported the idea.
", "votes": null, "replies": [] }, { "id": 857114, "author_name": "shonenkov", "author_url": "", "post_date": "05/22/2020 10:11:12", "content": "I don't think that only started competition needs in this correction. I am agree It is really good rule for middle and last stage of competition (2-3 weeks before merge deadline). But here is absolutely another situation.
\n\nI repeat, stop downvoting my messages! I don't share my weights without need
", "votes": null, "replies": [] }, { "id": 857124, "author_name": "shonenkov", "author_url": "", "post_date": "05/22/2020 10:23:58", "content": "@pestipeti also, first of all kaggle team has to support the competition spirit. Without correcting precedents such as DeepFake from facebook, any rules for keeping competition spirit are artificial.
", "votes": null, "replies": [] }, { "id": 857125, "author_name": "shonenkov", "author_url": "", "post_date": "05/22/2020 10:25:33", "content": "Today you are afraid of my kernel, and tomorrow you will be disqualified for no any reason after hardworking 2 months! Think about it, community
", "votes": null, "replies": [] } ] }, { "id": 857222, "author_name": "alexandersoare", "author_url": "", "post_date": "05/22/2020 12:29:11", "content": "Nice interesting thread here. Upvote for that!\nSince I'm so invested in this comp here are my 2 cents.\nA) The number one goal of the comp organizers is to get the best score possible.\nB) My number zero goal is to learn.\nC) My number one goal is to get the best score of all competitors.
\n\nSharing great solutions helps A) for sure because it instantly raises the bar. Anyone who was already doing well can incorporate the extra knowledge into their solution and maybe do even better.
\n\nSharing solutions helps B) for sure. No questions about that.
\n\nSharing solutions has little effect on C). It just raises the floor. If you were standing tall before, you will probably be standing tall after (as long as you decide to make use of the shared knowledge). The only situation when sharing solutions screws you over is when that solution renders your own tricks moot. That can hurt... but I would take that hit any day in exchange for the benefits.
\n\nAnd lastly, I don't care about the people up the leaderboard who copied the solution. You think I'm sitting here in 248th place because I don't know how to fork a kernel? It's actually easier to experiment without using bigger better models, or wasting GPU time doing pseudo labelling. I have to innovate if I want to win. I can apply the big guns later.
", "votes": null, "replies": [ { "id": 857256, "author_name": "kaushal2896", "author_url": "", "post_date": "05/22/2020 13:04:55", "content": "@alexandersoare you are right. I shared what I felt. No offense to anyone. Sharing absolutely helps to learn new things.
", "votes": null, "replies": [] } ] }, { "id": 857329, "author_name": "inversion", "author_url": "", "post_date": "05/22/2020 14:23:55", "content": "Let's look at one extreme. There is no leaderboard, and no way to share. Teams compete privately, and a winner is announced at the end of the competition. No one shares code or ideas. Does that seem like it would be a better format?
\n\nMost don't think so. (BTW, there are challenges on other platforms that run this way, for people would rather compete in such a format. They're just not nearly as popular.)
\n\nThe reason people don't prefer that format is that there is a tremendous personal benefit from seeing what others are doing and learning from from their approach. Does it make it hard? Yes. Can it be more stressful and frustrating for people trying to rank highly? Of course. (I've been there, many times.) Bet it is much better than the alternative.
\n\nSo, we've ruled out that extreme. Once we've ruled out that extreme, it's just a matter of degree. Everyone is going to have a different opinion about how much should be shared and when it's appropriate for it to be shared. There's no way to satisfy all of these conditions.
\n\nSo, we continue to emphasize that people shouldn't post high-scoring public notebooks the last week of the competition. But, other than that, the benefit from people openly sharing their work seems to vastly outweigh the downsides.
", "votes": null, "replies": [ { "id": 857336, "author_name": "inversion", "author_url": "", "post_date": "05/22/2020 14:27:59", "content": "With that said, I also find it annoying when people are merely copy/pasting and not putting their own effort into the competition. But that's a small downside compared to the incredible amount of learning I've personally gained from Kaggle. (And @kaushal2896, I similarly hope you find tremendous benefit here.)
", "votes": null, "replies": [] }, { "id": 857339, "author_name": "rohitsingh9990", "author_url": "", "post_date": "05/22/2020 14:32:02", "content": "Yes, you are right we should stick to not sharing high scoring kernels in last couple of weeks of the competition. before that it doesn't matter as everyone has equal opportunity to improve their models.
", "votes": null, "replies": [] }, { "id": 857345, "author_name": "inversion", "author_url": "", "post_date": "05/22/2020 14:39:18", "content": "@rohitsingh9990 Before I joined Kaggle, and when I was still relatively new in machine learning, I entered an image competition. I was doing pretty well. But 3 days before the competition ended, someone posted code in the forums that was an end-to-end pipeline for the problem, using their company's open-source platform. (This was before notebooks even, so it was a post of the raw code.)
\n\nPeople started using it and I dropped on the leaderboard quickly. But, I couldn't get the dependencies to work, and was never able to make use of the code. I ended up dropping significantly, and was very frustrated.
\n\nSo, I'm a big fan of sharing, but also for respecting the competitive spirit as well. :-)
", "votes": null, "replies": [] }, { "id": 857349, "author_name": "rohitsingh9990", "author_url": "", "post_date": "05/22/2020 14:46:21", "content": "Same here i also like to share a lot, but i think there is another reason for sharing high scoring kernel as it is the easiest way to gain attention and quick upvotes, which i think is fine. if they worked hard they deserve that.
", "votes": null, "replies": [] }, { "id": 857362, "author_name": "kaushal2896", "author_url": "", "post_date": "05/22/2020 14:51:14", "content": "@inversion you are absolutely right. Public kernels are the source of learning. If no one shares any ideas and just compete privately, beginners like me won't be able to learn. Because of sharing, I can look into what others are trying and most importantly why. I'm neither requesting to stop sharing kernels publicly nor I'm questioning this notebook sharing feature of Kaggle. I just felt that we should understand and not share just the inference kernels which scores very high. Sharing baselines kernels, ideas of approaching the problems, data preprocessing, etc. are super helpful. Thanks for your reply BTW.
", "votes": null, "replies": [] }, { "id": 857366, "author_name": "inversion", "author_url": "", "post_date": "05/22/2020 14:53:57", "content": "@kaushal2896 I definitely see your point. Sharing an inference kernel is effectively the same as just sharing a submission file. No one learns from that.
", "votes": null, "replies": [] }, { "id": 857382, "author_name": "pestipeti", "author_url": "", "post_date": "05/22/2020 15:06:08", "content": "@inversion \nI think disabling/removing the \"Best score\" sorting option during a competition would solve most of the issues. These copy-paste-fork gurus won't bother to look through dozens of pages searching for high scores.
\n\nHigh-scoring notebooks 99% of the time is all about getting lots of votes and a gold kernel medal. There are millions of ensemble of ensemble notebook with zero additional value.
\n\nThey are annoying for those who want to compete, and they are ruining the reputation of the kernel tier.
\n\nPlease consider these:\n- remove the best score sorting option (during a competition)\n- do not advertise the public scores (during a competition)
\n\nHere is some related topic (probably you know all about these):\n- It's all about upvotes and medals\n- (Open solution journal) friendly request for team \"Neptune ML\"\n- The leaderboard has been destroyed
\n\nAll of the frustration is because of the advertised public scores during a competition.
", "votes": null, "replies": [] } ] }, { "id": 857350, "author_name": "mks2192", "author_url": "", "post_date": "05/22/2020 14:46:23", "content": "This Comp is in its beginning phase. Sharing, in the beginning, makes it possible for problem to get solved to its true potential solution. But one should refrain releasing kernals towards the end of the comptetions.
", "votes": null, "replies": [ { "id": 857358, "author_name": "rohitsingh9990", "author_url": "", "post_date": "05/22/2020 14:50:12", "content": "Yes, that's what we should follow.
", "votes": null, "replies": [] } ] }, { "id": 857529, "author_name": "proxyy", "author_url": "", "post_date": "05/22/2020 17:25:51", "content": "I think making the kernels public during the initial stage of a competition is useful for the beginners as they get an idea about how to appraoch towards a good solution.In the end the kernels should be kept private so that the kernel with the best ideas wins.
", "votes": null, "replies": [] }, { "id": 857903, "author_name": "xujingzhao", "author_url": "", "post_date": "05/23/2020 03:37:04", "content": "Actually this a custom in Kaggle . In any competitions , guys will keep publishing kernels (include inference kernel) in the early stage , but during the last month , they will cease to do it .
", "votes": null, "replies": [] }, { "id": 858209, "author_name": "ratthachat", "author_url": "", "post_date": "05/23/2020 09:45:59", "content": "Since the competition just gets started, why do we worry so much on high-score kernels. I think everyone have time to learn from them.
", "votes": null, "replies": [] }, { "id": 858272, "author_name": "florianwaldner", "author_url": "", "post_date": "05/23/2020 10:49:02", "content": "I am doing an academic study on this issue. Whether or not the outcome of a contest improves when contestants share knowledge. The argument against it in the creativty literature is that it creates a form of cognitive fixation - once you saw an idea your mind is focusing on that idea, and you are less likely to come up with your own original idea. Especially when you see others ideas before you have come up with an own. \nI am wondering whether sharing behavior also depends on the prize, the contest owner, or the type of problem (commercial vs. social).
", "votes": null, "replies": [] }, { "id": 858362, "author_name": "ipythonx", "author_url": "", "post_date": "05/23/2020 12:06:07", "content": "@kaushal2896 I think I get your point exactly, and I agree on that. In Bengali.Ai competition when @iafoss shared his wonderful kernel that eventually brought some competitor in the medal zone at the end, and funny things were some of the competitors weren't too active to that competition. And in this competition @shonenkov has some wonderful high scored kernel and maybe the same things can repeat here too.
\n\nIMHO, sharing a high score kernel is OK as long as the weights are in private. In addition, sharing ideas that can reach the gold zone is always controversial, but it may be ok since we have 2 months till now. @shonenkov notebooks are just awesome and we need more and more high-quality notebooks from him.
", "votes": null, "replies": [ { "id": 859920, "author_name": "danoozy44", "author_url": "", "post_date": "05/24/2020 21:13:50", "content": "Great God, all this time I thought it was Lafoss.
", "votes": null, "replies": [] } ] }, { "id": 858659, "author_name": "tasnimnishatislam", "author_url": "", "post_date": "05/23/2020 17:04:42", "content": "Actually high scoring kernels being public is a good thing! You can learn many things. Also I have tried test submitting those high scored kernels, It turns out that people already have work on this, this does not make so much difference on the leaderboard. Moreover, you can learn stuff
", "votes": null, "replies": [] }, { "id": 858747, "author_name": "surpraka", "author_url": "", "post_date": "05/23/2020 19:10:09", "content": "As a beginner, Notebook reference is fantastic to learn but at the same time, I think if some user is referring to notebook related to a particular competition then code matching must be done to check the plagarism.
\n\nThanks to everyone who shared. I can't bear to give you downvotes. Sharing is free, and at the same time, the greater the ability, the greater the responsibility, and more things to consider, such as fairness. I think you should maintain fairness while sharing, and don't let those lazy people beat those who work hard.
\n\nThanks to everyone who shared again.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "856409": "Can we stop making high-scoring \"inference\" kernels public? (not starters, they are great and help beginners like me)\nIt just feels unfair that some people submit other people's model weights and place higher than people who are trying their best. It's quite depressing.\nCan we work around this anyhow?\n\nThere are currently ~38 people on top of the leaderboard having the same score as highest scoring public inference kernel (in some cases this might be a coincidence)\n\nNo offense to anyone.\n\nThanks!\n\n**EDIT:**\nI have not downvoted anyone's comments and also this post is not to target any individuals. It is just a humble request. I felt this in the last two competitions (this one and Bengali.AI and that's why shared it). If this section is not for such discussions, please let me know, I'll delete the post.", "856432": "You are right we should avoid posting high scoring inference kernels, but still the competition is in its early stage, so it doesn't matter. you should focus more on improving your model, all these copycats will eventually leave behind.", "856494": "Now, they on top, but this does not have matter. They don't win medal without own ideas.", "856632": "Publishing any kernel is useful for any competitor! It is your choice: make sharing or not your ideas! High score shows really good idea or not. You should enjoy the gift of fate, that you can learn new techniques, doesnt it?\n\nEveryone has time to improve own solutions. You have 2 months.\n\nAfter [DeepFake competition](https://www.kaggle.com/c/deepfake-detection-challenge) I don't see any point in continuing to do any competitions. Organizers made big fault with running notebooks and [I proved it](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/145982)! But kaggle team closed eyes on this precedent. Huge number of ideas and techniques could help anyone competitor, but I lost without a mark of participation even. I didn't break the rules.\n\nKaggle Team needs to pay attention on quality of competitions and organizers of big contests. \n\nI am going to continue publishing kernels, more more and more.\n\nThank you for your patience.", "856641": "I think we should share good inference kernel. just put our models in private dataset.", "856755": "Yes this is a good idea!", "856760": "\"You should enjoy the gift of fate, that you can learn new techniques\": I'm just talking about inference kernels and I don't think you learn new techniques from inference kernels. Other than this, I don't understand what you are talking about as I'm bit new to this challenges. Anyways this was just a request. Thanks for the opinion. @shonenkov", "856858": "I'm confused what happened with your notebook in the deepfake competition. You scored very high on the public LB but got 113th on the private LB?", "856871": "I 100% understand your frustration, and I agree with you, but publishing gold-level, high-scoring kernels won't solve anything. By doing this, you will only ruin the competition for a lot of people. Please do not do that. Any great idea, trick, or solution is welcome, but please do not share the trained weights.", "856895": "Yes exactly! That's what I wanted to communicate with this thread!", "856932": "matthewmasters of course NO! Facebook crushed working kernel 113 public in the private stage and kicked many participants from competition without explanation!", "856957": "I don't understand why [my message](https://www.kaggle.com/c/global-wheat-detection/discussion/152846#856632) was downvoted!? WHY? Kaggle community, competition run less month! You should think about team spirit and use jump start for solving this competition. But you whine and think only about yourself and your positions on the leaderboard. It is disheartening, really!\n\nMoreover, [last kernel](https://www.kaggle.com/shonenkov/oof-evaluation-efficientdet) that I published, I don't share weights - I am able to listen to your requests. But you gave downvotes without reason. 👎", "856973": "Yeah, I don't understand why they're downvoting your reply to me either. I just looked into what happened in the FB DeepFake detection challenge and that's terrible and definitely discouraging for future competitions :(\n\nAnyways, I appreciate all your selfless contributions to this competition! Hopefully you will have better luck here", "857044": "Alex, I think your message was downvoted because of your tone (not because of what you shared): `\"I am going to continue publishing kernels, more more and more.\"` It feels like you don't give a shit about what other competitors asked (nicely).\n\nI saw your latest kernel (without weights). I think that is a good way to share. \n\nI think a Kaggle competition is like running a marathon. I ran once, and I had to train almost a year before. Before a Kaggle competition, I learned lots of things, studying hours, days, taking courses, etc. I prepared myself to compete, just like before the marathon.\n\nI felt ready, I felt motivated. After the first few miles, my position started to decrease. That was ok. I knew there were lots of runners better than I was. Same as here, I don't care about my position, there are dozens of Kagglers better than I am. \n\nIf a world-class Kenyan runner shares anything, I will listen 100%. Same as here, If you (or any other Kaggler) share anything useful, I will thank, and I will use that information.\n\nBut imagine this: After the third of that marathon, that Kenyan runner started to Uber dozens of fat, lazy, untrained \"competitors\". And he said, don't worry, it is only the first third, there is plenty of time to beat them.\n(I can not say how fu\\*\\*ing annoying that would be.)\n\nMany Kagglers put lots of time, work, and effort into whatever they've achieved. By sharing high-scoring weights, you only destroy all of their motivation. The problem is the high scores (more precisely, the forks and resubmits those high scores without ANY effort), **not your sharings**.\n\nSo, keep up the good work and share your thoughts, tricks, ideas, and code (without any weights). I am sure you will get hundreds of upvotes.", "857077": "Thank you for explanation @pestipeti ! \n\nMy words mean that nobody's opinion will not be able stop publications of my ideas and techniques.\n\nI am ready sharing weights only by request in comments. If need I will share dataset! It is really important for economy money and save the environment (nature), because using my weights you can tune your model ~5*8h=40h faster! Also I see perspective use my weights for increasing image_size by original size. I believe it gives boost.", "857098": "This is not a new problem on Kaggle. One or two years ago, Kamil and Kuba from the Neptune.ai team came up with an acceptable solution: only share a solution with a score below medal level.\n\n[Take a look at this thread](https://www.kaggle.com/c/tgs-salt-identification-challenge/discussion/62162#365436) Lots of respected GMs (Anokas, Dieter, CPMP, and many more) supported the idea.", "857114": "I don't think that only started competition needs in this correction. I am agree It is really good rule for middle and last stage of competition (2-3 weeks before merge deadline). But here is absolutely another situation. \n\nI repeat, stop downvoting my messages! I don't share my weights without need", "857124": "pestipeti also, first of all kaggle team has to support the competition spirit. Without correcting precedents such as DeepFake from facebook, any rules for keeping competition spirit are artificial.", "857125": "Today you are afraid of my kernel, and tomorrow you will be disqualified for no any reason after hardworking 2 months! Think about it, community", "857222": "Nice interesting thread here. Upvote for that!\nSince I'm so invested in this comp here are my 2 cents.\nA) The number one goal of the comp organizers is to get the best score possible.\nB) My number zero goal is to learn.\nC) My number one goal is to get the best score of all competitors.\n\nSharing great solutions helps A) for sure because it instantly raises the bar. Anyone who was already doing well can incorporate the extra knowledge into their solution and maybe do even better.\n\nSharing solutions helps B) for sure. No questions about that.\n\nSharing solutions has little effect on C). It just raises the floor. If you were standing tall before, you will probably be standing tall after (as long as you decide to make use of the shared knowledge). The only situation when sharing solutions screws you over is when that solution renders your own tricks moot. That can hurt... but I would take that hit any day in exchange for the benefits.\n\nAnd lastly, I don't care about the people up the leaderboard who copied the solution. You think I'm sitting here in 248th place because I don't know how to fork a kernel? It's actually easier to experiment without using bigger better models, or wasting GPU time doing pseudo labelling. I have to innovate if I want to win. I can apply the big guns later.", "857256": "alexandersoare you are right. I shared what I felt. No offense to anyone. Sharing absolutely helps to learn new things.", "857329": "Let's look at one extreme. There is no leaderboard, and no way to share. Teams compete privately, and a winner is announced at the end of the competition. No one shares code or ideas. Does that seem like it would be a better format?\n\nMost don't think so. (BTW, there are challenges on other platforms that run this way, for people would rather compete in such a format. They're just not nearly as popular.) \n\nThe reason people don't prefer that format is that there is a _tremendous_ personal benefit from seeing what others are doing and learning from from their approach. Does it make it hard? Yes. Can it be more stressful and frustrating for people trying to rank highly? Of course. (I've been there, many times.) Bet it is much better than the alternative.\n\nSo, we've ruled out that extreme. Once we've ruled out that extreme, it's just a matter of degree. Everyone is going to have a different opinion about how much should be shared and when it's appropriate for it to be shared. There's no way to satisfy all of these conditions.\n\nSo, we continue to emphasize that people shouldn't post high-scoring public notebooks the last week of the competition. But, other than that, the benefit from people openly sharing their work seems to vastly outweigh the downsides.", "857336": "With that said, I also find it annoying when people are merely copy/pasting and not putting their own effort into the competition. But that's a small downside compared to the incredible amount of learning I've personally gained from Kaggle. (And @kaushal2896, I similarly hope you find tremendous benefit here.)", "857339": "Yes, you are right we should stick to not sharing high scoring kernels in last couple of weeks of the competition. before that it doesn't matter as everyone has equal opportunity to improve their models.", "857345": "rohitsingh9990 Before I joined Kaggle, and when I was still relatively new in machine learning, I entered an image competition. I was doing pretty well. But 3 days before the competition ended, someone posted code in the forums that was an end-to-end pipeline for the problem, using their company's open-source platform. (This was before notebooks even, so it was a post of the raw code.)\n\nPeople started using it and I dropped on the leaderboard quickly. But, I couldn't get the dependencies to work, and was never able to make use of the code. I ended up dropping significantly, and was very frustrated.\n\nSo, I'm a big fan of sharing, but also for respecting the competitive spirit as well. :-)", "857349": "Same here i also like to share a lot, but i think there is another reason for sharing high scoring kernel as it is the easiest way to gain attention and quick upvotes, which i think is fine. if they worked hard they deserve that.", "857350": "This Comp is in its beginning phase. Sharing, in the beginning, makes it possible for problem to get solved to its true potential solution. But one should refrain releasing kernals towards the end of the comptetions.", "857358": "Yes, that's what we should follow.", "857362": "inversion you are absolutely right. Public kernels are the source of learning. If no one shares any ideas and just compete privately, beginners like me won't be able to learn. Because of sharing, I can look into what others are trying and most importantly why. I'm neither requesting to stop sharing kernels publicly nor I'm questioning this notebook sharing feature of Kaggle. I just felt that we should understand and not share just the inference kernels which scores very high. Sharing baselines kernels, ideas of approaching the problems, data preprocessing, etc. are super helpful. Thanks for your reply BTW.", "857366": "kaushal2896 I definitely see your point. Sharing an inference kernel is effectively the same as just sharing a submission file. No one learns from that.", "857382": "inversion \nI think disabling/removing the \"Best score\" sorting option during a competition would solve most of the issues. These copy-paste-fork gurus won't bother to look through dozens of pages searching for high scores. \n\nHigh-scoring notebooks 99% of the time is all about getting lots of votes and a gold kernel medal. There are millions of ensemble of ensemble notebook with zero additional value. \n\nThey are annoying for those who want to compete, and they are ruining the reputation of the kernel tier.\n\nPlease consider these:\n- remove the best score sorting option (during a competition)\n- do not advertise the public scores (during a competition)\n\nHere is some related topic (probably you know all about these):\n- [It's all about upvotes and medals](https://www.kaggle.com/c/m5-forecasting-accuracy/discussion/150568)\n- [(Open solution journal) friendly request for team \"Neptune ML\"](https://www.kaggle.com/c/tgs-salt-identification-challenge/discussion/62162)\n- [The leaderboard has been destroyed](https://www.kaggle.com/c/severstal-steel-defect-detection/discussion/110670)\n\nAll of the frustration is because of the advertised public scores during a competition.", "857529": "I think making the kernels public during the initial stage of a competition is useful for the beginners as they get an idea about how to appraoch towards a good solution.In the end the kernels should be kept private so that the kernel with the best ideas wins.", "857903": "Actually this a custom in Kaggle . In any competitions , guys will keep publishing kernels (include inference kernel) in the early stage , but during the last month , they will cease to do it .", "858209": "Since the competition just gets started, why do we worry so much on high-score kernels. I think everyone have time to learn from them.", "858272": "I am doing an academic study on this issue. Whether or not the outcome of a contest improves when contestants share knowledge. The argument against it in the creativty literature is that it creates a form of cognitive fixation - once you saw an idea your mind is focusing on that idea, and you are less likely to come up with your own original idea. Especially when you see others ideas before you have come up with an own. \nI am wondering whether sharing behavior also depends on the prize, the contest owner, or the type of problem (commercial vs. social).", "858362": "kaushal2896 I think I get your point exactly, and I agree on that. In Bengali.Ai competition when @iafoss shared his wonderful kernel that eventually brought some competitor in the medal zone at the end, and funny things were some of the competitors weren't too active to that competition. And in this competition @shonenkov has some wonderful high scored kernel and maybe the same things can repeat here too. \n\nIMHO, sharing a high score kernel is OK as long as the weights are in private. In addition, sharing ideas that can reach the gold zone is always controversial, but it may be ok since we have 2 months till now. @shonenkov notebooks are just awesome and we need more and more high-quality notebooks from him.", "858659": "Actually high scoring kernels being public is a good thing! You can learn many things. Also I have tried test submitting those high scored kernels, It turns out that people already have work on this, this does not make so much difference on the leaderboard. Moreover, you can learn stuff", "858747": "As a beginner, Notebook reference is fantastic to learn but at the same time, I think if some user is referring to notebook related to a particular competition then code matching must be done to check the plagarism.\n#HappyLearning", "859920": "Great God, all this time I thought it was Lafoss.", "880417": "Thanks to everyone who shared. I can't bear to give you downvotes. Sharing is free, and at the same time, the greater the ability, the greater the responsibility, and more things to consider, such as fairness. I think you should maintain fairness while sharing, and don't let those lazy people beat those who work hard. \n\nThanks to everyone who shared again." }, "source": "meta" }