Is there a reason this competition seems to have so few grandmasters taking part?
", "messages": [ { "id": "1326548", "postDate": "05/28/2021 14:32:39", "content": "Is there a reason this competition seems to have so few grandmasters taking part?
", "rawMarkdown": "Is there a reason this competition seems to have so few grandmasters taking part?", "votes": null }, { "id": "1326615", "postDate": "05/28/2021 15:14:34", "content": "Lottery ticket problem: there's no way to perform meaningful local CV and the public LB is drawn from a different distribution than the private LB. This means that there's a high risk of a random shake-up that's out of your control. Jane Street's another competition which GMs avoided due to the same issue.
No good public starter notebook: many GMs tend to work off a good starter notebook and try to rush a competition in the last few weeks before the deadline. None has been shared… so far.
Many are currently tied up in BirdClef and BMS: there should be a minor influx of GMs after both competitions finish next week.
I'd be leery of working on this competition myself except I do NLP for a living and interesting NLP competitions are few and far between ;) Commonlit is the other NLP competition and is looking to be an uninteresting exercise in tweaking straight-forward Transformer models.
", "rawMarkdown": "1. Lottery ticket problem: there's no way to perform meaningful local CV and the public LB is drawn from a different distribution than the private LB. This means that there's a high risk of a random shake-up that's out of your control. Jane Street's another competition which GMs avoided due to the same issue.\n\n\n2. No good public starter notebook: many GMs tend to work off a good starter notebook and try to rush a competition in the last few weeks before the deadline. None has been shared... so far. \n\n\n3. Many are currently tied up in BirdClef and BMS: there should be a minor influx of GMs after both competitions finish next week. \n\nI'd be leery of working on this competition myself except I do NLP for a living and interesting NLP competitions are few and far between ;) Commonlit is the other NLP competition and is looking to be an uninteresting exercise in tweaking straight-forward Transformer models.", "votes": null }, { "id": "1327155", "postDate": "05/29/2021 02:10:30", "content": "Have a look at the posts where I tried to get answers from host and Kaggle staff on metric definition and target definition. I haven't seen any satisfactory answer. And many GMs read these posts and other posts reporting similar issues.
\nIt is a pity because I think the topic is extremely interesting.
", "rawMarkdown": "Have a look at the posts where I tried to get answers from host and Kaggle staff on metric definition and target definition. I haven't seen any satisfactory answer. And many GMs read these posts and other posts reporting similar issues.\n\nIt is a pity because I think the topic is extremely interesting.", "votes": null }, { "id": "1327386", "postDate": "05/29/2021 08:14:19", "content": "\n\nNo good public starter notebook: many GMs tend to work off a good starter notebook and try to rush a competition in the last few weeks before the deadline. None has been shared… so far.
\n
I don't agree with that. Many top GMs rarely work off from public kernels.
\nYou other points are spot on. GMs often try to avoid random competitions.
", "rawMarkdown": "> No good public starter notebook: many GMs tend to work off a good starter notebook and try to rush a competition in the last few weeks before the deadline. None has been shared… so far.\n\nI don't agree with that. Many top GMs rarely work off from public kernels.\n\nYou other points are spot on. GMs often try to avoid random competitions.", "votes": null }, { "id": "1327441", "postDate": "05/29/2021 09:35:05", "content": "@philippsinger some well ranked GMs mostly run public notebooks though. But I hope they are exceptions.
", "rawMarkdown": "philippsinger some well ranked GMs mostly run public notebooks though. But I hope they are exceptions.", "votes": null }, { "id": "1327636", "postDate": "05/29/2021 13:33:13", "content": "@cpmpml I know, I know. Some do it, some don't. I try to avoid it as much as possible. Often times I do not even look at them. If you start from public kernels you are most likely way more similar to everyone else than if you don't which reduces your chances of having an outstanding solution. I unscientifically believe there is a correlation between consistently doing well on private LB and writing your own code / solutions.
", "rawMarkdown": "cpmpml I know, I know. Some do it, some don't. I try to avoid it as much as possible. Often times I do not even look at them. If you start from public kernels you are most likely way more similar to everyone else than if you don't which reduces your chances of having an outstanding solution. I unscientifically believe there is a correlation between consistently doing well on private LB and writing your own code / solutions.", "votes": null }, { "id": "1327653", "postDate": "05/29/2021 13:55:42", "content": "@philippsinger this applies to me as is:
\n\n", "rawMarkdown": "philippsinger this applies to me as is:\n\n> I try to avoid it as much as possible. Often times I do not even look at them. If you start from public kernels you are most likely way more similar to everyone else than if you don't which reduces your chances of having an outstanding solution. I unscientifically believe there is a correlation between consistently doing well on private LB and writing your own code / solutions.", "votes": null }, { "id": "1328140", "postDate": "05/30/2021 00:14:34", "content": "I try to avoid it as much as possible. Often times I do not even look at them. If you start from public kernels you are most likely way more similar to everyone else than if you don't which reduces your chances of having an outstanding solution. I unscientifically believe there is a correlation between consistently doing well on private LB and writing your own code / solutions.
\n
To be fair, being able to write your own high public LB code/solution from scratch is an indicator of competency.
\nOn the flip side, high public LB shared notebooks might be biased towards performing poorly on private LB because they are self-selected to overfit to the public LB.
", "rawMarkdown": "To be fair, being able to write your own high public LB code/solution from scratch is an indicator of competency. \n\nOn the flip side, high public LB shared notebooks might be biased towards performing poorly on private LB because they are self-selected to overfit to the public LB.", "votes": null }, { "id": "1328144", "postDate": "05/30/2021 00:27:07", "content": "\n\nhigh public LB shared notebooks might be biased towards performing poorly on private LB because they are self-selected to overfit to the public LB.
\n
It is why I don't use them in general. They seldomly include proper cross validation, although it is improving lately.
", "rawMarkdown": "> high public LB shared notebooks might be biased towards performing poorly on private LB because they are self-selected to overfit to the public LB. \n\nIt is why I don't use them in general. They seldomly include proper cross validation, although it is improving lately.", "votes": null }, { "id": "1344717", "postDate": "06/11/2021 05:11:34", "content": "thats 100% the reason for me not to have entered
", "rawMarkdown": "thats 100% the reason for me not to have entered", "votes": null }, { "id": "1344721", "postDate": "06/11/2021 05:12:48", "content": "and with 12days to go, as far as I know bugs in metric have not been fixed, and labels are still tempered with
\nhttps://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/245126
", "rawMarkdown": "and with 12days to go, as far as I know bugs in metric have not been fixed, and labels are still tempered with\n\nhttps://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/245126", "votes": null } ], "comments": [ { "id": 1326615, "author_name": "leecming", "author_url": "", "post_date": "05/28/2021 15:14:34", "content": "Lottery ticket problem: there's no way to perform meaningful local CV and the public LB is drawn from a different distribution than the private LB. This means that there's a high risk of a random shake-up that's out of your control. Jane Street's another competition which GMs avoided due to the same issue.
No good public starter notebook: many GMs tend to work off a good starter notebook and try to rush a competition in the last few weeks before the deadline. None has been shared… so far.
Many are currently tied up in BirdClef and BMS: there should be a minor influx of GMs after both competitions finish next week.
I'd be leery of working on this competition myself except I do NLP for a living and interesting NLP competitions are few and far between ;) Commonlit is the other NLP competition and is looking to be an uninteresting exercise in tweaking straight-forward Transformer models.
", "votes": null, "replies": [ { "id": 1327386, "author_name": "philippsinger", "author_url": "", "post_date": "05/29/2021 08:14:19", "content": "\n\nNo good public starter notebook: many GMs tend to work off a good starter notebook and try to rush a competition in the last few weeks before the deadline. None has been shared… so far.
\n
I don't agree with that. Many top GMs rarely work off from public kernels.
\nYou other points are spot on. GMs often try to avoid random competitions.
", "votes": null, "replies": [] }, { "id": 1327441, "author_name": "cpmpml", "author_url": "", "post_date": "05/29/2021 09:35:05", "content": "@philippsinger some well ranked GMs mostly run public notebooks though. But I hope they are exceptions.
", "votes": null, "replies": [] }, { "id": 1327636, "author_name": "philippsinger", "author_url": "", "post_date": "05/29/2021 13:33:13", "content": "@cpmpml I know, I know. Some do it, some don't. I try to avoid it as much as possible. Often times I do not even look at them. If you start from public kernels you are most likely way more similar to everyone else than if you don't which reduces your chances of having an outstanding solution. I unscientifically believe there is a correlation between consistently doing well on private LB and writing your own code / solutions.
", "votes": null, "replies": [] }, { "id": 1327653, "author_name": "cpmpml", "author_url": "", "post_date": "05/29/2021 13:55:42", "content": "@philippsinger this applies to me as is:
\n\n", "votes": null, "replies": [] }, { "id": 1328140, "author_name": "leecming", "author_url": "", "post_date": "05/30/2021 00:14:34", "content": "I try to avoid it as much as possible. Often times I do not even look at them. If you start from public kernels you are most likely way more similar to everyone else than if you don't which reduces your chances of having an outstanding solution. I unscientifically believe there is a correlation between consistently doing well on private LB and writing your own code / solutions.
\n
To be fair, being able to write your own high public LB code/solution from scratch is an indicator of competency.
\nOn the flip side, high public LB shared notebooks might be biased towards performing poorly on private LB because they are self-selected to overfit to the public LB.
", "votes": null, "replies": [] }, { "id": 1328144, "author_name": "cpmpml", "author_url": "", "post_date": "05/30/2021 00:27:07", "content": "\n\nhigh public LB shared notebooks might be biased towards performing poorly on private LB because they are self-selected to overfit to the public LB.
\n
It is why I don't use them in general. They seldomly include proper cross validation, although it is improving lately.
", "votes": null, "replies": [] } ] }, { "id": 1327155, "author_name": "cpmpml", "author_url": "", "post_date": "05/29/2021 02:10:30", "content": "Have a look at the posts where I tried to get answers from host and Kaggle staff on metric definition and target definition. I haven't seen any satisfactory answer. And many GMs read these posts and other posts reporting similar issues.
\nIt is a pity because I think the topic is extremely interesting.
", "votes": null, "replies": [ { "id": 1344717, "author_name": "christofhenkel", "author_url": "", "post_date": "06/11/2021 05:11:34", "content": "thats 100% the reason for me not to have entered
", "votes": null, "replies": [] } ] }, { "id": 1344721, "author_name": "christofhenkel", "author_url": "", "post_date": "06/11/2021 05:12:48", "content": "and with 12days to go, as far as I know bugs in metric have not been fixed, and labels are still tempered with
\nhttps://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/245126
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1326548": "Is there a reason this competition seems to have so few grandmasters taking part?", "1326615": "1. Lottery ticket problem: there's no way to perform meaningful local CV and the public LB is drawn from a different distribution than the private LB. This means that there's a high risk of a random shake-up that's out of your control. Jane Street's another competition which GMs avoided due to the same issue.\n\n\n2. No good public starter notebook: many GMs tend to work off a good starter notebook and try to rush a competition in the last few weeks before the deadline. None has been shared... so far. \n\n\n3. Many are currently tied up in BirdClef and BMS: there should be a minor influx of GMs after both competitions finish next week. \n\nI'd be leery of working on this competition myself except I do NLP for a living and interesting NLP competitions are few and far between ;) Commonlit is the other NLP competition and is looking to be an uninteresting exercise in tweaking straight-forward Transformer models.", "1327155": "Have a look at the posts where I tried to get answers from host and Kaggle staff on metric definition and target definition. I haven't seen any satisfactory answer. And many GMs read these posts and other posts reporting similar issues.\n\nIt is a pity because I think the topic is extremely interesting.", "1327386": "> No good public starter notebook: many GMs tend to work off a good starter notebook and try to rush a competition in the last few weeks before the deadline. None has been shared… so far.\n\nI don't agree with that. Many top GMs rarely work off from public kernels.\n\nYou other points are spot on. GMs often try to avoid random competitions.", "1327441": "philippsinger some well ranked GMs mostly run public notebooks though. But I hope they are exceptions.", "1327636": "cpmpml I know, I know. Some do it, some don't. I try to avoid it as much as possible. Often times I do not even look at them. If you start from public kernels you are most likely way more similar to everyone else than if you don't which reduces your chances of having an outstanding solution. I unscientifically believe there is a correlation between consistently doing well on private LB and writing your own code / solutions.", "1327653": "philippsinger this applies to me as is:\n\n> I try to avoid it as much as possible. Often times I do not even look at them. If you start from public kernels you are most likely way more similar to everyone else than if you don't which reduces your chances of having an outstanding solution. I unscientifically believe there is a correlation between consistently doing well on private LB and writing your own code / solutions.", "1328140": "To be fair, being able to write your own high public LB code/solution from scratch is an indicator of competency. \n\nOn the flip side, high public LB shared notebooks might be biased towards performing poorly on private LB because they are self-selected to overfit to the public LB.", "1328144": "> high public LB shared notebooks might be biased towards performing poorly on private LB because they are self-selected to overfit to the public LB. \n\nIt is why I don't use them in general. They seldomly include proper cross validation, although it is improving lately.", "1344717": "thats 100% the reason for me not to have entered", "1344721": "and with 12days to go, as far as I know bugs in metric have not been fixed, and labels are still tempered with\n\nhttps://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/245126" }, "source": "meta" }