{
  "id": 228386,
  "title": "What is happening??!! 🤣",
  "url": "/competitions/coleridgeinitiative-show-us-the-data/discussion/228386",
  "author_name": "",
  "post_date": "2021-03-24T13:26:52.529704Z",
  "votes": 5,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hello fellow Kagglers,</p>\n<p>Huge scores on Day-1 essentially means either 1. There is a huge leakage in the datasets or 2. Scoring metric is not being calculated properly.<br>\n<br>\n<strong>UPDATE:- I have found out it's actually both!</strong></p>\n<p>I have a hunch that the algorithm is not penalizing FN. Thus even if we put the whole body of text as predicted value we will get a perfect score (Because obviously the citation will be a part of the full text).</p>\n<p>I can only think of these two possibilities.<br>\nWhat do you people think?</p>\n<p><strong>UPDATE (24 Mar 2021 19:15 IST):- Submissions have been disabled for the competition.</strong><br>\n<strong>UPDATE (25 Mar 2021 01:00 IST):- Submissions have been enabled again for the competition.</strong></p>",
  "messages": [
    {
      "id": "1251071",
      "postDate": "03/24/2021 13:26:52",
      "content": "<p>Hello fellow Kagglers,</p>\n<p>Huge scores on Day-1 essentially means either 1. There is a huge leakage in the datasets or 2. Scoring metric is not being calculated properly.<br>\n<br>\n<strong>UPDATE:- I have found out it's actually both!</strong></p>\n<p>I have a hunch that the algorithm is not penalizing FN. Thus even if we put the whole body of text as predicted value we will get a perfect score (Because obviously the citation will be a part of the full text).</p>\n<p>I can only think of these two possibilities.<br>\nWhat do you people think?</p>\n<p><strong>UPDATE (24 Mar 2021 19:15 IST):- Submissions have been disabled for the competition.</strong><br>\n<strong>UPDATE (25 Mar 2021 01:00 IST):- Submissions have been enabled again for the competition.</strong></p>",
      "rawMarkdown": "Hello fellow Kagglers,\n\nHuge scores on Day-1 essentially means either 1. There is a huge leakage in the datasets or 2. Scoring metric is not being calculated properly.\n~~I am guessing it is the later.~~\n**UPDATE:- I have found out it's actually both!**\n\nI have a hunch that the algorithm is not penalizing FN. Thus even if we put the whole body of text as predicted value we will get a perfect score (Because obviously the citation will be a part of the full text).\n\nI can only think of these two possibilities.\nWhat do you people think?\n\n**UPDATE (24 Mar 2021 19:15 IST):- Submissions have been disabled for the competition.**\n**UPDATE (25 Mar 2021 01:00 IST):- Submissions have been enabled again for the competition.**",
      "votes": null
    },
    {
      "id": "1251423",
      "postDate": "03/24/2021 18:54:24",
      "content": "<p>I think there is a possibility to go through the test dataset(as it is very small)and understand the datasets used and somehow make it as the result using code.  </p>\n<p>However there is a twist:-), Once the private leaderboard having 8K publications results are released the leaderboard is going to change a lot. </p>\n<p>Also public leader board might change after the evaluation metric is modified by the kaggle team(in progress). Lets see.</p>",
      "rawMarkdown": "I think there is a possibility to go through the test dataset(as it is very small)and understand the datasets used and somehow make it as the result using code.  \n\nHowever there is a twist:-), Once the private leaderboard having 8K publications results are released the leaderboard is going to change a lot. \n\nAlso public leader board might change after the evaluation metric is modified by the kaggle team(in progress). Lets see.",
      "votes": null
    },
    {
      "id": "1251433",
      "postDate": "03/24/2021 19:03:37",
      "content": "<p>I agree <a href=\"https://www.kaggle.com/archana20\" target=\"_blank\">@archana20</a> , Let's hope that's the case. Having said that, it's also very difficult to evaluate yourself if the CV and LB have such a huge gap 😅 </p>\n<p>I see that Kaggle has already started modifying the metric in background and people have started to move around. Let's hope it gets fixed soon. 😊</p>",
      "rawMarkdown": "I agree @archana20 , Let's hope that's the case. Having said that, it's also very difficult to evaluate yourself if the CV and LB have such a huge gap 😅 \n\nI see that Kaggle has already started modifying the metric in background and people have started to move around. Let's hope it gets fixed soon. 😊",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1251423,
      "author_name": "archana20",
      "author_url": "",
      "post_date": "03/24/2021 18:54:24",
      "content": "<p>I think there is a possibility to go through the test dataset(as it is very small)and understand the datasets used and somehow make it as the result using code.  </p>\n<p>However there is a twist:-), Once the private leaderboard having 8K publications results are released the leaderboard is going to change a lot. </p>\n<p>Also public leader board might change after the evaluation metric is modified by the kaggle team(in progress). Lets see.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1251433,
          "author_name": "manabendrarout",
          "author_url": "",
          "post_date": "03/24/2021 19:03:37",
          "content": "<p>I agree <a href=\"https://www.kaggle.com/archana20\" target=\"_blank\">@archana20</a> , Let's hope that's the case. Having said that, it's also very difficult to evaluate yourself if the CV and LB have such a huge gap 😅 </p>\n<p>I see that Kaggle has already started modifying the metric in background and people have started to move around. Let's hope it gets fixed soon. 😊</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1251071": "Hello fellow Kagglers,\n\nHuge scores on Day-1 essentially means either 1. There is a huge leakage in the datasets or 2. Scoring metric is not being calculated properly.\n~~I am guessing it is the later.~~\n**UPDATE:- I have found out it's actually both!**\n\nI have a hunch that the algorithm is not penalizing FN. Thus even if we put the whole body of text as predicted value we will get a perfect score (Because obviously the citation will be a part of the full text).\n\nI can only think of these two possibilities.\nWhat do you people think?\n\n**UPDATE (24 Mar 2021 19:15 IST):- Submissions have been disabled for the competition.**\n**UPDATE (25 Mar 2021 01:00 IST):- Submissions have been enabled again for the competition.**",
    "1251423": "I think there is a possibility to go through the test dataset(as it is very small)and understand the datasets used and somehow make it as the result using code.  \n\nHowever there is a twist:-), Once the private leaderboard having 8K publications results are released the leaderboard is going to change a lot. \n\nAlso public leader board might change after the evaluation metric is modified by the kaggle team(in progress). Lets see.",
    "1251433": "I agree @archana20 , Let's hope that's the case. Having said that, it's also very difficult to evaluate yourself if the CV and LB have such a huge gap 😅 \n\nI see that Kaggle has already started modifying the metric in background and people have started to move around. Let's hope it gets fixed soon. 😊"
  },
  "source": "meta"
}