{
  "id": 97918,
  "title": "Officially LB is broken",
  "url": "/competitions/generative-dog-images/discussion/97918",
  "author_name": "",
  "post_date": "2019-06-29T18:30:40.906726Z",
  "votes": 18,
  "comment_count": 10,
  "views": 0,
  "content": "<p></p>\n\n<p>Just in case I'm gonna remember:</p>\n\n<blockquote>\n  <p>Important Warning: This competition has an experimental format and submission style (images as submission). <strong>Competitors must use generative methods to create their submission images and are not permitted to make submissions that include any images already classified as dogs or altered versions of such images.</strong>\n  To enforce and prevent cheating, we reserve the right to: (a) Visually inspect all participants' submitted images, (b) review any submitted source code, (c) use these reviews to identify violators or determine winners, and (d) disqualify participants from the competition who are found in violation. This is also specified in the competition's rules</p>\n</blockquote>\n\n<p>PS: Maybe I'm wrong and this people developed a great GAN, if that's the case, sorry.</p>",
  "messages": [
    {
      "id": "564647",
      "postDate": "06/29/2019 18:30:40",
      "content": "<p></p>\n\n<p>Just in case I'm gonna remember:</p>\n\n<blockquote>\n  <p>Important Warning: This competition has an experimental format and submission style (images as submission). <strong>Competitors must use generative methods to create their submission images and are not permitted to make submissions that include any images already classified as dogs or altered versions of such images.</strong>\n  To enforce and prevent cheating, we reserve the right to: (a) Visually inspect all participants' submitted images, (b) review any submitted source code, (c) use these reviews to identify violators or determine winners, and (d) disqualify participants from the competition who are found in violation. This is also specified in the competition's rules</p>\n</blockquote>\n\n<p>PS: Maybe I'm wrong and this people developed a great GAN, if that's the case, sorry.</p>",
      "rawMarkdown": "<img src=\"https://i.ibb.co/RB8w9Tf/Captura-de-pantalla-de-2019-06-29-20-24-12.png\" alt=\"Captura-de-pantalla-de-2019-06-29-20-24-12\">\n\nJust in case I'm gonna remember:\n\n&gt;Important Warning: This competition has an experimental format and submission style (images as submission). **Competitors must use generative methods to create their submission images and are not permitted to make submissions that include any images already classified as dogs or altered versions of such images.**\n&gt;To enforce and prevent cheating, we reserve the right to: (a) Visually inspect all participants' submitted images, (b) review any submitted source code, (c) use these reviews to identify violators or determine winners, and (d) disqualify participants from the competition who are found in violation. This is also specified in the competition's rules\n\nPS: Maybe I'm wrong and this people developed a great GAN, if that's the case, sorry.",
      "votes": null
    },
    {
      "id": "564651",
      "postDate": "06/29/2019 18:36:42",
      "content": "<p>So much for respecting the rules.</p>",
      "rawMarkdown": "So much for respecting the rules.",
      "votes": null
    },
    {
      "id": "564820",
      "postDate": "06/30/2019 03:48:17",
      "content": "<p>@Nanashi</p>\n\n<p>I agree with you.</p>",
      "rawMarkdown": "Nanashi\n\nI agree with you.",
      "votes": null
    },
    {
      "id": "564905",
      "postDate": "06/30/2019 06:29:19",
      "content": "<p>You don't need to worry because the competition hosts will review all deliverables before they will declare the winner.  Maybe a tedious part from their end, but let's see how they can handle those risks. Btw, you can use that high-score kernel as part of your pre-processing or something then feed it to your NN. You need to be creative. But in the end, I think the winning solutions that Kaggle might consider are those full-pipeline creative Generative Adversarial Networks (GANs). I think that there's kaggle evaluation aside this LB metrics! </p>",
      "rawMarkdown": "You don't need to worry because the competition hosts will review all deliverables before they will declare the winner.  Maybe a tedious part from their end, but let's see how they can handle those risks. Btw, you can use that high-score kernel as part of your pre-processing or something then feed it to your NN. You need to be creative. But in the end, I think the winning solutions that Kaggle might consider are those full-pipeline creative Generative Adversarial Networks (GANs). I think that there's kaggle evaluation aside this LB metrics!",
      "votes": null
    },
    {
      "id": "564921",
      "postDate": "06/30/2019 07:01:22",
      "content": "<p>wow!</p>",
      "rawMarkdown": "wow!",
      "votes": null
    },
    {
      "id": "564928",
      "postDate": "06/30/2019 07:14:21",
      "content": "<p>only for the winners. </p>",
      "rawMarkdown": "only for the winners.",
      "votes": null
    },
    {
      "id": "565038",
      "postDate": "06/30/2019 10:36:08",
      "content": "<p>I don't think they will check all the kernels that could potentially be in the medal. I'm looking forward to seeing how they will deal with it.</p>",
      "rawMarkdown": "I don't think they will check all the kernels that could potentially be in the medal. I'm looking forward to seeing how they will deal with it.",
      "votes": null
    },
    {
      "id": "565133",
      "postDate": "06/30/2019 13:35:05",
      "content": "<p>I would consider adding one simple rule that kernels with best final submission private score must be made public in order to get medal/points for them. Kaggle can't review all models for sure, but community can. Someone who is just behind the line to get medals will definitely spend some time to review some kernels above.</p>",
      "rawMarkdown": "I would consider adding one simple rule that kernels with best final submission private score must be made public in order to get medal/points for them. Kaggle can't review all models for sure, but community can. Someone who is just behind the line to get medals will definitely spend some time to review some kernels above.",
      "votes": null
    },
    {
      "id": "565143",
      "postDate": "06/30/2019 13:58:04",
      "content": "<p>I think it is too early to worry.\nPeople saw the kernel, forked it and submitted. It doesn't mean they will choose it as a final or if chosen it will pass.\nHowever, this is an experimental competition and GAN is a slick to measure - no wonder LB is broken in less than a day</p>",
      "rawMarkdown": "I think it is too early to worry.\nPeople saw the kernel, forked it and submitted. It doesn't mean they will choose it as a final or if chosen it will pass.\nHowever, this is an experimental competition and GAN is a slick to measure - no wonder LB is broken in less than a day",
      "votes": null
    },
    {
      "id": "565158",
      "postDate": "06/30/2019 14:12:28",
      "content": "<p><a href=\"/venheads\">@venheads</a> right! I think the same, the main problem is kaggle, they have to clarify the situation and generative models. But, if we have to use GANs (as I think) then the LB is now broken.\nHow to make sure that people use GANs? \n- submit the images\n- submit models.</p>\n\n<p>as you can see in this kernel <a href=\"https://www.kaggle.com/wendykan/demo-mifid-metric-for-dog-image-generation-comp\">Demo MiFID metric for Dog image generation comp</a> </p>\n\n<p>Also, as <a href=\"/alijs1\">@alijs1</a> said, winners must share the models (or write a public paper about them)</p>\n\n<p>PS: I'm about to break the LB</p>",
      "rawMarkdown": "venheads right! I think the same, the main problem is kaggle, they have to clarify the situation and generative models. But, if we have to use GANs (as I think) then the LB is now broken.\nHow to make sure that people use GANs? \n- submit the images\n- submit models.\n\nas you can see in this kernel [Demo MiFID metric for Dog image generation comp](https://www.kaggle.com/wendykan/demo-mifid-metric-for-dog-image-generation-comp) \n\nAlso, as @alijs1 said, winners must share the models (or write a public paper about them)\n\nPS: I'm about to break the LB",
      "votes": null
    },
    {
      "id": "565176",
      "postDate": "06/30/2019 14:39:12",
      "content": "<p>That is sad when LB is broken but GAN is fun\nWhat community could do here? Try to figure out and propose how to make LB more robust and unbreakable</p>",
      "rawMarkdown": "That is sad when LB is broken but GAN is fun\nWhat community could do here? Try to figure out and propose how to make LB more robust and unbreakable",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 564651,
      "author_name": "abhishek",
      "author_url": "",
      "post_date": "06/29/2019 18:36:42",
      "content": "<p>So much for respecting the rules.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 564820,
      "author_name": "ymuroya47",
      "author_url": "",
      "post_date": "06/30/2019 03:48:17",
      "content": "<p>@Nanashi</p>\n\n<p>I agree with you.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 564905,
      "author_name": "projdev",
      "author_url": "",
      "post_date": "06/30/2019 06:29:19",
      "content": "<p>You don't need to worry because the competition hosts will review all deliverables before they will declare the winner.  Maybe a tedious part from their end, but let's see how they can handle those risks. Btw, you can use that high-score kernel as part of your pre-processing or something then feed it to your NN. You need to be creative. But in the end, I think the winning solutions that Kaggle might consider are those full-pipeline creative Generative Adversarial Networks (GANs). I think that there's kaggle evaluation aside this LB metrics! </p>",
      "votes": null,
      "replies": [
        {
          "id": 564928,
          "author_name": "abhishek",
          "author_url": "",
          "post_date": "06/30/2019 07:14:21",
          "content": "<p>only for the winners. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 565038,
          "author_name": "francoisdubois",
          "author_url": "",
          "post_date": "06/30/2019 10:36:08",
          "content": "<p>I don't think they will check all the kernels that could potentially be in the medal. I'm looking forward to seeing how they will deal with it.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 565133,
          "author_name": "alijs1",
          "author_url": "",
          "post_date": "06/30/2019 13:35:05",
          "content": "<p>I would consider adding one simple rule that kernels with best final submission private score must be made public in order to get medal/points for them. Kaggle can't review all models for sure, but community can. Someone who is just behind the line to get medals will definitely spend some time to review some kernels above.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 564921,
      "author_name": "frtgnn",
      "author_url": "",
      "post_date": "06/30/2019 07:01:22",
      "content": "<p>wow!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 565143,
      "author_name": "venheads",
      "author_url": "",
      "post_date": "06/30/2019 13:58:04",
      "content": "<p>I think it is too early to worry.\nPeople saw the kernel, forked it and submitted. It doesn't mean they will choose it as a final or if chosen it will pass.\nHowever, this is an experimental competition and GAN is a slick to measure - no wonder LB is broken in less than a day</p>",
      "votes": null,
      "replies": [
        {
          "id": 565158,
          "author_name": "jesucristo",
          "author_url": "",
          "post_date": "06/30/2019 14:12:28",
          "content": "<p><a href=\"/venheads\">@venheads</a> right! I think the same, the main problem is kaggle, they have to clarify the situation and generative models. But, if we have to use GANs (as I think) then the LB is now broken.\nHow to make sure that people use GANs? \n- submit the images\n- submit models.</p>\n\n<p>as you can see in this kernel <a href=\"https://www.kaggle.com/wendykan/demo-mifid-metric-for-dog-image-generation-comp\">Demo MiFID metric for Dog image generation comp</a> </p>\n\n<p>Also, as <a href=\"/alijs1\">@alijs1</a> said, winners must share the models (or write a public paper about them)</p>\n\n<p>PS: I'm about to break the LB</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 565176,
          "author_name": "venheads",
          "author_url": "",
          "post_date": "06/30/2019 14:39:12",
          "content": "<p>That is sad when LB is broken but GAN is fun\nWhat community could do here? Try to figure out and propose how to make LB more robust and unbreakable</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "564647": "<img src=\"https://i.ibb.co/RB8w9Tf/Captura-de-pantalla-de-2019-06-29-20-24-12.png\" alt=\"Captura-de-pantalla-de-2019-06-29-20-24-12\">\n\nJust in case I'm gonna remember:\n\n&gt;Important Warning: This competition has an experimental format and submission style (images as submission). **Competitors must use generative methods to create their submission images and are not permitted to make submissions that include any images already classified as dogs or altered versions of such images.**\n&gt;To enforce and prevent cheating, we reserve the right to: (a) Visually inspect all participants' submitted images, (b) review any submitted source code, (c) use these reviews to identify violators or determine winners, and (d) disqualify participants from the competition who are found in violation. This is also specified in the competition's rules\n\nPS: Maybe I'm wrong and this people developed a great GAN, if that's the case, sorry.",
    "564651": "So much for respecting the rules.",
    "564820": "Nanashi\n\nI agree with you.",
    "564905": "You don't need to worry because the competition hosts will review all deliverables before they will declare the winner.  Maybe a tedious part from their end, but let's see how they can handle those risks. Btw, you can use that high-score kernel as part of your pre-processing or something then feed it to your NN. You need to be creative. But in the end, I think the winning solutions that Kaggle might consider are those full-pipeline creative Generative Adversarial Networks (GANs). I think that there's kaggle evaluation aside this LB metrics!",
    "564921": "wow!",
    "564928": "only for the winners.",
    "565038": "I don't think they will check all the kernels that could potentially be in the medal. I'm looking forward to seeing how they will deal with it.",
    "565133": "I would consider adding one simple rule that kernels with best final submission private score must be made public in order to get medal/points for them. Kaggle can't review all models for sure, but community can. Someone who is just behind the line to get medals will definitely spend some time to review some kernels above.",
    "565143": "I think it is too early to worry.\nPeople saw the kernel, forked it and submitted. It doesn't mean they will choose it as a final or if chosen it will pass.\nHowever, this is an experimental competition and GAN is a slick to measure - no wonder LB is broken in less than a day",
    "565158": "venheads right! I think the same, the main problem is kaggle, they have to clarify the situation and generative models. But, if we have to use GANs (as I think) then the LB is now broken.\nHow to make sure that people use GANs? \n- submit the images\n- submit models.\n\nas you can see in this kernel [Demo MiFID metric for Dog image generation comp](https://www.kaggle.com/wendykan/demo-mifid-metric-for-dog-image-generation-comp) \n\nAlso, as @alijs1 said, winners must share the models (or write a public paper about them)\n\nPS: I'm about to break the LB",
    "565176": "That is sad when LB is broken but GAN is fun\nWhat community could do here? Try to figure out and propose how to make LB more robust and unbreakable"
  },
  "source": "meta"
}