{
  "id": 158947,
  "title": "A reminder to always make your kernels reproducible!",
  "url": "/competitions/global-wheat-detection/discussion/158947",
  "author_name": "",
  "post_date": "2020-06-15T21:21:03.383009900Z",
  "votes": 4,
  "comment_count": 2,
  "views": 0,
  "content": "<p>In one of my inference kernels, I used np.random.choice(glob('...')) and although I had set a static seed for numpy, the kernel's results were not reproducible because glob doesn't always return in the same order! This resulted in a sharp difference in results ±0.1. So just a friendly reminded to ensure your results are reproducible and don't leave anything up to chance</p>",
  "messages": [
    {
      "id": "887772",
      "postDate": "06/15/2020 21:21:03",
      "content": "<p>In one of my inference kernels, I used np.random.choice(glob('...')) and although I had set a static seed for numpy, the kernel's results were not reproducible because glob doesn't always return in the same order! This resulted in a sharp difference in results ±0.1. So just a friendly reminded to ensure your results are reproducible and don't leave anything up to chance</p>",
      "rawMarkdown": "In one of my inference kernels, I used np.random.choice(glob('...')) and although I had set a static seed for numpy, the kernel's results were not reproducible because glob doesn't always return in the same order! This resulted in a sharp difference in results ±0.1. So just a friendly reminded to ensure your results are reproducible and don't leave anything up to chance",
      "votes": null
    },
    {
      "id": "888246",
      "postDate": "06/16/2020 08:18:23",
      "content": "<p>Sorry if this seems to be a stupid question as I am new here but do we have to reproduce the same result of retraining the whole model if the organisers asked us to do so?</p>",
      "rawMarkdown": "Sorry if this seems to be a stupid question as I am new here but do we have to reproduce the same result of retraining the whole model if the organisers asked us to do so?",
      "votes": null
    },
    {
      "id": "889970",
      "postDate": "06/17/2020 08:29:21",
      "content": "<p>Good question! I did not know the answer either so I looked at <a href=\"https://www.kaggle.com/c/global-wheat-detection/rules\">the rules</a>. It states in section 11:\n&gt; As a condition to being awarded a Prize, a Prize winner must fulfill the following obligations:\n(a) deliver to the Competition Sponsor the final model's software code as used to generate the winning Submission and associated documentation. The delivered software code should follow <a href=\"https://www.kaggle.com/WinningModelDocumentationGuidelines\">these documentation guidelines</a>, must be capable of generating the winning Submission, and contain a description of resources required to build and/or run the executable code successfully. If the final model’s software code includes generally commercially available software that is not owned by you, but that can be procured by the Competition Sponsor without undue expense, then instead of delivering the code for that software to the Competition Sponsor, you must identify that software, method for procuring it, and any parameters or other information necessary to replicate the winning Submission</p>\n\n<p>So yes, you must be able to provide fully reproducible results if you intend on winning!</p>",
      "rawMarkdown": "Good question! I did not know the answer either so I looked at [the rules](https://www.kaggle.com/c/global-wheat-detection/rules). It states in section 11:\n&gt; As a condition to being awarded a Prize, a Prize winner must fulfill the following obligations:\n(a) deliver to the Competition Sponsor the final model's software code as used to generate the winning Submission and associated documentation. The delivered software code should follow [these documentation guidelines](https://www.kaggle.com/WinningModelDocumentationGuidelines), must be capable of generating the winning Submission, and contain a description of resources required to build and/or run the executable code successfully. If the final model’s software code includes generally commercially available software that is not owned by you, but that can be procured by the Competition Sponsor without undue expense, then instead of delivering the code for that software to the Competition Sponsor, you must identify that software, method for procuring it, and any parameters or other information necessary to replicate the winning Submission\n\nSo yes, you must be able to provide fully reproducible results if you intend on winning!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 888246,
      "author_name": "bendang",
      "author_url": "",
      "post_date": "06/16/2020 08:18:23",
      "content": "<p>Sorry if this seems to be a stupid question as I am new here but do we have to reproduce the same result of retraining the whole model if the organisers asked us to do so?</p>",
      "votes": null,
      "replies": [
        {
          "id": 889970,
          "author_name": "matthewmasters",
          "author_url": "",
          "post_date": "06/17/2020 08:29:21",
          "content": "<p>Good question! I did not know the answer either so I looked at <a href=\"https://www.kaggle.com/c/global-wheat-detection/rules\">the rules</a>. It states in section 11:\n&gt; As a condition to being awarded a Prize, a Prize winner must fulfill the following obligations:\n(a) deliver to the Competition Sponsor the final model's software code as used to generate the winning Submission and associated documentation. The delivered software code should follow <a href=\"https://www.kaggle.com/WinningModelDocumentationGuidelines\">these documentation guidelines</a>, must be capable of generating the winning Submission, and contain a description of resources required to build and/or run the executable code successfully. If the final model’s software code includes generally commercially available software that is not owned by you, but that can be procured by the Competition Sponsor without undue expense, then instead of delivering the code for that software to the Competition Sponsor, you must identify that software, method for procuring it, and any parameters or other information necessary to replicate the winning Submission</p>\n\n<p>So yes, you must be able to provide fully reproducible results if you intend on winning!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "887772": "In one of my inference kernels, I used np.random.choice(glob('...')) and although I had set a static seed for numpy, the kernel's results were not reproducible because glob doesn't always return in the same order! This resulted in a sharp difference in results ±0.1. So just a friendly reminded to ensure your results are reproducible and don't leave anything up to chance",
    "888246": "Sorry if this seems to be a stupid question as I am new here but do we have to reproduce the same result of retraining the whole model if the organisers asked us to do so?",
    "889970": "Good question! I did not know the answer either so I looked at [the rules](https://www.kaggle.com/c/global-wheat-detection/rules). It states in section 11:\n&gt; As a condition to being awarded a Prize, a Prize winner must fulfill the following obligations:\n(a) deliver to the Competition Sponsor the final model's software code as used to generate the winning Submission and associated documentation. The delivered software code should follow [these documentation guidelines](https://www.kaggle.com/WinningModelDocumentationGuidelines), must be capable of generating the winning Submission, and contain a description of resources required to build and/or run the executable code successfully. If the final model’s software code includes generally commercially available software that is not owned by you, but that can be procured by the Competition Sponsor without undue expense, then instead of delivering the code for that software to the Competition Sponsor, you must identify that software, method for procuring it, and any parameters or other information necessary to replicate the winning Submission\n\nSo yes, you must be able to provide fully reproducible results if you intend on winning!"
  },
  "source": "meta"
}