Hi All,
\n\nI recently submitted a couple of CSV files to Kaggle - one of them scored 0.002, and the other scored 0.004. Interestingly, the file which scored 0.004 missed out whole regions of nuclei from a few images such as 0f1f896d9ae5a04752d3239c690402c022db4d72c0d2c087d73380896f72c466.png & 17b9bf4356db24967c4677b8376ac38f826de73a88b93a8d73a8b452e399cdff.png (just to quote a few), while these images (which have considerably large numbers of nuclei) were identified and isolated in the other CSV file which scored 0.002!\nThis seemed to be an interesting finding, and I have attached both files to this post. I'd really like the Organizers for this Competition to provide some feedback on this matter, as it seems to me that there is an inherent flaw in the scoring methodology used for these submissions to the Competition. If this matter does not get resolved and/or addressed by the Organizers by tomorrow (11 April 2018, which represents the deadline for Stage one of this Competition), I'll withdraw from the Competition, since I have absolutely no intention to expend any additional time and effort on a Competition wherein there is potential bias in the scoring methodology, or if the Competition has been rigged in some way, even if Stage two might include other additional interesting photomicrographs captured through various other imaging modalities (such as darkfield, or DIC) for testing our algorithms with.
\n\nThanks & Regards,\nShiraz
", "messages": [ { "id": "311660", "postDate": "04/10/2018 12:55:32", "content": "Hi All,
\n\nI recently submitted a couple of CSV files to Kaggle - one of them scored 0.002, and the other scored 0.004. Interestingly, the file which scored 0.004 missed out whole regions of nuclei from a few images such as 0f1f896d9ae5a04752d3239c690402c022db4d72c0d2c087d73380896f72c466.png & 17b9bf4356db24967c4677b8376ac38f826de73a88b93a8d73a8b452e399cdff.png (just to quote a few), while these images (which have considerably large numbers of nuclei) were identified and isolated in the other CSV file which scored 0.002!\nThis seemed to be an interesting finding, and I have attached both files to this post. I'd really like the Organizers for this Competition to provide some feedback on this matter, as it seems to me that there is an inherent flaw in the scoring methodology used for these submissions to the Competition. If this matter does not get resolved and/or addressed by the Organizers by tomorrow (11 April 2018, which represents the deadline for Stage one of this Competition), I'll withdraw from the Competition, since I have absolutely no intention to expend any additional time and effort on a Competition wherein there is potential bias in the scoring methodology, or if the Competition has been rigged in some way, even if Stage two might include other additional interesting photomicrographs captured through various other imaging modalities (such as darkfield, or DIC) for testing our algorithms with.
\n\nThanks & Regards,\nShiraz
", "rawMarkdown": "Hi All,\n\nI recently submitted a couple of CSV files to Kaggle - one of them scored 0.002, and the other scored 0.004. Interestingly, the file which scored **0.004** missed out whole regions of nuclei from a few images such as **0f1f896d9ae5a04752d3239c690402c022db4d72c0d2c087d73380896f72c466.png** & **17b9bf4356db24967c4677b8376ac38f826de73a88b93a8d73a8b452e399cdff.png** (just to quote a few), while these images (which have considerably large numbers of nuclei) were identified and isolated in the other CSV file which scored **0.002**!\nThis seemed to be an interesting finding, and I have attached both files to this post. I'd really like the Organizers for this Competition to provide some feedback on this matter, as it seems to me that there is an inherent flaw in the scoring methodology used for these submissions to the Competition. If this matter does not get resolved and/or addressed by the Organizers by tomorrow (11 April 2018, which represents the deadline for Stage one of this Competition), I'll withdraw from the Competition, since I have absolutely *no* intention to expend any additional time and effort on a Competition wherein there is potential bias in the scoring methodology, or if the Competition has been rigged in some way, even if Stage two might include other additional interesting photomicrographs captured through various other imaging modalities (such as darkfield, or DIC) for testing our algorithms with.\n\nThanks & Regards,\nShiraz", "votes": null }, { "id": "311760", "postDate": "04/10/2018 16:25:36", "content": "I quickly took a look at your file and everything looks normal.
\n\nIf you have read how the score is evaluated, you would have noticed that it is not the number of nuclei which is important. Your submission shows a huge amount of unconnected pixels which will give a very large number of false positive and thus your really low score.
", "rawMarkdown": "I quickly took a look at your file and everything looks normal.\n\nIf you have read how the score is evaluated, you would have noticed that it is not the number of nuclei which is important. Your submission shows a huge amount of unconnected pixels which will give a very large number of false positive and thus your really low score.", "votes": null }, { "id": "311812", "postDate": "04/10/2018 17:58:55", "content": "Each nuclei must be placed at new line in .csv. Your submission contains only one nuclei per image.
", "rawMarkdown": "Each nuclei must be placed at new line in .csv. Your submission contains only one nuclei per image.", "votes": null }, { "id": "312300", "postDate": "04/11/2018 14:41:48", "content": "Hi Victor & jeandebleau,
\n\nThanks for your replies to my query & your clarifications.
\n\nHowever, due to time constraints, I won't be able to do much from here on now, hence I'll be withdrawing from this Competition.
\n\nGoodbye.
", "rawMarkdown": "Hi Victor & jeandebleau,\n\nThanks for your replies to my query & your clarifications.\n\nHowever, due to time constraints, I won't be able to do much from here on now, hence I'll be withdrawing from this Competition.\n\nGoodbye.", "votes": null }, { "id": "312374", "postDate": "04/11/2018 16:49:08", "content": "That's a big disappointment for the rest of us. :D
", "rawMarkdown": "That's a big disappointment for the rest of us. :D", "votes": null }, { "id": "312563", "postDate": "04/12/2018 01:33:52", "content": "@Vivek: 😃 ... Anyway, I joined the competition a little late, and I'm also pressed with several other matters to attend to during this period though, so maybe next year's Data Science Bowl would be a better choice ... 😉
", "rawMarkdown": "Vivek: 😃 ... Anyway, I joined the competition a little late, and I'm also pressed with several other matters to attend to during this period though, so maybe next year's Data Science Bowl would be a better choice ... 😉", "votes": null }, { "id": "314461", "postDate": "04/15/2018 16:07:21", "content": "The very fact that this post received downvotes confirms that some individuals condone rigging, which would be very likely in a competition such as this (i.e. Stage 2 of this Competition) where the public leaderboard does not show the true scores at the point of submission to prevent \"leaderboard probing\", when in fact, it also encourages rigging, as the Organizers would be potentially able to engage in unfair practices of scoring by manipulating the scores without the contestants' knowledge, and/or employing a potentially biased scoring methodology [for instance, removing some datasets from being scored to favour those contestants whom the Organizers have mutually agreed to award prizes to a priori, but whose models did not fare well for the said dataset/(s)].
", "rawMarkdown": "The very fact that this post received downvotes confirms that some individuals condone rigging, which would be very likely in a competition such as this (i.e. Stage 2 of this Competition) where the public leaderboard does ***not*** show the true scores at the point of submission to prevent \"leaderboard probing\", when in fact, it also encourages rigging, as the Organizers would be potentially able to engage in unfair practices of scoring by manipulating the scores without the contestants' knowledge, and/or employing a potentially biased scoring methodology [for instance, removing some datasets from being scored to favour those contestants whom the Organizers have mutually agreed to award prizes to *a priori*, but whose models did not fare well for the said dataset/(s)].", "votes": null }, { "id": "314485", "postDate": "04/15/2018 17:02:42", "content": "You bring an interesting point about the possibility of rigging by the organizers. One thing that they might consider to protect themselves is to hash the answer key and scoring algorithm and publish the hash before or at the competition start. This might help protect the scoring part, however, it doesn't prevent someone simply leaking the answer. I think ultimately trust is needed to those holding the key/knowledge as in any organization or institution. Things can fall apart without integrity regardless of best tools or methods.
", "rawMarkdown": "You bring an interesting point about the possibility of rigging by the organizers. One thing that they might consider to protect themselves is to hash the answer key and scoring algorithm and publish the hash before or at the competition start. This might help protect the scoring part, however, it doesn't prevent someone simply leaking the answer. I think ultimately trust is needed to those holding the key/knowledge as in any organization or institution. Things can fall apart without integrity regardless of best tools or methods.", "votes": null }, { "id": "314890", "postDate": "04/16/2018 12:43:31", "content": "It's possible I'm not following the logic, but the images will all be made publicly available and the scoring metric is defined so it doesn't seem possible to rig the competition (without it being obvious, that is). Regardless, I didn't rig the competition :)
", "rawMarkdown": "It's possible I'm not following the logic, but the images will all be made publicly available and the scoring metric is defined so it doesn't seem possible to rig the competition (without it being obvious, that is). Regardless, I didn't rig the competition :)", "votes": null }, { "id": "314895", "postDate": "04/16/2018 13:00:33", "content": "@Anne: Thanks for the reply. However, there exists a possibility of competition rigging on the part of the organizers for Stage 2, since the public LB scores are concealed/annulled, so that none of the participants really know how they fare individually in Stage 2. For all of the contestants concerned, the organizers in this context have the full authority to decide (and freely change their decision) on which images are scored & which aren't, without having to inform anyone else and/or seek prior approval. Through this privilege, there exists a possibility of the organizers (possibly not including you here) deciding to allow an image to be scored on the pretext that it favours one contestant over another, thereby promoting rigging, hence my post in this context. Say, for example, Contestant A scores higher than Contestant B on Image 1, but performs more poorly than Contestant B on Image 2. If the organizers wanted to favour Contestant A over Contestant B, they could easily decide to allow Image 1 to be scored, but Image 2 to be omitted from the scoring, so that Contestant A seemingly attains a higher overall score than Contestant B on the private LB, when the results are released. This would be a classic example of rigging in this context.
", "rawMarkdown": "Anne: Thanks for the reply. However, there exists a possibility of competition rigging on the part of the organizers for Stage 2, since the public LB scores are concealed/annulled, so that none of the participants really know how they fare individually in Stage 2. For all of the contestants concerned, the organizers in this context have the full authority to decide (and ***freely*** change their decision) on which images are scored & which aren't, without having to inform anyone else and/or seek prior approval. Through this privilege, there exists a possibility of the organizers (possibly not including you here) deciding to allow an image to be scored on the pretext that it favours one contestant over another, thereby promoting rigging, hence my post in this context. Say, for example, Contestant A scores higher than Contestant B on Image 1, but performs more poorly than Contestant B on Image 2. If the organizers wanted to favour Contestant A over Contestant B, they could easily decide to allow Image 1 to be scored, but Image 2 to be omitted from the scoring, so that Contestant A seemingly attains a higher overall score than Contestant B on the private LB, when the results are released. This would be a classic example of rigging in this context.", "votes": null }, { "id": "314917", "postDate": "04/16/2018 13:55:15", "content": "I don't see the single reason for Anne to rig the competition and I certainly would not expect her to do anything like this. In addition to obvious moral reasons, what's the point to receive an inferior solution by giving it a prize instead of the best one?
", "rawMarkdown": "I don't see the single reason for Anne to rig the competition and I certainly would not expect her to do anything like this. In addition to obvious moral reasons, what's the point to receive an inferior solution by giving it a prize instead of the best one?", "votes": null }, { "id": "314924", "postDate": "04/16/2018 14:11:29", "content": "@Dmytro: Yes indeed, I was just indicating the possibility of rigging existing in this context (where the public LB scores are being concealed) and the actual mode of scoring in Stage 2 is not being made transparent. As for why anyone would want to do this (and by anyone, I'm not referring to Anne or anyone else in particular in this context), the answer is the same as to the question on why corruption is rife in many sectors of society today, namely because of mutual benefits, money, etc. And don't forget that the organizers already have the models of all the contestants (since they have submitted their code at the end of Stage 1), so they can easily get the best model and fine-tune it to improve it further (even if they may decide not to award the best contestant the prize). So, the organizers would have nothing to lose in this context ...
", "rawMarkdown": "Dmytro: Yes indeed, I was just indicating the possibility of rigging existing in this context (where the public LB scores are being concealed) and the actual mode of scoring in Stage 2 is not being made transparent. As for why anyone would want to do this (and by anyone, I'm not referring to Anne or anyone else in particular in this context), the answer is the same as to the question on why corruption is rife in many sectors of society today, namely because of mutual benefits, money, etc. And don't forget that the organizers already have the models of all the contestants (since they have submitted their code at the end of Stage 1), so they can easily get the best model and fine-tune it to improve it further (even if they may decide not to award the best contestant the prize). So, the organizers would have nothing to lose in this context ...", "votes": null }, { "id": "314939", "postDate": "04/16/2018 14:33:27", "content": "Well, there is a significant difference between the quality of model uploaded to prove the result is reproducible and provided after the cleanup for the winning solution. I know.
", "rawMarkdown": "Well, there is a significant difference between the quality of model uploaded to prove the result is reproducible and provided after the cleanup for the winning solution. I know.", "votes": null }, { "id": "314960", "postDate": "04/16/2018 15:07:15", "content": "Yes, but why is there a significant difference in this model quality after the cleanup? Because the model has been re-trained? There's a separate thread actively discussing this right now actually, it's at https://www.kaggle.com/c/data-science-bowl-2018/discussion/54575. Thanks.
", "rawMarkdown": "Yes, but why is there a significant difference in this model quality after the cleanup? Because the model has been re-trained? There's a separate thread actively discussing this right now actually, it's at https://www.kaggle.com/c/data-science-bowl-2018/discussion/54575. Thanks.", "votes": null }, { "id": "315195", "postDate": "04/16/2018 21:12:51", "content": "I only mean the quality of the source code uploaded, documentation etc. I have uploaded over 10k lines of code only the subset of it being relevant/used.
", "rawMarkdown": "I only mean the quality of the source code uploaded, documentation etc. I have uploaded over 10k lines of code only the subset of it being relevant/used.", "votes": null } ], "comments": [ { "id": 311760, "author_name": "jeandebleau", "author_url": "", "post_date": "04/10/2018 16:25:36", "content": "I quickly took a look at your file and everything looks normal.
\n\nIf you have read how the score is evaluated, you would have noticed that it is not the number of nuclei which is important. Your submission shows a huge amount of unconnected pixels which will give a very large number of false positive and thus your really low score.
", "votes": null, "replies": [] }, { "id": 311812, "author_name": "nedelko", "author_url": "", "post_date": "04/10/2018 17:58:55", "content": "Each nuclei must be placed at new line in .csv. Your submission contains only one nuclei per image.
", "votes": null, "replies": [] }, { "id": 312300, "author_name": "shiraz14", "author_url": "", "post_date": "04/11/2018 14:41:48", "content": "Hi Victor & jeandebleau,
\n\nThanks for your replies to my query & your clarifications.
\n\nHowever, due to time constraints, I won't be able to do much from here on now, hence I'll be withdrawing from this Competition.
\n\nGoodbye.
", "votes": null, "replies": [ { "id": 312374, "author_name": "thevivekpandey", "author_url": "", "post_date": "04/11/2018 16:49:08", "content": "That's a big disappointment for the rest of us. :D
", "votes": null, "replies": [] }, { "id": 312563, "author_name": "shiraz14", "author_url": "", "post_date": "04/12/2018 01:33:52", "content": "@Vivek: 😃 ... Anyway, I joined the competition a little late, and I'm also pressed with several other matters to attend to during this period though, so maybe next year's Data Science Bowl would be a better choice ... 😉
", "votes": null, "replies": [] } ] }, { "id": 314461, "author_name": "shiraz14", "author_url": "", "post_date": "04/15/2018 16:07:21", "content": "The very fact that this post received downvotes confirms that some individuals condone rigging, which would be very likely in a competition such as this (i.e. Stage 2 of this Competition) where the public leaderboard does not show the true scores at the point of submission to prevent \"leaderboard probing\", when in fact, it also encourages rigging, as the Organizers would be potentially able to engage in unfair practices of scoring by manipulating the scores without the contestants' knowledge, and/or employing a potentially biased scoring methodology [for instance, removing some datasets from being scored to favour those contestants whom the Organizers have mutually agreed to award prizes to a priori, but whose models did not fare well for the said dataset/(s)].
", "votes": null, "replies": [ { "id": 314485, "author_name": "pliptor", "author_url": "", "post_date": "04/15/2018 17:02:42", "content": "You bring an interesting point about the possibility of rigging by the organizers. One thing that they might consider to protect themselves is to hash the answer key and scoring algorithm and publish the hash before or at the competition start. This might help protect the scoring part, however, it doesn't prevent someone simply leaking the answer. I think ultimately trust is needed to those holding the key/knowledge as in any organization or institution. Things can fall apart without integrity regardless of best tools or methods.
", "votes": null, "replies": [] }, { "id": 314890, "author_name": "drannecarpenter", "author_url": "", "post_date": "04/16/2018 12:43:31", "content": "It's possible I'm not following the logic, but the images will all be made publicly available and the scoring metric is defined so it doesn't seem possible to rig the competition (without it being obvious, that is). Regardless, I didn't rig the competition :)
", "votes": null, "replies": [] }, { "id": 314895, "author_name": "shiraz14", "author_url": "", "post_date": "04/16/2018 13:00:33", "content": "@Anne: Thanks for the reply. However, there exists a possibility of competition rigging on the part of the organizers for Stage 2, since the public LB scores are concealed/annulled, so that none of the participants really know how they fare individually in Stage 2. For all of the contestants concerned, the organizers in this context have the full authority to decide (and freely change their decision) on which images are scored & which aren't, without having to inform anyone else and/or seek prior approval. Through this privilege, there exists a possibility of the organizers (possibly not including you here) deciding to allow an image to be scored on the pretext that it favours one contestant over another, thereby promoting rigging, hence my post in this context. Say, for example, Contestant A scores higher than Contestant B on Image 1, but performs more poorly than Contestant B on Image 2. If the organizers wanted to favour Contestant A over Contestant B, they could easily decide to allow Image 1 to be scored, but Image 2 to be omitted from the scoring, so that Contestant A seemingly attains a higher overall score than Contestant B on the private LB, when the results are released. This would be a classic example of rigging in this context.
", "votes": null, "replies": [] }, { "id": 314917, "author_name": "dmytropoplavskiy", "author_url": "", "post_date": "04/16/2018 13:55:15", "content": "I don't see the single reason for Anne to rig the competition and I certainly would not expect her to do anything like this. In addition to obvious moral reasons, what's the point to receive an inferior solution by giving it a prize instead of the best one?
", "votes": null, "replies": [] }, { "id": 314924, "author_name": "shiraz14", "author_url": "", "post_date": "04/16/2018 14:11:29", "content": "@Dmytro: Yes indeed, I was just indicating the possibility of rigging existing in this context (where the public LB scores are being concealed) and the actual mode of scoring in Stage 2 is not being made transparent. As for why anyone would want to do this (and by anyone, I'm not referring to Anne or anyone else in particular in this context), the answer is the same as to the question on why corruption is rife in many sectors of society today, namely because of mutual benefits, money, etc. And don't forget that the organizers already have the models of all the contestants (since they have submitted their code at the end of Stage 1), so they can easily get the best model and fine-tune it to improve it further (even if they may decide not to award the best contestant the prize). So, the organizers would have nothing to lose in this context ...
", "votes": null, "replies": [] }, { "id": 314939, "author_name": "dmytropoplavskiy", "author_url": "", "post_date": "04/16/2018 14:33:27", "content": "Well, there is a significant difference between the quality of model uploaded to prove the result is reproducible and provided after the cleanup for the winning solution. I know.
", "votes": null, "replies": [] }, { "id": 314960, "author_name": "shiraz14", "author_url": "", "post_date": "04/16/2018 15:07:15", "content": "Yes, but why is there a significant difference in this model quality after the cleanup? Because the model has been re-trained? There's a separate thread actively discussing this right now actually, it's at https://www.kaggle.com/c/data-science-bowl-2018/discussion/54575. Thanks.
", "votes": null, "replies": [] }, { "id": 315195, "author_name": "dmytropoplavskiy", "author_url": "", "post_date": "04/16/2018 21:12:51", "content": "I only mean the quality of the source code uploaded, documentation etc. I have uploaded over 10k lines of code only the subset of it being relevant/used.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "311660": "Hi All,\n\nI recently submitted a couple of CSV files to Kaggle - one of them scored 0.002, and the other scored 0.004. Interestingly, the file which scored **0.004** missed out whole regions of nuclei from a few images such as **0f1f896d9ae5a04752d3239c690402c022db4d72c0d2c087d73380896f72c466.png** & **17b9bf4356db24967c4677b8376ac38f826de73a88b93a8d73a8b452e399cdff.png** (just to quote a few), while these images (which have considerably large numbers of nuclei) were identified and isolated in the other CSV file which scored **0.002**!\nThis seemed to be an interesting finding, and I have attached both files to this post. I'd really like the Organizers for this Competition to provide some feedback on this matter, as it seems to me that there is an inherent flaw in the scoring methodology used for these submissions to the Competition. If this matter does not get resolved and/or addressed by the Organizers by tomorrow (11 April 2018, which represents the deadline for Stage one of this Competition), I'll withdraw from the Competition, since I have absolutely *no* intention to expend any additional time and effort on a Competition wherein there is potential bias in the scoring methodology, or if the Competition has been rigged in some way, even if Stage two might include other additional interesting photomicrographs captured through various other imaging modalities (such as darkfield, or DIC) for testing our algorithms with.\n\nThanks & Regards,\nShiraz", "311760": "I quickly took a look at your file and everything looks normal.\n\nIf you have read how the score is evaluated, you would have noticed that it is not the number of nuclei which is important. Your submission shows a huge amount of unconnected pixels which will give a very large number of false positive and thus your really low score.", "311812": "Each nuclei must be placed at new line in .csv. Your submission contains only one nuclei per image.", "312300": "Hi Victor & jeandebleau,\n\nThanks for your replies to my query & your clarifications.\n\nHowever, due to time constraints, I won't be able to do much from here on now, hence I'll be withdrawing from this Competition.\n\nGoodbye.", "312374": "That's a big disappointment for the rest of us. :D", "312563": "Vivek: 😃 ... Anyway, I joined the competition a little late, and I'm also pressed with several other matters to attend to during this period though, so maybe next year's Data Science Bowl would be a better choice ... 😉", "314461": "The very fact that this post received downvotes confirms that some individuals condone rigging, which would be very likely in a competition such as this (i.e. Stage 2 of this Competition) where the public leaderboard does ***not*** show the true scores at the point of submission to prevent \"leaderboard probing\", when in fact, it also encourages rigging, as the Organizers would be potentially able to engage in unfair practices of scoring by manipulating the scores without the contestants' knowledge, and/or employing a potentially biased scoring methodology [for instance, removing some datasets from being scored to favour those contestants whom the Organizers have mutually agreed to award prizes to *a priori*, but whose models did not fare well for the said dataset/(s)].", "314485": "You bring an interesting point about the possibility of rigging by the organizers. One thing that they might consider to protect themselves is to hash the answer key and scoring algorithm and publish the hash before or at the competition start. This might help protect the scoring part, however, it doesn't prevent someone simply leaking the answer. I think ultimately trust is needed to those holding the key/knowledge as in any organization or institution. Things can fall apart without integrity regardless of best tools or methods.", "314890": "It's possible I'm not following the logic, but the images will all be made publicly available and the scoring metric is defined so it doesn't seem possible to rig the competition (without it being obvious, that is). Regardless, I didn't rig the competition :)", "314895": "Anne: Thanks for the reply. However, there exists a possibility of competition rigging on the part of the organizers for Stage 2, since the public LB scores are concealed/annulled, so that none of the participants really know how they fare individually in Stage 2. For all of the contestants concerned, the organizers in this context have the full authority to decide (and ***freely*** change their decision) on which images are scored & which aren't, without having to inform anyone else and/or seek prior approval. Through this privilege, there exists a possibility of the organizers (possibly not including you here) deciding to allow an image to be scored on the pretext that it favours one contestant over another, thereby promoting rigging, hence my post in this context. Say, for example, Contestant A scores higher than Contestant B on Image 1, but performs more poorly than Contestant B on Image 2. If the organizers wanted to favour Contestant A over Contestant B, they could easily decide to allow Image 1 to be scored, but Image 2 to be omitted from the scoring, so that Contestant A seemingly attains a higher overall score than Contestant B on the private LB, when the results are released. This would be a classic example of rigging in this context.", "314917": "I don't see the single reason for Anne to rig the competition and I certainly would not expect her to do anything like this. In addition to obvious moral reasons, what's the point to receive an inferior solution by giving it a prize instead of the best one?", "314924": "Dmytro: Yes indeed, I was just indicating the possibility of rigging existing in this context (where the public LB scores are being concealed) and the actual mode of scoring in Stage 2 is not being made transparent. As for why anyone would want to do this (and by anyone, I'm not referring to Anne or anyone else in particular in this context), the answer is the same as to the question on why corruption is rife in many sectors of society today, namely because of mutual benefits, money, etc. And don't forget that the organizers already have the models of all the contestants (since they have submitted their code at the end of Stage 1), so they can easily get the best model and fine-tune it to improve it further (even if they may decide not to award the best contestant the prize). So, the organizers would have nothing to lose in this context ...", "314939": "Well, there is a significant difference between the quality of model uploaded to prove the result is reproducible and provided after the cleanup for the winning solution. I know.", "314960": "Yes, but why is there a significant difference in this model quality after the cleanup? Because the model has been re-trained? There's a separate thread actively discussing this right now actually, it's at https://www.kaggle.com/c/data-science-bowl-2018/discussion/54575. Thanks.", "315195": "I only mean the quality of the source code uploaded, documentation etc. I have uploaded over 10k lines of code only the subset of it being relevant/used." }, "source": "meta" }