I wonder if the organizers of this contest could clarify the rules for stage 2 of this contest in regard to choosing the data for training the models that we uploaded at the end of stage 1.
\n\nWithout making any changes to the training model, which of the following approaches are allowed under the rules?
\n\n1) Adding stage 1 test data to the training data using the labels provided in stage1_solution.csv
\n\n2) Adding training data not used in stage 1 training obtained from external public sources without any particular selection
\n\n3) Adding training data not used in stage 1 training obtained from external public sources selected specifically to look similar to some of the samples in the test dataset of stage 2
\n\n4) Choosing a subset of stage 1 training data for stage 2 training based on the similarity of images to stage 2 test data
\n\n5) Modifying the images in stage 1 training data to make them more similar to stage 2 test data before training with the model submitted at the end of stage 1
\n\nThank you in advance
", "messages": [ { "id": "313745", "postDate": "04/13/2018 19:24:29", "content": "I wonder if the organizers of this contest could clarify the rules for stage 2 of this contest in regard to choosing the data for training the models that we uploaded at the end of stage 1.
\n\nWithout making any changes to the training model, which of the following approaches are allowed under the rules?
\n\n1) Adding stage 1 test data to the training data using the labels provided in stage1_solution.csv
\n\n2) Adding training data not used in stage 1 training obtained from external public sources without any particular selection
\n\n3) Adding training data not used in stage 1 training obtained from external public sources selected specifically to look similar to some of the samples in the test dataset of stage 2
\n\n4) Choosing a subset of stage 1 training data for stage 2 training based on the similarity of images to stage 2 test data
\n\n5) Modifying the images in stage 1 training data to make them more similar to stage 2 test data before training with the model submitted at the end of stage 1
\n\nThank you in advance
", "rawMarkdown": "I wonder if the organizers of this contest could clarify the rules for stage 2 of this contest in regard to choosing the data for training the models that we uploaded at the end of stage 1. \n\nWithout making any changes to the training model, which of the following approaches are allowed under the rules?\n\n1) Adding stage 1 test data to the training data using the labels provided in stage1_solution.csv\n\n2) Adding training data not used in stage 1 training obtained from external public sources without any particular selection \n\n3) Adding training data not used in stage 1 training obtained from external public sources selected specifically to look similar to some of the samples in the test dataset of stage 2\n\n4) Choosing a subset of stage 1 training data for stage 2 training based on the similarity of images to stage 2 test data\n\n5) Modifying the images in stage 1 training data to make them more similar to stage 2 test data before training with the model submitted at the end of stage 1\n\nThank you in advance", "votes": null }, { "id": "313753", "postDate": "04/13/2018 19:37:29", "content": "Forgot to add another one:
\n\n6) Hand labeling some images from stage 2 test data and using these for stage 2 training.
", "rawMarkdown": "Forgot to add another one:\n\n6) Hand labeling some images from stage 2 test data and using these for stage 2 training.", "votes": null }, { "id": "313754", "postDate": "04/13/2018 19:37:29", "content": "Actually these questions really make sense. What's the motive that they didn't release the stage1 test masks as images rather than asking us to reverse engineer the csv file?
", "rawMarkdown": "Actually these questions really make sense. What's the motive that they didn't release the stage1 test masks as images rather than asking us to reverse engineer the csv file?", "votes": null }, { "id": "313777", "postDate": "04/13/2018 20:26:40", "content": "You can retrain with the provided stg1 test images. Every other decision should be already fixed and explained in your uploaded model file.
", "rawMarkdown": "You can retrain with the provided stg1 test images. Every other decision should be already fixed and explained in your uploaded model file.", "votes": null }, { "id": "313780", "postDate": "04/13/2018 20:31:46", "content": "bobutils, how do you know this? Are you one of the organizers? Or can you provide a link to an answer from an organizer clearly stating this? Thank you.
", "rawMarkdown": "bobutils, how do you know this? Are you one of the organizers? Or can you provide a link to an answer from an organizer clearly stating this? Thank you.", "votes": null }, { "id": "313781", "postDate": "04/13/2018 20:35:53", "content": "Here's another thread with a kaggle official saying that it's ok to retrain with stage 1 test images/masks:\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54299
", "rawMarkdown": "Here's another thread with a kaggle official saying that it's ok to retrain with stage 1 test images/masks:\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54299", "votes": null }, { "id": "313786", "postDate": "04/13/2018 20:41:48", "content": "\n\n\ncan you provide a link to an answer from an organizer clearly stating this?
\n
Solution should be completely fixed before the second stage starts, clearly what data to train on is also part of the solution (just as important as the code of your solution). Please see https://www.kaggle.com/two-stage-faq
", "rawMarkdown": "> can you provide a link to an answer from an organizer clearly stating this? \n\nSolution should be completely fixed before the second stage starts, clearly what data to train on is also part of the solution (just as important as the code of your solution). Please see https://www.kaggle.com/two-stage-faq", "votes": null }, { "id": "313789", "postDate": "04/13/2018 20:43:44", "content": "I've participated in other 2-stage competitions before and it was always like this. I'd suggest Kaggle team to have a pinned topic explicitly stating this in every 2-stage comp.
", "rawMarkdown": "I've participated in other 2-stage competitions before and it was always like this. I'd suggest Kaggle team to have a pinned topic explicitly stating this in every 2-stage comp.", "votes": null }, { "id": "313790", "postDate": "04/13/2018 20:44:14", "content": "Yes, part 1 was there mostly for completeness, I am most interested in the other 5 parts. Is it certain that none of these techniques are allowed?
\n\nIt would be really hard to enforce this rule even for the prize winning teams (how would you know what data they trained on, other than relying on their word?), let alone for teams like mine that are just hoping to get a medal. I am sure that at least some teams will be using these techniques to improve their score, and I don't want to miss a chance to get the bronze medal because other teams used something that we didn't use because we assumed an implicit rule to be true. I don't want to violate the rules either, so I would like to have a clear answer.
", "rawMarkdown": "Yes, part 1 was there mostly for completeness, I am most interested in the other 5 parts. Is it certain that none of these techniques are allowed?\n\n It would be really hard to enforce this rule even for the prize winning teams (how would you know what data they trained on, other than relying on their word?), let alone for teams like mine that are just hoping to get a medal. I am sure that at least some teams will be using these techniques to improve their score, and I don't want to miss a chance to get the bronze medal because other teams used something that we didn't use because we assumed an implicit rule to be true. I don't want to violate the rules either, so I would like to have a clear answer.", "votes": null }, { "id": "313791", "postDate": "04/13/2018 20:49:57", "content": "How can any modification of your solution based on what you see in stage 2 data or didn't specify in the model upload, be allowed? There are a million ways to break the rules, it's not possible to enumerate them all.
", "rawMarkdown": "How can any modification of your solution based on what you see in stage 2 data or didn't specify in the model upload, be allowed? There are a million ways to break the rules, it's not possible to enumerate them all.", "votes": null }, { "id": "313792", "postDate": "04/13/2018 20:50:11", "content": "\n\n\nclearly what data to train on is also part of the solution
\n
This is not quite clear to me. Does this mean that when submitting our model after stage 1, we had to specify explicitly in the submission what data we would be training it on, e.g. just the training data from stage 1 as provided by the organizers, or training + test data from stage 1, so in stage 2 we can't even decide between these two? At least in our model submissions we didn't mention the training data at all, so how would the organizers know what we were planning to train it on?
", "rawMarkdown": "> clearly what data to train on is also part of the solution\n\nThis is not quite clear to me. Does this mean that when submitting our model after stage 1, we had to specify explicitly in the submission what data we would be training it on, e.g. just the training data from stage 1 as provided by the organizers, or training + test data from stage 1, so in stage 2 we can't even decide between these two? At least in our model submissions we didn't mention the training data at all, so how would the organizers know what we were planning to train it on?", "votes": null }, { "id": "313798", "postDate": "04/13/2018 20:58:23", "content": "Yes, chosen training data is part of your methodology (a very important part too) which has to be explained in the model uploaded.
", "rawMarkdown": "Yes, chosen training data is part of your methodology (a very important part too) which has to be explained in the model uploaded.", "votes": null }, { "id": "313799", "postDate": "04/13/2018 21:00:39", "content": "And also, if we are not even allowed to change the training data, why do the organizers have 25 submissions available to each team in stage 2 (5 submissions per day for 5 days)? Is it so that people can retrain the same model on the same data 25 times and submit each result? This doesn't make much sense to me.
", "rawMarkdown": "And also, if we are not even allowed to change the training data, why do the organizers have 25 submissions available to each team in stage 2 (5 submissions per day for 5 days)? Is it so that people can retrain the same model on the same data 25 times and submit each result? This doesn't make much sense to me.", "votes": null }, { "id": "313800", "postDate": "04/13/2018 21:04:41", "content": "\n\n\nYes, chosen training data is part of your methodology (a very important part too) which has to be explained in the model uploaded.
\n
Ok, I guess we did actually include a readme file which mentioned that we would combine stage 1 training and test data.
", "rawMarkdown": "> Yes, chosen training data is part of your methodology (a very important part too) which has to be explained in the model uploaded.\n\nOk, I guess we did actually include a readme file which mentioned that we would combine stage 1 training and test data.", "votes": null }, { "id": "313802", "postDate": "04/13/2018 21:06:45", "content": "About the 25 submissions - yeh, I agree it's not intuitive and I actually asked the same exact question a year ago. Again, I'd suggest Kaggle to include a clear explanation about 2-stage rules as a separate topic.
", "rawMarkdown": "About the 25 submissions - yeh, I agree it's not intuitive and I actually asked the same exact question a year ago. Again, I'd suggest Kaggle to include a clear explanation about 2-stage rules as a separate topic.", "votes": null }, { "id": "313807", "postDate": "04/13/2018 21:14:51", "content": "Yes, it just seems that the presence of 25 submissions rewards the teams that are willing to cheat and change their models by giving them extra opportunities to submit and get feedback on these submissions. Which leads to the question that I posted in another thread: how will the submissions be chosen if the team doesn't choose them manually? If it's done by the highest score on the private or even the public dataset, the cheating teams submitting multiple results from tweaked models would clearly have an advantage.
", "rawMarkdown": "Yes, it just seems that the presence of 25 submissions rewards the teams that are willing to cheat and change their models by giving them extra opportunities to submit and get feedback on these submissions. Which leads to the question that I posted in another thread: how will the submissions be chosen if the team doesn't choose them manually? If it's done by the highest score on the private or even the public dataset, the cheating teams submitting multiple results from tweaked models would clearly have an advantage.", "votes": null }, { "id": "313886", "postDate": "04/14/2018 02:17:11", "content": "I also think these questions make sense. In a strict interpretation of the rule, all the external data used at the stage 1 should also be included in the submission of the stage 1 becuase they are the part of parameter tuning and mode development, but it is not cleary mentioned in the rule. I am even wonefing how many people who used the external data included them in the submission. There are a lot of gray zone in the rule and I think that it might no be certain only based on the past experience. What I cleary read in the faq page - https://www.kaggle.com/two-stage-faq - is
\n\n\n", "rawMarkdown": "I also think these questions make sense. In a strict interpretation of the rule, all the external data used at the stage 1 should also be included in the submission of the stage 1 becuase they are the part of parameter tuning and mode development, but it is not cleary mentioned in the rule. I am even wonefing how many people who used the external data included them in the submission. There are a lot of gray zone in the rule and I think that it might no be certain only based on the past experience. What I cleary read in the faq page - https://www.kaggle.com/two-stage-faq - is\n\n> What should I upload?\n> When you upload a model, you pack all the code that you are eventually going to use to generate your submission csv file.", "votes": null }, { "id": "313916", "postDate": "04/14/2018 04:22:54", "content": "What should I upload?\n When you upload a model, you pack all the code that you are eventually going to use to generate your submission csv file.
\n
There is no public test dataset here, all images are either ignored or used for the private LB. See https://www.kaggle.com/c/data-science-bowl-2018/discussion/54470#latest-313815
\n\nAnd automated selection is never made on the private LB score. Never.
", "rawMarkdown": "There is no public test dataset here, all images are either ignored or used for the private LB. See https://www.kaggle.com/c/data-science-bowl-2018/discussion/54470#latest-313815\n\nAnd automated selection is never made on the private LB score. Never.", "votes": null }, { "id": "313928", "postDate": "04/14/2018 05:06:30", "content": "So, how will it be made here? The first two submitted?
", "rawMarkdown": "So, how will it be made here? The first two submitted?", "votes": null }, { "id": "313936", "postDate": "04/14/2018 05:18:25", "content": "I don't know, which is why our team will select our two submissions explicitly. This should be easy for everybody as we are supposed to reuse the two submissions we selected in stage1, possibly with models retrained using stage1 test model.
", "rawMarkdown": "I don't know, which is why our team will select our two submissions explicitly. This should be easy for everybody as we are supposed to reuse the two submissions we selected in stage1, possibly with models retrained using stage1 test model.", "votes": null }, { "id": "313997", "postDate": "04/14/2018 10:08:45", "content": "We know! see https://www.kaggle.com/c/web-traffic-time-series-forecasting/discussion/39356#220604
", "rawMarkdown": "We know! see https://www.kaggle.com/c/web-traffic-time-series-forecasting/discussion/39356#220604", "votes": null } ], "comments": [ { "id": 313753, "author_name": "skriventsov", "author_url": "", "post_date": "04/13/2018 19:37:29", "content": "Forgot to add another one:
\n\n6) Hand labeling some images from stage 2 test data and using these for stage 2 training.
", "votes": null, "replies": [] }, { "id": 313754, "author_name": "vijaybj", "author_url": "", "post_date": "04/13/2018 19:37:29", "content": "Actually these questions really make sense. What's the motive that they didn't release the stage1 test masks as images rather than asking us to reverse engineer the csv file?
", "votes": null, "replies": [] }, { "id": 313777, "author_name": "bobutis", "author_url": "", "post_date": "04/13/2018 20:26:40", "content": "You can retrain with the provided stg1 test images. Every other decision should be already fixed and explained in your uploaded model file.
", "votes": null, "replies": [ { "id": 313780, "author_name": "skriventsov", "author_url": "", "post_date": "04/13/2018 20:31:46", "content": "bobutils, how do you know this? Are you one of the organizers? Or can you provide a link to an answer from an organizer clearly stating this? Thank you.
", "votes": null, "replies": [] }, { "id": 313781, "author_name": "qrjviyidaedaxoxuazei", "author_url": "", "post_date": "04/13/2018 20:35:53", "content": "Here's another thread with a kaggle official saying that it's ok to retrain with stage 1 test images/masks:\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54299
", "votes": null, "replies": [] }, { "id": 313786, "author_name": "lopuhin", "author_url": "", "post_date": "04/13/2018 20:41:48", "content": "\n\n\ncan you provide a link to an answer from an organizer clearly stating this?
\n
Solution should be completely fixed before the second stage starts, clearly what data to train on is also part of the solution (just as important as the code of your solution). Please see https://www.kaggle.com/two-stage-faq
", "votes": null, "replies": [] }, { "id": 313789, "author_name": "bobutis", "author_url": "", "post_date": "04/13/2018 20:43:44", "content": "I've participated in other 2-stage competitions before and it was always like this. I'd suggest Kaggle team to have a pinned topic explicitly stating this in every 2-stage comp.
", "votes": null, "replies": [] }, { "id": 313790, "author_name": "skriventsov", "author_url": "", "post_date": "04/13/2018 20:44:14", "content": "Yes, part 1 was there mostly for completeness, I am most interested in the other 5 parts. Is it certain that none of these techniques are allowed?
\n\nIt would be really hard to enforce this rule even for the prize winning teams (how would you know what data they trained on, other than relying on their word?), let alone for teams like mine that are just hoping to get a medal. I am sure that at least some teams will be using these techniques to improve their score, and I don't want to miss a chance to get the bronze medal because other teams used something that we didn't use because we assumed an implicit rule to be true. I don't want to violate the rules either, so I would like to have a clear answer.
", "votes": null, "replies": [] }, { "id": 313791, "author_name": "lopuhin", "author_url": "", "post_date": "04/13/2018 20:49:57", "content": "How can any modification of your solution based on what you see in stage 2 data or didn't specify in the model upload, be allowed? There are a million ways to break the rules, it's not possible to enumerate them all.
", "votes": null, "replies": [] }, { "id": 313792, "author_name": "skriventsov", "author_url": "", "post_date": "04/13/2018 20:50:11", "content": "\n\n\nclearly what data to train on is also part of the solution
\n
This is not quite clear to me. Does this mean that when submitting our model after stage 1, we had to specify explicitly in the submission what data we would be training it on, e.g. just the training data from stage 1 as provided by the organizers, or training + test data from stage 1, so in stage 2 we can't even decide between these two? At least in our model submissions we didn't mention the training data at all, so how would the organizers know what we were planning to train it on?
", "votes": null, "replies": [] }, { "id": 313798, "author_name": "bobutis", "author_url": "", "post_date": "04/13/2018 20:58:23", "content": "Yes, chosen training data is part of your methodology (a very important part too) which has to be explained in the model uploaded.
", "votes": null, "replies": [] }, { "id": 313799, "author_name": "skriventsov", "author_url": "", "post_date": "04/13/2018 21:00:39", "content": "And also, if we are not even allowed to change the training data, why do the organizers have 25 submissions available to each team in stage 2 (5 submissions per day for 5 days)? Is it so that people can retrain the same model on the same data 25 times and submit each result? This doesn't make much sense to me.
", "votes": null, "replies": [] }, { "id": 313800, "author_name": "skriventsov", "author_url": "", "post_date": "04/13/2018 21:04:41", "content": "\n\n\nYes, chosen training data is part of your methodology (a very important part too) which has to be explained in the model uploaded.
\n
Ok, I guess we did actually include a readme file which mentioned that we would combine stage 1 training and test data.
", "votes": null, "replies": [] }, { "id": 313802, "author_name": "bobutis", "author_url": "", "post_date": "04/13/2018 21:06:45", "content": "About the 25 submissions - yeh, I agree it's not intuitive and I actually asked the same exact question a year ago. Again, I'd suggest Kaggle to include a clear explanation about 2-stage rules as a separate topic.
", "votes": null, "replies": [] }, { "id": 313807, "author_name": "skriventsov", "author_url": "", "post_date": "04/13/2018 21:14:51", "content": "Yes, it just seems that the presence of 25 submissions rewards the teams that are willing to cheat and change their models by giving them extra opportunities to submit and get feedback on these submissions. Which leads to the question that I posted in another thread: how will the submissions be chosen if the team doesn't choose them manually? If it's done by the highest score on the private or even the public dataset, the cheating teams submitting multiple results from tweaked models would clearly have an advantage.
", "votes": null, "replies": [] }, { "id": 313916, "author_name": "cpmpml", "author_url": "", "post_date": "04/14/2018 04:22:54", "content": "There is no public test dataset here, all images are either ignored or used for the private LB. See https://www.kaggle.com/c/data-science-bowl-2018/discussion/54470#latest-313815
\n\nAnd automated selection is never made on the private LB score. Never.
", "votes": null, "replies": [] }, { "id": 313928, "author_name": "skriventsov", "author_url": "", "post_date": "04/14/2018 05:06:30", "content": "So, how will it be made here? The first two submitted?
", "votes": null, "replies": [] }, { "id": 313936, "author_name": "cpmpml", "author_url": "", "post_date": "04/14/2018 05:18:25", "content": "I don't know, which is why our team will select our two submissions explicitly. This should be easy for everybody as we are supposed to reuse the two submissions we selected in stage1, possibly with models retrained using stage1 test model.
", "votes": null, "replies": [] }, { "id": 313997, "author_name": "cpmpml", "author_url": "", "post_date": "04/14/2018 10:08:45", "content": "We know! see https://www.kaggle.com/c/web-traffic-time-series-forecasting/discussion/39356#220604
", "votes": null, "replies": [] } ] }, { "id": 313886, "author_name": "makio323", "author_url": "", "post_date": "04/14/2018 02:17:11", "content": "I also think these questions make sense. In a strict interpretation of the rule, all the external data used at the stage 1 should also be included in the submission of the stage 1 becuase they are the part of parameter tuning and mode development, but it is not cleary mentioned in the rule. I am even wonefing how many people who used the external data included them in the submission. There are a lot of gray zone in the rule and I think that it might no be certain only based on the past experience. What I cleary read in the faq page - https://www.kaggle.com/two-stage-faq - is
\n\n\n", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "313745": "I wonder if the organizers of this contest could clarify the rules for stage 2 of this contest in regard to choosing the data for training the models that we uploaded at the end of stage 1. \n\nWithout making any changes to the training model, which of the following approaches are allowed under the rules?\n\n1) Adding stage 1 test data to the training data using the labels provided in stage1_solution.csv\n\n2) Adding training data not used in stage 1 training obtained from external public sources without any particular selection \n\n3) Adding training data not used in stage 1 training obtained from external public sources selected specifically to look similar to some of the samples in the test dataset of stage 2\n\n4) Choosing a subset of stage 1 training data for stage 2 training based on the similarity of images to stage 2 test data\n\n5) Modifying the images in stage 1 training data to make them more similar to stage 2 test data before training with the model submitted at the end of stage 1\n\nThank you in advance", "313753": "Forgot to add another one:\n\n6) Hand labeling some images from stage 2 test data and using these for stage 2 training.", "313754": "Actually these questions really make sense. What's the motive that they didn't release the stage1 test masks as images rather than asking us to reverse engineer the csv file?", "313777": "You can retrain with the provided stg1 test images. Every other decision should be already fixed and explained in your uploaded model file.", "313780": "bobutils, how do you know this? Are you one of the organizers? Or can you provide a link to an answer from an organizer clearly stating this? Thank you.", "313781": "Here's another thread with a kaggle official saying that it's ok to retrain with stage 1 test images/masks:\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54299", "313786": "> can you provide a link to an answer from an organizer clearly stating this? \n\nSolution should be completely fixed before the second stage starts, clearly what data to train on is also part of the solution (just as important as the code of your solution). Please see https://www.kaggle.com/two-stage-faq", "313789": "I've participated in other 2-stage competitions before and it was always like this. I'd suggest Kaggle team to have a pinned topic explicitly stating this in every 2-stage comp.", "313790": "Yes, part 1 was there mostly for completeness, I am most interested in the other 5 parts. Is it certain that none of these techniques are allowed?\n\n It would be really hard to enforce this rule even for the prize winning teams (how would you know what data they trained on, other than relying on their word?), let alone for teams like mine that are just hoping to get a medal. I am sure that at least some teams will be using these techniques to improve their score, and I don't want to miss a chance to get the bronze medal because other teams used something that we didn't use because we assumed an implicit rule to be true. I don't want to violate the rules either, so I would like to have a clear answer.", "313791": "How can any modification of your solution based on what you see in stage 2 data or didn't specify in the model upload, be allowed? There are a million ways to break the rules, it's not possible to enumerate them all.", "313792": "> clearly what data to train on is also part of the solution\n\nThis is not quite clear to me. Does this mean that when submitting our model after stage 1, we had to specify explicitly in the submission what data we would be training it on, e.g. just the training data from stage 1 as provided by the organizers, or training + test data from stage 1, so in stage 2 we can't even decide between these two? At least in our model submissions we didn't mention the training data at all, so how would the organizers know what we were planning to train it on?", "313798": "Yes, chosen training data is part of your methodology (a very important part too) which has to be explained in the model uploaded.", "313799": "And also, if we are not even allowed to change the training data, why do the organizers have 25 submissions available to each team in stage 2 (5 submissions per day for 5 days)? Is it so that people can retrain the same model on the same data 25 times and submit each result? This doesn't make much sense to me.", "313800": "> Yes, chosen training data is part of your methodology (a very important part too) which has to be explained in the model uploaded.\n\nOk, I guess we did actually include a readme file which mentioned that we would combine stage 1 training and test data.", "313802": "About the 25 submissions - yeh, I agree it's not intuitive and I actually asked the same exact question a year ago. Again, I'd suggest Kaggle to include a clear explanation about 2-stage rules as a separate topic.", "313807": "Yes, it just seems that the presence of 25 submissions rewards the teams that are willing to cheat and change their models by giving them extra opportunities to submit and get feedback on these submissions. Which leads to the question that I posted in another thread: how will the submissions be chosen if the team doesn't choose them manually? If it's done by the highest score on the private or even the public dataset, the cheating teams submitting multiple results from tweaked models would clearly have an advantage.", "313886": "I also think these questions make sense. In a strict interpretation of the rule, all the external data used at the stage 1 should also be included in the submission of the stage 1 becuase they are the part of parameter tuning and mode development, but it is not cleary mentioned in the rule. I am even wonefing how many people who used the external data included them in the submission. There are a lot of gray zone in the rule and I think that it might no be certain only based on the past experience. What I cleary read in the faq page - https://www.kaggle.com/two-stage-faq - is\n\n> What should I upload?\n> When you upload a model, you pack all the code that you are eventually going to use to generate your submission csv file.", "313916": "There is no public test dataset here, all images are either ignored or used for the private LB. See https://www.kaggle.com/c/data-science-bowl-2018/discussion/54470#latest-313815\n\nAnd automated selection is never made on the private LB score. Never.", "313928": "So, how will it be made here? The first two submitted?", "313936": "I don't know, which is why our team will select our two submissions explicitly. This should be easy for everybody as we are supposed to reuse the two submissions we selected in stage1, possibly with models retrained using stage1 test model.", "313997": "We know! see https://www.kaggle.com/c/web-traffic-time-series-forecasting/discussion/39356#220604" }, "source": "meta" }What should I upload?\n When you upload a model, you pack all the code that you are eventually going to use to generate your submission csv file.
\n