I noticed that on the timeline page: After this deadline, the test set is released, the answers to the validation set are released, and participants make predictions on the test set.
\n\nMy question is: Can we use the stage 1 test set for training in stage 2? Should we freeze our code by the end of stage 1 or freeze our trained model?
\n\nThanks.
", "messages": [ { "id": "303140", "postDate": "03/25/2018 15:47:41", "content": "I noticed that on the timeline page: After this deadline, the test set is released, the answers to the validation set are released, and participants make predictions on the test set.
\n\nMy question is: Can we use the stage 1 test set for training in stage 2? Should we freeze our code by the end of stage 1 or freeze our trained model?
\n\nThanks.
", "rawMarkdown": "I noticed that on the timeline page: After this deadline, the test set is released, the answers to the validation set are released, and participants make predictions on the test set.\n\nMy question is: Can we use the stage 1 test set for training in stage 2? Should we freeze our code by the end of stage 1 or freeze our trained model?\n\nThanks.", "votes": null }, { "id": "303761", "postDate": "03/26/2018 16:21:37", "content": "have the same question
", "rawMarkdown": "have the same question", "votes": null }, { "id": "303802", "postDate": "03/26/2018 17:16:01", "content": "yes, you should freeze your code, you can re-train your model.
", "rawMarkdown": "yes, you should freeze your code, you can re-train your model.", "votes": null }, { "id": "303851", "postDate": "03/26/2018 18:12:33", "content": "thanks for the clarification
", "rawMarkdown": "thanks for the clarification", "votes": null }, { "id": "304091", "postDate": "03/27/2018 02:13:39", "content": "Hi Wendy, what if we want to cluster the stage 2 test set into groups and maybe do some different pre-processing for each group? I mean, maybe we need to look at the clustering results to decide which pre-processing should be applied to each group. Of course, all the candidate pre-processing methods and the rule of choosing them (which is very simple but may require we take a look at the clustering results since we do not know what (and how many) kinds of images will appear in test 2) will be uploaded before stage 2.
", "rawMarkdown": "Hi Wendy, what if we want to cluster the stage 2 test set into groups and maybe do some different pre-processing for each group? I mean, maybe we need to look at the clustering results to decide which pre-processing should be applied to each group. Of course, all the candidate pre-processing methods and the rule of choosing them (which is very simple but may require we take a look at the clustering results since we do not know what (and how many) kinds of images will appear in test 2) will be uploaded before stage 2.", "votes": null }, { "id": "304128", "postDate": "03/27/2018 03:55:56", "content": "i think as long as the process is automatic it is fine. If your method needs user decision or user input, then it is not ok.
", "rawMarkdown": "i think as long as the process is automatic it is fine. If your method needs user decision or user input, then it is not ok.", "votes": null }, { "id": "304299", "postDate": "03/27/2018 10:22:49", "content": "Can we manually fix the (then released) stage-1-test masks before re-training (e.g. split non-split nuclei and fill holes in the masks)?
", "rawMarkdown": "Can we manually fix the (then released) stage-1-test masks before re-training (e.g. split non-split nuclei and fill holes in the masks)?", "votes": null }, { "id": "304721", "postDate": "03/27/2018 23:41:30", "content": "Hi Wendy, I read the competition overview section, it says:
\n\nApril 11, 2018 - Stage one deadline and stage two data release. Your model must be finalized and uploaded to Kaggle by this deadline.\nI am not sure where to upload model or are you/the admin team going to add the link later?
\n\nAlso, I assume it's not required to run the prediction of stage 2 via the uploaded model, we will still upload the stage 2 submission file directly at the end. Is this correct?
", "rawMarkdown": "Hi Wendy, I read the competition overview section, it says:\n\n April 11, 2018 - Stage one deadline and stage two data release. Your model must be finalized and uploaded to Kaggle by this deadline.\n\nI am not sure where to upload model or are you/the admin team going to add the link later? \n\nAlso, I assume it's not required to run the prediction of stage 2 via the uploaded model, we will still upload the stage 2 submission file directly at the end. Is this correct?", "votes": null }, { "id": "304731", "postDate": "03/28/2018 00:18:59", "content": "https://www.kaggle.com/c/data-science-bowl-2018/team There is a window.
", "rawMarkdown": "https://www.kaggle.com/c/data-science-bowl-2018/team There is a window.", "votes": null }, { "id": "304733", "postDate": "03/28/2018 00:31:17", "content": "That's good to know. Thanks Shujian!
", "rawMarkdown": "That's good to know. Thanks Shujian!", "votes": null }, { "id": "305183", "postDate": "03/28/2018 15:11:58", "content": "The Rules say:\n You may select up to 2 final submissions for judging.
\n\nDoes it apply to stage 2 as well? If so, do we have to upload two models thru the team page before the deadline?
", "rawMarkdown": "The Rules say:\n *You may select up to 2 final submissions for judging.*\n\nDoes it apply to stage 2 as well? If so, do we have to upload two models thru the team page before the deadline?", "votes": null }, { "id": "306838", "postDate": "03/31/2018 05:44:20", "content": "Do we need to use specific programming languages like Python , R ? or We can use any other programming languages like java ? In the competition rules its mentioned we have to develop the \"algorithm\" they have not specified any language restrictions hence I assume that any programming language other than java will also work right ?
", "rawMarkdown": "Do we need to use specific programming languages like Python , R ? or We can use any other programming languages like java ? In the competition rules its mentioned we have to develop the \"algorithm\" they have not specified any language restrictions hence I assume that any programming language other than java will also work right ?", "votes": null }, { "id": "306864", "postDate": "03/31/2018 07:30:57", "content": "hence I assume that any programming language other than Python like java etc will also work right ?
", "rawMarkdown": "hence I assume that any programming language other than Python like java etc will also work right ?", "votes": null }, { "id": "306945", "postDate": "03/31/2018 11:43:39", "content": "Many questions about stage 2 are answered in the FAQ:
\n\nhttps://www.kaggle.com/two-stage-faq
", "rawMarkdown": "Many questions about stage 2 are answered in the FAQ:\n\nhttps://www.kaggle.com/two-stage-faq", "votes": null }, { "id": "306959", "postDate": "03/31/2018 12:38:51", "content": "Thanks for the information !
", "rawMarkdown": "Thanks for the information !", "votes": null }, { "id": "307282", "postDate": "04/01/2018 09:21:22", "content": "@Heng CherKeng: Sorry to bother, is it not possible to manually group the test images type and process them ? Then separating images into gray\\color and train\\predict with different models as discussed in the thread (https://www.kaggle.com/c/data-science-bowl-2018/discussion/49692#296759) seems impossible to do that here... I thought the rule is to just freeze the algorithm like model architectures and hyper-parameters. Thanks in advance.
", "rawMarkdown": "Heng CherKeng: Sorry to bother, is it not possible to manually group the test images type and process them ? Then separating images into gray\\color and train\\predict with different models as discussed in the thread (https://www.kaggle.com/c/data-science-bowl-2018/discussion/49692#296759) seems impossible to do that here... I thought the rule is to just freeze the algorithm like model architectures and hyper-parameters. Thanks in advance.", "votes": null } ], "comments": [ { "id": 303761, "author_name": "watersmart", "author_url": "", "post_date": "03/26/2018 16:21:37", "content": "have the same question
", "votes": null, "replies": [] }, { "id": 303802, "author_name": "wendykan", "author_url": "", "post_date": "03/26/2018 17:16:01", "content": "yes, you should freeze your code, you can re-train your model.
", "votes": null, "replies": [ { "id": 303851, "author_name": "shujian", "author_url": "", "post_date": "03/26/2018 18:12:33", "content": "thanks for the clarification
", "votes": null, "replies": [] }, { "id": 304091, "author_name": "wayne999", "author_url": "", "post_date": "03/27/2018 02:13:39", "content": "Hi Wendy, what if we want to cluster the stage 2 test set into groups and maybe do some different pre-processing for each group? I mean, maybe we need to look at the clustering results to decide which pre-processing should be applied to each group. Of course, all the candidate pre-processing methods and the rule of choosing them (which is very simple but may require we take a look at the clustering results since we do not know what (and how many) kinds of images will appear in test 2) will be uploaded before stage 2.
", "votes": null, "replies": [] }, { "id": 304128, "author_name": "hengck23", "author_url": "", "post_date": "03/27/2018 03:55:56", "content": "i think as long as the process is automatic it is fine. If your method needs user decision or user input, then it is not ok.
", "votes": null, "replies": [] }, { "id": 304299, "author_name": "seesee", "author_url": "", "post_date": "03/27/2018 10:22:49", "content": "Can we manually fix the (then released) stage-1-test masks before re-training (e.g. split non-split nuclei and fill holes in the masks)?
", "votes": null, "replies": [] }, { "id": 307282, "author_name": "khyeh0719", "author_url": "", "post_date": "04/01/2018 09:21:22", "content": "@Heng CherKeng: Sorry to bother, is it not possible to manually group the test images type and process them ? Then separating images into gray\\color and train\\predict with different models as discussed in the thread (https://www.kaggle.com/c/data-science-bowl-2018/discussion/49692#296759) seems impossible to do that here... I thought the rule is to just freeze the algorithm like model architectures and hyper-parameters. Thanks in advance.
", "votes": null, "replies": [] } ] }, { "id": 304721, "author_name": "dongxu027", "author_url": "", "post_date": "03/27/2018 23:41:30", "content": "Hi Wendy, I read the competition overview section, it says:
\n\nApril 11, 2018 - Stage one deadline and stage two data release. Your model must be finalized and uploaded to Kaggle by this deadline.\nI am not sure where to upload model or are you/the admin team going to add the link later?
\n\nAlso, I assume it's not required to run the prediction of stage 2 via the uploaded model, we will still upload the stage 2 submission file directly at the end. Is this correct?
", "votes": null, "replies": [ { "id": 304731, "author_name": "shujian", "author_url": "", "post_date": "03/28/2018 00:18:59", "content": "https://www.kaggle.com/c/data-science-bowl-2018/team There is a window.
", "votes": null, "replies": [] }, { "id": 304733, "author_name": "dongxu027", "author_url": "", "post_date": "03/28/2018 00:31:17", "content": "That's good to know. Thanks Shujian!
", "votes": null, "replies": [] } ] }, { "id": 305183, "author_name": "alexz0", "author_url": "", "post_date": "03/28/2018 15:11:58", "content": "The Rules say:\n You may select up to 2 final submissions for judging.
\n\nDoes it apply to stage 2 as well? If so, do we have to upload two models thru the team page before the deadline?
", "votes": null, "replies": [] }, { "id": 306838, "author_name": "pradnyan", "author_url": "", "post_date": "03/31/2018 05:44:20", "content": "Do we need to use specific programming languages like Python , R ? or We can use any other programming languages like java ? In the competition rules its mentioned we have to develop the \"algorithm\" they have not specified any language restrictions hence I assume that any programming language other than java will also work right ?
", "votes": null, "replies": [] }, { "id": 306864, "author_name": "pradnyan", "author_url": "", "post_date": "03/31/2018 07:30:57", "content": "hence I assume that any programming language other than Python like java etc will also work right ?
", "votes": null, "replies": [] }, { "id": 306945, "author_name": "alexz0", "author_url": "", "post_date": "03/31/2018 11:43:39", "content": "Many questions about stage 2 are answered in the FAQ:
\n\nhttps://www.kaggle.com/two-stage-faq
", "votes": null, "replies": [] }, { "id": 306959, "author_name": "pradnyan", "author_url": "", "post_date": "03/31/2018 12:38:51", "content": "Thanks for the information !
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "303140": "I noticed that on the timeline page: After this deadline, the test set is released, the answers to the validation set are released, and participants make predictions on the test set.\n\nMy question is: Can we use the stage 1 test set for training in stage 2? Should we freeze our code by the end of stage 1 or freeze our trained model?\n\nThanks.", "303761": "have the same question", "303802": "yes, you should freeze your code, you can re-train your model.", "303851": "thanks for the clarification", "304091": "Hi Wendy, what if we want to cluster the stage 2 test set into groups and maybe do some different pre-processing for each group? I mean, maybe we need to look at the clustering results to decide which pre-processing should be applied to each group. Of course, all the candidate pre-processing methods and the rule of choosing them (which is very simple but may require we take a look at the clustering results since we do not know what (and how many) kinds of images will appear in test 2) will be uploaded before stage 2.", "304128": "i think as long as the process is automatic it is fine. If your method needs user decision or user input, then it is not ok.", "304299": "Can we manually fix the (then released) stage-1-test masks before re-training (e.g. split non-split nuclei and fill holes in the masks)?", "304721": "Hi Wendy, I read the competition overview section, it says:\n\n April 11, 2018 - Stage one deadline and stage two data release. Your model must be finalized and uploaded to Kaggle by this deadline.\n\nI am not sure where to upload model or are you/the admin team going to add the link later? \n\nAlso, I assume it's not required to run the prediction of stage 2 via the uploaded model, we will still upload the stage 2 submission file directly at the end. Is this correct?", "304731": "https://www.kaggle.com/c/data-science-bowl-2018/team There is a window.", "304733": "That's good to know. Thanks Shujian!", "305183": "The Rules say:\n *You may select up to 2 final submissions for judging.*\n\nDoes it apply to stage 2 as well? If so, do we have to upload two models thru the team page before the deadline?", "306838": "Do we need to use specific programming languages like Python , R ? or We can use any other programming languages like java ? In the competition rules its mentioned we have to develop the \"algorithm\" they have not specified any language restrictions hence I assume that any programming language other than java will also work right ?", "306864": "hence I assume that any programming language other than Python like java etc will also work right ?", "306945": "Many questions about stage 2 are answered in the FAQ:\n\nhttps://www.kaggle.com/two-stage-faq", "306959": "Thanks for the information !", "307282": "Heng CherKeng: Sorry to bother, is it not possible to manually group the test images type and process them ? Then separating images into gray\\color and train\\predict with different models as discussed in the thread (https://www.kaggle.com/c/data-science-bowl-2018/discussion/49692#296759) seems impossible to do that here... I thought the rule is to just freeze the algorithm like model architectures and hyper-parameters. Thanks in advance." }, "source": "meta" }