{
  "id": 54299,
  "title": "Retrain with stage 1 test data allowed? Are masks provided?",
  "url": "/competitions/data-science-bowl-2018/discussion/54299",
  "author_name": "",
  "post_date": "2018-04-12T01:24:51.727555100Z",
  "votes": 6,
  "comment_count": 13,
  "views": 0,
  "content": "<p>Hi, everyone who stayed for stage 2, (the number of team down from 3600+ to 720+, wow...):</p>\n\n<p>Is retrain with stage 1 test data allowed? It seems that the masks of stage 1 test data is not provided as masks, although the labeled csv file is provided and technically we can restore the masks from that...</p>",
  "messages": [
    {
      "id": "312559",
      "postDate": "04/12/2018 01:24:51",
      "content": "<p>Hi, everyone who stayed for stage 2, (the number of team down from 3600+ to 720+, wow...):</p>\n\n<p>Is retrain with stage 1 test data allowed? It seems that the masks of stage 1 test data is not provided as masks, although the labeled csv file is provided and technically we can restore the masks from that...</p>",
      "rawMarkdown": "Hi, everyone who stayed for stage 2, (the number of team down from 3600+ to 720+, wow...):\n\nIs retrain with stage 1 test data allowed? It seems that the masks of stage 1 test data is not provided as masks, although the labeled csv file is provided and technically we can restore the masks from that...",
      "votes": null
    },
    {
      "id": "312568",
      "postDate": "04/12/2018 01:53:52",
      "content": "<p>I haven't downloaded the file yet, but there is a stage1_solution file.  I guess it contains the masks for all stage1 test images.</p>\n\n<p>Therefore your question is very relevant: can we add these to the stage1 train images and retrain our models?</p>\n\n<p>It is very tempting, and I hope it is allowed.</p>",
      "rawMarkdown": "I haven't downloaded the file yet, but there is a stage1_solution file.  I guess it contains the masks for all stage1 test images.\n\nTherefore your question is very relevant: can we add these to the stage1 train images and retrain our models?\n\nIt is very tempting, and I hope it is allowed.",
      "votes": null
    },
    {
      "id": "312585",
      "postDate": "04/12/2018 02:13:33",
      "content": "<p>We only have 4 teams now. lol</p>",
      "rawMarkdown": "We only have 4 teams now. lol",
      "votes": null
    },
    {
      "id": "312589",
      "postDate": "04/12/2018 02:25:44",
      "content": "<p>How do you know if you made the deadline? I'm pretty sure I submitted on time. </p>",
      "rawMarkdown": "How do you know if you made the deadline? I'm pretty sure I submitted on time.",
      "votes": null
    },
    {
      "id": "312591",
      "postDate": "04/12/2018 02:31:40",
      "content": "<p>I guess they just cleaned all teams. Now we can submit new .csv based on the new data.</p>",
      "rawMarkdown": "I guess they just cleaned all teams. Now we can submit new .csv based on the new data.",
      "votes": null
    },
    {
      "id": "312638",
      "postDate": "04/12/2018 04:36:25",
      "content": "<p>I'm sure it is allowed. Not sure weather it will help though. So far, every submission scored 0.</p>",
      "rawMarkdown": "I'm sure it is allowed. Not sure weather it will help though. So far, every submission scored 0.",
      "votes": null
    },
    {
      "id": "312640",
      "postDate": "04/12/2018 04:39:54",
      "content": "<p>It should help. </p>",
      "rawMarkdown": "It should help.",
      "votes": null
    },
    {
      "id": "312675",
      "postDate": "04/12/2018 06:13:50",
      "content": "<p>@CPMP </p>\n\n<p>@Qing Yuan already generated masks from the CSV file. We should be able to use it as an external source\n<a href=\"https://www.kaggle.com/c/data-science-bowl-2018/discussion/54306\">https://www.kaggle.com/c/data-science-bowl-2018/discussion/54306</a>, \n her github -  <a href=\"https://github.com/yuanqing811/DSB2018_stage1_test\">https://github.com/yuanqing811/DSB2018_stage1_test</a>  </p>",
      "rawMarkdown": "CPMP \n\n@Qing Yuan already generated masks from the CSV file. We should be able to use it as an external source\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54306, \n her github -  https://github.com/yuanqing811/DSB2018_stage1_test",
      "votes": null
    },
    {
      "id": "312708",
      "postDate": "04/12/2018 07:09:18",
      "content": "<p>@Nicole,  I have code to read csv solution files, via @Heng.</p>",
      "rawMarkdown": "Nicole,  I have code to read csv solution files, via @Heng.",
      "votes": null
    },
    {
      "id": "312722",
      "postDate": "04/12/2018 07:31:27",
      "content": "<p>@CPMP Perfect, I'm glad we're on the same team. Team work rocks! </p>",
      "rawMarkdown": "CPMP Perfect, I'm glad we're on the same team. Team work rocks!",
      "votes": null
    },
    {
      "id": "313731",
      "postDate": "04/13/2018 19:06:28",
      "content": "<p>Yes, you can retrain with stage 1 test data, as long as your modeling code doesn't change. </p>\n\n<p>You are allowed to add code to translate RLE encoded masks into masks. </p>",
      "rawMarkdown": "Yes, you can retrain with stage 1 test data, as long as your modeling code doesn't change. \n\nYou are allowed to add code to translate RLE encoded masks into masks.",
      "votes": null
    },
    {
      "id": "313735",
      "postDate": "04/13/2018 19:11:38",
      "content": "<p>Excellent, thanks for the confirmation!</p>",
      "rawMarkdown": "Excellent, thanks for the confirmation!",
      "votes": null
    },
    {
      "id": "313743",
      "postDate": "04/13/2018 19:23:31",
      "content": "<p>@Wendy,\nCould you please confirm if we can select additional data from \"Official External Data Thread\" to retrain after we have seen stage2 data? According to this discussion:\n<a href=\"https://www.kaggle.com/c/data-science-bowl-2018/discussion/54409#313366\">https://www.kaggle.com/c/data-science-bowl-2018/discussion/54409#313366</a></p>",
      "rawMarkdown": "Wendy,\nCould you please confirm if we can select additional data from \"Official External Data Thread\" to retrain after we have seen stage2 data? According to this discussion:\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54409#313366",
      "votes": null
    },
    {
      "id": "313841",
      "postDate": "04/13/2018 22:58:13",
      "content": "<p>maybe the right way is to use stage1 test data as validation only</p>",
      "rawMarkdown": "maybe the right way is to use stage1 test data as validation only",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 312568,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "04/12/2018 01:53:52",
      "content": "<p>I haven't downloaded the file yet, but there is a stage1_solution file.  I guess it contains the masks for all stage1 test images.</p>\n\n<p>Therefore your question is very relevant: can we add these to the stage1 train images and retrain our models?</p>\n\n<p>It is very tempting, and I hope it is allowed.</p>",
      "votes": null,
      "replies": [
        {
          "id": 312638,
          "author_name": "bsp2020",
          "author_url": "",
          "post_date": "04/12/2018 04:36:25",
          "content": "<p>I'm sure it is allowed. Not sure weather it will help though. So far, every submission scored 0.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 312640,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "04/12/2018 04:39:54",
          "content": "<p>It should help. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 312675,
          "author_name": "nicolefinnie",
          "author_url": "",
          "post_date": "04/12/2018 06:13:50",
          "content": "<p>@CPMP </p>\n\n<p>@Qing Yuan already generated masks from the CSV file. We should be able to use it as an external source\n<a href=\"https://www.kaggle.com/c/data-science-bowl-2018/discussion/54306\">https://www.kaggle.com/c/data-science-bowl-2018/discussion/54306</a>, \n her github -  <a href=\"https://github.com/yuanqing811/DSB2018_stage1_test\">https://github.com/yuanqing811/DSB2018_stage1_test</a>  </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 312708,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "04/12/2018 07:09:18",
          "content": "<p>@Nicole,  I have code to read csv solution files, via @Heng.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 312722,
          "author_name": "nicolefinnie",
          "author_url": "",
          "post_date": "04/12/2018 07:31:27",
          "content": "<p>@CPMP Perfect, I'm glad we're on the same team. Team work rocks! </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 312585,
      "author_name": "shujian",
      "author_url": "",
      "post_date": "04/12/2018 02:13:33",
      "content": "<p>We only have 4 teams now. lol</p>",
      "votes": null,
      "replies": [
        {
          "id": 312589,
          "author_name": "dskswu",
          "author_url": "",
          "post_date": "04/12/2018 02:25:44",
          "content": "<p>How do you know if you made the deadline? I'm pretty sure I submitted on time. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 312591,
          "author_name": "shujian",
          "author_url": "",
          "post_date": "04/12/2018 02:31:40",
          "content": "<p>I guess they just cleaned all teams. Now we can submit new .csv based on the new data.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 313731,
      "author_name": "wendykan",
      "author_url": "",
      "post_date": "04/13/2018 19:06:28",
      "content": "<p>Yes, you can retrain with stage 1 test data, as long as your modeling code doesn't change. </p>\n\n<p>You are allowed to add code to translate RLE encoded masks into masks. </p>",
      "votes": null,
      "replies": [
        {
          "id": 313735,
          "author_name": "jliamfinnie",
          "author_url": "",
          "post_date": "04/13/2018 19:11:38",
          "content": "<p>Excellent, thanks for the confirmation!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 313743,
          "author_name": "victorsd",
          "author_url": "",
          "post_date": "04/13/2018 19:23:31",
          "content": "<p>@Wendy,\nCould you please confirm if we can select additional data from \"Official External Data Thread\" to retrain after we have seen stage2 data? According to this discussion:\n<a href=\"https://www.kaggle.com/c/data-science-bowl-2018/discussion/54409#313366\">https://www.kaggle.com/c/data-science-bowl-2018/discussion/54409#313366</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 313841,
      "author_name": "andre1233",
      "author_url": "",
      "post_date": "04/13/2018 22:58:13",
      "content": "<p>maybe the right way is to use stage1 test data as validation only</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "312559": "Hi, everyone who stayed for stage 2, (the number of team down from 3600+ to 720+, wow...):\n\nIs retrain with stage 1 test data allowed? It seems that the masks of stage 1 test data is not provided as masks, although the labeled csv file is provided and technically we can restore the masks from that...",
    "312568": "I haven't downloaded the file yet, but there is a stage1_solution file.  I guess it contains the masks for all stage1 test images.\n\nTherefore your question is very relevant: can we add these to the stage1 train images and retrain our models?\n\nIt is very tempting, and I hope it is allowed.",
    "312585": "We only have 4 teams now. lol",
    "312589": "How do you know if you made the deadline? I'm pretty sure I submitted on time.",
    "312591": "I guess they just cleaned all teams. Now we can submit new .csv based on the new data.",
    "312638": "I'm sure it is allowed. Not sure weather it will help though. So far, every submission scored 0.",
    "312640": "It should help.",
    "312675": "CPMP \n\n@Qing Yuan already generated masks from the CSV file. We should be able to use it as an external source\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54306, \n her github -  https://github.com/yuanqing811/DSB2018_stage1_test",
    "312708": "Nicole,  I have code to read csv solution files, via @Heng.",
    "312722": "CPMP Perfect, I'm glad we're on the same team. Team work rocks!",
    "313731": "Yes, you can retrain with stage 1 test data, as long as your modeling code doesn't change. \n\nYou are allowed to add code to translate RLE encoded masks into masks.",
    "313735": "Excellent, thanks for the confirmation!",
    "313743": "Wendy,\nCould you please confirm if we can select additional data from \"Official External Data Thread\" to retrain after we have seen stage2 data? According to this discussion:\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54409#313366",
    "313841": "maybe the right way is to use stage1 test data as validation only"
  },
  "source": "meta"
}