{
  "id": 54703,
  "title": "Did retraining with stage1 test labels help?",
  "url": "/competitions/data-science-bowl-2018/discussion/54703",
  "author_name": "",
  "post_date": "2018-04-17T01:08:15.351806700Z",
  "votes": 3,
  "comment_count": 8,
  "views": 0,
  "content": "<p>I am not sure it helped, my local CV decreased, but given we said we would retrain, we did retrain...  What about others?</p>",
  "messages": [
    {
      "id": "315312",
      "postDate": "04/17/2018 01:08:15",
      "content": "<p>I am not sure it helped, my local CV decreased, but given we said we would retrain, we did retrain...  What about others?</p>",
      "rawMarkdown": "I am not sure it helped, my local CV decreased, but given we said we would retrain, we did retrain...  What about others?",
      "votes": null
    },
    {
      "id": "315319",
      "postDate": "04/17/2018 01:11:49",
      "content": "<p>It decreased our score from .504 to .493\n1. Trained for same number of epochs\n2. Did not remove any image based on clustering</p>",
      "rawMarkdown": "It decreased our score from .504 to .493\n1. Trained for same number of epochs\n2. Did not remove any image based on clustering",
      "votes": null
    },
    {
      "id": "315321",
      "postDate": "04/17/2018 01:13:39",
      "content": "<p>It did in my case. 0.470 including 0.439 excluding.</p>",
      "rawMarkdown": "It did in my case. 0.470 including 0.439 excluding.",
      "votes": null
    },
    {
      "id": "315426",
      "postDate": "04/17/2018 05:01:18",
      "content": "<p>Same here; From 0.503 to 0.493 and I didn't select the 0.503 submission :(. I guess that cost me a silver medal.    (I guess the same thing happened to you too.)</p>",
      "rawMarkdown": "Same here; From 0.503 to 0.493 and I didn't select the 0.503 submission :(. I guess that cost me a silver medal.    (I guess the same thing happened to you too.)",
      "votes": null
    },
    {
      "id": "315427",
      "postDate": "04/17/2018 05:05:03",
      "content": "<p>@CPMP; we had a weird experience; with re-training of Mask_RCNN val_loss went up from 0.75 to 2.5!! \nSo we selected model without re-training. but on stage2_test data old model gave 0.472 and new model gave 0.489!! \n(And we selected the old one); also new model has fewer detections .. </p>\n\n<p>One thing i wish we had done; old model was trained for 8 epochs and new model for 22 epoch. i wish we did same number of epochs.  we were on K80 GPU so was not able to train for 50-60 epochs like some other teams....</p>",
      "rawMarkdown": "CPMP; we had a weird experience; with re-training of Mask_RCNN val_loss went up from 0.75 to 2.5!! \nSo we selected model without re-training. but on stage2_test data old model gave 0.472 and new model gave 0.489!! \n(And we selected the old one); also new model has fewer detections .. \n\nOne thing i wish we had done; old model was trained for 8 epochs and new model for 22 epoch. i wish we did same number of epochs.  we were on K80 GPU so was not able to train for 50-60 epochs like some other teams....",
      "votes": null
    },
    {
      "id": "315509",
      "postDate": "04/17/2018 07:39:28",
      "content": "<p>I had the same, in some folds loss went up like crazy.  I reran, got it back to normal, but the crazy model performed better in the end!</p>",
      "rawMarkdown": "I had the same, in some folds loss went up like crazy.  I reran, got it back to normal, but the crazy model performed better in the end!",
      "votes": null
    },
    {
      "id": "315513",
      "postDate": "04/17/2018 07:44:54",
      "content": "<p>I am not sure what was the loss function, but maybe the increase in val_loss is related to size of validation set. It happens for example, if the normalization term is related to the number of positive classes, not total examples seen. So it might not be a good idea to compare them unless you are sure it is normalized by the validation size.</p>",
      "rawMarkdown": "I am not sure what was the loss function, but maybe the increase in val_loss is related to size of validation set. It happens for example, if the normalization term is related to the number of positive classes, not total examples seen. So it might not be a good idea to compare them unless you are sure it is normalized by the validation size.",
      "votes": null
    },
    {
      "id": "315525",
      "postDate": "04/17/2018 08:03:24",
      "content": "<p>In my case retraining with stage1 test data increased my score from 0.532 to 0.539</p>",
      "rawMarkdown": "In my case retraining with stage1 test data increased my score from 0.532 to 0.539",
      "votes": null
    },
    {
      "id": "317405",
      "postDate": "04/21/2018 12:58:25",
      "content": "<p>Increased from 0.484 to 0.503 after retraining</p>",
      "rawMarkdown": "Increased from 0.484 to 0.503 after retraining",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 315319,
      "author_name": "outliar",
      "author_url": "",
      "post_date": "04/17/2018 01:11:49",
      "content": "<p>It decreased our score from .504 to .493\n1. Trained for same number of epochs\n2. Did not remove any image based on clustering</p>",
      "votes": null,
      "replies": [
        {
          "id": 315426,
          "author_name": "akilaw",
          "author_url": "",
          "post_date": "04/17/2018 05:01:18",
          "content": "<p>Same here; From 0.503 to 0.493 and I didn't select the 0.503 submission :(. I guess that cost me a silver medal.    (I guess the same thing happened to you too.)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 315321,
      "author_name": "pliptor",
      "author_url": "",
      "post_date": "04/17/2018 01:13:39",
      "content": "<p>It did in my case. 0.470 including 0.439 excluding.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 315427,
      "author_name": "mpsampat",
      "author_url": "",
      "post_date": "04/17/2018 05:05:03",
      "content": "<p>@CPMP; we had a weird experience; with re-training of Mask_RCNN val_loss went up from 0.75 to 2.5!! \nSo we selected model without re-training. but on stage2_test data old model gave 0.472 and new model gave 0.489!! \n(And we selected the old one); also new model has fewer detections .. </p>\n\n<p>One thing i wish we had done; old model was trained for 8 epochs and new model for 22 epoch. i wish we did same number of epochs.  we were on K80 GPU so was not able to train for 50-60 epochs like some other teams....</p>",
      "votes": null,
      "replies": [
        {
          "id": 315509,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "04/17/2018 07:39:28",
          "content": "<p>I had the same, in some folds loss went up like crazy.  I reran, got it back to normal, but the crazy model performed better in the end!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 315513,
          "author_name": "mrtzmilani",
          "author_url": "",
          "post_date": "04/17/2018 07:44:54",
          "content": "<p>I am not sure what was the loss function, but maybe the increase in val_loss is related to size of validation set. It happens for example, if the normalization term is related to the number of positive classes, not total examples seen. So it might not be a good idea to compare them unless you are sure it is normalized by the validation size.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 315525,
      "author_name": "ramzes2",
      "author_url": "",
      "post_date": "04/17/2018 08:03:24",
      "content": "<p>In my case retraining with stage1 test data increased my score from 0.532 to 0.539</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 317405,
      "author_name": "joek47",
      "author_url": "",
      "post_date": "04/21/2018 12:58:25",
      "content": "<p>Increased from 0.484 to 0.503 after retraining</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "315312": "I am not sure it helped, my local CV decreased, but given we said we would retrain, we did retrain...  What about others?",
    "315319": "It decreased our score from .504 to .493\n1. Trained for same number of epochs\n2. Did not remove any image based on clustering",
    "315321": "It did in my case. 0.470 including 0.439 excluding.",
    "315426": "Same here; From 0.503 to 0.493 and I didn't select the 0.503 submission :(. I guess that cost me a silver medal.    (I guess the same thing happened to you too.)",
    "315427": "CPMP; we had a weird experience; with re-training of Mask_RCNN val_loss went up from 0.75 to 2.5!! \nSo we selected model without re-training. but on stage2_test data old model gave 0.472 and new model gave 0.489!! \n(And we selected the old one); also new model has fewer detections .. \n\nOne thing i wish we had done; old model was trained for 8 epochs and new model for 22 epoch. i wish we did same number of epochs.  we were on K80 GPU so was not able to train for 50-60 epochs like some other teams....",
    "315509": "I had the same, in some folds loss went up like crazy.  I reran, got it back to normal, but the crazy model performed better in the end!",
    "315513": "I am not sure what was the loss function, but maybe the increase in val_loss is related to size of validation set. It happens for example, if the normalization term is related to the number of positive classes, not total examples seen. So it might not be a good idea to compare them unless you are sure it is normalized by the validation size.",
    "315525": "In my case retraining with stage1 test data increased my score from 0.532 to 0.539",
    "317405": "Increased from 0.484 to 0.503 after retraining"
  },
  "source": "meta"
}