{
  "id": 294462,
  "title": "Tensorflow neural network",
  "url": "/competitions/cs5785-fall-2021-final/discussion/294462",
  "author_name": "Danish Virani",
  "post_date": "2021-12-10T14:57:28.067000",
  "votes": 0,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I've (kindof) successfully built a neural network. </p>\n<p>It works fantastically on the training and validation set. </p>\n<p>If you try to input new captions and map it to anything in the entire training set it works very well.  But if you bring in entirely new data (testing set) and encode the testing images and try the testing captions, the performance is very bad.</p>\n<p>I think it's safe to say that even with a validation set It ended up overfitting.</p>\n<p>batch size = 32<br>\nepochs = 30</p>\n<p>history = dual_encoder.fit(<br>\n    train_dataset,<br>\n    epochs=num_epochs,<br>\n    validation_data=valid_dataset<br>\n)</p>\n<p>Anyone have any tips or tricks to fix this? </p>",
  "messages": [
    {
      "id": 1614245,
      "postDate": "2021-12-10T19:44:03.747Z",
      "content": "<p>I tried neural network as well. Adding to James's comment, trying different sequences to find out the best one also helps.</p>",
      "rawMarkdown": "I tried neural network as well. Adding to James's comment, trying different sequences to find out the best one also helps.",
      "votes": 1,
      "replies": [
        {
          "id": 1614283,
          "postDate": "2021-12-10T21:45:29.260Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    },
    {
      "id": 1614272,
      "postDate": "2021-12-10T21:16:10.673Z",
      "content": "<p>Got in the same spot and recommend this article <a href=\"https://www.kdnuggets.com/2019/12/5-techniques-prevent-overfitting-neural-networks.html\" target=\"_blank\">https://www.kdnuggets.com/2019/12/5-techniques-prevent-overfitting-neural-networks.html</a> </p>",
      "rawMarkdown": "Got in the same spot and recommend this article https://www.kdnuggets.com/2019/12/5-techniques-prevent-overfitting-neural-networks.html "
    },
    {
      "id": 1614068,
      "postDate": "2021-12-10T14:57:28.067Z",
      "content": "<p>I've (kindof) successfully built a neural network. </p>\n<p>It works fantastically on the training and validation set. </p>\n<p>If you try to input new captions and map it to anything in the entire training set it works very well.  But if you bring in entirely new data (testing set) and encode the testing images and try the testing captions, the performance is very bad.</p>\n<p>I think it's safe to say that even with a validation set It ended up overfitting.</p>\n<p>batch size = 32<br>\nepochs = 30</p>\n<p>history = dual_encoder.fit(<br>\n    train_dataset,<br>\n    epochs=num_epochs,<br>\n    validation_data=valid_dataset<br>\n)</p>\n<p>Anyone have any tips or tricks to fix this? </p>",
      "rawMarkdown": "I've (kindof) successfully built a neural network. \n\nIt works fantastically on the training and validation set. \n\nIf you try to input new captions and map it to anything in the entire training set it works very well.  But if you bring in entirely new data (testing set) and encode the testing images and try the testing captions, the performance is very bad.\n\nI think it's safe to say that even with a validation set It ended up overfitting.\n\nbatch size = 32\nepochs = 30\n\nhistory = dual_encoder.fit(\n    train_dataset,\n    epochs=num_epochs,\n    validation_data=valid_dataset\n)\n\nAnyone have any tips or tricks to fix this? "
    },
    {
      "id": 1614092,
      "postDate": "2021-12-10T15:47:20.630Z",
      "rawMarkdown": "",
      "votes": 1,
      "isDeleted": true,
      "replies": [
        {
          "id": 1614247,
          "postDate": "2021-12-10T19:52:43.847Z",
          "content": "<p>I'm retraining my model. </p>\n<p>I had to disable gpu because that wasn't working right. It would work for a while but just crashed from time to time. </p>\n<p>I'll find out around 6:10 if the new model is generalizable! </p>\n<p>Thanks for your help with this.</p>",
          "rawMarkdown": "I'm retraining my model. \n\nI had to disable gpu because that wasn't working right. It would work for a while but just crashed from time to time. \n\nI'll find out around 6:10 if the new model is generalizable! \n\nThanks for your help with this."
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1614245,
      "author_name": "TGCF",
      "author_url": "",
      "post_date": "2021-12-10T19:44:03.747000",
      "content": "<p>I tried neural network as well. Adding to James's comment, trying different sequences to find out the best one also helps.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1614283,
          "author_name": "",
          "author_url": "",
          "post_date": "2021-12-10T21:45:29.260000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1614272,
      "author_name": "Anna462",
      "author_url": "",
      "post_date": "2021-12-10T21:16:10.673000",
      "content": "<p>Got in the same spot and recommend this article <a href=\"https://www.kdnuggets.com/2019/12/5-techniques-prevent-overfitting-neural-networks.html\" target=\"_blank\">https://www.kdnuggets.com/2019/12/5-techniques-prevent-overfitting-neural-networks.html</a> </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1614092,
      "author_name": "",
      "author_url": "",
      "post_date": "2021-12-10T15:47:20.630000",
      "content": "",
      "votes": 1,
      "replies": [
        {
          "id": 1614247,
          "author_name": "Danish Virani",
          "author_url": "",
          "post_date": "2021-12-10T19:52:43.847000",
          "content": "<p>I'm retraining my model. </p>\n<p>I had to disable gpu because that wasn't working right. It would work for a while but just crashed from time to time. </p>\n<p>I'll find out around 6:10 if the new model is generalizable! </p>\n<p>Thanks for your help with this.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1614245": "I tried neural network as well. Adding to James's comment, trying different sequences to find out the best one also helps.",
    "1614272": "Got in the same spot and recommend this article https://www.kdnuggets.com/2019/12/5-techniques-prevent-overfitting-neural-networks.html ",
    "1614068": "I've (kindof) successfully built a neural network. \n\nIt works fantastically on the training and validation set. \n\nIf you try to input new captions and map it to anything in the entire training set it works very well.  But if you bring in entirely new data (testing set) and encode the testing images and try the testing captions, the performance is very bad.\n\nI think it's safe to say that even with a validation set It ended up overfitting.\n\nbatch size = 32\nepochs = 30\n\nhistory = dual_encoder.fit(\n    train_dataset,\n    epochs=num_epochs,\n    validation_data=valid_dataset\n)\n\nAnyone have any tips or tricks to fix this? ",
    "1614092": ""
  }
}