{
  "id": 416792,
  "title": "Matcha performance",
  "url": "/competitions/benetech-making-graphs-accessible/discussion/416792",
  "author_name": "xxxjx250",
  "post_date": "2023-06-13T03:09:15.781000",
  "votes": 1,
  "comment_count": 10,
  "views": 0,
  "content": "<p>Hi,</p>\n<p>I have a question regarding Matcha performance. I saw in previous discussion that Matcha would possibly reach 0.6 LB. I tried to implement and only got local CV and LB under 0.45. Here is my training setup</p>\n<p>epoch: 2<br>\nlr: 3e-5 (no scheduler)<br>\noptimizer: Adam <br>\nmax patch: 1024<br>\nbatch size: 2 (the best my setup could offer)<br>\ntransformer version: 4.31.0.dev0</p>\n<p>In generation, I am using:<br>\nnum_beams = 1,<br>\ntemperature = 0.8,<br>\nmax_length = 512,</p>\n<p>I was not able to record loss diagram, but it generally goes down under 0.07 batch-wise. I suppose there must be something I missed. </p>\n<p>Any suggestion would be helpful. Thank you very much! </p>",
  "messages": [
    {
      "id": 2302557,
      "postDate": "2023-06-14T16:05:59.677Z",
      "content": "<p>do you use extra 500k images dataset ?.  I train only competitive dataset for 5 epoch with matcha for score 0.64.</p>",
      "rawMarkdown": "do you use extra 500k images dataset ?.  I train only competitive dataset for 5 epoch with matcha for score 0.64.",
      "votes": 1,
      "replies": [
        {
          "id": 2302566,
          "postDate": "2023-06-14T16:09:51.577Z",
          "content": "<p>500k would be quite a lot to add but for some it has been helpful. I would expect that most are only adding part of the 500k images. As the more there is the longer training would take and there is only so much variance that can be gathered from more charts.</p>",
          "rawMarkdown": "500k would be quite a lot to add but for some it has been helpful. I would expect that most are only adding part of the 500k images. As the more there is the longer training would take and there is only so much variance that can be gathered from more charts.",
          "votes": 2,
          "replies": [
            {
              "id": 2304014,
              "postDate": "2023-06-15T16:07:20.583Z",
              "content": "<p><a href=\"https://www.kaggle.com/cody11null\" target=\"_blank\">@cody11null</a> Do you use all chart-types or exclude 'scatter' during training?</p>",
              "rawMarkdown": "@cody11null Do you use all chart-types or exclude 'scatter' during training?"
            },
            {
              "id": 2304021,
              "postDate": "2023-06-15T16:14:51.290Z",
              "content": "<p>So we actually have both versions and they preform similarly well. So I dont think one is more correct than another.</p>",
              "rawMarkdown": "So we actually have both versions and they preform similarly well. So I dont think one is more correct than another.",
              "votes": 1
            },
            {
              "id": 2304031,
              "postDate": "2023-06-15T16:20:22.493Z",
              "content": "<p>Thnx a lot for clarifying.</p>",
              "rawMarkdown": "Thnx a lot for clarifying.",
              "votes": 1
            }
          ]
        }
      ]
    },
    {
      "id": 2301025,
      "postDate": "2023-06-13T15:15:43.537Z",
      "content": "<p>Yes unfortunately, with this competition it is very difficult to train these mass models without using a Google Colab Pro or other larger system access </p>",
      "rawMarkdown": "Yes unfortunately, with this competition it is very difficult to train these mass models without using a Google Colab Pro or other larger system access ",
      "votes": 1,
      "replies": [
        {
          "id": 2301424,
          "postDate": "2023-06-13T23:08:50.717Z",
          "content": "<p>thank you!!! Unfortunately yes, especially given limited RAM I could only use very small bs</p>",
          "rawMarkdown": "thank you!!! Unfortunately yes, especially given limited RAM I could only use very small bs",
          "votes": 1,
          "replies": [
            {
              "id": 2301434,
              "postDate": "2023-06-13T23:55:59.610Z",
              "rawMarkdown": "",
              "isDeleted": true
            }
          ]
        }
      ]
    },
    {
      "id": 2300566,
      "postDate": "2023-06-13T08:56:29.823Z",
      "content": "<p>you can try to train it longer</p>",
      "rawMarkdown": "you can try to train it longer",
      "votes": 1,
      "replies": [
        {
          "id": 2301422,
          "postDate": "2023-06-13T23:08:17.143Z",
          "content": "<p>thank you!</p>",
          "rawMarkdown": "thank you!",
          "votes": 1
        }
      ]
    },
    {
      "id": 2300088,
      "postDate": "2023-06-13T03:09:15.783Z",
      "content": "<p>Hi,</p>\n<p>I have a question regarding Matcha performance. I saw in previous discussion that Matcha would possibly reach 0.6 LB. I tried to implement and only got local CV and LB under 0.45. Here is my training setup</p>\n<p>epoch: 2<br>\nlr: 3e-5 (no scheduler)<br>\noptimizer: Adam <br>\nmax patch: 1024<br>\nbatch size: 2 (the best my setup could offer)<br>\ntransformer version: 4.31.0.dev0</p>\n<p>In generation, I am using:<br>\nnum_beams = 1,<br>\ntemperature = 0.8,<br>\nmax_length = 512,</p>\n<p>I was not able to record loss diagram, but it generally goes down under 0.07 batch-wise. I suppose there must be something I missed. </p>\n<p>Any suggestion would be helpful. Thank you very much! </p>",
      "rawMarkdown": "Hi,\n\nI have a question regarding Matcha performance. I saw in previous discussion that Matcha would possibly reach 0.6 LB. I tried to implement and only got local CV and LB under 0.45. Here is my training setup\n\nepoch: 2\nlr: 3e-5 (no scheduler)\noptimizer: Adam \nmax patch: 1024\nbatch size: 2 (the best my setup could offer)\ntransformer version: 4.31.0.dev0\n\nIn generation, I am using:\nnum_beams = 1,\ntemperature = 0.8,\nmax_length = 512,\n\nI was not able to record loss diagram, but it generally goes down under 0.07 batch-wise. I suppose there must be something I missed. \n\nAny suggestion would be helpful. Thank you very much! ",
      "votes": 1
    }
  ],
  "comments": [
    {
      "id": 2302557,
      "author_name": "datnt114",
      "author_url": "",
      "post_date": "2023-06-14T16:05:59.677000",
      "content": "<p>do you use extra 500k images dataset ?.  I train only competitive dataset for 5 epoch with matcha for score 0.64.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2302566,
          "author_name": "Cody_Null",
          "author_url": "",
          "post_date": "2023-06-14T16:09:51.577000",
          "content": "<p>500k would be quite a lot to add but for some it has been helpful. I would expect that most are only adding part of the 500k images. As the more there is the longer training would take and there is only so much variance that can be gathered from more charts.</p>",
          "votes": 2,
          "replies": [
            {
              "id": 2304014,
              "author_name": "Sandy",
              "author_url": "",
              "post_date": "2023-06-15T16:07:20.583000",
              "content": "<p><a href=\"https://www.kaggle.com/cody11null\" target=\"_blank\">@cody11null</a> Do you use all chart-types or exclude 'scatter' during training?</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2304021,
              "author_name": "Cody_Null",
              "author_url": "",
              "post_date": "2023-06-15T16:14:51.290000",
              "content": "<p>So we actually have both versions and they preform similarly well. So I dont think one is more correct than another.</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 2304031,
              "author_name": "Sandy",
              "author_url": "",
              "post_date": "2023-06-15T16:20:22.493000",
              "content": "<p>Thnx a lot for clarifying.</p>",
              "votes": 1,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2301025,
      "author_name": "Cody_Null",
      "author_url": "",
      "post_date": "2023-06-13T15:15:43.537000",
      "content": "<p>Yes unfortunately, with this competition it is very difficult to train these mass models without using a Google Colab Pro or other larger system access </p>",
      "votes": 1,
      "replies": [
        {
          "id": 2301424,
          "author_name": "xxxjx250",
          "author_url": "",
          "post_date": "2023-06-13T23:08:50.717000",
          "content": "<p>thank you!!! Unfortunately yes, especially given limited RAM I could only use very small bs</p>",
          "votes": 1,
          "replies": [
            {
              "id": 2301434,
              "author_name": "",
              "author_url": "",
              "post_date": "2023-06-13T23:55:59.610000",
              "content": "",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2300566,
      "author_name": "Sinan Calisir",
      "author_url": "",
      "post_date": "2023-06-13T08:56:29.823000",
      "content": "<p>you can try to train it longer</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2301422,
          "author_name": "xxxjx250",
          "author_url": "",
          "post_date": "2023-06-13T23:08:17.143000",
          "content": "<p>thank you!</p>",
          "votes": 1,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2302557": "do you use extra 500k images dataset ?.  I train only competitive dataset for 5 epoch with matcha for score 0.64.",
    "2301025": "Yes unfortunately, with this competition it is very difficult to train these mass models without using a Google Colab Pro or other larger system access ",
    "2300566": "you can try to train it longer",
    "2300088": "Hi,\n\nI have a question regarding Matcha performance. I saw in previous discussion that Matcha would possibly reach 0.6 LB. I tried to implement and only got local CV and LB under 0.45. Here is my training setup\n\nepoch: 2\nlr: 3e-5 (no scheduler)\noptimizer: Adam \nmax patch: 1024\nbatch size: 2 (the best my setup could offer)\ntransformer version: 4.31.0.dev0\n\nIn generation, I am using:\nnum_beams = 1,\ntemperature = 0.8,\nmax_length = 512,\n\nI was not able to record loss diagram, but it generally goes down under 0.07 batch-wise. I suppose there must be something I missed. \n\nAny suggestion would be helpful. Thank you very much! "
  }
}