{
  "id": 418706,
  "title": "8th place solution",
  "url": "/competitions/benetech-making-graphs-accessible/discussion/418706",
  "author_name": "",
  "post_date": "2023-06-22T07:03:18.543991600Z",
  "votes": 10,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I respect all the teams that fought to the end.</p>\n<h1>overview</h1>\n<p>・classification + object detection + ocr + pp<br>\nSince I was not inclined to use an end to end model for unknown data, I divided the data into smaller tasks and applied a post process using human knowledge to each of them. All processes were trained on all data</p>\n<h1>1, Classify chart type</h1>\n<p>・I used convnext xlarge.</p>\n<h1>2, Detect each parts</h1>\n<p>・chart value such as bar and scatter<br>\n・chart body<br>\n・x-tick-label<br>\n・y-tick-label</p>\n<h1>3, Classify text direction</h1>\n<p>・Post processing depends on the orientation of the x text.</p>\n<h1>4, ocr each x/y texts by TROCR</h1>\n<p>・Detected x-tick-label / y-tick-label converted to text by ocr one by one.</p>\n<h1>5, pp</h1>\n<p>・The coordinates of the detected chart are converted to a value using the all results. Several hundred lines of code were required for each type. </p>\n<h1>point</h1>\n<p>・I spent the most time on the scatter, which is most often included in private test set.<br>\n・Scatter is detected at the same time as ticks to reduce false positives.<br>\n・Increasing the image size when detecting Scatter has greatly improved the score. This was due to the fact that the number of small overlapping dots could be more accurately estimated. (I used 4 sizes for Scatter. Train by 768 and inference by 864/960, and train by 1024 and inference by 1156/1280 and apply wbf.)<br>\n・The $ and % have been replaced with ''. However, for $, I apply a preprocessing to account for the fact that it is often mistaken for 3, 5, s, and S.<br>\n・Many other modifications using human knowledge are incorporated in the post process.<br>\n・In line chart-type, a vertical line was drawn from the x-tick-label, and the intersection with the line chart was detectied.</p>\n<h1>Things that didn't work</h1>\n<p>・finetune during submit<br>\nSince I had about 8 extra hours during the submission, I figured there was room to relearn scatter: after detecting the points in scatter, generate a new image with copy-paste augmentation (I figured it would be better to generate a challenging image with overlapping points), The image is then used to finetune yolox. The automatic generation of the image seemed to work very well, but did not increase the score.<br>\n・1000 other experiments</p>\n<hr>\n<p>Good work guys! Let's fight again in another competition!</p>",
  "messages": [
    {
      "id": "2312751",
      "postDate": "06/22/2023 07:03:18",
      "content": "<p>I respect all the teams that fought to the end.</p>\n<h1>overview</h1>\n<p>・classification + object detection + ocr + pp<br>\nSince I was not inclined to use an end to end model for unknown data, I divided the data into smaller tasks and applied a post process using human knowledge to each of them. All processes were trained on all data</p>\n<h1>1, Classify chart type</h1>\n<p>・I used convnext xlarge.</p>\n<h1>2, Detect each parts</h1>\n<p>・chart value such as bar and scatter<br>\n・chart body<br>\n・x-tick-label<br>\n・y-tick-label</p>\n<h1>3, Classify text direction</h1>\n<p>・Post processing depends on the orientation of the x text.</p>\n<h1>4, ocr each x/y texts by TROCR</h1>\n<p>・Detected x-tick-label / y-tick-label converted to text by ocr one by one.</p>\n<h1>5, pp</h1>\n<p>・The coordinates of the detected chart are converted to a value using the all results. Several hundred lines of code were required for each type. </p>\n<h1>point</h1>\n<p>・I spent the most time on the scatter, which is most often included in private test set.<br>\n・Scatter is detected at the same time as ticks to reduce false positives.<br>\n・Increasing the image size when detecting Scatter has greatly improved the score. This was due to the fact that the number of small overlapping dots could be more accurately estimated. (I used 4 sizes for Scatter. Train by 768 and inference by 864/960, and train by 1024 and inference by 1156/1280 and apply wbf.)<br>\n・The $ and % have been replaced with ''. However, for $, I apply a preprocessing to account for the fact that it is often mistaken for 3, 5, s, and S.<br>\n・Many other modifications using human knowledge are incorporated in the post process.<br>\n・In line chart-type, a vertical line was drawn from the x-tick-label, and the intersection with the line chart was detectied.</p>\n<h1>Things that didn't work</h1>\n<p>・finetune during submit<br>\nSince I had about 8 extra hours during the submission, I figured there was room to relearn scatter: after detecting the points in scatter, generate a new image with copy-paste augmentation (I figured it would be better to generate a challenging image with overlapping points), The image is then used to finetune yolox. The automatic generation of the image seemed to work very well, but did not increase the score.<br>\n・1000 other experiments</p>\n<hr>\n<p>Good work guys! Let's fight again in another competition!</p>",
      "rawMarkdown": "I respect all the teams that fought to the end.\n\n# overview\n・classification + object detection + ocr + pp\nSince I was not inclined to use an end to end model for unknown data, I divided the data into smaller tasks and applied a post process using human knowledge to each of them. All processes were trained on all data\n\n# 1, Classify chart type \n・I used convnext xlarge.\n\n# 2, Detect each parts \n・chart value such as bar and scatter\n・chart body\n・x-tick-label\n・y-tick-label\n\n# 3, Classify text direction\n・Post processing depends on the orientation of the x text.\n\n# 4, ocr each x/y texts by TROCR\n・Detected x-tick-label / y-tick-label converted to text by ocr one by one.\n\n# 5, pp\n・The coordinates of the detected chart are converted to a value using the all results. Several hundred lines of code were required for each type. \n\n# point\n・I spent the most time on the scatter, which is most often included in private test set.\n・Scatter is detected at the same time as ticks to reduce false positives.\n・Increasing the image size when detecting Scatter has greatly improved the score. This was due to the fact that the number of small overlapping dots could be more accurately estimated. (I used 4 sizes for Scatter. Train by 768 and inference by 864/960, and train by 1024 and inference by 1156/1280 and apply wbf.)\n・The $ and % have been replaced with ''. However, for $, I apply a preprocessing to account for the fact that it is often mistaken for 3, 5, s, and S.\n・Many other modifications using human knowledge are incorporated in the post process.\n・In line chart-type, a vertical line was drawn from the x-tick-label, and the intersection with the line chart was detectied.\n\n# Things that didn't work\n・finetune during submit\nSince I had about 8 extra hours during the submission, I figured there was room to relearn scatter: after detecting the points in scatter, generate a new image with copy-paste augmentation (I figured it would be better to generate a challenging image with overlapping points), The image is then used to finetune yolox. The automatic generation of the image seemed to work very well, but did not increase the score.\n・1000 other experiments\n\n---------\nGood work guys! Let's fight again in another competition!",
      "votes": null
    },
    {
      "id": "2313103",
      "postDate": "06/22/2023 12:25:10",
      "content": "<p>Congrats on the gold modal! <br>\nThanks for sharing!<br>\nWhich algorithm did you use for object detection?</p>",
      "rawMarkdown": "Congrats on the gold modal! \nThanks for sharing!\nWhich algorithm did you use for object detection?",
      "votes": null
    },
    {
      "id": "2313223",
      "postDate": "06/22/2023 14:35:50",
      "content": "<p>Thank you! Of course I used yolox.</p>",
      "rawMarkdown": "Thank you! Of course I used yolox.",
      "votes": null
    },
    {
      "id": "2319386",
      "postDate": "06/27/2023 05:25:48",
      "content": "<p><a href=\"https://www.kaggle.com/yujiariyasu\" target=\"_blank\">@yujiariyasu</a> congrats on the gold medal! Nice topic! Thanks!</p>",
      "rawMarkdown": "yujiariyasu congrats on the gold medal! Nice topic! Thanks!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2313103,
      "author_name": "zhugds",
      "author_url": "",
      "post_date": "06/22/2023 12:25:10",
      "content": "<p>Congrats on the gold modal! <br>\nThanks for sharing!<br>\nWhich algorithm did you use for object detection?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2313223,
          "author_name": "yujiariyasu",
          "author_url": "",
          "post_date": "06/22/2023 14:35:50",
          "content": "<p>Thank you! Of course I used yolox.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2319386,
      "author_name": "serangu",
      "author_url": "",
      "post_date": "06/27/2023 05:25:48",
      "content": "<p><a href=\"https://www.kaggle.com/yujiariyasu\" target=\"_blank\">@yujiariyasu</a> congrats on the gold medal! Nice topic! Thanks!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2312751": "I respect all the teams that fought to the end.\n\n# overview\n・classification + object detection + ocr + pp\nSince I was not inclined to use an end to end model for unknown data, I divided the data into smaller tasks and applied a post process using human knowledge to each of them. All processes were trained on all data\n\n# 1, Classify chart type \n・I used convnext xlarge.\n\n# 2, Detect each parts \n・chart value such as bar and scatter\n・chart body\n・x-tick-label\n・y-tick-label\n\n# 3, Classify text direction\n・Post processing depends on the orientation of the x text.\n\n# 4, ocr each x/y texts by TROCR\n・Detected x-tick-label / y-tick-label converted to text by ocr one by one.\n\n# 5, pp\n・The coordinates of the detected chart are converted to a value using the all results. Several hundred lines of code were required for each type. \n\n# point\n・I spent the most time on the scatter, which is most often included in private test set.\n・Scatter is detected at the same time as ticks to reduce false positives.\n・Increasing the image size when detecting Scatter has greatly improved the score. This was due to the fact that the number of small overlapping dots could be more accurately estimated. (I used 4 sizes for Scatter. Train by 768 and inference by 864/960, and train by 1024 and inference by 1156/1280 and apply wbf.)\n・The $ and % have been replaced with ''. However, for $, I apply a preprocessing to account for the fact that it is often mistaken for 3, 5, s, and S.\n・Many other modifications using human knowledge are incorporated in the post process.\n・In line chart-type, a vertical line was drawn from the x-tick-label, and the intersection with the line chart was detectied.\n\n# Things that didn't work\n・finetune during submit\nSince I had about 8 extra hours during the submission, I figured there was room to relearn scatter: after detecting the points in scatter, generate a new image with copy-paste augmentation (I figured it would be better to generate a challenging image with overlapping points), The image is then used to finetune yolox. The automatic generation of the image seemed to work very well, but did not increase the score.\n・1000 other experiments\n\n---------\nGood work guys! Let's fight again in another competition!",
    "2313103": "Congrats on the gold modal! \nThanks for sharing!\nWhich algorithm did you use for object detection?",
    "2313223": "Thank you! Of course I used yolox.",
    "2319386": "yujiariyasu congrats on the gold medal! Nice topic! Thanks!"
  },
  "source": "meta"
}