{ "id": 418376, "title": "51st Place Solution - Single Matcha Model", "url": "/competitions/benetech-making-graphs-accessible/writeups/one-more-epoch-51st-place-solution-single-matcha-m", "author_name": "", "post_date": "2023-06-20T09:25:37.987Z", "votes": 19, "comment_count": 4, "views": 0, "content": "

Hi everyone,

\n

First of all, huge congratulations to the winners. I would like to share our 51st-place solution briefly as well. Our team is @batuhanm and @emirkocak

\n
\n

We started working on this competition just two weeks prior to the deadline and dived into the discussions immediately. After having an initial understanding we found these two baseline notebooks:

\n\n

huge thanks to @nbroad @heyytanay for their contributions. I merged these two code baselines and trained an end-to-end Matcha model for two epochs. The result of this pipeline was .43 - .25. After some investigations, I realized that the model was underfitting so we trained the same model for 8 more epochs (so 10 in total). This experiment reached .66 in the public LB and .37 in the private. This whole process took a week or so.

\n

Then after having lots of debates on using the ICDAR dataset and discussions, it was banned at first so we didn't focus on that part. Instead, we included this dataset:

\n\n

from @brendanartley (again huge thanks here). With around 300k~ images we trained 3 more epochs in total and reached out to .73 in the public LB.

\n

Since the matcha is a generative model, for final post-processing, we did some cleaning with the numbers and got a slight boost (+0.001).

\n

Things didn't work

\n\n

We also trained a separate scatter model but due to time limit, we couldn't manage it to incorporate our inference pipeline. After reading couple of solutions, it looks this would improve the results a lot.

\n

Well, thanks for reading, and again congrats to the winners!

", "messages": [ { "id": "2310321", "postDate": "06/20/2023 09:23:17", "content": "

Hi everyone,

\n

First of all, huge congratulations to the winners. I would like to share our 51st-place solution briefly as well. Our team is @batuhanm and @emirkocak

\n
\n

We started working on this competition just two weeks prior to the deadline and dived into the discussions immediately. After having an initial understanding we found these two baseline notebooks:

\n\n

huge thanks to @nbroad @heyytanay for their contributions. I merged these two code baselines and trained an end-to-end Matcha model for two epochs. The result of this pipeline was .43 - .25. After some investigations, I realized that the model was underfitting so we trained the same model for 8 more epochs (so 10 in total). This experiment reached .66 in the public LB and .37 in the private. This whole process took a week or so.

\n

Then after having lots of debates on using the ICDAR dataset and discussions, it was banned at first so we didn't focus on that part. Instead, we included this dataset:

\n\n

from @brendanartley (again huge thanks here). With around 300k~ images we trained 3 more epochs in total and reached out to .73 in the public LB.

\n

Since the matcha is a generative model, for final post-processing, we did some cleaning with the numbers and got a slight boost (+0.001).

\n

Things didn't work

\n\n

We also trained a separate scatter model but due to time limit, we couldn't manage it to incorporate our inference pipeline. After reading couple of solutions, it looks this would improve the results a lot.

\n

Well, thanks for reading, and again congrats to the winners!

", "rawMarkdown": "Hi everyone,\n\nFirst of all, huge congratulations to the winners. I would like to share our 51st-place solution briefly as well. Our team is @batuhanm and @emirkocak \n\n---\n\nWe started working on this competition just two weeks prior to the deadline and dived into the discussions immediately. After having an initial understanding we found these two baseline notebooks:\n\n- https://www.kaggle.com/code/nbroad/donut-train-benetech \n- https://www.kaggle.com/code/heyytanay/training-pix2struct-pytorch-amp-w-b \n\nhuge thanks to @nbroad @heyytanay for their contributions. I merged these two code baselines and trained an end-to-end Matcha model for two epochs. The result of this pipeline was .43 - .25. After some investigations, I realized that the model was underfitting so we trained the same model for 8 more epochs (so 10 in total). This experiment reached .66 in the public LB and .37 in the private. This whole process took a week or so.\n\nThen after having lots of debates on using the ICDAR dataset and discussions, it was banned at first so we didn't focus on that part. Instead, we included this dataset:\n\n- https://www.kaggle.com/datasets/brendanartley/benetech-extra-generated-data\n\nfrom @brendanartley (again huge thanks here). With around 300k~ images we trained 3 more epochs in total and reached out to .73 in the public LB.\n\nSince the matcha is a generative model, for final post-processing, we did some cleaning with the numbers and got a slight boost (+0.001).\n\n#### Things didn't work\n- We couldn't manage deplot to work as matcha. Our goal was to combine these two models and improve on confused examples, but 1 week of effort was not enough and it just didn't work.\n\nWe also trained a separate scatter model but due to time limit, we couldn't manage it to incorporate our inference pipeline. After reading couple of solutions, it looks this would improve the results a lot.\n\nWell, thanks for reading, and again congrats to the winners!", "votes": null }, { "id": "2310369", "postDate": "06/20/2023 10:09:02", "content": "

@snnclsr thank you for your story!

", "rawMarkdown": "snnclsr thank you for your story!", "votes": null }, { "id": "2310807", "postDate": "06/20/2023 16:04:26", "content": "

Thanks for the kind words @snnclsr. I am glad you found the data useful :)

", "rawMarkdown": "Thanks for the kind words @snnclsr. I am glad you found the data useful :)", "votes": null }, { "id": "2310918", "postDate": "06/20/2023 17:30:48", "content": "

Congratulations on your Bronze finish @snnclsr!
\nI'm glad you found my kernel helpful!

", "rawMarkdown": "Congratulations on your Bronze finish @snnclsr!\nI'm glad you found my kernel helpful!", "votes": null }, { "id": "2311095", "postDate": "06/20/2023 22:53:22", "content": "

Thank you! @heyytanay

", "rawMarkdown": "Thank you! @heyytanay", "votes": null } ], "comments": [ { "id": 2310369, "author_name": "serangu", "author_url": "", "post_date": "06/20/2023 10:09:02", "content": "

@snnclsr thank you for your story!

", "votes": null, "replies": [] }, { "id": 2310807, "author_name": "brendanartley", "author_url": "", "post_date": "06/20/2023 16:04:26", "content": "

Thanks for the kind words @snnclsr. I am glad you found the data useful :)

", "votes": null, "replies": [] }, { "id": 2310918, "author_name": "heyytanay", "author_url": "", "post_date": "06/20/2023 17:30:48", "content": "

Congratulations on your Bronze finish @snnclsr!
\nI'm glad you found my kernel helpful!

", "votes": null, "replies": [ { "id": 2311095, "author_name": "snnclsr", "author_url": "", "post_date": "06/20/2023 22:53:22", "content": "

Thank you! @heyytanay

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2310321": "Hi everyone,\n\nFirst of all, huge congratulations to the winners. I would like to share our 51st-place solution briefly as well. Our team is @batuhanm and @emirkocak \n\n---\n\nWe started working on this competition just two weeks prior to the deadline and dived into the discussions immediately. After having an initial understanding we found these two baseline notebooks:\n\n- https://www.kaggle.com/code/nbroad/donut-train-benetech \n- https://www.kaggle.com/code/heyytanay/training-pix2struct-pytorch-amp-w-b \n\nhuge thanks to @nbroad @heyytanay for their contributions. I merged these two code baselines and trained an end-to-end Matcha model for two epochs. The result of this pipeline was .43 - .25. After some investigations, I realized that the model was underfitting so we trained the same model for 8 more epochs (so 10 in total). This experiment reached .66 in the public LB and .37 in the private. This whole process took a week or so.\n\nThen after having lots of debates on using the ICDAR dataset and discussions, it was banned at first so we didn't focus on that part. Instead, we included this dataset:\n\n- https://www.kaggle.com/datasets/brendanartley/benetech-extra-generated-data\n\nfrom @brendanartley (again huge thanks here). With around 300k~ images we trained 3 more epochs in total and reached out to .73 in the public LB.\n\nSince the matcha is a generative model, for final post-processing, we did some cleaning with the numbers and got a slight boost (+0.001).\n\n#### Things didn't work\n- We couldn't manage deplot to work as matcha. Our goal was to combine these two models and improve on confused examples, but 1 week of effort was not enough and it just didn't work.\n\nWe also trained a separate scatter model but due to time limit, we couldn't manage it to incorporate our inference pipeline. After reading couple of solutions, it looks this would improve the results a lot.\n\nWell, thanks for reading, and again congrats to the winners!", "2310369": "snnclsr thank you for your story!", "2310807": "Thanks for the kind words @snnclsr. I am glad you found the data useful :)", "2310918": "Congratulations on your Bronze finish @snnclsr!\nI'm glad you found my kernel helpful!", "2311095": "Thank you! @heyytanay" }, "source": "existing_discussion", "complete": true, "competition_id": 43873, "competition_slug": "benetech-making-graphs-accessible", "team_id": 10454966, "solution_url": "/competitions/benetech-making-graphs-accessible/writeups/one-more-epoch-51st-place-solution-single-matcha-m" }