![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2125251%2F67c9167e0ad00e783e6aeada277b7263%2FScreenshot%202023-06-11%20at%202.22.30%20PM.png?generation=1686464571539128&alt=media\")
Pre-trained backbones such as Matcha and Deplot are quite popular in this competition. Recent clarifications regarding the competition rules made me re-think acceptability of such models. For Matcha, the composition of training dataset are as below (https://arxiv.org/pdf/2212.09662v2.pdf):
\n
I wonder if these datasets are compliant with competition rules. If not, it will make a large portion of the LB submissions invalid.
\nNote: Deplot model (https://arxiv.org/pdf/2212.10505.pdf) was initialized with weights from Matcha. Hence, its admissibility is contingent on Matcha.
", "messages": [ { "id": "2295617", "postDate": "06/11/2023 06:35:19", "content": "Pre-trained backbones such as Matcha and Deplot are quite popular in this competition. Recent clarifications regarding the competition rules made me re-think acceptability of such models. For Matcha, the composition of training dataset are as below (https://arxiv.org/pdf/2212.09662v2.pdf):
\n
I wonder if these datasets are compliant with competition rules. If not, it will make a large portion of the LB submissions invalid.
\nNote: Deplot model (https://arxiv.org/pdf/2212.10505.pdf) was initialized with weights from Matcha. Hence, its admissibility is contingent on Matcha.
", "rawMarkdown": "Pre-trained backbones such as Matcha and Deplot are quite popular in this competition. Recent clarifications regarding the competition rules made me re-think acceptability of such models. For Matcha, the composition of training dataset are as below (https://arxiv.org/pdf/2212.09662v2.pdf):\n\n\n\n* MATH dataset (https://paperswithcode.com/dataset/math) - MIT\n* DROP dataset (https://paperswithcode.com/dataset/drop) - CC BY-SA 4.0\n* ChartQA dataset (https://github.com/vis-nlp/ChartQA) - GPL-3.0 license\n* PlotQA dataset (https://paperswithcode.com/dataset/plotqa) - Public\n* Screenshot parsing & other internal datasets - I'm not sure\n\nI wonder if these datasets are compliant with competition rules. If not, it will make a large portion of the LB submissions invalid.\n\nNote: Deplot model (https://arxiv.org/pdf/2212.10505.pdf) was initialized with weights from Matcha. Hence, its admissibility is contingent on Matcha.", "votes": null }, { "id": "2295633", "postDate": "06/11/2023 06:56:30", "content": "Many OCR models are pre-trained on the SynthText (https://www.robots.ox.ac.uk/~vgg/terms/dataset-group-2-access.html) and this dataset is for non-commercial use. Also the ResNet backbone of so many models are trained on ImageNet. Are all these pre-trained models not available in this competition?
", "rawMarkdown": "Many OCR models are pre-trained on the SynthText (https://www.robots.ox.ac.uk/~vgg/terms/dataset-group-2-access.html) and this dataset is for non-commercial use. Also the ResNet backbone of so many models are trained on ImageNet. Are all these pre-trained models not available in this competition?", "votes": null }, { "id": "2295666", "postDate": "06/11/2023 07:30:48", "content": "I would like to raise also the question on the availability of mmdetection/mmocr models. I came across @junkoda 's notebook, where he warned that some parts of mmcv do not have MIT/Apache license, but NVIDIA's non-commercial one. From the wording here, it can be understood that as long as the model you use doesn't call upon these specific ops, it should be fine.
\nI would like to think this is the case since mm models have been used frequently in past competitions. But in this competition one is not sure about anything anymore 😂
", "rawMarkdown": "I would like to raise also the question on the availability of mmdetection/mmocr models. I came across @junkoda 's [notebook](https://www.kaggle.com/code/junkoda/cached-chart-element-detection), where he warned that some parts of mmcv do not have MIT/Apache license, but NVIDIA's non-commercial one. From the wording [here](https://github.com/open-mmlab/mmcv/blob/main/LICENSES.md), it can be understood that as long as the model you use doesn't call upon these specific ops, it should be fine.\n\nI would like to think this is the case since mm models have been used frequently in past competitions. But in this competition one is not sure about anything anymore 😂", "votes": null }, { "id": "2296148", "postDate": "06/11/2023 15:17:01", "content": "At this point competition hosts should accept that this was all a product of their lack of research into the matter and nobody should be eliminated from the leaderboard after working for months on optimizing for their problem.
", "rawMarkdown": "At this point competition hosts should accept that this was all a product of their lack of research into the matter and nobody should be eliminated from the leaderboard after working for months on optimizing for their problem.", "votes": null } ], "comments": [ { "id": 2295633, "author_name": "matrix127", "author_url": "", "post_date": "06/11/2023 06:56:30", "content": "Many OCR models are pre-trained on the SynthText (https://www.robots.ox.ac.uk/~vgg/terms/dataset-group-2-access.html) and this dataset is for non-commercial use. Also the ResNet backbone of so many models are trained on ImageNet. Are all these pre-trained models not available in this competition?
", "votes": null, "replies": [] }, { "id": 2295666, "author_name": "qdv206", "author_url": "", "post_date": "06/11/2023 07:30:48", "content": "I would like to raise also the question on the availability of mmdetection/mmocr models. I came across @junkoda 's notebook, where he warned that some parts of mmcv do not have MIT/Apache license, but NVIDIA's non-commercial one. From the wording here, it can be understood that as long as the model you use doesn't call upon these specific ops, it should be fine.
\nI would like to think this is the case since mm models have been used frequently in past competitions. But in this competition one is not sure about anything anymore 😂
", "votes": null, "replies": [] }, { "id": 2296148, "author_name": "cody11null", "author_url": "", "post_date": "06/11/2023 15:17:01", "content": "At this point competition hosts should accept that this was all a product of their lack of research into the matter and nobody should be eliminated from the leaderboard after working for months on optimizing for their problem.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2295617": "Pre-trained backbones such as Matcha and Deplot are quite popular in this competition. Recent clarifications regarding the competition rules made me re-think acceptability of such models. For Matcha, the composition of training dataset are as below (https://arxiv.org/pdf/2212.09662v2.pdf):\n\n\n\n* MATH dataset (https://paperswithcode.com/dataset/math) - MIT\n* DROP dataset (https://paperswithcode.com/dataset/drop) - CC BY-SA 4.0\n* ChartQA dataset (https://github.com/vis-nlp/ChartQA) - GPL-3.0 license\n* PlotQA dataset (https://paperswithcode.com/dataset/plotqa) - Public\n* Screenshot parsing & other internal datasets - I'm not sure\n\nI wonder if these datasets are compliant with competition rules. If not, it will make a large portion of the LB submissions invalid.\n\nNote: Deplot model (https://arxiv.org/pdf/2212.10505.pdf) was initialized with weights from Matcha. Hence, its admissibility is contingent on Matcha.", "2295633": "Many OCR models are pre-trained on the SynthText (https://www.robots.ox.ac.uk/~vgg/terms/dataset-group-2-access.html) and this dataset is for non-commercial use. Also the ResNet backbone of so many models are trained on ImageNet. Are all these pre-trained models not available in this competition?", "2295666": "I would like to raise also the question on the availability of mmdetection/mmocr models. I came across @junkoda 's [notebook](https://www.kaggle.com/code/junkoda/cached-chart-element-detection), where he warned that some parts of mmcv do not have MIT/Apache license, but NVIDIA's non-commercial one. From the wording [here](https://github.com/open-mmlab/mmcv/blob/main/LICENSES.md), it can be understood that as long as the model you use doesn't call upon these specific ops, it should be fine.\n\nI would like to think this is the case since mm models have been used frequently in past competitions. But in this competition one is not sure about anything anymore 😂", "2296148": "At this point competition hosts should accept that this was all a product of their lack of research into the matter and nobody should be eliminated from the leaderboard after working for months on optimizing for their problem." }, "source": "meta" }