![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4359632%2F98c407b0f80cb83876cfc21de9604f57%2FScreenshot%202023-04-07%20at%206.06.03%20PM.png?generation=1680871109257950&alt=media\")
A new plot-to-text model is just released by Google named as DePlot. Recently, Pix2Struct model by Google was released on HuggingFace hub. It showed great visual-language ability. It came with a very simple architecture (a vision encoder and a language decoder, with no OCR involved).
\nDeplot is based on the same architecture as Pix2Struct. It is designed to convert visual plots or charts into text. The approach taken by DePlot involves breaking down the visual language reasoning challenge into two stages: plot-to-text translation and reasoning over the resulting text. This is achieved through a modality conversion module (named as DePlot), which translates images into linearized tables. The resulting text can then be used to prompt a large language model (LLM), which can reason over the graph data using its few-shot reasoning capabilities.
\n
Source: Deplot Paper
\nThis model can be useful for this competition.
", "messages": [ { "id": "2213232", "postDate": "04/07/2023 12:35:14", "content": "A new plot-to-text model is just released by Google named as DePlot. Recently, Pix2Struct model by Google was released on HuggingFace hub. It showed great visual-language ability. It came with a very simple architecture (a vision encoder and a language decoder, with no OCR involved).
\nDeplot is based on the same architecture as Pix2Struct. It is designed to convert visual plots or charts into text. The approach taken by DePlot involves breaking down the visual language reasoning challenge into two stages: plot-to-text translation and reasoning over the resulting text. This is achieved through a modality conversion module (named as DePlot), which translates images into linearized tables. The resulting text can then be used to prompt a large language model (LLM), which can reason over the graph data using its few-shot reasoning capabilities.
\n
Source: Deplot Paper
\nThis model can be useful for this competition.
", "rawMarkdown": "# DePlot\n\nA new plot-to-text model is just released by Google named as [**DePlot**](https://huggingface.co/docs/transformers/main/en/model_doc/deplot#deplot). Recently, [**Pix2Struct**](https://huggingface.co/docs/transformers/main/en/model_doc/pix2struct) model by Google was released on HuggingFace hub. It showed great visual-language ability. It came with a very simple architecture (a vision encoder and a language decoder, with no OCR involved).\n\n**Deplot** is based on the same architecture as Pix2Struct. It is designed to convert visual plots or charts into text. The approach taken by DePlot involves breaking down the visual language reasoning challenge into two stages: plot-to-text translation and reasoning over the resulting text. This is achieved through a modality conversion module (named as DePlot), which translates images into linearized tables. The resulting text can then be used to prompt a large language model (LLM), which can reason over the graph data using its few-shot reasoning capabilities.\n\n\n\nSource: [Deplot Paper](https://arxiv.org/pdf/2212.10505.pdf)\n\n# Resources\n- [Announcement post](https://www.linkedin.com/posts/niels-rogge-a3b7a3127_ai-google-artificialintelligence-activity-7050015993594540033-VoVA?utm_source=share&utm_medium=member_desktop)\n- [Deplot documentation](https://huggingface.co/docs/transformers/main/en/model_doc/deplot#deplot)\n- [DePlot+LLM: Multimodal chain-of-thought reasoning on plots (HuggingFace Space demo)](https://huggingface.co/spaces/fl399/deplot_plus_llm)\n- [Finetuning DePlot on custom data](https://huggingface.co/docs/transformers/main/en/model_doc/deplot#finetuning)\n\nThis model can be useful for this competition.", "votes": null }, { "id": "2213862", "postDate": "04/07/2023 22:48:57", "content": "Looks promising!
", "rawMarkdown": "Looks promising!", "votes": null }, { "id": "2213978", "postDate": "04/08/2023 02:55:24", "content": "Thanks for sharing! I tried
\nhttps://www.kaggle.com/code/gomaki/deplot-trying-out
Interesting contribution @atharvaingle! Thanks for sharing💥💪
", "rawMarkdown": "Interesting contribution @atharvaingle! Thanks for sharing💥💪", "votes": null } ], "comments": [ { "id": 2213862, "author_name": "zhugds", "author_url": "", "post_date": "04/07/2023 22:48:57", "content": "Looks promising!
", "votes": null, "replies": [] }, { "id": 2213978, "author_name": "gomaki", "author_url": "", "post_date": "04/08/2023 02:55:24", "content": "Thanks for sharing! I tried
\nhttps://www.kaggle.com/code/gomaki/deplot-trying-out
Interesting contribution @atharvaingle! Thanks for sharing💥💪
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2213232": "# DePlot\n\nA new plot-to-text model is just released by Google named as [**DePlot**](https://huggingface.co/docs/transformers/main/en/model_doc/deplot#deplot). Recently, [**Pix2Struct**](https://huggingface.co/docs/transformers/main/en/model_doc/pix2struct) model by Google was released on HuggingFace hub. It showed great visual-language ability. It came with a very simple architecture (a vision encoder and a language decoder, with no OCR involved).\n\n**Deplot** is based on the same architecture as Pix2Struct. It is designed to convert visual plots or charts into text. The approach taken by DePlot involves breaking down the visual language reasoning challenge into two stages: plot-to-text translation and reasoning over the resulting text. This is achieved through a modality conversion module (named as DePlot), which translates images into linearized tables. The resulting text can then be used to prompt a large language model (LLM), which can reason over the graph data using its few-shot reasoning capabilities.\n\n\n\nSource: [Deplot Paper](https://arxiv.org/pdf/2212.10505.pdf)\n\n# Resources\n- [Announcement post](https://www.linkedin.com/posts/niels-rogge-a3b7a3127_ai-google-artificialintelligence-activity-7050015993594540033-VoVA?utm_source=share&utm_medium=member_desktop)\n- [Deplot documentation](https://huggingface.co/docs/transformers/main/en/model_doc/deplot#deplot)\n- [DePlot+LLM: Multimodal chain-of-thought reasoning on plots (HuggingFace Space demo)](https://huggingface.co/spaces/fl399/deplot_plus_llm)\n- [Finetuning DePlot on custom data](https://huggingface.co/docs/transformers/main/en/model_doc/deplot#finetuning)\n\nThis model can be useful for this competition.", "2213862": "Looks promising!", "2213978": "Thanks for sharing! I tried\nhttps://www.kaggle.com/code/gomaki/deplot-trying-out", "2300908": "Interesting contribution @atharvaingle! Thanks for sharing💥💪" }, "source": "meta" }