![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F5837437%2F73fc4006430735336f56ea7064a4935f%2Fdata_dis.png?generation=1769686226547614&alt=media\")
\nIt can be concluded that: \nIt can be concluded that: Therefore, using the Tweedie loss function is most suitable to describe this distribution. It is a compound Poisson-Gamma distribution. You can see more about the Tweedie distribution function here: https://en.wikipedia.org/wiki/Tweedie_distribution
\nOne of the mistake is relying too much on the overall R2 score provided by the organizers. A high overall R2 can be misleading and may hide significant bias in your model.\nSince we need a model that generalizes well across all three targets \"Dry_Green\", \"Dry_Clover\", and \"Dry_Dead\". We should calculate the R2 for each category separately. To prevent the model from performing well on one category while failing on others, I have introduced a custom evaluation metric:\n$$S = {(R^2_{\\text{Green}}, R^2_{\\text{Clover}}, R^2_{\\text{Dead}})}$$$$\\text{Score} = \\text{mean}(S) - \\alpha \\times \\text{std}(S)$$\nWhy this works:\nBy subtracting the standard deviation (std) , we penalize inconsistency. This forces the model to achieve a balance between all categories, ensuring the predictions are stable and unbiased across the entire dataset.
\nThe complete source code is here. Hopefully, it will be useful to everyone:\nhttps://github.com/sunny442k3/CSIRO-Image2Biomass
", "messages": [ { "id": "3398612", "postDate": "01/29/2026 11:59:44", "content": "\nIt can be concluded that: Therefore, using the Tweedie loss function is most suitable to describe this distribution. It is a compound Poisson-Gamma distribution. You can see more about the Tweedie distribution function here: https://en.wikipedia.org/wiki/Tweedie_distribution
\nOne of the mistake is relying too much on the overall R2 score provided by the organizers. A high overall R2 can be misleading and may hide significant bias in your model.\nSince we need a model that generalizes well across all three targets \"Dry_Green\", \"Dry_Clover\", and \"Dry_Dead\". We should calculate the R2 for each category separately. To prevent the model from performing well on one category while failing on others, I have introduced a custom evaluation metric:\n$$S = {(R^2_{\\text{Green}}, R^2_{\\text{Clover}}, R^2_{\\text{Dead}})}$$$$\\text{Score} = \\text{mean}(S) - \\alpha \\times \\text{std}(S)$$\nWhy this works:\nBy subtracting the standard deviation (std) , we penalize inconsistency. This forces the model to achieve a balance between all categories, ensuring the predictions are stable and unbiased across the entire dataset.
\nThe complete source code is here. Hopefully, it will be useful to everyone:\nhttps://github.com/sunny442k3/CSIRO-Image2Biomass
", "rawMarkdown": "# Introduction\n- In this writeup, I will focus on some key points in this method. The baseline model is inspired by https://www.kaggle.com/code/mattiaangeli/dinov3-baseline-lb-0-70\n\n# Preprocessing\n- Use StratifiedGroupKFold to divide into 5 corresponding folds based on **labels=\"State**\" and **groups=\"Sampling_Date\"**. However, brute-force analysis can be performed on multiple random_seed to find the **golden_seed**. \n- The metric used to find the golden_seed is the Mean calculated based on the **\"Dry_Total\"** column of the training and validation sets.\n\n# Model fine-tuning\n- Use DoRA instead of LoRA. In short, LoRA (Low-Rank Adaptation) updates a model by adding a low-rank matrix to the original weights, focusing primarily on efficient fine-tuning. DoRA (Weight-Decomposed Low-Rank Adaptation) improves this by decomposing the weights into two components—magnitude and direction and applying LoRA only to the directional part. This allows DoRA to mimic full fine-tuning more accurately by learning changes in scale and orientation separately.\n- Objective: Use Tweedie instead of L1 or L2 or Hubert Loss. You can look at the distribution of the original data through the image below:\n\nIt can be concluded that: \n- Zero-inflation: A significant number of samples have exactly 0g biomass. \n- Right-skewness: Non-zero values are highly skewed with a long tail.\n\nTherefore, using the Tweedie loss function is most suitable to describe this distribution. It is a compound Poisson-Gamma distribution. You can see more about the Tweedie distribution function here: https://en.wikipedia.org/wiki/Tweedie_distribution\n\n# Ballance metric\nOne of the mistake is relying too much on the overall **R2** score provided by the organizers. A high overall **R2** can be misleading and may hide significant bias in your model.\nSince we need a model that generalizes well across all three targets **\"Dry_Green\"**, **\"Dry_Clover\"**, and **\"Dry_Dead\"**. We should calculate the **R2** for each category separately. To prevent the model from performing well on one category while failing on others, I have introduced a custom evaluation metric:\n$$S = \\{(R^2_{\\text{Green}}, R^2_{\\text{Clover}}, R^2_{\\text{Dead}})\\}$$$$\\text{Score} = \\text{mean}(S) - \\alpha \\times \\text{std}(S)$$\nWhy this works:\nBy subtracting the standard deviation (**std**) , we penalize inconsistency. This forces the model to achieve a balance between all categories, ensuring the predictions are stable and unbiased across the entire dataset.\n\nThe complete source code is here. Hopefully, it will be useful to everyone:\nhttps://github.com/sunny442k3/CSIRO-Image2Biomass", "votes": null }, { "id": "3398707", "postDate": "01/29/2026 15:22:15", "content": "cool approach , it's my first time learning about Dora ,thanks for that
", "rawMarkdown": "cool approach , it's my first time learning about Dora ,thanks for that", "votes": null }, { "id": "3398956", "postDate": "01/30/2026 01:00:33", "content": "perfect approach specialy you metric thank you
", "rawMarkdown": "perfect approach specialy you metric thank you", "votes": null }, { "id": "3399165", "postDate": "01/30/2026 10:30:07", "content": "Nice work. Thank you for shearing.\nI have a question. \nHow did you submit a notebook using \"facebook/dinov3-vitl16-pretrain-lvd1689m\".\nBecause I was not able to find a way to submit a notebook using a model without internet.
", "rawMarkdown": "Nice work. Thank you for shearing.\nI have a question. \nHow did you submit a notebook using \"facebook/dinov3-vitl16-pretrain-lvd1689m\".\nBecause I was not able to find a way to submit a notebook using a model without internet.", "votes": null }, { "id": "3399179", "postDate": "01/30/2026 10:51:04", "content": "You can load it and save it locally so you can reuse it when the internet is off
", "rawMarkdown": "You can load it and save it locally so you can reuse it when the internet is off", "votes": null }, { "id": "3400834", "postDate": "02/02/2026 11:43:15", "content": "Thank you so much!
", "rawMarkdown": "Thank you so much!", "votes": null }, { "id": "3400873", "postDate": "02/02/2026 14:08:10", "content": "I really enjoyed your solution.\nI’d like to adapt parts of your GitHub code, run new experiments, and share the modified version publicly in Japanese with your github url.\nWould that be okay?
", "rawMarkdown": "I really enjoyed your solution.\nI’d like to adapt parts of your GitHub code, run new experiments, and share the modified version publicly in Japanese with your github url.\nWould that be okay?", "votes": null }, { "id": "3400880", "postDate": "02/02/2026 14:22:58", "content": "Feel free to use it. I'm glad someone is using my code 🥳
", "rawMarkdown": "Feel free to use it. I'm glad someone is using my code 🥳", "votes": null }, { "id": "3400883", "postDate": "02/02/2026 14:38:32", "content": "Thank you!!
", "rawMarkdown": "Thank you!!", "votes": null } ], "comments": [ { "id": 3398707, "author_name": "rajraman83", "author_url": "", "post_date": "01/29/2026 15:22:15", "content": "cool approach , it's my first time learning about Dora ,thanks for that
", "votes": null, "replies": [] }, { "id": 3398956, "author_name": "bassetkerouche", "author_url": "", "post_date": "01/30/2026 01:00:33", "content": "perfect approach specialy you metric thank you
", "votes": null, "replies": [] }, { "id": 3399165, "author_name": "hhorimoto", "author_url": "", "post_date": "01/30/2026 10:30:07", "content": "Nice work. Thank you for shearing.\nI have a question. \nHow did you submit a notebook using \"facebook/dinov3-vitl16-pretrain-lvd1689m\".\nBecause I was not able to find a way to submit a notebook using a model without internet.
", "votes": null, "replies": [ { "id": 3399179, "author_name": "nguynphcvn", "author_url": "", "post_date": "01/30/2026 10:51:04", "content": "You can load it and save it locally so you can reuse it when the internet is off
", "votes": null, "replies": [ { "id": 3400834, "author_name": "hhorimoto", "author_url": "", "post_date": "02/02/2026 11:43:15", "content": "Thank you so much!
", "votes": null, "replies": [] }, { "id": 3400873, "author_name": "hhorimoto", "author_url": "", "post_date": "02/02/2026 14:08:10", "content": "I really enjoyed your solution.\nI’d like to adapt parts of your GitHub code, run new experiments, and share the modified version publicly in Japanese with your github url.\nWould that be okay?
", "votes": null, "replies": [ { "id": 3400880, "author_name": "nguynphcvn", "author_url": "", "post_date": "02/02/2026 14:22:58", "content": "Feel free to use it. I'm glad someone is using my code 🥳
", "votes": null, "replies": [ { "id": 3400883, "author_name": "hhorimoto", "author_url": "", "post_date": "02/02/2026 14:38:32", "content": "Thank you!!
", "votes": null, "replies": [] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "3398612": "# Introduction\n- In this writeup, I will focus on some key points in this method. The baseline model is inspired by https://www.kaggle.com/code/mattiaangeli/dinov3-baseline-lb-0-70\n\n# Preprocessing\n- Use StratifiedGroupKFold to divide into 5 corresponding folds based on **labels=\"State**\" and **groups=\"Sampling_Date\"**. However, brute-force analysis can be performed on multiple random_seed to find the **golden_seed**. \n- The metric used to find the golden_seed is the Mean calculated based on the **\"Dry_Total\"** column of the training and validation sets.\n\n# Model fine-tuning\n- Use DoRA instead of LoRA. In short, LoRA (Low-Rank Adaptation) updates a model by adding a low-rank matrix to the original weights, focusing primarily on efficient fine-tuning. DoRA (Weight-Decomposed Low-Rank Adaptation) improves this by decomposing the weights into two components—magnitude and direction and applying LoRA only to the directional part. This allows DoRA to mimic full fine-tuning more accurately by learning changes in scale and orientation separately.\n- Objective: Use Tweedie instead of L1 or L2 or Hubert Loss. You can look at the distribution of the original data through the image below:\n\nIt can be concluded that: \n- Zero-inflation: A significant number of samples have exactly 0g biomass. \n- Right-skewness: Non-zero values are highly skewed with a long tail.\n\nTherefore, using the Tweedie loss function is most suitable to describe this distribution. It is a compound Poisson-Gamma distribution. You can see more about the Tweedie distribution function here: https://en.wikipedia.org/wiki/Tweedie_distribution\n\n# Ballance metric\nOne of the mistake is relying too much on the overall **R2** score provided by the organizers. A high overall **R2** can be misleading and may hide significant bias in your model.\nSince we need a model that generalizes well across all three targets **\"Dry_Green\"**, **\"Dry_Clover\"**, and **\"Dry_Dead\"**. We should calculate the **R2** for each category separately. To prevent the model from performing well on one category while failing on others, I have introduced a custom evaluation metric:\n$$S = \\{(R^2_{\\text{Green}}, R^2_{\\text{Clover}}, R^2_{\\text{Dead}})\\}$$$$\\text{Score} = \\text{mean}(S) - \\alpha \\times \\text{std}(S)$$\nWhy this works:\nBy subtracting the standard deviation (**std**) , we penalize inconsistency. This forces the model to achieve a balance between all categories, ensuring the predictions are stable and unbiased across the entire dataset.\n\nThe complete source code is here. Hopefully, it will be useful to everyone:\nhttps://github.com/sunny442k3/CSIRO-Image2Biomass", "3398707": "cool approach , it's my first time learning about Dora ,thanks for that", "3398956": "perfect approach specialy you metric thank you", "3399165": "Nice work. Thank you for shearing.\nI have a question. \nHow did you submit a notebook using \"facebook/dinov3-vitl16-pretrain-lvd1689m\".\nBecause I was not able to find a way to submit a notebook using a model without internet.", "3399179": "You can load it and save it locally so you can reuse it when the internet is off", "3400834": "Thank you so much!", "3400873": "I really enjoyed your solution.\nI’d like to adapt parts of your GitHub code, run new experiments, and share the modified version publicly in Japanese with your github url.\nWould that be okay?", "3400880": "Feel free to use it. I'm glad someone is using my code 🥳", "3400883": "Thank you!!" }, "source": "meta" }