![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16362079%2Fc2bf83499653e5e07e1131724b4dd6d3%2Ftraining_curve_fold0.png?generation=1766626406418133&alt=media\")
I am getting 0.70 validation score while training but Kaggle score is much lower (0.58). I am not sure what I might be missing. Is it data splitting or just because of small validation set? It may be also because overall training set itself is much smaller than Kaggle test set. The competition is purposefully made difficult to force participants to work with limited data. Possibly to find a way to train model when dataset is actually small.
\n
This dataset is a classic - it has almost everything wrong that you can get.
\nIn other posts I have shared the opinion that when the data was collected a decade ago there was never an intention to use visual image methods. The images have lots of issues that go well beyond any attempt to generalize the model; and I am not sure that term even existed at the time this study was initiated.
\nYou need to read ALL the discussion posts to jot down all the issues that you might be facing. The most likely is your fold method is leaking information resulting in the over-fit your seeing on your loss plot and the gap your getting on your LB score. Since you have not shared a notebook its hard to tell what other things you might be missing. Your Dry_green R2 plot is a bit strange, Dry_Dead is my consistent worst label over a wide range of models.
\nYou have 1 down vote and you might end up with a few more - that's probably folks telling you to read all the discussion posts before you ask a question that kind of shouts \"help me with my homework\" :)
", "rawMarkdown": "This dataset is a classic - it has almost everything wrong that you can get.\n\nIn other posts I have shared the opinion that when the data was collected a decade ago there was never an intention to use visual image methods. The images have lots of issues that go well beyond any attempt to generalize the model; and I am not sure that term even existed at the time this study was initiated.\n\nYou need to read ALL the discussion posts to jot down all the issues that you might be facing. The most likely is your fold method is leaking information resulting in the over-fit your seeing on your loss plot and the gap your getting on your LB score. Since you have not shared a notebook its hard to tell what other things you might be missing. Your Dry_green R2 plot is a bit strange, Dry_Dead is my consistent worst label over a wide range of models.\n\nYou have 1 down vote and you might end up with a few more - that's probably folks telling you to read all the discussion posts before you ask a question that kind of shouts \"help me with my homework\" :)", "votes": 5 }, { "id": 3381588, "postDate": "2025-12-25T01:23:31.717Z", "content": "I am getting 0.70 validation score while training but Kaggle score is much lower (0.58). I am not sure what I might be missing. Is it data splitting or just because of small validation set? It may be also because overall training set itself is much smaller than Kaggle test set. The competition is purposefully made difficult to force participants to work with limited data. Possibly to find a way to train model when dataset is actually small.
\n
Heyy dude … This situation is very common in Kaggle competitions, especially when the dataset is small. A validation R² of 0.70 usually means your local split is not fully representative of the hidden test set.\nDifferences in data distribution, random train–validation splits, or subtle data leakage can make local scores look optimistic. Small validation sets also add a lot of variance to the metric.\nUsing k-fold cross-validation instead of a single split, validating feature engineering carefully, and watching for overfitting often help reduce the gap. In difficult competitions with limited data, some score drop on Kaggle’s leaderboard is expected.\nI hope it will help you bro👍.
", "rawMarkdown": "Heyy dude ... This situation is very common in Kaggle competitions, especially when the dataset is small. A validation R² of 0.70 usually means your local split is not fully representative of the hidden test set.\nDifferences in data distribution, random train–validation splits, or subtle data leakage can make local scores look optimistic. Small validation sets also add a lot of variance to the metric.\nUsing k-fold cross-validation instead of a single split, validating feature engineering carefully, and watching for overfitting often help reduce the gap. In difficult competitions with limited data, some score drop on Kaggle’s leaderboard is expected.\nI hope it will help you bro👍.", "votes": 4 }, { "id": 3381589, "postDate": "2025-12-25T01:32:56.310Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 3381652, "author_name": "PC Jimmmy", "author_url": "", "post_date": "2025-12-25T06:32:51.537000", "content": "This dataset is a classic - it has almost everything wrong that you can get.
\nIn other posts I have shared the opinion that when the data was collected a decade ago there was never an intention to use visual image methods. The images have lots of issues that go well beyond any attempt to generalize the model; and I am not sure that term even existed at the time this study was initiated.
\nYou need to read ALL the discussion posts to jot down all the issues that you might be facing. The most likely is your fold method is leaking information resulting in the over-fit your seeing on your loss plot and the gap your getting on your LB score. Since you have not shared a notebook its hard to tell what other things you might be missing. Your Dry_green R2 plot is a bit strange, Dry_Dead is my consistent worst label over a wide range of models.
\nYou have 1 down vote and you might end up with a few more - that's probably folks telling you to read all the discussion posts before you ask a question that kind of shouts \"help me with my homework\" :)
", "votes": 5, "replies": [] }, { "id": 3381776, "author_name": "Mani Ratan", "author_url": "", "post_date": "2025-12-25T13:56:05.207000", "content": "Heyy dude … This situation is very common in Kaggle competitions, especially when the dataset is small. A validation R² of 0.70 usually means your local split is not fully representative of the hidden test set.\nDifferences in data distribution, random train–validation splits, or subtle data leakage can make local scores look optimistic. Small validation sets also add a lot of variance to the metric.\nUsing k-fold cross-validation instead of a single split, validating feature engineering carefully, and watching for overfitting often help reduce the gap. In difficult competitions with limited data, some score drop on Kaggle’s leaderboard is expected.\nI hope it will help you bro👍.
", "votes": 4, "replies": [] }, { "id": 3381589, "author_name": "", "author_url": "", "post_date": "2025-12-25T01:32:56.310000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "3381652": "This dataset is a classic - it has almost everything wrong that you can get.\n\nIn other posts I have shared the opinion that when the data was collected a decade ago there was never an intention to use visual image methods. The images have lots of issues that go well beyond any attempt to generalize the model; and I am not sure that term even existed at the time this study was initiated.\n\nYou need to read ALL the discussion posts to jot down all the issues that you might be facing. The most likely is your fold method is leaking information resulting in the over-fit your seeing on your loss plot and the gap your getting on your LB score. Since you have not shared a notebook its hard to tell what other things you might be missing. Your Dry_green R2 plot is a bit strange, Dry_Dead is my consistent worst label over a wide range of models.\n\nYou have 1 down vote and you might end up with a few more - that's probably folks telling you to read all the discussion posts before you ask a question that kind of shouts \"help me with my homework\" :)", "3381588": "I am getting 0.70 validation score while training but Kaggle score is much lower (0.58). I am not sure what I might be missing. Is it data splitting or just because of small validation set? It may be also because overall training set itself is much smaller than Kaggle test set. The competition is purposefully made difficult to force participants to work with limited data. Possibly to find a way to train model when dataset is actually small.\n\n", "3381776": "Heyy dude ... This situation is very common in Kaggle competitions, especially when the dataset is small. A validation R² of 0.70 usually means your local split is not fully representative of the hidden test set.\nDifferences in data distribution, random train–validation splits, or subtle data leakage can make local scores look optimistic. Small validation sets also add a lot of variance to the metric.\nUsing k-fold cross-validation instead of a single split, validating feature engineering carefully, and watching for overfitting often help reduce the gap. In difficult competitions with limited data, some score drop on Kaggle’s leaderboard is expected.\nI hope it will help you bro👍.", "3381589": "" } }