{ "id": 669120, "title": "Is CV even useful here?", "url": "/competitions/csiro-biomass/discussion/669120", "author_name": "", "post_date": "2026-01-20T17:21:36.168050400Z", "votes": 9, "comment_count": 11, "views": 0, "content": "

Just started this competition and trained some baseline models using stratified folds and stratified grouped (Sampling_Date) folds. It seems like CV score reaches 0.8 without any problem regardless of the CV scheme, but my LB score is 0.5x. At this point, I'm thinking about training on the entire training dataset and validate on LB.

", "messages": [ { "id": "3394236", "postDate": "01/20/2026 17:21:36", "content": "

Just started this competition and trained some baseline models using stratified folds and stratified grouped (Sampling_Date) folds. It seems like CV score reaches 0.8 without any problem regardless of the CV scheme, but my LB score is 0.5x. At this point, I'm thinking about training on the entire training dataset and validate on LB.

", "rawMarkdown": "Just started this competition and trained some baseline models using stratified folds and stratified grouped (Sampling_Date) folds. It seems like CV score reaches 0.8 without any problem regardless of the CV scheme, but my LB score is 0.5x. At this point, I'm thinking about training on the entire training dataset and validate on LB.", "votes": null }, { "id": "3394251", "postDate": "01/20/2026 17:52:26", "content": "

I think it's still useful for choosing the best epoch during training. For me it's better than best loss or a set number of epochs. But it's not useful for choosing between different models.

", "rawMarkdown": "I think it's still useful for choosing the best epoch during training. For me it's better than best loss or a set number of epochs. But it's not useful for choosing between different models.", "votes": null }, { "id": "3394291", "postDate": "01/20/2026 19:35:59", "content": "

cv 0.67 lb 0.66

\n

cv 0.69 lb 0.67

\n

cv 0.73 lb 0.69

\n

cv 0.75 lb 0.70

\n

cv 0.76 lb 0.72

\n

cv 0.76 lb 0.73

\n

took me a bit of experiments until the cv worked (it is actually correlating very well up to now). Not sure how it will behave in the private, but I can say at least changing the seed doesn't affect my lb score. I hope this is robust enough lol

", "rawMarkdown": "cv 0.67 lb 0.66\n\ncv 0.69 lb 0.67\n\ncv 0.73 lb 0.69\n\ncv 0.75 lb 0.70\n\ncv 0.76 lb 0.72\n\ncv 0.76 lb 0.73\n\ntook me a bit of experiments until the cv worked (it is actually correlating very well up to now). Not sure how it will behave in the private, but I can say at least changing the seed doesn't affect my lb score. I hope this is robust enough lol", "votes": null }, { "id": "3394376", "postDate": "01/20/2026 23:29:20", "content": "

That's pretty good, my cv behaves nothing like that. I hope I can learn some things when the top solutions are revealed.

", "rawMarkdown": "That's pretty good, my cv behaves nothing like that. I hope I can learn some things when the top solutions are revealed.", "votes": null }, { "id": "3394510", "postDate": "01/21/2026 06:56:55", "content": "

Same here, but some of my folds are actually around 0.5x range. I think some of them are leaking while others are not. Manual fold creation could be useful.

", "rawMarkdown": "Same here, but some of my folds are actually around 0.5x range. I think some of them are leaking while others are not. Manual fold creation could be useful.", "votes": null }, { "id": "3394531", "postDate": "01/21/2026 08:04:54", "content": "

What score are you using? CV mean score or out-of-folds (OOF) score?\nMy OOF score is 0.825, and when it increases, I also see an improvement on the LB.

", "rawMarkdown": "What score are you using? CV mean score or out-of-folds (OOF) score?\nMy OOF score is 0.825, and when it increases, I also see an improvement on the LB.", "votes": null }, { "id": "3394553", "postDate": "01/21/2026 09:28:54", "content": "

Wouldn't actual achievements and project cases be more useful than CV? What's your take on this?

", "rawMarkdown": "Wouldn't actual achievements and project cases be more useful than CV? What's your take on this?", "votes": null }, { "id": "3394643", "postDate": "01/21/2026 12:15:57", "content": "
\n

Manual fold creation could be useful.

\n
\n

You're right. For me, manually creating folds turned out to be the most effective.

", "rawMarkdown": ">Manual fold creation could be useful.\n\nYou're right. For me, manually creating folds turned out to be the most effective.", "votes": null }, { "id": "3395182", "postDate": "01/22/2026 12:18:50", "content": "

My CV and LB trends are generally consistent, but I'm just not sure whether the final private leaderboard score will follow the same trend.

", "rawMarkdown": "My CV and LB trends are generally consistent, but I'm just not sure whether the final private leaderboard score will follow the same trend.", "votes": null }, { "id": "3395284", "postDate": "01/22/2026 16:29:22", "content": "

it's very chaotic between folds in my case, but lb score generally improves when cv also goes up

", "rawMarkdown": "it's very chaotic between folds in my case, but lb score generally improves when cv also goes up", "votes": null }, { "id": "3395707", "postDate": "01/23/2026 14:27:05", "content": "

The data is extremely noisy. Based on my understanding of the paper, I used stratified GroupKFold cross-validation, grouping by date and stratifying by state. I tried random seeds up to 2,100,000; eventually, I found one that minimized the standard deviation in the number of samples across folds :)

", "rawMarkdown": "The data is extremely noisy. Based on my understanding of the paper, I used stratified GroupKFold cross-validation, grouping by date and stratifying by state. I tried random seeds up to 2,100,000; eventually, I found one that minimized the standard deviation in the number of samples across folds :)", "votes": null }, { "id": "3395958", "postDate": "01/24/2026 02:54:29", "content": "

Use seed 963. The LB might drop slightly (by 0.0001), but LB > CV.

", "rawMarkdown": "Use seed 963. The LB might drop slightly (by 0.0001), but LB > CV.", "votes": null } ], "comments": [ { "id": 3394251, "author_name": "tennogh", "author_url": "", "post_date": "01/20/2026 17:52:26", "content": "

I think it's still useful for choosing the best epoch during training. For me it's better than best loss or a set number of epochs. But it's not useful for choosing between different models.

", "votes": null, "replies": [] }, { "id": 3394291, "author_name": "mohammad2012191", "author_url": "", "post_date": "01/20/2026 19:35:59", "content": "

cv 0.67 lb 0.66

\n

cv 0.69 lb 0.67

\n

cv 0.73 lb 0.69

\n

cv 0.75 lb 0.70

\n

cv 0.76 lb 0.72

\n

cv 0.76 lb 0.73

\n

took me a bit of experiments until the cv worked (it is actually correlating very well up to now). Not sure how it will behave in the private, but I can say at least changing the seed doesn't affect my lb score. I hope this is robust enough lol

", "votes": null, "replies": [ { "id": 3394376, "author_name": "tennogh", "author_url": "", "post_date": "01/20/2026 23:29:20", "content": "

That's pretty good, my cv behaves nothing like that. I hope I can learn some things when the top solutions are revealed.

", "votes": null, "replies": [ { "id": 3394510, "author_name": "gunesevitan", "author_url": "", "post_date": "01/21/2026 06:56:55", "content": "

Same here, but some of my folds are actually around 0.5x range. I think some of them are leaking while others are not. Manual fold creation could be useful.

", "votes": null, "replies": [ { "id": 3394643, "author_name": "dmitryuarov", "author_url": "", "post_date": "01/21/2026 12:15:57", "content": "
\n

Manual fold creation could be useful.

\n
\n

You're right. For me, manually creating folds turned out to be the most effective.

", "votes": null, "replies": [] } ] } ] }, { "id": 3395284, "author_name": "gaoxuesong123", "author_url": "", "post_date": "01/22/2026 16:29:22", "content": "

it's very chaotic between folds in my case, but lb score generally improves when cv also goes up

", "votes": null, "replies": [] } ] }, { "id": 3394531, "author_name": "eliork", "author_url": "", "post_date": "01/21/2026 08:04:54", "content": "

What score are you using? CV mean score or out-of-folds (OOF) score?\nMy OOF score is 0.825, and when it increases, I also see an improvement on the LB.

", "votes": null, "replies": [ { "id": 3395182, "author_name": "sweetyheehee", "author_url": "", "post_date": "01/22/2026 12:18:50", "content": "

My CV and LB trends are generally consistent, but I'm just not sure whether the final private leaderboard score will follow the same trend.

", "votes": null, "replies": [] } ] }, { "id": 3394553, "author_name": "frankchenhunterworld", "author_url": "", "post_date": "01/21/2026 09:28:54", "content": "

Wouldn't actual achievements and project cases be more useful than CV? What's your take on this?

", "votes": null, "replies": [] }, { "id": 3395707, "author_name": "aliozanmemetoglu", "author_url": "", "post_date": "01/23/2026 14:27:05", "content": "

The data is extremely noisy. Based on my understanding of the paper, I used stratified GroupKFold cross-validation, grouping by date and stratifying by state. I tried random seeds up to 2,100,000; eventually, I found one that minimized the standard deviation in the number of samples across folds :)

", "votes": null, "replies": [] }, { "id": 3395958, "author_name": "giovannyrodrguez", "author_url": "", "post_date": "01/24/2026 02:54:29", "content": "

Use seed 963. The LB might drop slightly (by 0.0001), but LB > CV.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3394236": "Just started this competition and trained some baseline models using stratified folds and stratified grouped (Sampling_Date) folds. It seems like CV score reaches 0.8 without any problem regardless of the CV scheme, but my LB score is 0.5x. At this point, I'm thinking about training on the entire training dataset and validate on LB.", "3394251": "I think it's still useful for choosing the best epoch during training. For me it's better than best loss or a set number of epochs. But it's not useful for choosing between different models.", "3394291": "cv 0.67 lb 0.66\n\ncv 0.69 lb 0.67\n\ncv 0.73 lb 0.69\n\ncv 0.75 lb 0.70\n\ncv 0.76 lb 0.72\n\ncv 0.76 lb 0.73\n\ntook me a bit of experiments until the cv worked (it is actually correlating very well up to now). Not sure how it will behave in the private, but I can say at least changing the seed doesn't affect my lb score. I hope this is robust enough lol", "3394376": "That's pretty good, my cv behaves nothing like that. I hope I can learn some things when the top solutions are revealed.", "3394510": "Same here, but some of my folds are actually around 0.5x range. I think some of them are leaking while others are not. Manual fold creation could be useful.", "3394531": "What score are you using? CV mean score or out-of-folds (OOF) score?\nMy OOF score is 0.825, and when it increases, I also see an improvement on the LB.", "3394553": "Wouldn't actual achievements and project cases be more useful than CV? What's your take on this?", "3394643": ">Manual fold creation could be useful.\n\nYou're right. For me, manually creating folds turned out to be the most effective.", "3395182": "My CV and LB trends are generally consistent, but I'm just not sure whether the final private leaderboard score will follow the same trend.", "3395284": "it's very chaotic between folds in my case, but lb score generally improves when cv also goes up", "3395707": "The data is extremely noisy. Based on my understanding of the paper, I used stratified GroupKFold cross-validation, grouping by date and stratifying by state. I tried random seeds up to 2,100,000; eventually, I found one that minimized the standard deviation in the number of samples across folds :)", "3395958": "Use seed 963. The LB might drop slightly (by 0.0001), but LB > CV." }, "source": "meta" }