First; A big thanks to Kaggle + the organizers for putting together a fun competition and dataset.
\nI was aiming for top 10%, didn’t get there — but the biggest takeaway was learning how important experiment discipline is: get a strong baseline + evaluation plan first, then explore riskier ideas only if you’ve got runway.
\nI spent time on experimental routes (TTT and other ideas I never even ended up using). It wasn’t that they’re bad ideas — it’s that I didn’t earn them yet.
\nI think a better rule would’ve been: get one “boring but strong” candidate to a state you’d actually ship, then experiment if you still have time. Without that baseline, experimental work turns into procrastination disguised as ambition.
\nDuring the competition I treated “a good CV run” as the end goal. But in the end, my solution was basically one big ensemble of multiple cv runs. The problem: I only had a few days left to evaluate the ensemble properly.
\nNext time I’m going to treat the ensemble setup as the real “final candidate” from much earlier, not just the individual models. That means the ensemble details (which models go in, weights, any post-processing, blending strategy, etc.) get tuned and validated just as carefully as the model training itself, instead of being rushed together at the very end.
\nI picked two submissions that were among my better public LB results… but they landed around ~top 12 out of my own submissions on private. Nothing scandalous, just the usual: public LB noise + overfitting your selection strategy.
\nStarted about halfway into the competition, read a few strong Kaggle notebooks, copied the key insights, then went straight into “train mode.” It worked… but it kept me shallow. I didn’t build enough first-principles understanding of what was actually in the dataset, what the weird cases were, where the leakage risks might be, etc.
\nAnd I’m pretty sure that happened because I started late and felt like I had to “catch up” by training models immediately.
\nI treated stuff like EMA/SWA, warmup schedules, etc. as “safe obvious gains.” Sometimes they weren’t. And when performance dropped, my first instinct was “this must be a bug”, so I wasted time debugging something that was possibly just: “this trick doesn’t help in this setting.”
\nNext time I want to treat even “standard tricks” as just… hyperparameters. If it doesn’t help, drop it and move on unless I have real evidence it’s broken.
\nI didn’t hit my target, but I’m walking away with something more valuable than a rank: I now understand where time actually goes, and what I’d do differently next time to spend it on higher-leverage work.
", "messages": [ { "id": "3398496", "postDate": "01/29/2026 08:22:19", "content": "First; A big thanks to Kaggle + the organizers for putting together a fun competition and dataset.
\nI was aiming for top 10%, didn’t get there — but the biggest takeaway was learning how important experiment discipline is: get a strong baseline + evaluation plan first, then explore riskier ideas only if you’ve got runway.
\nI spent time on experimental routes (TTT and other ideas I never even ended up using). It wasn’t that they’re bad ideas — it’s that I didn’t earn them yet.
\nI think a better rule would’ve been: get one “boring but strong” candidate to a state you’d actually ship, then experiment if you still have time. Without that baseline, experimental work turns into procrastination disguised as ambition.
\nDuring the competition I treated “a good CV run” as the end goal. But in the end, my solution was basically one big ensemble of multiple cv runs. The problem: I only had a few days left to evaluate the ensemble properly.
\nNext time I’m going to treat the ensemble setup as the real “final candidate” from much earlier, not just the individual models. That means the ensemble details (which models go in, weights, any post-processing, blending strategy, etc.) get tuned and validated just as carefully as the model training itself, instead of being rushed together at the very end.
\nI picked two submissions that were among my better public LB results… but they landed around ~top 12 out of my own submissions on private. Nothing scandalous, just the usual: public LB noise + overfitting your selection strategy.
\nStarted about halfway into the competition, read a few strong Kaggle notebooks, copied the key insights, then went straight into “train mode.” It worked… but it kept me shallow. I didn’t build enough first-principles understanding of what was actually in the dataset, what the weird cases were, where the leakage risks might be, etc.
\nAnd I’m pretty sure that happened because I started late and felt like I had to “catch up” by training models immediately.
\nI treated stuff like EMA/SWA, warmup schedules, etc. as “safe obvious gains.” Sometimes they weren’t. And when performance dropped, my first instinct was “this must be a bug”, so I wasted time debugging something that was possibly just: “this trick doesn’t help in this setting.”
\nNext time I want to treat even “standard tricks” as just… hyperparameters. If it doesn’t help, drop it and move on unless I have real evidence it’s broken.
\nI didn’t hit my target, but I’m walking away with something more valuable than a rank: I now understand where time actually goes, and what I’d do differently next time to spend it on higher-leverage work.
", "rawMarkdown": "First; A big thanks to Kaggle + the organizers for putting together a fun competition and dataset.\n\nI was aiming for top 10%, didn’t get there — but the biggest takeaway was learning how important experiment discipline is: get a strong baseline + evaluation plan first, then explore riskier ideas only if you’ve got runway. \n\n---\n\n### The biggest mistake: going “research mode” before I had a solid final candidate\n\nI spent time on experimental routes (TTT and other ideas I never even ended up using). It wasn’t that they’re bad ideas — it’s that I didn’t earn them yet.\n\nI think a better rule would’ve been: **get one “boring but strong” candidate to a state you’d actually ship**, then experiment if you still have time. Without that baseline, experimental work turns into procrastination disguised as ambition.\n\n---\n\n### I underestimated how late-stage the *ensemble* becomes\n\nDuring the competition I treated “a good CV run” as the end goal. But in the end, my solution was basically one big ensemble of multiple cv runs. The problem: I only had a few days left to evaluate the ensemble properly.\n\nNext time I’m going to treat the ensemble setup as the real “final candidate” from much earlier, not just the individual models. That means the ensemble details (which models go in, weights, any post-processing, blending strategy, etc.) get tuned and validated just as carefully as the model training itself, instead of being rushed together at the very end.\n\n---\n\n### Public leaderboard bait got me (classic)\n\nI picked two submissions that were among my better public LB results… but they landed around ~top 12 out of my own submissions on private. Nothing scandalous, just the usual: public LB noise + overfitting your selection strategy.\n\n---\n\n### I rushed into training and didn’t understand the data deeply enough\n\nStarted about halfway into the competition, read a few strong Kaggle notebooks, copied the key insights, then went straight into “train mode.” It worked… but it kept me shallow. I didn’t build enough first-principles understanding of what was actually in the dataset, what the weird cases were, where the leakage risks might be, etc.\n\nAnd I’m pretty sure that happened because I started late and felt like I had to “catch up” by training models immediately.\n\n---\n\n### I also had too many assumptions about what “should work”\n\nI treated stuff like EMA/SWA, warmup schedules, etc. as “safe obvious gains.” Sometimes they weren’t. And when performance dropped, my first instinct was “this must be a bug”, so I wasted time debugging something that was possibly just: “this trick doesn’t help in this setting.”\n\nNext time I want to treat even “standard tricks” as just… hyperparameters. If it doesn’t help, drop it and move on unless I have real evidence it’s broken.\n\n---\n\n### TL;DR\n\nI didn’t hit my target, but I’m walking away with something more valuable than a rank: I now understand where time actually goes, and what I’d do differently next time to spend it on higher-leverage work.", "votes": null }, { "id": "3398729", "postDate": "01/29/2026 15:58:56", "content": "Don't treat this competition too seriously; it is too poorly designed to be sure of most of your conclusions.\nI personally think you did well!
\na. For example, a public score often represents a good estimate of a private score. A lot of participants say that \"you should ignore the public LB\", but it isn't true in most cases. Sometimes it is even the only way to validate your models (if the test distribution is completely different from the train distribution and public/private split is purely random)!
\nb. Most of the tricks didn't work because the dataset is small and noisy. And, again, there are unseen data distribution in the test set. I believe there is no genuine solid CV setup for this competition, so it is difficult to evaluate your models/tricks/approaches properly. What you did may possibly give very good improvement in the different (and better designed) competition.
", "rawMarkdown": "Don't treat this competition too seriously; it is too poorly designed to be sure of most of your conclusions.\nI personally think you did well!\n\na. For example, a public score often represents a good estimate of a private score. A lot of participants say that \"you should ignore the public LB\", but it isn't true in most cases. Sometimes it is even the only way to validate your models (if the test distribution is completely different from the train distribution and public/private split is purely random)!\n\nb. Most of the tricks didn't work because the dataset is small and noisy. And, again, there are unseen data distribution in the test set. I believe there is no genuine solid CV setup for this competition, so it is difficult to evaluate your models/tricks/approaches properly. What you did may possibly give very good improvement in the different (and better designed) competition.", "votes": null } ], "comments": [ { "id": 3398729, "author_name": "churkinnikita", "author_url": "", "post_date": "01/29/2026 15:58:56", "content": "Don't treat this competition too seriously; it is too poorly designed to be sure of most of your conclusions.\nI personally think you did well!
\na. For example, a public score often represents a good estimate of a private score. A lot of participants say that \"you should ignore the public LB\", but it isn't true in most cases. Sometimes it is even the only way to validate your models (if the test distribution is completely different from the train distribution and public/private split is purely random)!
\nb. Most of the tricks didn't work because the dataset is small and noisy. And, again, there are unseen data distribution in the test set. I believe there is no genuine solid CV setup for this competition, so it is difficult to evaluate your models/tricks/approaches properly. What you did may possibly give very good improvement in the different (and better designed) competition.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3398496": "First; A big thanks to Kaggle + the organizers for putting together a fun competition and dataset.\n\nI was aiming for top 10%, didn’t get there — but the biggest takeaway was learning how important experiment discipline is: get a strong baseline + evaluation plan first, then explore riskier ideas only if you’ve got runway. \n\n---\n\n### The biggest mistake: going “research mode” before I had a solid final candidate\n\nI spent time on experimental routes (TTT and other ideas I never even ended up using). It wasn’t that they’re bad ideas — it’s that I didn’t earn them yet.\n\nI think a better rule would’ve been: **get one “boring but strong” candidate to a state you’d actually ship**, then experiment if you still have time. Without that baseline, experimental work turns into procrastination disguised as ambition.\n\n---\n\n### I underestimated how late-stage the *ensemble* becomes\n\nDuring the competition I treated “a good CV run” as the end goal. But in the end, my solution was basically one big ensemble of multiple cv runs. The problem: I only had a few days left to evaluate the ensemble properly.\n\nNext time I’m going to treat the ensemble setup as the real “final candidate” from much earlier, not just the individual models. That means the ensemble details (which models go in, weights, any post-processing, blending strategy, etc.) get tuned and validated just as carefully as the model training itself, instead of being rushed together at the very end.\n\n---\n\n### Public leaderboard bait got me (classic)\n\nI picked two submissions that were among my better public LB results… but they landed around ~top 12 out of my own submissions on private. Nothing scandalous, just the usual: public LB noise + overfitting your selection strategy.\n\n---\n\n### I rushed into training and didn’t understand the data deeply enough\n\nStarted about halfway into the competition, read a few strong Kaggle notebooks, copied the key insights, then went straight into “train mode.” It worked… but it kept me shallow. I didn’t build enough first-principles understanding of what was actually in the dataset, what the weird cases were, where the leakage risks might be, etc.\n\nAnd I’m pretty sure that happened because I started late and felt like I had to “catch up” by training models immediately.\n\n---\n\n### I also had too many assumptions about what “should work”\n\nI treated stuff like EMA/SWA, warmup schedules, etc. as “safe obvious gains.” Sometimes they weren’t. And when performance dropped, my first instinct was “this must be a bug”, so I wasted time debugging something that was possibly just: “this trick doesn’t help in this setting.”\n\nNext time I want to treat even “standard tricks” as just… hyperparameters. If it doesn’t help, drop it and move on unless I have real evidence it’s broken.\n\n---\n\n### TL;DR\n\nI didn’t hit my target, but I’m walking away with something more valuable than a rank: I now understand where time actually goes, and what I’d do differently next time to spend it on higher-leverage work.", "3398729": "Don't treat this competition too seriously; it is too poorly designed to be sure of most of your conclusions.\nI personally think you did well!\n\na. For example, a public score often represents a good estimate of a private score. A lot of participants say that \"you should ignore the public LB\", but it isn't true in most cases. Sometimes it is even the only way to validate your models (if the test distribution is completely different from the train distribution and public/private split is purely random)!\n\nb. Most of the tricks didn't work because the dataset is small and noisy. And, again, there are unseen data distribution in the test set. I believe there is no genuine solid CV setup for this competition, so it is difficult to evaluate your models/tricks/approaches properly. What you did may possibly give very good improvement in the different (and better designed) competition." }, "source": "meta" }