Hi everyone,
\nI'm a complete beginner on Kaggle, and this CMI competition is my very first challenge.
This might not be the best place to post this, but after reading this discussion about “Copy to Win?”, it made me reflect a bit—and I’d love to hear your thoughts.
\nAt first, I tried to build a model using LightGBM, simply because GBDT is the only method I’m somewhat familiar with. But I couldn’t improve my score, and in the end, I ended up referencing and copying a public notebook just to get something that works and submit it.
\nThat made me wonder: Will I really grow if I continue like this?
\nHow do people actually acquire the skills needed to succeed on Kaggle?
My goal for joining Kaggle was to improve my data analysis skills, and I saw this competition as a great opportunity to learn Transformers. So, I referred to this amazing notebook using BERT:
\nHowever, I struggled to follow it. Since parts of the model are preloaded and defined elsewhere, I couldn’t trace the full pipeline from scratch—and as a result, I couldn’t really understand what was happening.
\nSo here's my question to the community:
\nHow did you grow your Kaggle skills?
\nEven a simple piece of advice would mean a lot to me.
\nThanks in advance!
Hi everyone,
\nI'm a complete beginner on Kaggle, and this CMI competition is my very first challenge.
This might not be the best place to post this, but after reading this discussion about “Copy to Win?”, it made me reflect a bit—and I’d love to hear your thoughts.
\nAt first, I tried to build a model using LightGBM, simply because GBDT is the only method I’m somewhat familiar with. But I couldn’t improve my score, and in the end, I ended up referencing and copying a public notebook just to get something that works and submit it.
\nThat made me wonder: Will I really grow if I continue like this?
\nHow do people actually acquire the skills needed to succeed on Kaggle?
My goal for joining Kaggle was to improve my data analysis skills, and I saw this competition as a great opportunity to learn Transformers. So, I referred to this amazing notebook using BERT:
\nHowever, I struggled to follow it. Since parts of the model are preloaded and defined elsewhere, I couldn’t trace the full pipeline from scratch—and as a result, I couldn’t really understand what was happening.
\nSo here's my question to the community:
\nHow did you grow your Kaggle skills?
\nEven a simple piece of advice would mean a lot to me.
\nThanks in advance!
When I was interviewing several month ago, interviewer wonders why I'm really sensitive to evaluation. I said I'm a kaggler. So you know, analyzing whether your model can handle every case to survive in shake-up is what we need to put a lot of effort in kaggle competition.
\nYou can copy some public notebooks to try out.
I suppose you will still grow somewhat. You're at least learning acronyms you didn't know before! 😁
\nThis is my first competition here on Kaggle as well, and it has become very consuming for me. My first submission was a simple XGBoost because that was all I had ever used before. After week 1 or 2 when phalanx posted about quickly achieving 0.82 by iterating with the LLM agent, I fell down the rabbit hole and started a broad search over about 70+ different experiments at this point.
\nOriginally I was running my notebook on Kaggle, and I soon decided I wanted to train multiple models in parallel. I stitched together some Dollar Store version of an MLOps pipeline using vast.ai and some custom tooling on top of a tool called skypilot. It was pretty bad.
\nAfter a bunch of research this pipeline has evolved and I've been using Ray distributed training to parallelize jobs (and folds), stream parquet datasets, store model checkpoints in s3, etc etc.
\nI go back and forth with the AI (Claude, o3, and Grok) asking questions, making my own ideas for Feature Engineering, asking for citations to papers, references to infrastructure tooling, etc. That's how I found out about Full Stack Deep Learning, LSTMs, transfer learning, loss functions, optimization algorithms, hyperparameter tuning, so many different papers, books like Understanding Deep Learning, Dive Into Deep Learning, Deep Learning and Mathematics for Machine Learning, the list goes on
\nI was able to iterate on my model to get to 0.80 LB and then when the public notebooks started to drop I realized that there is a bit of a meta-strategy to the competition where maybe it makes more sense to ease into the longer competitions and not spend so much time and GPU hours up front doing the experimenting yourself. I don't think I realized that so much would get published during an active competition. Since it's not only about competition medals and notebook medals are so valuable as well, etc. I haven't studied the published notebooks yet, but I guess in theory there should be some elements in there I can tack on or ensemble with my own models which should help too.
\nAnyway, it's been really impressive to see how the top names have managed to keep improving their scores at the top of the LB. Someday it would be great to get there. I think it's easy to underestimate how much talent and strategy is involved in these competitions, so as first-timers I suppose we should give ourselves some grace.
\nIf you find some good resources or want to bounce some ideas off of someone, I'm around! Good luck on the grind.
", "rawMarkdown": "I suppose you will still grow somewhat. You're at least learning acronyms you didn't know before! 😁\n\nThis is my first competition here on Kaggle as well, and it has become very consuming for me. My first submission was a simple XGBoost because that was all I had ever used before. After week 1 or 2 when phalanx posted about quickly achieving 0.82 by iterating with the LLM agent, I fell down the rabbit hole and started a broad search over about 70+ different experiments at this point.\n\nOriginally I was running my notebook on Kaggle, and I soon decided I wanted to train multiple models in parallel. I stitched together some Dollar Store version of an MLOps pipeline using vast.ai and some custom tooling on top of a tool called skypilot. It was pretty bad.\n\nAfter a bunch of research this pipeline has evolved and I've been using Ray distributed training to parallelize jobs (and folds), stream parquet datasets, store model checkpoints in s3, etc etc.\n\nI go back and forth with the AI (Claude, o3, and Grok) asking questions, making my own ideas for Feature Engineering, asking for citations to papers, references to infrastructure tooling, etc. That's how I found out about [Full Stack Deep Learning](https://fullstackdeeplearning.com/course/2022/lecture-1-course-vision-and-when-to-use-ml/), LSTMs, transfer learning, loss functions, optimization algorithms, hyperparameter tuning, so many different papers, books like [Understanding Deep Learning](https://udlbook.github.io/udlbook/), [Dive Into Deep Learning](https://d2l.ai/), [Deep Learning](https://www.deeplearningbook.org/) and [Mathematics for Machine Learning](https://mml-book.github.io/), the list goes on\n\nI was able to iterate on my model to get to 0.80 LB and then when the public notebooks started to drop I realized that there is a bit of a meta-strategy to the competition where maybe it makes more sense to ease into the longer competitions and not spend so much time and GPU hours up front doing the experimenting yourself. I don't think I realized that so much would get published during an active competition. Since it's not only about competition medals and notebook medals are so valuable as well, etc. I haven't studied the published notebooks yet, but I guess in theory there should be some elements in there I can tack on or ensemble with my own models which should help too.\n\nAnyway, it's been really impressive to see how the top names have managed to keep improving their scores at the top of the LB. Someday it would be great to get there. I think it's easy to underestimate how much talent and strategy is involved in these competitions, so as first-timers I suppose we should give ourselves some grace.\n\nIf you find some good resources or want to bounce some ideas off of someone, I'm around! Good luck on the grind.", "votes": null }, { "id": "3249940", "postDate": "07/17/2025 11:55:37", "content": "I think how to do feature engineering and improve model selection is the key to score higher in this competition. Of course, you need to pay attention to overfitting. You can try to do some feature crossover, which may help you score higher.
", "rawMarkdown": "I think how to do feature engineering and improve model selection is the key to score higher in this competition. Of course, you need to pay attention to overfitting. You can try to do some feature crossover, which may help you score higher.", "votes": null } ], "comments": [ { "id": 3249723, "author_name": "tom99763", "author_url": "", "post_date": "07/17/2025 02:50:06", "content": "When I was interviewing several month ago, interviewer wonders why I'm really sensitive to evaluation. I said I'm a kaggler. So you know, analyzing whether your model can handle every case to survive in shake-up is what we need to put a lot of effort in kaggle competition.
\nYou can copy some public notebooks to try out.
I suppose you will still grow somewhat. You're at least learning acronyms you didn't know before! 😁
\nThis is my first competition here on Kaggle as well, and it has become very consuming for me. My first submission was a simple XGBoost because that was all I had ever used before. After week 1 or 2 when phalanx posted about quickly achieving 0.82 by iterating with the LLM agent, I fell down the rabbit hole and started a broad search over about 70+ different experiments at this point.
\nOriginally I was running my notebook on Kaggle, and I soon decided I wanted to train multiple models in parallel. I stitched together some Dollar Store version of an MLOps pipeline using vast.ai and some custom tooling on top of a tool called skypilot. It was pretty bad.
\nAfter a bunch of research this pipeline has evolved and I've been using Ray distributed training to parallelize jobs (and folds), stream parquet datasets, store model checkpoints in s3, etc etc.
\nI go back and forth with the AI (Claude, o3, and Grok) asking questions, making my own ideas for Feature Engineering, asking for citations to papers, references to infrastructure tooling, etc. That's how I found out about Full Stack Deep Learning, LSTMs, transfer learning, loss functions, optimization algorithms, hyperparameter tuning, so many different papers, books like Understanding Deep Learning, Dive Into Deep Learning, Deep Learning and Mathematics for Machine Learning, the list goes on
\nI was able to iterate on my model to get to 0.80 LB and then when the public notebooks started to drop I realized that there is a bit of a meta-strategy to the competition where maybe it makes more sense to ease into the longer competitions and not spend so much time and GPU hours up front doing the experimenting yourself. I don't think I realized that so much would get published during an active competition. Since it's not only about competition medals and notebook medals are so valuable as well, etc. I haven't studied the published notebooks yet, but I guess in theory there should be some elements in there I can tack on or ensemble with my own models which should help too.
\nAnyway, it's been really impressive to see how the top names have managed to keep improving their scores at the top of the LB. Someday it would be great to get there. I think it's easy to underestimate how much talent and strategy is involved in these competitions, so as first-timers I suppose we should give ourselves some grace.
\nIf you find some good resources or want to bounce some ideas off of someone, I'm around! Good luck on the grind.
", "votes": null, "replies": [ { "id": 3249940, "author_name": "majiaqi111", "author_url": "", "post_date": "07/17/2025 11:55:37", "content": "I think how to do feature engineering and improve model selection is the key to score higher in this competition. Of course, you need to pay attention to overfitting. You can try to do some feature crossover, which may help you score higher.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3249682": "Hi everyone,\nI'm a complete beginner on Kaggle, and this CMI competition is my very first challenge.\n\nThis might not be the best place to post this, but after reading this discussion about [“Copy to Win?”](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/discussion/589894), it made me reflect a bit—and I’d love to hear your thoughts.\n\nAt first, I tried to build a model using LightGBM, simply because GBDT is the only method I’m somewhat familiar with. But I couldn’t improve my score, and in the end, I ended up referencing and copying a public notebook just to get something that works and submit it.\n\nThat made me wonder: Will I really grow if I continue like this?\nHow do people actually acquire the skills needed to succeed on Kaggle?\n\nMy goal for joining Kaggle was to improve my data analysis skills, and I saw this competition as a great opportunity to learn Transformers. So, I referred to this amazing [notebook using BERT](https://www.kaggle.com/code/wasupandceacar/lb-0-82-5fold-single-bert-model):\n\nHowever, I struggled to follow it. Since parts of the model are preloaded and defined elsewhere, I couldn’t trace the full pipeline from scratch—and as a result, I couldn’t really understand what was happening.\n\nSo here's my question to the community:\nHow did you grow your Kaggle skills?\nEven a simple piece of advice would mean a lot to me.\nThanks in advance!", "3249723": "When I was interviewing several month ago, interviewer wonders why I'm really sensitive to evaluation. I said I'm a kaggler. So you know, analyzing whether your model can handle every case to survive in shake-up is what we need to put a lot of effort in kaggle competition.\nYou can copy some public notebooks to try out.", "3249731": "I suppose you will still grow somewhat. You're at least learning acronyms you didn't know before! 😁\n\nThis is my first competition here on Kaggle as well, and it has become very consuming for me. My first submission was a simple XGBoost because that was all I had ever used before. After week 1 or 2 when phalanx posted about quickly achieving 0.82 by iterating with the LLM agent, I fell down the rabbit hole and started a broad search over about 70+ different experiments at this point.\n\nOriginally I was running my notebook on Kaggle, and I soon decided I wanted to train multiple models in parallel. I stitched together some Dollar Store version of an MLOps pipeline using vast.ai and some custom tooling on top of a tool called skypilot. It was pretty bad.\n\nAfter a bunch of research this pipeline has evolved and I've been using Ray distributed training to parallelize jobs (and folds), stream parquet datasets, store model checkpoints in s3, etc etc.\n\nI go back and forth with the AI (Claude, o3, and Grok) asking questions, making my own ideas for Feature Engineering, asking for citations to papers, references to infrastructure tooling, etc. That's how I found out about [Full Stack Deep Learning](https://fullstackdeeplearning.com/course/2022/lecture-1-course-vision-and-when-to-use-ml/), LSTMs, transfer learning, loss functions, optimization algorithms, hyperparameter tuning, so many different papers, books like [Understanding Deep Learning](https://udlbook.github.io/udlbook/), [Dive Into Deep Learning](https://d2l.ai/), [Deep Learning](https://www.deeplearningbook.org/) and [Mathematics for Machine Learning](https://mml-book.github.io/), the list goes on\n\nI was able to iterate on my model to get to 0.80 LB and then when the public notebooks started to drop I realized that there is a bit of a meta-strategy to the competition where maybe it makes more sense to ease into the longer competitions and not spend so much time and GPU hours up front doing the experimenting yourself. I don't think I realized that so much would get published during an active competition. Since it's not only about competition medals and notebook medals are so valuable as well, etc. I haven't studied the published notebooks yet, but I guess in theory there should be some elements in there I can tack on or ensemble with my own models which should help too.\n\nAnyway, it's been really impressive to see how the top names have managed to keep improving their scores at the top of the LB. Someday it would be great to get there. I think it's easy to underestimate how much talent and strategy is involved in these competitions, so as first-timers I suppose we should give ourselves some grace.\n\nIf you find some good resources or want to bounce some ideas off of someone, I'm around! Good luck on the grind.", "3249940": "I think how to do feature engineering and improve model selection is the key to score higher in this competition. Of course, you need to pay attention to overfitting. You can try to do some feature crossover, which may help you score higher." }, "source": "meta" }