{
  "id": 651498,
  "title": "How do you use Cross Validation with NN?",
  "url": "/competitions/csiro-biomass/discussion/651498",
  "author_name": "YannMnc",
  "post_date": "2025-12-03T21:46:04.355000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Sorry if that question has been answered a million times already but I am new to Kaggle and new to Neural Networks. I have used CV with sklearn \"regular\" regression models many times but somehow with NN it seems somewhat different.</p>\n<p>In the normal world I used CV to choose my model parameters in order to avoid overfitting (high stdev of scores across different folds) and find the best fit (lower average score). With NN models and especially with Kaggle LB I am a bit lost:</p>\n<ol>\n<li><p>I am not sure what parameters to test. Is it different LR for different part of the model? different configurations like patience for the plateau scheduler? different architecture? \nThere is so many things to tweak.</p></li>\n<li><p>How does that help to reconcile with the LB score? \nI guess the more stable the model is, the higher chance to match with LB, but given how sensitive NN are to the training data I never know what will come out of it if I choose one fold or another for my final fitting.</p></li>\n<li><p>NN training is inherently non-deterministic has it is so dependant on the starting weights. I seed PyTorch, but do you use CV to find the best seed? \nReading between the lines of some posts I would think so but I find it quite scary. Especially if you are not even sure it will reconcile with LB or if changing a parameter will throw you off, not because the parameter is worse but because the seed is not good for that particular case…</p></li>\n</ol>\n<p>Anyway, as a first with NN this competition is quite fun, and I think I am building a 7 heads monsters.</p>",
  "messages": [
    {
      "id": 3361845,
      "postDate": "2025-12-03T21:46:04.357Z",
      "content": "<p>Sorry if that question has been answered a million times already but I am new to Kaggle and new to Neural Networks. I have used CV with sklearn \"regular\" regression models many times but somehow with NN it seems somewhat different.</p>\n<p>In the normal world I used CV to choose my model parameters in order to avoid overfitting (high stdev of scores across different folds) and find the best fit (lower average score). With NN models and especially with Kaggle LB I am a bit lost:</p>\n<ol>\n<li><p>I am not sure what parameters to test. Is it different LR for different part of the model? different configurations like patience for the plateau scheduler? different architecture? \nThere is so many things to tweak.</p></li>\n<li><p>How does that help to reconcile with the LB score? \nI guess the more stable the model is, the higher chance to match with LB, but given how sensitive NN are to the training data I never know what will come out of it if I choose one fold or another for my final fitting.</p></li>\n<li><p>NN training is inherently non-deterministic has it is so dependant on the starting weights. I seed PyTorch, but do you use CV to find the best seed? \nReading between the lines of some posts I would think so but I find it quite scary. Especially if you are not even sure it will reconcile with LB or if changing a parameter will throw you off, not because the parameter is worse but because the seed is not good for that particular case…</p></li>\n</ol>\n<p>Anyway, as a first with NN this competition is quite fun, and I think I am building a 7 heads monsters.</p>",
      "rawMarkdown": "Sorry if that question has been answered a million times already but I am new to Kaggle and new to Neural Networks. I have used CV with sklearn \"regular\" regression models many times but somehow with NN it seems somewhat different.\n\nIn the normal world I used CV to choose my model parameters in order to avoid overfitting (high stdev of scores across different folds) and find the best fit (lower average score). With NN models and especially with Kaggle LB I am a bit lost:\n1. I am not sure what parameters to test. Is it different LR for different part of the model? different configurations like patience for the plateau scheduler? different architecture? \nThere is so many things to tweak.\n\n2. How does that help to reconcile with the LB score? \nI guess the more stable the model is, the higher chance to match with LB, but given how sensitive NN are to the training data I never know what will come out of it if I choose one fold or another for my final fitting.\n\n3. NN training is inherently non-deterministic has it is so dependant on the starting weights. I seed PyTorch, but do you use CV to find the best seed? \nReading between the lines of some posts I would think so but I find it quite scary. Especially if you are not even sure it will reconcile with LB or if changing a parameter will throw you off, not because the parameter is worse but because the seed is not good for that particular case...\n\nAnyway, as a first with NN this competition is quite fun, and I think I am building a 7 heads monsters.",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "3361845": "Sorry if that question has been answered a million times already but I am new to Kaggle and new to Neural Networks. I have used CV with sklearn \"regular\" regression models many times but somehow with NN it seems somewhat different.\n\nIn the normal world I used CV to choose my model parameters in order to avoid overfitting (high stdev of scores across different folds) and find the best fit (lower average score). With NN models and especially with Kaggle LB I am a bit lost:\n1. I am not sure what parameters to test. Is it different LR for different part of the model? different configurations like patience for the plateau scheduler? different architecture? \nThere is so many things to tweak.\n\n2. How does that help to reconcile with the LB score? \nI guess the more stable the model is, the higher chance to match with LB, but given how sensitive NN are to the training data I never know what will come out of it if I choose one fold or another for my final fitting.\n\n3. NN training is inherently non-deterministic has it is so dependant on the starting weights. I seed PyTorch, but do you use CV to find the best seed? \nReading between the lines of some posts I would think so but I find it quite scary. Especially if you are not even sure it will reconcile with LB or if changing a parameter will throw you off, not because the parameter is worse but because the seed is not good for that particular case...\n\nAnyway, as a first with NN this competition is quite fun, and I think I am building a 7 heads monsters."
  }
}