The following presents my solution, which got 43rd place on the private leaderboard and earned me a silver medal.
\nYou can see the code for my solution here.
\nI constructed my solution using protein data and visit_month only. I did not use the peptide data at all. I felt that all of the necessary information in the peptide data was summarized in the protein data as every peptide has a 1:1 relationship with a protein. Moreover, because a protein has many peptides, using the peptide data could lead to a situation where a particular protein was detected for 2 different observations using different peptides; thus, using peptide data could have led to a situation where the same information (the detection of a protein) could be obfuscated by seemingly different features (different peptides).
This competition was very difficult because of the very large number of targets which competitors are forced to predict. In particular for all 4 updrs scores, we were asked to predict that score 0, 6, 12, and 24 months ahead. Moreover, we must make these predictions from every possible visit_month. In other words, if, for instance, visit_month == 36, then we must predict all 4 updrs for the 36th month, the 42nd month, the 48th month and the 60th month. To make things even worse, no conclusive list of possible visit_months was ever provided throughout the competition. I strongly believe that this extraordinary complexity made this competition extremely challenging and interesting.
To tackle the complexity, for every combination of updrs (except for updrs_4), visit_month, and months ahead, I trained a separate CatBoost model. For each of these models, I optimized the hyperparameters using Optuna.
For updrs_4 I used a linear model provided by https://www.kaggle.com/code/ambrosm/pdpp-almost-only-trends by Ambrosm which itself was a variation of code originally posted by https://www.kaggle.com/code/vitalykudelya/only-trends by Vitaly Kudelya. I used a clip month of 54 for updrs_4.
Moreover, because of the large number of targets, it was possible for some observations in the test set and for some targets the prediction ought to be outside the range of what was seen in training. Because tree-based models extrapolate very poorly, I felt that if the prediction updrs score provided by CatBoost was at the boundary of what was seen in training, then that prediction might be very poor. As such, in cases where CatBoost predicted a score at the boundary of the training data, that score was disregarded and the linear model described in the previous bullet point was used instead.
A combination of CatBoost and the linear model in instances where CatBoost predicts the boundary appeared to work best in GroupKFold cross-validation. Due to the large number of targets CatBoost did a worse job by itself without the linear component.
Setting the objective to MAE for CatBoost appeared to provide better results than MSE.
updrs_4 and for CatBoost predictions outside the bounds of what was seen in trainingvisit_month alone. I missed the trick all top notebooks used, but absent the trick, I found that including the protein data gave superior results.I really enjoyed this competition and I hope the organizers will find our solutions useful.
\nCongrats to all winners and good luck to all in future competitions!!!
The following presents my solution, which got 43rd place on the private leaderboard and earned me a silver medal.
\nYou can see the code for my solution here.
\nI constructed my solution using protein data and visit_month only. I did not use the peptide data at all. I felt that all of the necessary information in the peptide data was summarized in the protein data as every peptide has a 1:1 relationship with a protein. Moreover, because a protein has many peptides, using the peptide data could lead to a situation where a particular protein was detected for 2 different observations using different peptides; thus, using peptide data could have led to a situation where the same information (the detection of a protein) could be obfuscated by seemingly different features (different peptides).
This competition was very difficult because of the very large number of targets which competitors are forced to predict. In particular for all 4 updrs scores, we were asked to predict that score 0, 6, 12, and 24 months ahead. Moreover, we must make these predictions from every possible visit_month. In other words, if, for instance, visit_month == 36, then we must predict all 4 updrs for the 36th month, the 42nd month, the 48th month and the 60th month. To make things even worse, no conclusive list of possible visit_months was ever provided throughout the competition. I strongly believe that this extraordinary complexity made this competition extremely challenging and interesting.
To tackle the complexity, for every combination of updrs (except for updrs_4), visit_month, and months ahead, I trained a separate CatBoost model. For each of these models, I optimized the hyperparameters using Optuna.
For updrs_4 I used a linear model provided by https://www.kaggle.com/code/ambrosm/pdpp-almost-only-trends by Ambrosm which itself was a variation of code originally posted by https://www.kaggle.com/code/vitalykudelya/only-trends by Vitaly Kudelya. I used a clip month of 54 for updrs_4.
Moreover, because of the large number of targets, it was possible for some observations in the test set and for some targets the prediction ought to be outside the range of what was seen in training. Because tree-based models extrapolate very poorly, I felt that if the prediction updrs score provided by CatBoost was at the boundary of what was seen in training, then that prediction might be very poor. As such, in cases where CatBoost predicted a score at the boundary of the training data, that score was disregarded and the linear model described in the previous bullet point was used instead.
A combination of CatBoost and the linear model in instances where CatBoost predicts the boundary appeared to work best in GroupKFold cross-validation. Due to the large number of targets CatBoost did a worse job by itself without the linear component.
Setting the objective to MAE for CatBoost appeared to provide better results than MSE.
updrs_4 and for CatBoost predictions outside the bounds of what was seen in trainingvisit_month alone. I missed the trick all top notebooks used, but absent the trick, I found that including the protein data gave superior results.I really enjoyed this competition and I hope the organizers will find our solutions useful.
\nCongrats to all winners and good luck to all in future competitions!!!