![\"b1\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7917824%2Fac8539db5efe8bbbc2136e8e8bd6a240%2Fb1.png?generation=1684456007327684&alt=media\")
When I selected my two final submissions a few hours ago, I decided for
\nIt turned out that the public leaderboard was the better indicator than the cv, and the peptide/protein feature engineering was useless.
\nIf we plot the median updrs scores for every month, we see that the months which are multiples of 12 (the cyan markers on the gridlines) are usually lower than the non-multiples of 12 (the magenta markers between the gridlines). This cannot be a coincidence.
\n
A scatterplot of the 248 patients versus the months of their updrs assessments reveals that there are three groups of patients:
\nThe patients of the green group had their first visits in months 0, 3, 6, 9, 12.\nThe patients of the orange group had their first visits in months 0, 6, 12, 18, 24 and the last visit in month 60.\nThe patients of the red group had their first visits in months 0, 12, 24.\n![\"b2\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7917824%2F6be1bdd4328f7ddfe3cadabc12ea7609%2Fb2.png?generation=1684456024639136&alt=media\")
If we plot the updrs scores over time of every patient, we see differences among the groups. The red group in particular has the lowest updrs scores, which means that these are the healthiest people, and updrs_4 has rarely been measured for them.
\nWe can hypothesize that the red group is the control group (a group of people without Parkinson's disease), and the experimenters decided to test the control group only once a year and to skip the updrs_4 test for this group. The real patients (green and orange groups) were tested more often and with all four updrs tests.
\n![\"b3\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7917824%2F1fe6d7bdde18dc308ad5ad6b27cab16b%2Fb3.png?generation=1684456038645918&alt=media\")
Conclusion: We can distinguish the control group from the real patients according to their first non-zero visit_month: If the first non-zero visit_month is <12, we have a real patient; if the first non-zero visit_month equals 12, the person belongs to the healthy control group. This distinction has high predictive value for the updrs scores.
\nThe model has only two features:
\nDepending on the group, it predicts a linear or isotonic regression:
\n![\"b4\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7917824%2Fd26b9a640de7558a2cfc229abb21c567%2Fb4.png?generation=1684456323453532&alt=media\")
Source code is here.
", "comments": [ { "id": 2271947, "author_name": "Gaurav Srivastava", "author_url": "", "post_date": "2023-05-24T07:47:37.817000", "content": "Very Insightful ๐ฅ Congratulations ๐ and Thanks for sharing @ambrosm ๐ค
", "votes": 1, "replies": [] }, { "id": 2265803, "author_name": "Oscar Aguilar", "author_url": "", "post_date": "2023-05-19T14:14:30.383000", "content": "Congrats @ambrosm and thanks for sharing your approach. Nice discovery๐
", "votes": 1, "replies": [] }, { "id": 2265334, "author_name": "steubk", "author_url": "", "post_date": "2023-05-19T07:08:02.853000", "content": "Congratulations and thank you for sharing @ambrosm ! A very clever solution !
", "votes": 1, "replies": [] }, { "id": 2265328, "author_name": "Remek Kinas", "author_url": "", "post_date": "2023-05-19T06:59:39.770000", "content": "Congratulations @ambrosm. ๐๐๐
", "votes": 1, "replies": [] }, { "id": 2265290, "author_name": "Quim Quadrada", "author_url": "", "post_date": "2023-05-19T06:16:01.220000", "content": "Hi, congratulations!
\nMay I ask you, how did you do the predictions? Did you check each patient's non-zero first month and the based on this you predicted the updrs values?
", "votes": 1, "replies": [ { "id": 2267103, "author_name": "AmbrosM", "author_url": "", "post_date": "2023-05-20T15:37:40.627000", "content": "Yes, @quimquadrada, in month zero I do not yet know the group of the patient, but starting from month 6 the group is known, and then the model predicts either red or green according to the bottommost diagram of the post.
", "votes": 0, "replies": [] } ] }, { "id": 2265124, "author_name": "Yoobin", "author_url": "", "post_date": "2023-05-19T02:15:45.830000", "content": "Thank you for your notes. this is indeed great insight! I have to say this competition is really ruined by the organization. The aim should be finding useful info for patient, like protein stuff. But at the end, it becomes a data issue, lol.
", "votes": 1, "replies": [] }, { "id": 2265087, "author_name": "Cody_Null", "author_url": "", "post_date": "2023-05-19T01:13:45.177000", "content": "Lesson learned. Plot all variables against the target.
", "votes": 1, "replies": [] }, { "id": 2265075, "author_name": "AC", "author_url": "", "post_date": "2023-05-19T01:00:21.993000", "content": "Hey! Congratulations! :)
\nThank you for sharing your findings. If it's okay, could you please tell what's the private LB score of your other notebook that uses protein/peptide.
Hi @ahsuna123 I've submitted 7 notebooks based on groups and proteins with private scores from 60.0 to 60.8. The one I chose for the final submission has 60.4.
", "votes": 2, "replies": [ { "id": 2265102, "author_name": "AC", "author_url": "", "post_date": "2023-05-19T01:28:44.040000", "content": "Oh, yes! That's a huge difference. I had the same experience of notebooks using only visit_months performing much better than those using protein/peptide features on private LB. Thank you for sharing. Learnt a lot from your public notebooks and of course, this one! Congratulations once again. Well Deserved! :)
", "votes": 1, "replies": [] }, { "id": 2265117, "author_name": "", "author_url": "", "post_date": "2023-05-19T01:58:44.910000", "content": "", "votes": 2, "replies": [] }, { "id": 2265134, "author_name": "AC", "author_url": "", "post_date": "2023-05-19T02:30:49.680000", "content": "Likewise! After seeing some of the great solutions made public! It's definitely intriguing! Will go more in depth for EDA from next time! :)
", "votes": 0, "replies": [] } ] } ] }, { "id": 2265069, "author_name": "Makotu", "author_url": "", "post_date": "2023-05-19T00:49:00.667000", "content": "Congrats!
\nThank you for sharing. I learned a lot from your notebook and discussion in this competition.
Great work, excellent thoughts on EDA.
", "votes": 1, "replies": [] }, { "id": 2267209, "author_name": "SSS", "author_url": "", "post_date": "2023-05-20T17:13:08.180000", "content": "I played with the source code for a bit. It is a typical Kaggle dark magicโฆ Though, @ambrosm it does not undermine your analytical skills. Congratulations!
\nI could not find any signal and practical use of the possible solution, so I skipped the competition. Looking forward to crossing the swords with you on the leaderboard anytime soon.
@ambrosm , one best thing I learnt from you is how to make dig into the data & extract required info smartly .
\nGreat work . Congratulations on gold . could you please tell me how many hours did you spent in this comp !!(Out Of Blue )
", "votes": 2, "replies": [ { "id": 2267098, "author_name": "AmbrosM", "author_url": "", "post_date": "2023-05-20T15:34:47.030000", "content": "Thanks, @soumendraprasad. As I started late into the competition, it's perhaps five weeks times three hours per evening = 100 hours.
", "votes": 4, "replies": [] } ] }, { "id": 2265183, "author_name": "Vincent Schuler", "author_url": "", "post_date": "2023-05-19T03:50:19.263000", "content": "Congrats @ambrosm !
\nAnd thanks for the graphs and the explanation.
", "votes": 2, "replies": [] }, { "id": 2265074, "author_name": "Ankur Limbashia", "author_url": "", "post_date": "2023-05-19T00:55:49.470000", "content": "Wow, excellent explanation!! Congratulations!!
", "votes": 2, "replies": [] }, { "id": 2265064, "author_name": "Chris Deotte", "author_url": "", "post_date": "2023-05-19T00:38:59.073000", "content": "Wow, great discovery! Congratulations @ambrosm !!
", "votes": 2, "replies": [ { "id": 2265079, "author_name": "Ankur Limbashia", "author_url": "", "post_date": "2023-05-19T01:07:22.643000", "content": "Eagar to learn about your approach @cdeotte
", "votes": 1, "replies": [] } ] }, { "id": 2266684, "author_name": "yuda lee", "author_url": "", "post_date": "2023-05-20T09:14:28.303000", "content": "Insightful! Thank you for sharing!
", "votes": 1, "replies": [] }, { "id": 2265802, "author_name": "Xiao-Su (Frank) Hu", "author_url": "", "post_date": "2023-05-19T14:14:22.887000", "content": "Impressive! Thank you for sharing.
", "votes": 1, "replies": [] }, { "id": 2265213, "author_name": "Rocha Erik", "author_url": "", "post_date": "2023-05-19T04:45:46.053000", "content": "Great work and thank for sharing it!!
", "votes": 1, "replies": [] }, { "id": 2265070, "author_name": "Woodpecker", "author_url": "", "post_date": "2023-05-19T00:49:15.993000", "content": "I have learnt a lot! Thanks!
", "votes": 1, "replies": [] } ], "comment_count": 26, "messages": [ { "id": 2265054, "postDate": "2023-05-19T00:33:50.590Z", "content": "When I selected my two final submissions a few hours ago, I decided for
\nIt turned out that the public leaderboard was the better indicator than the cv, and the peptide/protein feature engineering was useless.
\nIf we plot the median updrs scores for every month, we see that the months which are multiples of 12 (the cyan markers on the gridlines) are usually lower than the non-multiples of 12 (the magenta markers between the gridlines). This cannot be a coincidence.
\n
A scatterplot of the 248 patients versus the months of their updrs assessments reveals that there are three groups of patients:
\nThe patients of the green group had their first visits in months 0, 3, 6, 9, 12.\nThe patients of the orange group had their first visits in months 0, 6, 12, 18, 24 and the last visit in month 60.\nThe patients of the red group had their first visits in months 0, 12, 24.\n
If we plot the updrs scores over time of every patient, we see differences among the groups. The red group in particular has the lowest updrs scores, which means that these are the healthiest people, and updrs_4 has rarely been measured for them.
\nWe can hypothesize that the red group is the control group (a group of people without Parkinson's disease), and the experimenters decided to test the control group only once a year and to skip the updrs_4 test for this group. The real patients (green and orange groups) were tested more often and with all four updrs tests.
\n
Conclusion: We can distinguish the control group from the real patients according to their first non-zero visit_month: If the first non-zero visit_month is <12, we have a real patient; if the first non-zero visit_month equals 12, the person belongs to the healthy control group. This distinction has high predictive value for the updrs scores.
\nThe model has only two features:
\nDepending on the group, it predicts a linear or isotonic regression:
\n
Source code is here.
", "rawMarkdown": "When I selected my two final submissions a few hours ago, I decided for\n- one model which uses only the month data for the predictions (no peptides and proteins) and has a public lb score of 54.7.\n- another model, which additionally uses the peptides and proteins, has a better cv but a bad public lb score (55.3).\n\nIt turned out that the public leaderboard was the better indicator than the cv, and the peptide/protein feature engineering was useless.\n\n# Recognizing the control group\n\nIf we plot the median updrs scores for every month, we see that the months which are multiples of 12 (the cyan markers on the gridlines) are usually lower than the non-multiples of 12 (the magenta markers between the gridlines). This cannot be a coincidence.\n\n\n\nA scatterplot of the 248 patients versus the months of their updrs assessments reveals that there are three groups of patients:\n\n The patients of the green group had their first visits in months 0, 3, 6, 9, 12.\n The patients of the orange group had their first visits in months 0, 6, 12, 18, 24 and the last visit in month 60.\n The patients of the red group had their first visits in months 0, 12, 24.\n\n\n\nIf we plot the updrs scores over time of every patient, we see differences among the groups. The red group in particular has the lowest updrs scores, which means that these are the healthiest people, and updrs_4 has rarely been measured for them.\n\nWe can hypothesize that the red group is the control group (a group of people without Parkinson's disease), and the experimenters decided to test the control group only once a year and to skip the updrs_4 test for this group. The real patients (green and orange groups) were tested more often and with all four updrs tests.\n\n\n\nConclusion: We can distinguish the control group from the real patients according to their first non-zero visit_month: If the first non-zero visit_month is <12, we have a real patient; if the first non-zero visit_month equals 12, the person belongs to the healthy control group. This distinction has high predictive value for the updrs scores.\n\n# The model\n\nThe model has only two features:\n- the group to which a patient belongs\n- the month of the prediction\n\nDepending on the group, it predicts a linear or isotonic regression:\n\n\n\n# Lessons learned\n- A thorough EDA is important, and the EDA must be adapted to the dataset. Automated EDA tools don't find the hidden information.\n- Unusual metrics (smape plus one) require unusual methods.\n- If the training dataset is small, simple models turn out best.\n- Medical data is scarce and expensive. If we haven't been able to prove a connection between proteins and Parkinson symptoms, this doesn't mean there is none. It only means that another thousand patients must be convinced to participate in a five-year study, and we might see a follow-up competition in 2028...\n- In biology and medicine, we usually search for very weak effects: Protein measurements are imprecise and updrs scores depend on the mood of the patient and the doctor. If anybody was expecting SMAPE scores far below 50, this expectation was unrealistic.\n \nSource code is [here](https://www.kaggle.com/code/ambrosm/pdpp-linear-and-isotonic-groups).\n", "votes": 75 }, { "id": 2271947, "postDate": "2023-05-24T07:47:37.817Z", "content": "Very Insightful ๐ฅ Congratulations ๐ and Thanks for sharing @ambrosm ๐ค
", "rawMarkdown": "Very Insightful ๐ฅ Congratulations ๐ and Thanks for sharing @ambrosm ๐ค", "votes": 1 }, { "id": 2265803, "postDate": "2023-05-19T14:14:30.383Z", "content": "Congrats @ambrosm and thanks for sharing your approach. Nice discovery๐
", "rawMarkdown": "Congrats @ambrosm and thanks for sharing your approach. Nice discovery๐", "votes": 1 }, { "id": 2265334, "postDate": "2023-05-19T07:08:02.853Z", "content": "Congratulations and thank you for sharing @ambrosm ! A very clever solution !
", "rawMarkdown": "Congratulations and thank you for sharing @ambrosm ! A very clever solution !", "votes": 1 }, { "id": 2265328, "postDate": "2023-05-19T06:59:39.770Z", "content": "Congratulations @ambrosm. ๐๐๐
", "rawMarkdown": "Congratulations @ambrosm. ๐๐๐", "votes": 1 }, { "id": 2265290, "postDate": "2023-05-19T06:16:01.220Z", "content": "Hi, congratulations!
\nMay I ask you, how did you do the predictions? Did you check each patient's non-zero first month and the based on this you predicted the updrs values?
", "rawMarkdown": "Hi, congratulations!\n\nMay I ask you, how did you do the predictions? Did you check each patient's non-zero first month and the based on this you predicted the updrs values?", "votes": 1, "replies": [ { "id": 2267103, "postDate": "2023-05-20T15:37:40.627Z", "content": "Yes, @quimquadrada, in month zero I do not yet know the group of the patient, but starting from month 6 the group is known, and then the model predicts either red or green according to the bottommost diagram of the post.
", "rawMarkdown": "Yes, @quimquadrada, in month zero I do not yet know the group of the patient, but starting from month 6 the group is known, and then the model predicts either red or green according to the bottommost diagram of the post." } ] }, { "id": 2265124, "postDate": "2023-05-19T02:15:45.830Z", "content": "Thank you for your notes. this is indeed great insight! I have to say this competition is really ruined by the organization. The aim should be finding useful info for patient, like protein stuff. But at the end, it becomes a data issue, lol.
", "rawMarkdown": "Thank you for your notes. this is indeed great insight! I have to say this competition is really ruined by the organization. The aim should be finding useful info for patient, like protein stuff. But at the end, it becomes a data issue, lol.", "votes": 1 }, { "id": 2265087, "postDate": "2023-05-19T01:13:45.177Z", "content": "Lesson learned. Plot all variables against the target.
", "rawMarkdown": "Lesson learned. Plot all variables against the target.", "votes": 1 }, { "id": 2265075, "postDate": "2023-05-19T01:00:21.993Z", "content": "Hey! Congratulations! :)
\nThank you for sharing your findings. If it's okay, could you please tell what's the private LB score of your other notebook that uses protein/peptide.
Hi @ahsuna123 I've submitted 7 notebooks based on groups and proteins with private scores from 60.0 to 60.8. The one I chose for the final submission has 60.4.
", "rawMarkdown": "Hi @ahsuna123 I've submitted 7 notebooks based on groups and proteins with private scores from 60.0 to 60.8. The one I chose for the final submission has 60.4.", "votes": 2, "replies": [ { "id": 2265102, "postDate": "2023-05-19T01:28:44.040Z", "content": "Oh, yes! That's a huge difference. I had the same experience of notebooks using only visit_months performing much better than those using protein/peptide features on private LB. Thank you for sharing. Learnt a lot from your public notebooks and of course, this one! Congratulations once again. Well Deserved! :)
", "rawMarkdown": "Oh, yes! That's a huge difference. I had the same experience of notebooks using only visit_months performing much better than those using protein/peptide features on private LB. Thank you for sharing. Learnt a lot from your public notebooks and of course, this one! Congratulations once again. Well Deserved! :) ", "votes": 1 }, { "id": 2265117, "postDate": "2023-05-19T01:58:44.910Z", "rawMarkdown": "", "votes": 2, "isDeleted": true }, { "id": 2265134, "postDate": "2023-05-19T02:30:49.680Z", "content": "Likewise! After seeing some of the great solutions made public! It's definitely intriguing! Will go more in depth for EDA from next time! :)
", "rawMarkdown": "Likewise! After seeing some of the great solutions made public! It's definitely intriguing! Will go more in depth for EDA from next time! :)" } ] } ] }, { "id": 2265069, "postDate": "2023-05-19T00:49:00.667Z", "content": "Congrats!
\nThank you for sharing. I learned a lot from your notebook and discussion in this competition.
Great work, excellent thoughts on EDA.
", "rawMarkdown": "Great work, excellent thoughts on EDA. ", "votes": 1 }, { "id": 2267209, "postDate": "2023-05-20T17:13:08.180Z", "content": "I played with the source code for a bit. It is a typical Kaggle dark magicโฆ Though, @ambrosm it does not undermine your analytical skills. Congratulations!
\nI could not find any signal and practical use of the possible solution, so I skipped the competition. Looking forward to crossing the swords with you on the leaderboard anytime soon.
@ambrosm , one best thing I learnt from you is how to make dig into the data & extract required info smartly .
\nGreat work . Congratulations on gold . could you please tell me how many hours did you spent in this comp !!(Out Of Blue )
", "rawMarkdown": "@ambrosm , one best thing I learnt from you is how to make dig into the data & extract required info smartly .\n\n\nGreat work . Congratulations on gold . could you please tell me how many hours did you spent in this comp !!(Out Of Blue )", "votes": 2, "replies": [ { "id": 2267098, "postDate": "2023-05-20T15:34:47.030Z", "content": "Thanks, @soumendraprasad. As I started late into the competition, it's perhaps five weeks times three hours per evening = 100 hours.
", "rawMarkdown": "Thanks, @soumendraprasad. As I started late into the competition, it's perhaps five weeks times three hours per evening = 100 hours.", "votes": 4 } ] }, { "id": 2265183, "postDate": "2023-05-19T03:50:19.263Z", "content": "Congrats @ambrosm !
\nAnd thanks for the graphs and the explanation.
", "rawMarkdown": "Congrats @ambrosm !\n\nAnd thanks for the graphs and the explanation.", "votes": 2 }, { "id": 2265074, "postDate": "2023-05-19T00:55:49.470Z", "content": "Wow, excellent explanation!! Congratulations!!
", "rawMarkdown": "Wow, excellent explanation!! Congratulations!!", "votes": 2 }, { "id": 2265064, "postDate": "2023-05-19T00:38:59.073Z", "content": "Wow, great discovery! Congratulations @ambrosm !!
", "rawMarkdown": "Wow, great discovery! Congratulations @ambrosm !! ", "votes": 2, "replies": [ { "id": 2265079, "postDate": "2023-05-19T01:07:22.643Z", "content": "Eagar to learn about your approach @cdeotte
", "rawMarkdown": "Eagar to learn about your approach @cdeotte ", "votes": 1 } ] }, { "id": 2266684, "postDate": "2023-05-20T09:14:28.303Z", "content": "Insightful! Thank you for sharing!
", "rawMarkdown": "Insightful! Thank you for sharing!", "votes": 1 }, { "id": 2265802, "postDate": "2023-05-19T14:14:22.887Z", "content": "Impressive! Thank you for sharing.
", "rawMarkdown": "Impressive! Thank you for sharing.", "votes": 1 }, { "id": 2265213, "postDate": "2023-05-19T04:45:46.053Z", "content": "Great work and thank for sharing it!!
", "rawMarkdown": "Great work and thank for sharing it!!", "votes": 1 }, { "id": 2265070, "postDate": "2023-05-19T00:49:15.993Z", "content": "I have learnt a lot! Thanks!
", "rawMarkdown": "I have learnt a lot! Thanks!", "votes": 1 } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T14:12:54Z", "competition_id": 37794, "competition_slug": "amp-parkinsons-disease-progression-prediction", "team_id": 10232751, "solution_url": "/competitions/amp-parkinsons-disease-progression-prediction/writeups/ambrosm-5-find-the-control-group" }