![\"cross_val_score](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2107080%2Fc3ff437473f9b175b62dd1a8873bdc7d%2FCapture.PNG?generation=1583830769590062&alt=media\")
\nlooks good but when I upload my prediction I get only 0.71635 Public Score on Leaderboard I don't know why my score has huge diffrence.Hi. I trained my model with some newly created feature and I used cross_val_score and the results of cross_val_score are \nlooks good but when I upload my prediction I get only 0.71635 Public Score on Leaderboard I don't know why my score has huge diffrence.
Hi. I trained my model with some newly created feature and I used cross_val_score and the results of cross_val_score are \nlooks good but when I upload my prediction I get only 0.71635 Public Score on Leaderboard I don't know why my score has huge diffrence.
Maybe crossvalscore is note the best, on evaluation is write : \"Submissions are evaluated on area under the ROC curve between the predicted probability and the observed target.\"
\n\nI use roc_auc_score and is not very different. But I just start on kaggle, maybe I'm wrong... Your score 0.91 is very high !!
", "rawMarkdown": "Maybe crossvalscore is note the best, on evaluation is write : \"Submissions are evaluated on area under the ROC curve between the predicted probability and the observed target.\"\n\nI use roc_auc_score and is not very different. But I just start on kaggle, maybe I'm wrong... Your score 0.91 is very high !!", "votes": null }, { "id": "768172", "postDate": "03/10/2020 14:08:58", "content": "I agree with @priv375. I think you might not be using the roc-auc metric for validation.
", "rawMarkdown": "I agree with @priv375. I think you might not be using the roc-auc metric for validation.", "votes": null }, { "id": "768252", "postDate": "03/10/2020 15:33:16", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2107080%2F9d195c15e9da541efc351bd57dd386ea%2FCapture.PNG?generation=1583854353051022&alt=media\")
\nused Scoring = roc_auc \n@priv375
--> Its actually very strange. Can you try the same with sklearn.StratifiedKFold?
", "rawMarkdown": "> Its actually very strange. Can you try the same with sklearn.StratifiedKFold?", "votes": null }, { "id": "770145", "postDate": "03/12/2020 16:02:34", "content": "There could be some data leakage in the way you preprocess the data. \nFor example, you talked about some newly created features. If some of them have been created using the target, it could be an example of data leakage (depending on the way you created them).
", "rawMarkdown": "There could be some data leakage in the way you preprocess the data. \nFor example, you talked about some newly created features. If some of them have been created using the target, it could be an example of data leakage (depending on the way you created them).", "votes": null } ], "comments": [ { "id": 767968, "author_name": "priv375", "author_url": "", "post_date": "03/10/2020 10:08:01", "content": "Maybe crossvalscore is note the best, on evaluation is write : \"Submissions are evaluated on area under the ROC curve between the predicted probability and the observed target.\"
\n\nI use roc_auc_score and is not very different. But I just start on kaggle, maybe I'm wrong... Your score 0.91 is very high !!
", "votes": null, "replies": [] }, { "id": 768172, "author_name": "siimtarbe", "author_url": "", "post_date": "03/10/2020 14:08:58", "content": "I agree with @priv375. I think you might not be using the roc-auc metric for validation.
", "votes": null, "replies": [ { "id": 768252, "author_name": "jayesh4520", "author_url": "", "post_date": "03/10/2020 15:33:16", "content": "\nused Scoring = roc_auc \n@priv375
--> Its actually very strange. Can you try the same with sklearn.StratifiedKFold?
", "votes": null, "replies": [] }, { "id": 770145, "author_name": "robinmunier", "author_url": "", "post_date": "03/12/2020 16:02:34", "content": "There could be some data leakage in the way you preprocess the data. \nFor example, you talked about some newly created features. If some of them have been created using the target, it could be an example of data leakage (depending on the way you created them).
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "767926": "Hi. I trained my model with some newly created feature and I used cross_val_score and the results of cross_val_score are \nlooks good but when I upload my prediction I get only 0.71635 Public Score on Leaderboard I don't know why my score has huge diffrence.", "767968": "Maybe crossvalscore is note the best, on evaluation is write : \"Submissions are evaluated on area under the ROC curve between the predicted probability and the observed target.\"\n\nI use roc_auc_score and is not very different. But I just start on kaggle, maybe I'm wrong... Your score 0.91 is very high !!", "768172": "I agree with @priv375. I think you might not be using the roc-auc metric for validation.", "768252": "\nused Scoring = roc_auc \n@priv375", "769643": "> Its actually very strange. Can you try the same with sklearn.StratifiedKFold?", "770145": "There could be some data leakage in the way you preprocess the data. \nFor example, you talked about some newly created features. If some of them have been created using the target, it could be an example of data leakage (depending on the way you created them)." }, "source": "meta" }