Hi all,
\nThe competition will end in a few days. Once the final ranking is released, to be eligible for prizes, you will need to share with us your runnable solution (private or public, up to you). Your solution should follow the rules.
\nPlease share your solution if you are in the top 10 as some solutions may not be eligible. Prizes attribution will be done one week after the end of the competition.
\nGood luck!
", "messages": [ { "id": "1775669", "postDate": "05/03/2022 08:50:36", "content": "Hi all,
\nThe competition will end in a few days. Once the final ranking is released, to be eligible for prizes, you will need to share with us your runnable solution (private or public, up to you). Your solution should follow the rules.
\nPlease share your solution if you are in the top 10 as some solutions may not be eligible. Prizes attribution will be done one week after the end of the competition.
\nGood luck!
", "rawMarkdown": "Hi all,\n\nThe competition will end in a few days. Once the final ranking is released, to be eligible for prizes, you will need to share with us your runnable solution (private or public, up to you). Your solution should follow [the rules](https://www.kaggle.com/competitions/football-match-probability-prediction/rules).\n\n**Please share your solution if you are in the top 10** as some solutions may not be eligible. Prizes attribution will be done one week after the end of the competition.\n\nGood luck!", "votes": null }, { "id": "1780096", "postDate": "05/07/2022 04:41:10", "content": "Do we need to add you as colaborator or we should make notebook public for the purpose of sharing notebook with octosport
", "rawMarkdown": "Do we need to add you as colaborator or we should make notebook public for the purpose of sharing notebook with octosport", "votes": null }, { "id": "1780184", "postDate": "05/07/2022 07:15:55", "content": "It is up to you but it is probably better to make it public.
", "rawMarkdown": "It is up to you but it is probably better to make it public.", "votes": null }, { "id": "1780194", "postDate": "05/07/2022 07:30:12", "content": "Congratulation to everyone. We will now review the solution. Note that the true final ranking may not be reflected by the leaderboard if some solutions do not follow the rules even if you are in the top 10. If you are on the list below, can you share your solution so we can review it and announce the winner in a week?
\n@seraquevence
\n@alexandreurwald
\n@tangtunyu
\n@jbomitchell
\n@youneseloiarm
\n@himanshunayal
\n@hamzaouammou
\n@claraljm
\n@mathurinache
\n@lonnieqin
Thank you
", "rawMarkdown": "Congratulation to everyone. We will now review the solution. Note that the true final ranking may not be reflected by the leaderboard if some solutions do not follow the rules even if you are in the top 10. If you are on the list below, can you share your solution so we can review it and announce the winner in a week?\n\n@seraquevence\n@alexandreurwald\n@tangtunyu\n@jbomitchell \n@youneseloiarm\n@himanshunayal \n@hamzaouammou \n@claraljm\n@mathurinache\n@lonnieqin \n\nThank you", "votes": null }, { "id": "1780277", "postDate": "05/07/2022 09:00:07", "content": "this notebook maybe be important for you, but however, for me, I don't think so, because I have personal data of Premier League Football between 2015 to 2022, I can with naive models get a score higher than when I use your data exactly (0.9+-0.05 log loss). with taking the attention before 2020 is for a train set and valid set,
\nthe year 2021 is for the test set.
\n So, you should clean your data from missing values first, then organize it for each league, to remove the noise as possible, so that it is going to get you the high score maybe that I wish (higher than 0.7 for accuracy) and/or (less than 0.7 for log loss).
\nBest regards
Hi @octosportio
\nHow the vote for the best notebook will be done?
Most votes, comments and community impact.
", "rawMarkdown": "Most votes, comments and community impact.", "votes": null }, { "id": "1780781", "postDate": "05/07/2022 21:03:19", "content": "Thank you @octosportio for organizing this competition and the special mention.
\nI have shared the notebook. Cheers.
", "rawMarkdown": "Thank you @octosportio for organizing this competition and the special mention.\n\nI have shared the notebook. Cheers.", "votes": null }, { "id": "1781361", "postDate": "05/08/2022 12:47:34", "content": "Hello @octosportio, thanks for the dataset. I participated in the competition for fun and to become more acquainted with LSTM. I've seen some of the top solutions and I believe many of them have the same issues: 1) isn't the usage of Bidirectional LSTM a case of data leakage? 2) in the case of an LSTM implementation, is fitting the model with attributes that are identical to the target attribute in the test case wrong? i.e. having attributes for home/away win/draw/loss
", "rawMarkdown": "Hello @octosportio, thanks for the dataset. I participated in the competition for fun and to become more acquainted with LSTM. I've seen some of the top solutions and I believe many of them have the same issues: 1) isn't the usage of Bidirectional LSTM a case of data leakage? 2) in the case of an LSTM implementation, is fitting the model with attributes that are identical to the target attribute in the test case wrong? i.e. having attributes for home/away win/draw/loss", "votes": null }, { "id": "1781427", "postDate": "05/08/2022 13:46:38", "content": "Hi, the LSTM do not leak because it uses pre match information even if it is bidirectional. The idea being to infer the team strength given their past results. So it does not see the future.
\nI am not sure to understand the second question. Do you have an example?
", "rawMarkdown": "Hi, the LSTM do not leak because it uses pre match information even if it is bidirectional. The idea being to infer the team strength given their past results. So it does not see the future.\n\nI am not sure to understand the second question. Do you have an example?", "votes": null }, { "id": "1781443", "postDate": "05/08/2022 14:03:17", "content": "Hello @octosportio, I have one question about the method that you used to split test data into a public leaderboard and a private leaderboard?
\nbecause I think the private leaderboard has a different distribution from the public leaderboard, and the last thing is how can you trust the best 5 models that have a high score on the private leader board (maybe these ones can't generalization their predictions on the future data that you don't have yet.)
Hi, regarding Bidirectional LSTM's data leakage I haven't found any resource online except this part off tf's guide:
\n\n\nFor sequences other than time series (e.g. text), it is often the case that a RNN model can perform better if it not only processes sequence from start to end, but also backwards.
\n
which I'm not sure whether it suggests avoiding using Bidirectional LSTM with time series or that it does not bring better results. However, given that the input is a sequence of historical matches it makes sense to me that going through the historical matches in reverse order is data leakage. Meaning that there is data leakage between the historical matches not with the prediction set. However, that's just my interpretation.
\nRegarding the 2nd point, in example solutions for other problems, I have seen a different approach to the train set. For example, one of the top solutions uses a sequence of 10 historical matches in order to predict the 'target' variable i.e. 0/1/2 (winner: draw, home, away) for the next match, but the same variable (winner: draw, home, away) with a different name is also present in the train set (for the historical matches). In those example solutions/tutorials I looked at, this is avoided and I was wondering whether that is an incorrect approach. Thanks for any input.
\n*edit:
\nmaybe this is a clearer way for my second point:
\nif we were given a dataset with the same information, but in a long format with each instance having the 'target' attribute. If we were to use a sequence of 10 matches to predict only the 11th match, wouldn't we need to drop the 'target' variable from the sequence of 10 matches as this is what the output should be?
We don't have any leak situation if we apply the Sequences method on features variables, not on Target variable, another thing about adding new features has names like draw_…, home_…., away_…,
\nis just using the Stacking method, So maybe it gives us an overfitting situation if we ignore its rule to use it, but It's not a leak method.
Distribution change for time series, we need to live with. Split is just temporal here with 10k matches that can not be leaked at all.
\nBidirectional LSTM is useful in this case to better fit the expected rating. For your second point If the target variable is in the training data, it is clearly wrong. But this is not the case here otherwise you get a perfect score.
Neither person used the time series method, for splitting his data, to converge his CV with LB. I was stupid to don't think about that, However, all top 10 models will work better if we change their CV with Time-series split method CV.
", "rawMarkdown": "Neither person used the time series method, for splitting his data, to converge his CV with LB. I was stupid to don't think about that, However, all top 10 models will work better if we change their CV with Time-series split method CV.", "votes": null } ], "comments": [ { "id": 1780096, "author_name": "himanshunayal", "author_url": "", "post_date": "05/07/2022 04:41:10", "content": "Do we need to add you as colaborator or we should make notebook public for the purpose of sharing notebook with octosport
", "votes": null, "replies": [ { "id": 1780184, "author_name": "octosportio", "author_url": "", "post_date": "05/07/2022 07:15:55", "content": "It is up to you but it is probably better to make it public.
", "votes": null, "replies": [] } ] }, { "id": 1780194, "author_name": "octosportio", "author_url": "", "post_date": "05/07/2022 07:30:12", "content": "Congratulation to everyone. We will now review the solution. Note that the true final ranking may not be reflected by the leaderboard if some solutions do not follow the rules even if you are in the top 10. If you are on the list below, can you share your solution so we can review it and announce the winner in a week?
\n@seraquevence
\n@alexandreurwald
\n@tangtunyu
\n@jbomitchell
\n@youneseloiarm
\n@himanshunayal
\n@hamzaouammou
\n@claraljm
\n@mathurinache
\n@lonnieqin
Thank you
", "votes": null, "replies": [ { "id": 1780277, "author_name": "youneseloiarm", "author_url": "", "post_date": "05/07/2022 09:00:07", "content": "this notebook maybe be important for you, but however, for me, I don't think so, because I have personal data of Premier League Football between 2015 to 2022, I can with naive models get a score higher than when I use your data exactly (0.9+-0.05 log loss). with taking the attention before 2020 is for a train set and valid set,
\nthe year 2021 is for the test set.
\n So, you should clean your data from missing values first, then organize it for each league, to remove the noise as possible, so that it is going to get you the high score maybe that I wish (higher than 0.7 for accuracy) and/or (less than 0.7 for log loss).
\nBest regards
Hi @octosportio
\nHow the vote for the best notebook will be done?
Most votes, comments and community impact.
", "votes": null, "replies": [] } ] }, { "id": 1780781, "author_name": "hamzaouammou", "author_url": "", "post_date": "05/07/2022 21:03:19", "content": "Thank you @octosportio for organizing this competition and the special mention.
\nI have shared the notebook. Cheers.
", "votes": null, "replies": [] }, { "id": 1781361, "author_name": "azr25684276", "author_url": "", "post_date": "05/08/2022 12:47:34", "content": "Hello @octosportio, thanks for the dataset. I participated in the competition for fun and to become more acquainted with LSTM. I've seen some of the top solutions and I believe many of them have the same issues: 1) isn't the usage of Bidirectional LSTM a case of data leakage? 2) in the case of an LSTM implementation, is fitting the model with attributes that are identical to the target attribute in the test case wrong? i.e. having attributes for home/away win/draw/loss
", "votes": null, "replies": [ { "id": 1781427, "author_name": "octosportio", "author_url": "", "post_date": "05/08/2022 13:46:38", "content": "Hi, the LSTM do not leak because it uses pre match information even if it is bidirectional. The idea being to infer the team strength given their past results. So it does not see the future.
\nI am not sure to understand the second question. Do you have an example?
", "votes": null, "replies": [] }, { "id": 1781443, "author_name": "youneseloiarm", "author_url": "", "post_date": "05/08/2022 14:03:17", "content": "Hello @octosportio, I have one question about the method that you used to split test data into a public leaderboard and a private leaderboard?
\nbecause I think the private leaderboard has a different distribution from the public leaderboard, and the last thing is how can you trust the best 5 models that have a high score on the private leader board (maybe these ones can't generalization their predictions on the future data that you don't have yet.)
Hi, regarding Bidirectional LSTM's data leakage I haven't found any resource online except this part off tf's guide:
\n\n\nFor sequences other than time series (e.g. text), it is often the case that a RNN model can perform better if it not only processes sequence from start to end, but also backwards.
\n
which I'm not sure whether it suggests avoiding using Bidirectional LSTM with time series or that it does not bring better results. However, given that the input is a sequence of historical matches it makes sense to me that going through the historical matches in reverse order is data leakage. Meaning that there is data leakage between the historical matches not with the prediction set. However, that's just my interpretation.
\nRegarding the 2nd point, in example solutions for other problems, I have seen a different approach to the train set. For example, one of the top solutions uses a sequence of 10 historical matches in order to predict the 'target' variable i.e. 0/1/2 (winner: draw, home, away) for the next match, but the same variable (winner: draw, home, away) with a different name is also present in the train set (for the historical matches). In those example solutions/tutorials I looked at, this is avoided and I was wondering whether that is an incorrect approach. Thanks for any input.
\n*edit:
\nmaybe this is a clearer way for my second point:
\nif we were given a dataset with the same information, but in a long format with each instance having the 'target' attribute. If we were to use a sequence of 10 matches to predict only the 11th match, wouldn't we need to drop the 'target' variable from the sequence of 10 matches as this is what the output should be?
We don't have any leak situation if we apply the Sequences method on features variables, not on Target variable, another thing about adding new features has names like draw_…, home_…., away_…,
\nis just using the Stacking method, So maybe it gives us an overfitting situation if we ignore its rule to use it, but It's not a leak method.
Distribution change for time series, we need to live with. Split is just temporal here with 10k matches that can not be leaked at all.
\nBidirectional LSTM is useful in this case to better fit the expected rating. For your second point If the target variable is in the training data, it is clearly wrong. But this is not the case here otherwise you get a perfect score.
Neither person used the time series method, for splitting his data, to converge his CV with LB. I was stupid to don't think about that, However, all top 10 models will work better if we change their CV with Time-series split method CV.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1775669": "Hi all,\n\nThe competition will end in a few days. Once the final ranking is released, to be eligible for prizes, you will need to share with us your runnable solution (private or public, up to you). Your solution should follow [the rules](https://www.kaggle.com/competitions/football-match-probability-prediction/rules).\n\n**Please share your solution if you are in the top 10** as some solutions may not be eligible. Prizes attribution will be done one week after the end of the competition.\n\nGood luck!", "1780096": "Do we need to add you as colaborator or we should make notebook public for the purpose of sharing notebook with octosport", "1780184": "It is up to you but it is probably better to make it public.", "1780194": "Congratulation to everyone. We will now review the solution. Note that the true final ranking may not be reflected by the leaderboard if some solutions do not follow the rules even if you are in the top 10. If you are on the list below, can you share your solution so we can review it and announce the winner in a week?\n\n@seraquevence\n@alexandreurwald\n@tangtunyu\n@jbomitchell \n@youneseloiarm\n@himanshunayal \n@hamzaouammou \n@claraljm\n@mathurinache\n@lonnieqin \n\nThank you", "1780277": "this [notebook](https://www.kaggle.com/code/youneseloiarm/football-prob-prediction-lstm-002?scriptVersionId=94733415) maybe be important for you, but however, for me, I don't think so, because I have personal data of Premier League Football between 2015 to 2022, I can with naive models get a score higher than when I use your data exactly (0.9+-0.05 log loss). with taking the attention before 2020 is for a train set and valid set, \nthe year 2021 is for the test set.\n So, you should clean your data from missing values first, then organize it for each league, to remove the noise as possible, so that it is going to get you the high score maybe that I wish (higher than 0.7 for accuracy) and/or (less than 0.7 for log loss).\n*Best regards*", "1780447": "Hi @octosportio\nHow the vote for the best notebook will be done?", "1780493": "Most votes, comments and community impact.", "1780781": "Thank you @octosportio for organizing this competition and the special mention.\n\nI have shared the notebook. Cheers.", "1781361": "Hello @octosportio, thanks for the dataset. I participated in the competition for fun and to become more acquainted with LSTM. I've seen some of the top solutions and I believe many of them have the same issues: 1) isn't the usage of Bidirectional LSTM a case of data leakage? 2) in the case of an LSTM implementation, is fitting the model with attributes that are identical to the target attribute in the test case wrong? i.e. having attributes for home/away win/draw/loss", "1781427": "Hi, the LSTM do not leak because it uses pre match information even if it is bidirectional. The idea being to infer the team strength given their past results. So it does not see the future.\n\nI am not sure to understand the second question. Do you have an example?", "1781443": "Hello @octosportio, I have one question about the method that you used to split test data into a public leaderboard and a private leaderboard?\nbecause I think the private leaderboard has a different distribution from the public leaderboard, and the last thing is how can you trust the best 5 models that have a high score on the private leader board (maybe these ones can't generalization their predictions on the future data that you don't have yet.)", "1781478": "Hi, regarding Bidirectional LSTM's data leakage I haven't found any resource online except this part off [tf's guide](https://www.tensorflow.org/guide/keras/rnn):\n> For sequences other than time series (e.g. text), it is often the case that a RNN model can perform better if it not only processes sequence from start to end, but also backwards.\n\nwhich I'm not sure whether it suggests avoiding using Bidirectional LSTM with time series or that it does not bring better results. However, given that the input is a sequence of historical matches it makes sense to me that going through the historical matches in reverse order is data leakage. Meaning that there is data leakage between the historical matches not with the prediction set. However, that's just my interpretation.\n\nRegarding the 2nd point, in example solutions for other problems, I have seen a different approach to the train set. For example, one of the top solutions uses a sequence of 10 historical matches in order to predict the 'target' variable i.e. 0/1/2 (winner: draw, home, away) for the next match, but the same variable (winner: draw, home, away) with a different name is also present in the train set (for the historical matches). In those example solutions/tutorials I looked at, this is avoided and I was wondering whether that is an incorrect approach. Thanks for any input.\n\n*edit:\n\nmaybe this is a clearer way for my second point:\nif we were given a dataset with the same information, but in a long format with each instance having the 'target' attribute. If we were to use a sequence of 10 matches to predict only the 11th match, wouldn't we need to drop the 'target' variable from the sequence of 10 matches as this is what the output should be?", "1781499": "We don't have any leak situation if we apply the Sequences method on features variables, not on Target variable, another thing about adding new features has names like draw_..., home_...., away_...,\nis just using the Stacking method, So maybe it gives us an overfitting situation if we ignore its rule to use it, but It's not a leak method.", "1781565": "Distribution change for time series, we need to live with. Split is just temporal here with 10k matches that can not be leaked at all.\nBidirectional LSTM is useful in this case to better fit the expected rating. For your second point If the target variable is in the training data, it is clearly wrong. But this is not the case here otherwise you get a perfect score.", "1781607": "Neither person used the time series method, for splitting his data, to converge his CV with LB. I was stupid to don't think about that, However, all top 10 models will work better if we change their CV with Time-series split method CV." }, "source": "meta" }