The competition sponsors had decided that Precision was their metric of choice.
\nWithin the first two days of this competition, we achieved a 1.000 score on the leaderboard.
\nOn the first day of the competition, I looked at the dataset and saw a tabular data problem. Strategy is the Memory Patterns agent from Rock Paper Scissors. Solution is to build a lookup table. I didn't properly understand the competition metric.
\n\nOn the second day @josephassaker had a similar idea but simply searched for the formal dataset titles. This surprised him too when he got a 1.000 score. This method achieved 100% Precision.
\n\nOn the third day, Kaggle realized that this was not the solution the competition sponsors wanted, thus they reset the leaderboard and replaced the competition metric with the F0.5 score.
\nSo what exactly do Precision and Recall actually mean?
\n100% Precision means that every we gave an answer, it was always correct and we never got one wrong. We may however have failed to find all the items we were looking for.
\n100% Recall means that we have successfully found all the available matches in the dataset. but we may also have a big pile of additional false positives.
\nIf neither of these extremes is to your liking, then it is possible to blend them together. This is called the F score. This is most commonly implemented as the F1 score, which is the harmonic means of Precision and Recall.
\nF1 = 2 * (Precision * Recall) / (Precision + Recall)\nHowever, the competition sponsors were biased, they understood now the importance of Recall, but still, they had a preference for Precision. Thus they chose the F0.5 score, which is defined as thus:
\nF0.5 = ((1 + 0.5^2) * Precision * Recall) / (0.5^2 * Precision + Recall)\nF0.5 = (1.25 * Precision * Recall) / (0.25 * Precision + Recall)\nHad they wanted, they could have also chosen the F2 score, which is biased towards Recall. But they didn't.
\nF2 = ((1 + 2^2) * Precision * Recall) / (2^2 * Precision + Recall)\nF2 = (5 * Precision * Recall) / (4 * Precision + Recall)\nKaggle then reset the leaderboard. Ding! Ding! Round Two of the Competition!
\nJason Brownlee has done a more in-depth writeup of the F Score Metric here:
\n", "messages": [ { "id": 1258717, "postDate": "2021-03-31T20:30:46.150Z", "content": "The competition sponsors had decided that Precision was their metric of choice.
\nWithin the first two days of this competition, we achieved a 1.000 score on the leaderboard.
\nOn the first day of the competition, I looked at the dataset and saw a tabular data problem. Strategy is the Memory Patterns agent from Rock Paper Scissors. Solution is to build a lookup table. I didn't properly understand the competition metric.
\n\nOn the second day @josephassaker had a similar idea but simply searched for the formal dataset titles. This surprised him too when he got a 1.000 score. This method achieved 100% Precision.
\n\nOn the third day, Kaggle realized that this was not the solution the competition sponsors wanted, thus they reset the leaderboard and replaced the competition metric with the F0.5 score.
\nSo what exactly do Precision and Recall actually mean?
\n100% Precision means that every we gave an answer, it was always correct and we never got one wrong. We may however have failed to find all the items we were looking for.
\n100% Recall means that we have successfully found all the available matches in the dataset. but we may also have a big pile of additional false positives.
\nIf neither of these extremes is to your liking, then it is possible to blend them together. This is called the F score. This is most commonly implemented as the F1 score, which is the harmonic means of Precision and Recall.
\nF1 = 2 * (Precision * Recall) / (Precision + Recall)\nHowever, the competition sponsors were biased, they understood now the importance of Recall, but still, they had a preference for Precision. Thus they chose the F0.5 score, which is defined as thus:
\nF0.5 = ((1 + 0.5^2) * Precision * Recall) / (0.5^2 * Precision + Recall)\nF0.5 = (1.25 * Precision * Recall) / (0.25 * Precision + Recall)\nHad they wanted, they could have also chosen the F2 score, which is biased towards Recall. But they didn't.
\nF2 = ((1 + 2^2) * Precision * Recall) / (2^2 * Precision + Recall)\nF2 = (5 * Precision * Recall) / (4 * Precision + Recall)\nKaggle then reset the leaderboard. Ding! Ding! Round Two of the Competition!
\nJason Brownlee has done a more in-depth writeup of the F Score Metric here:
\n", "rawMarkdown": "The competition sponsors had decided that Precision was their metric of choice. \n\nWithin the first two days of this competition, we achieved a 1.000 score on the leaderboard. \n\nOn the first day of the competition, I looked at the dataset and saw a tabular data problem. Strategy is the Memory Patterns agent from Rock Paper Scissors. Solution is to build a lookup table. I didn't properly understand the competition metric.\n- https://www.kaggle.com/jamesmcguigan/coleridge-string-literals\n\nOn the second day @josephassaker had a similar idea but simply searched for the formal dataset titles. This surprised him too when he got a 1.000 score. This method achieved 100% Precision. \n- https://www.kaggle.com/josephassaker/coleridge-initiative-eda-na-ve-submission\n\nOn the third day, Kaggle realized that this was not the solution the competition sponsors wanted, thus they reset the leaderboard and replaced the competition metric with the F0.5 score.\n\nSo what exactly do Precision and Recall actually mean?\n\n100% Precision means that every we gave an answer, it was always correct and we never got one wrong. We may however have failed to find all the items we were looking for.\n\n100% Recall means that we have successfully found all the available matches in the dataset. but we may also have a big pile of additional false positives.\n\nIf neither of these extremes is to your liking, then it is possible to blend them together. This is called the F score. This is most commonly implemented as the F1 score, which is the harmonic means of Precision and Recall.\n\n```\nF1 = 2 * (Precision * Recall) / (Precision + Recall)\n```\n\nHowever, the competition sponsors were biased, they understood now the importance of Recall, but still, they had a preference for Precision. Thus they chose the F0.5 score, which is defined as thus:\n```\nF0.5 = ((1 + 0.5^2) * Precision * Recall) / (0.5^2 * Precision + Recall)\nF0.5 = (1.25 * Precision * Recall) / (0.25 * Precision + Recall)\n```\n\nHad they wanted, they could have also chosen the F2 score, which is biased towards Recall. But they didn't.\n```\nF2 = ((1 + 2^2) * Precision * Recall) / (2^2 * Precision + Recall)\nF2 = (5 * Precision * Recall) / (4 * Precision + Recall)\n```\n\nKaggle then reset the leaderboard. Ding! Ding! Round Two of the Competition!\n\n [Jason Brownlee](https://machinelearningmastery.com/) has done a more in-depth writeup of the F Score Metric here:\n- https://machinelearningmastery.com/fbeta-measure-for-machine-learning/", "votes": 4 } ], "comments": [], "raw_markdown_by_id": { "1258717": "The competition sponsors had decided that Precision was their metric of choice. \n\nWithin the first two days of this competition, we achieved a 1.000 score on the leaderboard. \n\nOn the first day of the competition, I looked at the dataset and saw a tabular data problem. Strategy is the Memory Patterns agent from Rock Paper Scissors. Solution is to build a lookup table. I didn't properly understand the competition metric.\n- https://www.kaggle.com/jamesmcguigan/coleridge-string-literals\n\nOn the second day @josephassaker had a similar idea but simply searched for the formal dataset titles. This surprised him too when he got a 1.000 score. This method achieved 100% Precision. \n- https://www.kaggle.com/josephassaker/coleridge-initiative-eda-na-ve-submission\n\nOn the third day, Kaggle realized that this was not the solution the competition sponsors wanted, thus they reset the leaderboard and replaced the competition metric with the F0.5 score.\n\nSo what exactly do Precision and Recall actually mean?\n\n100% Precision means that every we gave an answer, it was always correct and we never got one wrong. We may however have failed to find all the items we were looking for.\n\n100% Recall means that we have successfully found all the available matches in the dataset. but we may also have a big pile of additional false positives.\n\nIf neither of these extremes is to your liking, then it is possible to blend them together. This is called the F score. This is most commonly implemented as the F1 score, which is the harmonic means of Precision and Recall.\n\n```\nF1 = 2 * (Precision * Recall) / (Precision + Recall)\n```\n\nHowever, the competition sponsors were biased, they understood now the importance of Recall, but still, they had a preference for Precision. Thus they chose the F0.5 score, which is defined as thus:\n```\nF0.5 = ((1 + 0.5^2) * Precision * Recall) / (0.5^2 * Precision + Recall)\nF0.5 = (1.25 * Precision * Recall) / (0.25 * Precision + Recall)\n```\n\nHad they wanted, they could have also chosen the F2 score, which is biased towards Recall. But they didn't.\n```\nF2 = ((1 + 2^2) * Precision * Recall) / (2^2 * Precision + Recall)\nF2 = (5 * Precision * Recall) / (4 * Precision + Recall)\n```\n\nKaggle then reset the leaderboard. Ding! Ding! Round Two of the Competition!\n\n [Jason Brownlee](https://machinelearningmastery.com/) has done a more in-depth writeup of the F Score Metric here:\n- https://machinelearningmastery.com/fbeta-measure-for-machine-learning/" } }