I was carefully reviewing the evaluation metrics of this competition.\nAccording to the explanation, there are clear guidelines on how to calculate the IoU, and then there are steps of threshold that will be used to calculate the True Positive, False Positive, and False Negative. All of this is clear, however the evaluation state that a precision average will be used. \nThe formula for precision is given as TP /(TP +FP + FN) \nbased on my experience and formal data science study, the formal precision formula is TP/(TP+FP). Can someone from Kaggle approve that the formula which is given is the one that is used or the standard precision formula is used?\nI joined this competition recently, and this topic may have discussed, but I did not find any reference to this in the discussion, so I apologize in advance if it has been discussed and answered in the past
", "messages": [ { "id": "914430", "postDate": "07/03/2020 21:31:18", "content": "I was carefully reviewing the evaluation metrics of this competition.\nAccording to the explanation, there are clear guidelines on how to calculate the IoU, and then there are steps of threshold that will be used to calculate the True Positive, False Positive, and False Negative. All of this is clear, however the evaluation state that a precision average will be used. \nThe formula for precision is given as TP /(TP +FP + FN) \nbased on my experience and formal data science study, the formal precision formula is TP/(TP+FP). Can someone from Kaggle approve that the formula which is given is the one that is used or the standard precision formula is used?\nI joined this competition recently, and this topic may have discussed, but I did not find any reference to this in the discussion, so I apologize in advance if it has been discussed and answered in the past
", "rawMarkdown": "I was carefully reviewing the evaluation metrics of this competition.\nAccording to the explanation, there are clear guidelines on how to calculate the IoU, and then there are steps of threshold that will be used to calculate the True Positive, False Positive, and False Negative. All of this is clear, however the evaluation state that a precision average will be used. \nThe formula for precision is given as TP /(TP +FP + FN) \nbased on my experience and formal data science study, the formal precision formula is TP/(TP+FP). Can someone from Kaggle approve that the formula which is given is the one that is used or the standard precision formula is used?\nI joined this competition recently, and this topic may have discussed, but I did not find any reference to this in the discussion, so I apologize in advance if it has been discussed and answered in the past", "votes": null }, { "id": "914517", "postDate": "07/04/2020 01:54:11", "content": "This point has been brought up before (here and here), though it hasn't been addressed by the competition host. So I'm not sure if this is actually the metric they are using or if there is some reason behind selecting it.
", "rawMarkdown": "This point has been brought up before ([here](https://www.kaggle.com/c/global-wheat-detection/discussion/156874) and [here](https://www.kaggle.com/c/global-wheat-detection/discussion/159767)), though it hasn't been addressed by the competition host. So I'm not sure if this is actually the metric they are using or if there is some reason behind selecting it.", "votes": null }, { "id": "914726", "postDate": "07/04/2020 07:18:32", "content": "Thanks for providing the relevant links - maybe somone f from the organizer will comment on this topic this time
", "rawMarkdown": "Thanks for providing the relevant links - maybe somone f from the organizer will comment on this topic this time", "votes": null }, { "id": "915068", "postDate": "07/04/2020 13:02:01", "content": "Hi OmerS @omershect , My conclusion is that Kaggle simply wants to use this new metric which is not actually \"mAP\" but they call \"mAP\" anyway. This metric is sensible though and a bit less strict than the standard mAP metric.
", "rawMarkdown": "Hi OmerS @omershect , My conclusion is that Kaggle simply wants to use this new metric which is not actually \"mAP\" but they call \"mAP\" anyway. This metric is sensible though and a bit less strict than the standard mAP metric.", "votes": null }, { "id": "915423", "postDate": "07/04/2020 18:10:40", "content": "Thanks - I saw your detailed post. at this stage, I will stick with what they published
", "rawMarkdown": "Thanks - I saw your detailed post. at this stage, I will stick with what they published", "votes": null } ], "comments": [ { "id": 914517, "author_name": "matthewmasters", "author_url": "", "post_date": "07/04/2020 01:54:11", "content": "This point has been brought up before (here and here), though it hasn't been addressed by the competition host. So I'm not sure if this is actually the metric they are using or if there is some reason behind selecting it.
", "votes": null, "replies": [ { "id": 914726, "author_name": "omershect", "author_url": "", "post_date": "07/04/2020 07:18:32", "content": "Thanks for providing the relevant links - maybe somone f from the organizer will comment on this topic this time
", "votes": null, "replies": [] }, { "id": 915068, "author_name": "ratthachat", "author_url": "", "post_date": "07/04/2020 13:02:01", "content": "Hi OmerS @omershect , My conclusion is that Kaggle simply wants to use this new metric which is not actually \"mAP\" but they call \"mAP\" anyway. This metric is sensible though and a bit less strict than the standard mAP metric.
", "votes": null, "replies": [] }, { "id": 915423, "author_name": "omershect", "author_url": "", "post_date": "07/04/2020 18:10:40", "content": "Thanks - I saw your detailed post. at this stage, I will stick with what they published
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "914430": "I was carefully reviewing the evaluation metrics of this competition.\nAccording to the explanation, there are clear guidelines on how to calculate the IoU, and then there are steps of threshold that will be used to calculate the True Positive, False Positive, and False Negative. All of this is clear, however the evaluation state that a precision average will be used. \nThe formula for precision is given as TP /(TP +FP + FN) \nbased on my experience and formal data science study, the formal precision formula is TP/(TP+FP). Can someone from Kaggle approve that the formula which is given is the one that is used or the standard precision formula is used?\nI joined this competition recently, and this topic may have discussed, but I did not find any reference to this in the discussion, so I apologize in advance if it has been discussed and answered in the past", "914517": "This point has been brought up before ([here](https://www.kaggle.com/c/global-wheat-detection/discussion/156874) and [here](https://www.kaggle.com/c/global-wheat-detection/discussion/159767)), though it hasn't been addressed by the competition host. So I'm not sure if this is actually the metric they are using or if there is some reason behind selecting it.", "914726": "Thanks for providing the relevant links - maybe somone f from the organizer will comment on this topic this time", "915068": "Hi OmerS @omershect , My conclusion is that Kaggle simply wants to use this new metric which is not actually \"mAP\" but they call \"mAP\" anyway. This metric is sensible though and a bit less strict than the standard mAP metric.", "915423": "Thanks - I saw your detailed post. at this stage, I will stick with what they published" }, "source": "meta" }