![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1705102%2F583f904a410bdce9db2bdc1de461936b%2Fcode1.png?generation=1595486645825845&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1705102%2Fe7f89be64b1e2f426dea18e505abb647%2Fcode2.png?generation=1595486644524495&alt=media\")
Hi, everyone. I found something that doesn't make sense.
\n\nTo make it clear, post a few pieces of code. May look bloated😑
\n\nIn the inference, two thresholds are involved：
\n\nscore_threshold in the make_tta_predictions functionskip_box_threshold in the run_wbffunction. Because the make_tta_predictions function is executed first, as long as the score_threshold is any value less than skip_box_threshold, it is actually not effective, because run_wbf will filter the boxes again.
But from my experimental results, this is not the case. For example, when I set skip_box_threshold to 0.4 and I changed score_threshold from 0.25 to 0.37, my score improves. why? Does anyone have the same problem?
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1705102%2F8ef31c41dbb263231c977ea00ac58ba5%2Fcode3.png?generation=1595486693874657&alt=media\")
Hi, everyone. I found something that doesn't make sense.
\n\nTo make it clear, post a few pieces of code. May look bloated😑
\n\nIn the inference, two thresholds are involved：
\n\nscore_threshold in the make_tta_predictions functionskip_box_threshold in the run_wbffunction. Because the make_tta_predictions function is executed first, as long as the score_threshold is any value less than skip_box_threshold, it is actually not effective, because run_wbf will filter the boxes again.
But from my experimental results, this is not the case. For example, when I set skip_box_threshold to 0.4 and I changed score_threshold from 0.25 to 0.37, my score improves. why? Does anyone have the same problem?
\n
after run_wbf , the score is changed, for example:\n1 image -> 6 images( rotate , flip) , 2 of them got a 0.9 box each ,other 4 got nothing, then wbf out 1.8/6 = 0.3
", "rawMarkdown": "after run_wbf , the score is changed, for example:\n1 image -> 6 images( rotate , flip) , 2 of them got a 0.9 box each ,other 4 got nothing, then wbf out 1.8/6 = 0.3", "replies": [ { "id": 947057, "postDate": "2020-07-27T03:54:10.390Z", "content": "Yes, after run_wbf , the score is changed. But before boxes fusion, prefilter_boxes is executed, please see here
score_threshold from 0.25 to 0.37, is after run_wbf, prefilter_boxes is 0.4 is effective for input of run_wbf , so your result changes.
", "rawMarkdown": "score_threshold from 0.25 to 0.37, is after run_wbf, prefilter_boxes is 0.4 is effective for input of run_wbf , so your result changes." }, { "id": 947176, "postDate": "2020-07-27T05:46:11.880Z", "content": "Sorry, could you make it clear? My work follows this
", "rawMarkdown": "Sorry, could you make it clear? My work follows [this](https://www.kaggle.com/raininbox/efficientdet-with-pseduo-labeling?scriptVersionId=34833370)" }, { "id": 947337, "postDate": "2020-07-27T07:33:52.333Z", "content": "I think the code you use is not very good operations\nit should be : tta->collect all predictions-> wbf -> use score_threshold to filter again.
\n\nthe code you use : tta-> filter with socre_threshold-> wbf.
\n\nif you encounter the 0.4 not effective, maybe under the code you havn't show , uses score_threshold again.
", "rawMarkdown": "I think the code you use is not very good operations\nit should be : tta->collect all predictions-> wbf -> use score_threshold to filter again.\n\nthe code you use : tta-> filter with socre_threshold-> wbf.\n\nif you encounter the 0.4 not effective, maybe under the code you havn't show , uses score_threshold again." }, { "id": 947343, "postDate": "2020-07-27T07:40:04.647Z", "content": "Thanks, I'll try:)
", "rawMarkdown": "Thanks, I'll try:)" } ] }, { "id": 947002, "postDate": "2020-07-27T02:17:15.843Z", "content": "that is strange indeed, can you reproduce this? like you said, score_threshold from 0.25 to 0.37 should not change under skip_box_threshold 0.4 since they are both less than 0.4 and anything < 0.4 all get filtered in the prefilter_boxes function, was your seed fixed or anything possibly changing?
I haven't do a experiment to reproduce it. Seed was fixed.
", "rawMarkdown": "I haven't do a experiment to reproduce it. Seed was fixed." } ] }, { "id": 943428, "postDate": "2020-07-24T11:01:05.037Z", "content": "@guizengyou \nThanks for posting this..\nI will let you know if i find this
\n\nBtw. \nCan you help me understand if are training at less than full res image,then how bounding box predictions are scaled during inference to match ground truth that are based upon full res images ?
", "rawMarkdown": "@guizengyou \nThanks for posting this..\nI will let you know if i find this\n\nBtw. \nCan you help me understand if are training at less than full res image,then how bounding box predictions are scaled during inference to match ground truth that are based upon full res images ?", "replies": [ { "id": 944280, "postDate": "2020-07-25T01:54:10.660Z", "content": "Yes, you can reference shonenkov's notebook
", "rawMarkdown": "Yes, you can reference [shonenkov's](https://www.kaggle.com/shonenkov) [notebook](https://www.kaggle.com/shonenkov/inference-efficientdet/output)" } ] }, { "id": 939076, "postDate": "2020-07-22T02:40:16.017Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 947045, "author_name": "Alex_Wang", "author_url": "", "post_date": "2020-07-27T03:34:04.883000", "content": "after run_wbf , the score is changed, for example:\n1 image -> 6 images( rotate , flip) , 2 of them got a 0.9 box each ,other 4 got nothing, then wbf out 1.8/6 = 0.3
", "votes": 0, "replies": [ { "id": 947057, "author_name": "gzYou", "author_url": "", "post_date": "2020-07-27T03:54:10.390000", "content": "Yes, after run_wbf , the score is changed. But before boxes fusion, prefilter_boxes is executed, please see here
score_threshold from 0.25 to 0.37, is after run_wbf, prefilter_boxes is 0.4 is effective for input of run_wbf , so your result changes.
", "votes": 0, "replies": [] }, { "id": 947176, "author_name": "gzYou", "author_url": "", "post_date": "2020-07-27T05:46:11.880000", "content": "Sorry, could you make it clear? My work follows this
", "votes": 0, "replies": [] }, { "id": 947337, "author_name": "Alex_Wang", "author_url": "", "post_date": "2020-07-27T07:33:52.333000", "content": "I think the code you use is not very good operations\nit should be : tta->collect all predictions-> wbf -> use score_threshold to filter again.
\n\nthe code you use : tta-> filter with socre_threshold-> wbf.
\n\nif you encounter the 0.4 not effective, maybe under the code you havn't show , uses score_threshold again.
", "votes": 0, "replies": [] }, { "id": 947343, "author_name": "gzYou", "author_url": "", "post_date": "2020-07-27T07:40:04.647000", "content": "Thanks, I'll try:)
", "votes": 0, "replies": [] } ] }, { "id": 947002, "author_name": "samshipengs", "author_url": "", "post_date": "2020-07-27T02:17:15.843000", "content": "that is strange indeed, can you reproduce this? like you said, score_threshold from 0.25 to 0.37 should not change under skip_box_threshold 0.4 since they are both less than 0.4 and anything < 0.4 all get filtered in the prefilter_boxes function, was your seed fixed or anything possibly changing?
I haven't do a experiment to reproduce it. Seed was fixed.
", "votes": 0, "replies": [] } ] }, { "id": 943428, "author_name": "Jaideep", "author_url": "", "post_date": "2020-07-24T11:01:05.037000", "content": "@guizengyou \nThanks for posting this..\nI will let you know if i find this
\n\nBtw. \nCan you help me understand if are training at less than full res image,then how bounding box predictions are scaled during inference to match ground truth that are based upon full res images ?
", "votes": 0, "replies": [ { "id": 944280, "author_name": "gzYou", "author_url": "", "post_date": "2020-07-25T01:54:10.660000", "content": "Yes, you can reference shonenkov's notebook
", "votes": 0, "replies": [] } ] }, { "id": 939076, "author_name": "", "author_url": "", "post_date": "2020-07-22T02:40:16.017000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "939070": "Hi, everyone. I found something that doesn't make sense.\n\nTo make it clear, post a few pieces of code. May look bloated😑\n\nIn the inference, two thresholds are involved：\n\n1. `score_threshold` in the `make_tta_predictions` function\n2. `skip_box_threshold` in the `run_wbf `function. \n\nBecause the `make_tta_predictions` function is executed first, as long as the `score_threshold` is any value less than `skip_box_threshold`, it is actually not effective, because `run_wbf` will filter the boxes again. \n\nBut from my experimental results, this is not the case. For example, when I set `skip_box_threshold` to 0.4 and I changed `score_threshold` from 0.25 to 0.37, my score improves. why? Does anyone have the same problem?\n\n\n\n\n\n", "947045": "after run_wbf , the score is changed, for example:\n1 image -> 6 images( rotate , flip) , 2 of them got a 0.9 box each ,other 4 got nothing, then wbf out 1.8/6 = 0.3", "947002": "that is strange indeed, can you reproduce this? like you said, `score_threshold` from 0.25 to 0.37 should not change under `skip_box_threshold` 0.4 since they are both less than 0.4 and anything < 0.4 all get filtered in the `prefilter_boxes` function, was your seed fixed or anything possibly changing?", "943428": "@guizengyou \nThanks for posting this..\nI will let you know if i find this\n\nBtw. \nCan you help me understand if are training at less than full res image,then how bounding box predictions are scaled during inference to match ground truth that are based upon full res images ?", "939076": "" } }