![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2098454%2F948cd80d6ee074bf516547ed6c6b33f8%2F51f1be19e.jpg?generation=1593409916269235&alt=media\")
Hello everyone\ni trained yolov3 on wheat train dataset and visualized model performance on test dataset the model is detecting wheat heads on test dataset even though getting very low score of 0.0009 on leaderboard it would be very helpful if someone explain what's happening here.Here is the link of my kernel and prediction on test images.
\n\nhttps://www.kaggle.com/ravi02516/visualize-yolov3-darknet-prediction-on-test-images
\n\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2098454%2F5aea563b17c9e84db855e2bce87a3952%2F51b3e36ab.jpg?generation=1593409916611888&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2098454%2F6755356e8904396513609f955363defa%2F2fd875eaa.jpg?generation=1593409916883386&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2098454%2Fc48c0fed55e38a6707a9d462292c8911%2Fcb8d261a3.jpg?generation=1593409916873378&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2098454%2Fc22848a0254c6e7820627689a2e11e4f%2F348a992bb.jpg?generation=1593409917484315&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2098454%2Ff2597d2cd56dc984540a9288ce7c328d%2Faac893a91.jpg?generation=1593409918107349&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2098454%2F785c342a8b1c49d50bd448bebd397f53%2F53f253011.jpg?generation=1593409919265245&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2098454%2Ff06c1b85fcab76d1763536728949d895%2Fcc3532ff6.jpg?generation=1593409919913807&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2098454%2F8824956d5495663baee4a33f2c0b3caf%2Ff5a1f0358.jpg?generation=1593409920485333&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2098454%2Fbeba4f59f6c7ac916cb2c0a6f07d2214%2F796707dd7.jpg?generation=1593409936684123&alt=media\")
Hello everyone\ni trained yolov3 on wheat train dataset and visualized model performance on test dataset the model is detecting wheat heads on test dataset even though getting very low score of 0.0009 on leaderboard it would be very helpful if someone explain what's happening here.Here is the link of my kernel and prediction on test images.
\n\nhttps://www.kaggle.com/ravi02516/visualize-yolov3-darknet-prediction-on-test-images
\n\n
\n\n\n\n\n\n\n\n
there is a hidden dataset on which your model is tested which l gives the public score... \nI guess it is not able to detect the wheat heads over there...
", "rawMarkdown": "there is a hidden dataset on which your model is tested which l gives the public score... \nI guess it is not able to detect the wheat heads over there...", "votes": null }, { "id": "906433", "postDate": "06/29/2020 09:39:54", "content": "Dear @ravi02516 ,\nI think an error exists in the following cell: \n```python\nimport matplotlib.pyplot as plt\nprediction=[]\nimport pandas as pd\nsub_data=pd.read_csv(\"../input/global-wheat-detection/sample_submission.csv\")
\n\nfor img_id in sub_data[\"image_id\"]:\n imgpath=f\"../input/global-wheat-detection/test/{img_id}.jpg\"\n im=cv2.imread(imgpath)\n image, labels , cords= get_predictions(\"../input/trained-weights-and-cfg/labels.name\",im,net)\n plt.figure(figsize=(10,10))\n plt.imshow(image)\n plt.show()\n prediction.append(\" \".join(cords))
\n\nsub_data[\"PredictionString\"]=prediction\n```
\n\nIn this cell, you create your submission CSV file using the *sample_submission* as a way to loop over the dataset. However, at test time, this file is not populated with every image your algorithm should be run into. Instead, what you should do is loop over every existing image file of the input folder then make your submission. I would correct your code as follows:\n```\nimport os\nimg_ids = []\nfor f in os.listdir(\"/kaggle/input/global-wheat-detection/test/\"):\n if f.endswith(\".jpg\"):\n imgpath=f\"../input/global-wheat-detection/test/{f}\"\n im=cv2.imread(imgpath)\n image, labels , cords= get_predictions(\"../input/trained-weights-and-cfg/labels.name\",im,net)
\n\n # I advise NOT to plot your images at test time, there will be over 3000 images tested on, it may create an error in the final \n #plt.figure(figsize=(10,10))\n #plt.imshow(image)\n #plt.show()\n prediction.append(\" \".join(cords))\n img_ids.append(\n f.split(\".\")[0]\n )\nsub_data = pd.DataFrame({\n \"image_id\":img_ids,\n \"PredictionString\":prediction\n})
\n\nsub_data[\"PredictionString\"]=prediction\n```
", "rawMarkdown": "Dear @ravi02516 ,\nI think an error exists in the following cell: \n```python\nimport matplotlib.pyplot as plt\nprediction=[]\nimport pandas as pd\nsub_data=pd.read_csv(\"../input/global-wheat-detection/sample_submission.csv\")\n\nfor img_id in sub_data[\"image_id\"]:\n imgpath=f\"../input/global-wheat-detection/test/{img_id}.jpg\"\n im=cv2.imread(imgpath)\n image, labels , cords= get_predictions(\"../input/trained-weights-and-cfg/labels.name\",im,net)\n plt.figure(figsize=(10,10))\n plt.imshow(image)\n plt.show()\n prediction.append(\" \".join(cords))\n\n \n \nsub_data[\"PredictionString\"]=prediction\n```\n\nIn this cell, you create your submission CSV file using the *sample_submission* as a way to loop over the dataset. However, at test time, this file is not populated with every image your algorithm should be run into. Instead, what you should do is loop over every existing image file of the input folder then make your submission. I would correct your code as follows:\n```\nimport os\nimg_ids = []\nfor f in os.listdir(\"/kaggle/input/global-wheat-detection/test/\"):\n if f.endswith(\".jpg\"):\n imgpath=f\"../input/global-wheat-detection/test/{f}\"\n im=cv2.imread(imgpath)\n image, labels , cords= get_predictions(\"../input/trained-weights-and-cfg/labels.name\",im,net)\n\n # I advise NOT to plot your images at test time, there will be over 3000 images tested on, it may create an error in the final \n #plt.figure(figsize=(10,10))\n #plt.imshow(image)\n #plt.show()\n prediction.append(\" \".join(cords))\n img_ids.append(\n f.split(\".\")[0]\n )\n\nsub_data = pd.DataFrame({\n \"image_id\":img_ids,\n \"PredictionString\":prediction\n})\n\nsub_data[\"PredictionString\"]=prediction\n```", "votes": null }, { "id": "907729", "postDate": "06/30/2020 06:04:07", "content": "Thanks for answering but After replacing code snippet with your suggested snipped i am getting submission error how that possible??
", "rawMarkdown": "Thanks for answering but After replacing code snippet with your suggested snipped i am getting submission error how that possible??", "votes": null }, { "id": "907732", "postDate": "06/30/2020 06:08:05", "content": "Bro as per the detection in test images it should at least score more than 0.0009 it is so close to 0 means it did not detect anything.
", "rawMarkdown": "Bro as per the detection in test images it should at least score more than 0.0009 it is so close to 0 means it did not detect anything.", "votes": null }, { "id": "907742", "postDate": "06/30/2020 06:21:49", "content": "True that,\nI am not sure what's going wrong, there were some instances when my LB score was also 0.0018 because of image size mismatch...
", "rawMarkdown": "True that,\nI am not sure what's going wrong, there were some instances when my LB score was also 0.0018 because of image size mismatch...", "votes": null }, { "id": "907757", "postDate": "06/30/2020 06:35:58", "content": "Hm, sorry to hear that @ravi02516 ... I have struggled a lot with submission errors myself..
\n\nThey may be linked to the fact that, according to me, you were not executing your inference code on all test data. Indeed, once executing on all the dataset, you may encounter errors for any following reasons:\n- Not all images are sized 1024x1024\n- Not all images are predicted as having wheat heads (hence you need to take care of them by adding to your predictions an empty string for the image without any wheat heads)
\n\nA certain number of posts have been talking about issues related to these errors, you can check some out: https://www.kaggle.com/c/global-wheat-detection/discussion/149871
\n\nFurthermore, my code is not test-proof, I did not try executing it in any way, it stands more as a guideline for you to fix your issue. You should then make sure that it does what you want. For that intent, try executing your inference code on the training set to check if it lists correctly every image of the training set at the end of its computation.
\n\nWe can keep this thread to debug your code, I'd be happy to try to help.
\n\nBest regards and I wish you luck !
", "rawMarkdown": "Hm, sorry to hear that @ravi02516 ... I have struggled a lot with submission errors myself.. \n\nThey may be linked to the fact that, according to me, you were not executing your inference code on all test data. Indeed, once executing on all the dataset, you may encounter errors for any following reasons:\n- Not all images are sized 1024x1024\n- Not all images are predicted as having wheat heads (hence you need to take care of them by adding to your predictions an empty string for the image without any wheat heads)\n\nA certain number of posts have been talking about issues related to these errors, you can check some out: https://www.kaggle.com/c/global-wheat-detection/discussion/149871\n\nFurthermore, my code is not test-proof, I did not try executing it in any way, it stands more as a guideline for you to fix your issue. You should then make sure that it does what you want. For that intent, try executing your inference code on the **training set** to check if it lists correctly every image of the training set at the end of its computation.\n\nWe can keep this thread to debug your code, I'd be happy to try to help.\n\nBest regards and I wish you luck !", "votes": null }, { "id": "910635", "postDate": "07/01/2020 09:12:28", "content": "before writing to csv file using following code to format the string:\n```\ndef format_prediction_string(boxes, scores):\n pred_strings = []\n for j in zip(scores, boxes):\n pred_strings.append(\"{0:.4f} {1} {2} {3} {4}\".format(j[0], j[1][0], j[1][1], j[1][2], j[1][3]))
\n\nreturn \" \".join(pred_strings)\n```
\n\ncall the function here:
\n\n```\n result = {\n 'image_id': image_id,\n 'PredictionString': format_prediction_string(boxes, scores)\n }
\n\n results.append(result)\nreturn results\n```
", "rawMarkdown": "before writing to csv file using following code to format the string:\n```\ndef format_prediction_string(boxes, scores):\n pred_strings = []\n for j in zip(scores, boxes):\n pred_strings.append(\"{0:.4f} {1} {2} {3} {4}\".format(j[0], j[1][0], j[1][1], j[1][2], j[1][3]))\n\n return \" \".join(pred_strings)\n```\n\ncall the function here:\n\n ```\n result = {\n 'image_id': image_id,\n 'PredictionString': format_prediction_string(boxes, scores)\n }\n\n results.append(result)\n return results\n```", "votes": null }, { "id": "917132", "postDate": "07/06/2020 09:05:06", "content": "I was getting a similarly low score (0.0034) when I wasn't resizing the detection boxes correctly before submission
", "rawMarkdown": "I was getting a similarly low score (0.0034) when I wasn't resizing the detection boxes correctly before submission", "votes": null } ], "comments": [ { "id": 906228, "author_name": "nitindatta", "author_url": "", "post_date": "06/29/2020 06:21:31", "content": "there is a hidden dataset on which your model is tested which l gives the public score... \nI guess it is not able to detect the wheat heads over there...
", "votes": null, "replies": [ { "id": 907732, "author_name": "ravi02516", "author_url": "", "post_date": "06/30/2020 06:08:05", "content": "Bro as per the detection in test images it should at least score more than 0.0009 it is so close to 0 means it did not detect anything.
", "votes": null, "replies": [] }, { "id": 907742, "author_name": "nitindatta", "author_url": "", "post_date": "06/30/2020 06:21:49", "content": "True that,\nI am not sure what's going wrong, there were some instances when my LB score was also 0.0018 because of image size mismatch...
", "votes": null, "replies": [] }, { "id": 910635, "author_name": "ahmmadrashid", "author_url": "", "post_date": "07/01/2020 09:12:28", "content": "before writing to csv file using following code to format the string:\n```\ndef format_prediction_string(boxes, scores):\n pred_strings = []\n for j in zip(scores, boxes):\n pred_strings.append(\"{0:.4f} {1} {2} {3} {4}\".format(j[0], j[1][0], j[1][1], j[1][2], j[1][3]))
\n\nreturn \" \".join(pred_strings)\n```
\n\ncall the function here:
\n\n```\n result = {\n 'image_id': image_id,\n 'PredictionString': format_prediction_string(boxes, scores)\n }
\n\n results.append(result)\nreturn results\n```
", "votes": null, "replies": [] } ] }, { "id": 906433, "author_name": "dimartinot", "author_url": "", "post_date": "06/29/2020 09:39:54", "content": "Dear @ravi02516 ,\nI think an error exists in the following cell: \n```python\nimport matplotlib.pyplot as plt\nprediction=[]\nimport pandas as pd\nsub_data=pd.read_csv(\"../input/global-wheat-detection/sample_submission.csv\")
\n\nfor img_id in sub_data[\"image_id\"]:\n imgpath=f\"../input/global-wheat-detection/test/{img_id}.jpg\"\n im=cv2.imread(imgpath)\n image, labels , cords= get_predictions(\"../input/trained-weights-and-cfg/labels.name\",im,net)\n plt.figure(figsize=(10,10))\n plt.imshow(image)\n plt.show()\n prediction.append(\" \".join(cords))
\n\nsub_data[\"PredictionString\"]=prediction\n```
\n\nIn this cell, you create your submission CSV file using the *sample_submission* as a way to loop over the dataset. However, at test time, this file is not populated with every image your algorithm should be run into. Instead, what you should do is loop over every existing image file of the input folder then make your submission. I would correct your code as follows:\n```\nimport os\nimg_ids = []\nfor f in os.listdir(\"/kaggle/input/global-wheat-detection/test/\"):\n if f.endswith(\".jpg\"):\n imgpath=f\"../input/global-wheat-detection/test/{f}\"\n im=cv2.imread(imgpath)\n image, labels , cords= get_predictions(\"../input/trained-weights-and-cfg/labels.name\",im,net)
\n\n # I advise NOT to plot your images at test time, there will be over 3000 images tested on, it may create an error in the final \n #plt.figure(figsize=(10,10))\n #plt.imshow(image)\n #plt.show()\n prediction.append(\" \".join(cords))\n img_ids.append(\n f.split(\".\")[0]\n )\nsub_data = pd.DataFrame({\n \"image_id\":img_ids,\n \"PredictionString\":prediction\n})
\n\nsub_data[\"PredictionString\"]=prediction\n```
", "votes": null, "replies": [ { "id": 907729, "author_name": "ravi02516", "author_url": "", "post_date": "06/30/2020 06:04:07", "content": "Thanks for answering but After replacing code snippet with your suggested snipped i am getting submission error how that possible??
", "votes": null, "replies": [] }, { "id": 907757, "author_name": "dimartinot", "author_url": "", "post_date": "06/30/2020 06:35:58", "content": "Hm, sorry to hear that @ravi02516 ... I have struggled a lot with submission errors myself..
\n\nThey may be linked to the fact that, according to me, you were not executing your inference code on all test data. Indeed, once executing on all the dataset, you may encounter errors for any following reasons:\n- Not all images are sized 1024x1024\n- Not all images are predicted as having wheat heads (hence you need to take care of them by adding to your predictions an empty string for the image without any wheat heads)
\n\nA certain number of posts have been talking about issues related to these errors, you can check some out: https://www.kaggle.com/c/global-wheat-detection/discussion/149871
\n\nFurthermore, my code is not test-proof, I did not try executing it in any way, it stands more as a guideline for you to fix your issue. You should then make sure that it does what you want. For that intent, try executing your inference code on the training set to check if it lists correctly every image of the training set at the end of its computation.
\n\nWe can keep this thread to debug your code, I'd be happy to try to help.
\n\nBest regards and I wish you luck !
", "votes": null, "replies": [] } ] }, { "id": 917132, "author_name": "researn", "author_url": "", "post_date": "07/06/2020 09:05:06", "content": "I was getting a similarly low score (0.0034) when I wasn't resizing the detection boxes correctly before submission
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "906189": "Hello everyone\ni trained yolov3 on wheat train dataset and visualized model performance on test dataset the model is detecting wheat heads on test dataset even though getting very low score of 0.0009 on leaderboard it would be very helpful if someone explain what's happening here.Here is the link of my kernel and prediction on test images.\n\nhttps://www.kaggle.com/ravi02516/visualize-yolov3-darknet-prediction-on-test-images\n\n\n\n\n\n\n\n\n\n\n\n\n", "906228": "there is a hidden dataset on which your model is tested which l gives the public score... \nI guess it is not able to detect the wheat heads over there...", "906433": "Dear @ravi02516 ,\nI think an error exists in the following cell: \n```python\nimport matplotlib.pyplot as plt\nprediction=[]\nimport pandas as pd\nsub_data=pd.read_csv(\"../input/global-wheat-detection/sample_submission.csv\")\n\nfor img_id in sub_data[\"image_id\"]:\n imgpath=f\"../input/global-wheat-detection/test/{img_id}.jpg\"\n im=cv2.imread(imgpath)\n image, labels , cords= get_predictions(\"../input/trained-weights-and-cfg/labels.name\",im,net)\n plt.figure(figsize=(10,10))\n plt.imshow(image)\n plt.show()\n prediction.append(\" \".join(cords))\n\n \n \nsub_data[\"PredictionString\"]=prediction\n```\n\nIn this cell, you create your submission CSV file using the *sample_submission* as a way to loop over the dataset. However, at test time, this file is not populated with every image your algorithm should be run into. Instead, what you should do is loop over every existing image file of the input folder then make your submission. I would correct your code as follows:\n```\nimport os\nimg_ids = []\nfor f in os.listdir(\"/kaggle/input/global-wheat-detection/test/\"):\n if f.endswith(\".jpg\"):\n imgpath=f\"../input/global-wheat-detection/test/{f}\"\n im=cv2.imread(imgpath)\n image, labels , cords= get_predictions(\"../input/trained-weights-and-cfg/labels.name\",im,net)\n\n # I advise NOT to plot your images at test time, there will be over 3000 images tested on, it may create an error in the final \n #plt.figure(figsize=(10,10))\n #plt.imshow(image)\n #plt.show()\n prediction.append(\" \".join(cords))\n img_ids.append(\n f.split(\".\")[0]\n )\n\nsub_data = pd.DataFrame({\n \"image_id\":img_ids,\n \"PredictionString\":prediction\n})\n\nsub_data[\"PredictionString\"]=prediction\n```", "907729": "Thanks for answering but After replacing code snippet with your suggested snipped i am getting submission error how that possible??", "907732": "Bro as per the detection in test images it should at least score more than 0.0009 it is so close to 0 means it did not detect anything.", "907742": "True that,\nI am not sure what's going wrong, there were some instances when my LB score was also 0.0018 because of image size mismatch...", "907757": "Hm, sorry to hear that @ravi02516 ... I have struggled a lot with submission errors myself.. \n\nThey may be linked to the fact that, according to me, you were not executing your inference code on all test data. Indeed, once executing on all the dataset, you may encounter errors for any following reasons:\n- Not all images are sized 1024x1024\n- Not all images are predicted as having wheat heads (hence you need to take care of them by adding to your predictions an empty string for the image without any wheat heads)\n\nA certain number of posts have been talking about issues related to these errors, you can check some out: https://www.kaggle.com/c/global-wheat-detection/discussion/149871\n\nFurthermore, my code is not test-proof, I did not try executing it in any way, it stands more as a guideline for you to fix your issue. You should then make sure that it does what you want. For that intent, try executing your inference code on the **training set** to check if it lists correctly every image of the training set at the end of its computation.\n\nWe can keep this thread to debug your code, I'd be happy to try to help.\n\nBest regards and I wish you luck !", "910635": "before writing to csv file using following code to format the string:\n```\ndef format_prediction_string(boxes, scores):\n pred_strings = []\n for j in zip(scores, boxes):\n pred_strings.append(\"{0:.4f} {1} {2} {3} {4}\".format(j[0], j[1][0], j[1][1], j[1][2], j[1][3]))\n\n return \" \".join(pred_strings)\n```\n\ncall the function here:\n\n ```\n result = {\n 'image_id': image_id,\n 'PredictionString': format_prediction_string(boxes, scores)\n }\n\n results.append(result)\n return results\n```", "917132": "I was getting a similarly low score (0.0034) when I wasn't resizing the detection boxes correctly before submission" }, "source": "meta" }