Hi All,
\n\nI'm facing issue while submitting, I receive \"Submission scoring error\". I have an inference notebook for submitting predictions. So far I have checked the below:\n1. Internet on inference notebook is turned off\n2. Tried with GPU on / off\n3. Submission file has 10 rows apart from the headers\n4. Submission file has 2 columns (sample entry as below)\n5. There is an empty line at the end.
\n\nAny guidance / help would be appreciated. I have been struggling since a day
\n\nSample submission entry:\nimage_id,PredictionString\naac893a91,0.9916 614 913 82 110 0.9883 74 0 93 163\n51f1be19e,0.9780 282 473 128 118 0.9721 0 376 57 107 0.9664 768 884 152 101\nf5a1f0358,0.9701 152 246 79 96 0.9665 540 269 110 118\n796707dd7,0.9506 0 451 69 76 0.9462 894 331 106 95 0.9275 707 832 100 97\n51b3e36ab,0.9909 0 906 82 117 0.9889 417 930 89 93 0.9888 236 640 90 167\n348a992bb,0.9929 1 317 119 101 0.9873 0 471 137 100\ncc3532ff6,0.9869 0 404 151 110 0.9854 768 818 173 171\n2fd875eaa,0.9924 942 646 80 105 0.9905 103 587 143 86 0.9897 456 490 84 143\ncb8d261a3,0.9661 24 858 79 148 0.9402 751 710 88 91\n53f253011,0.9877 930 199 93 144 0.9807 910 743 74 88
\n\nCheers,\nBingi
", "messages": [ { "id": 932840, "postDate": "2020-07-17T10:30:02.663Z", "content": "Hi All,
\n\nI'm facing issue while submitting, I receive \"Submission scoring error\". I have an inference notebook for submitting predictions. So far I have checked the below:\n1. Internet on inference notebook is turned off\n2. Tried with GPU on / off\n3. Submission file has 10 rows apart from the headers\n4. Submission file has 2 columns (sample entry as below)\n5. There is an empty line at the end.
\n\nAny guidance / help would be appreciated. I have been struggling since a day
\n\nSample submission entry:\nimage_id,PredictionString\naac893a91,0.9916 614 913 82 110 0.9883 74 0 93 163\n51f1be19e,0.9780 282 473 128 118 0.9721 0 376 57 107 0.9664 768 884 152 101\nf5a1f0358,0.9701 152 246 79 96 0.9665 540 269 110 118\n796707dd7,0.9506 0 451 69 76 0.9462 894 331 106 95 0.9275 707 832 100 97\n51b3e36ab,0.9909 0 906 82 117 0.9889 417 930 89 93 0.9888 236 640 90 167\n348a992bb,0.9929 1 317 119 101 0.9873 0 471 137 100\ncc3532ff6,0.9869 0 404 151 110 0.9854 768 818 173 171\n2fd875eaa,0.9924 942 646 80 105 0.9905 103 587 143 86 0.9897 456 490 84 143\ncb8d261a3,0.9661 24 858 79 148 0.9402 751 710 88 91\n53f253011,0.9877 930 199 93 144 0.9807 910 743 74 88
\n\nCheers,\nBingi
", "rawMarkdown": "Hi All,\n\nI'm facing issue while submitting, I receive \"Submission scoring error\". I have an inference notebook for submitting predictions. So far I have checked the below:\n1. Internet on inference notebook is turned off\n2. Tried with GPU on / off\n3. Submission file has 10 rows apart from the headers\n4. Submission file has 2 columns (sample entry as below)\n5. There is an empty line at the end.\n\nAny guidance / help would be appreciated. I have been struggling since a day\n\nSample submission entry:\nimage_id,PredictionString\naac893a91,0.9916 614 913 82 110 0.9883 74 0 93 163\n51f1be19e,0.9780 282 473 128 118 0.9721 0 376 57 107 0.9664 768 884 152 101\nf5a1f0358,0.9701 152 246 79 96 0.9665 540 269 110 118\n796707dd7,0.9506 0 451 69 76 0.9462 894 331 106 95 0.9275 707 832 100 97\n51b3e36ab,0.9909 0 906 82 117 0.9889 417 930 89 93 0.9888 236 640 90 167\n348a992bb,0.9929 1 317 119 101 0.9873 0 471 137 100\ncc3532ff6,0.9869 0 404 151 110 0.9854 768 818 173 171\n2fd875eaa,0.9924 942 646 80 105 0.9905 103 587 143 86 0.9897 456 490 84 143\ncb8d261a3,0.9661 24 858 79 148 0.9402 751 710 88 91\n53f253011,0.9877 930 199 93 144 0.9807 910 743 74 88\n\nCheers,\nBingi", "votes": 4 }, { "id": 942923, "postDate": "2020-07-24T04:22:23.720Z", "content": "I could fix my problem at last, just posting it here hoping it might help others.\nThere were two things causing the problem:\n1. I was doing \"Quick Save\" most of the times while saving the versions while the instructions say that we have to do \"Save and Run all\"\n2. I was adding RGB values of all test images into a list to visualise the object detection using matplotlib, as the public test images were only 10, I never got an error when ran on 10 images but there were 1000s of images it failed as CUDA / CPU was not sufficient. So its worth a while to examine all variables being used and ensure they do not cause memory crashes
\n\nOne other thing I tried and was not sure if it made any difference was - I has my own datasets for offline packages and pretrained weights which were private before and I made them to public.
\n\nThanks for all your suggestions. Sorry if any of my \"cause of issue\" sound dumb to you :)
", "rawMarkdown": "I could fix my problem at last, just posting it here hoping it might help others.\nThere were two things causing the problem:\n1. I was doing \"Quick Save\" most of the times while saving the versions while the instructions say that we have to do \"Save and Run all\"\n2. I was adding RGB values of all test images into a list to visualise the object detection using matplotlib, as the public test images were only 10, I never got an error when ran on 10 images but there were 1000s of images it failed as CUDA / CPU was not sufficient. So its worth a while to examine all variables being used and ensure they do not cause memory crashes\n\nOne other thing I tried and was not sure if it made any difference was - I has my own datasets for offline packages and pretrained weights which were private before and I made them to public.\n\nThanks for all your suggestions. Sorry if any of my \"cause of issue\" sound dumb to you :)", "votes": 1 }, { "id": 933377, "postDate": "2020-07-17T17:17:28.747Z", "content": "Please try the suggestions here first: Submission scoring error/submission CSV not found
", "rawMarkdown": "Please try the suggestions here first: [Submission scoring error/submission CSV not found](https://www.kaggle.com/c/global-wheat-detection/discussion/149871)", "votes": 2 }, { "id": 941040, "postDate": "2020-07-23T06:07:25.887Z", "content": "I think the data in submission stage have only test folder. If you are using some codes like read_csv('sample.csv'), it can raise the issue. I'm not sure about this, but it worked for my case.
", "rawMarkdown": "I think the data in submission stage have only test folder. If you are using some codes like read_csv('sample.csv'), it can raise the issue. I'm not sure about this, but it worked for my case." }, { "id": 937553, "postDate": "2020-07-21T04:38:26.700Z", "content": "By now, I have tried all the things I can and I'm running out of ideas. Few things I have tried:
\n\nIt is really frustrating now :) any help would be really appreciated.
", "rawMarkdown": "By now, I have tried all the things I can and I'm running out of ideas. Few things I have tried:\n\n1. Took a working inference notebook - https://www.kaggle.com/shonenkov/inference-efficientdet and replaced the logic of the model to use mine and left the remaining part of the code intact. Still same issue.\n2. Ensure the co-ordinates are between 1 and 1023. No luck.\n3. Added try: except: while generating co-ordinates and formatting the csv. Tested the same by triggering some error and could see the csv file to have empty strings whenver error was occured. Still no luck.\n\nIt is really frustrating now :) any help would be really appreciated." }, { "id": 934042, "postDate": "2020-07-18T07:47:48.823Z", "content": "did you put empty string where nothing is detected??
", "rawMarkdown": "did you put empty string where nothing is detected??", "replies": [ { "id": 934066, "postDate": "2020-07-18T08:02:05.253Z", "content": "my test results always had something detected.
", "rawMarkdown": "my test results always had something detected." }, { "id": 934075, "postDate": "2020-07-18T08:04:09.053Z", "content": "You wouldn't be able to see that for the whole test set. When you submit, your kernel is rerun with the full test set, not just the 10 public images.
", "rawMarkdown": "You wouldn't be able to see that for the whole test set. When you submit, your kernel is rerun with the full test set, not just the 10 public images." }, { "id": 934115, "postDate": "2020-07-18T08:41:26.047Z", "content": "@matthewmasters can we see any log file generated after running the submission on the whole test set? It will be very helpful to know what is the exact issue causing the 'Submission CSV Not Found' error.
", "rawMarkdown": "@matthewmasters can we see any log file generated after running the submission on the whole test set? It will be very helpful to know what is the exact issue causing the 'Submission CSV Not Found' error." }, { "id": 934125, "postDate": "2020-07-18T08:48:41.690Z", "content": "I don't believe so. I think a log is problematic as it could be used to leak private test data. There is likely some code in your notebook that throws an error when run on the full set. My suggestion is to simplify your notebook as much as possible to isolate the issue or base your submission notebook off one of the public inference notebooks that works.
", "rawMarkdown": "I don't believe so. I think a log is problematic as it could be used to leak private test data. There is likely some code in your notebook that throws an error when run on the full set. My suggestion is to simplify your notebook as much as possible to isolate the issue or base your submission notebook off one of the public inference notebooks that works." }, { "id": 934133, "postDate": "2020-07-18T08:52:33.103Z", "content": "Logs are always helpful in debugging and we can encrypt sensitive information for not disclosing. If we do not face any issue with public data while submitting then how can we trace and diagnose the error in test data? Host can hide private test details and can show errors in the logs.
", "rawMarkdown": "Logs are always helpful in debugging and we can encrypt sensitive information for not disclosing. If we do not face any issue with public data while submitting then how can we trace and diagnose the error in test data? Host can hide private test details and can show errors in the logs." }, { "id": 934140, "postDate": "2020-07-18T08:57:19.710Z", "content": "I agree they would be helpful. But unfortunately I don't think it's something Kaggle will do, and for good reason. What I mean by leaking is that someone could cheat by outputting information about the private test data in the logs that would help them in their submission. This would be hard to protect against from Kaggle's side.
", "rawMarkdown": "I agree they would be helpful. But unfortunately I don't think it's something Kaggle will do, and for good reason. What I mean by leaking is that someone could cheat by outputting information about the private test data in the logs that would help them in their submission. This would be hard to protect against from Kaggle's side.", "votes": 2 }, { "id": 934148, "postDate": "2020-07-18T09:04:01.040Z", "content": "If the host can put read-only access to the logs file then maybe no one can alter this and they can show only errors in the logs, rest other things they can encrypt. Think about the username/password or credit card details- they all are in an application log but in encrypted form. In spite of security risk developers maintain the logs so that they can easily solve the issue in production. Maybe I am not right wrt Kaggle but it's my opinion to share the logs after the final submission.
", "rawMarkdown": "If the host can put read-only access to the logs file then maybe no one can alter this and they can show only errors in the logs, rest other things they can encrypt. Think about the username/password or credit card details- they all are in an application log but in encrypted form. In spite of security risk developers maintain the logs so that they can easily solve the issue in production. Maybe I am not right wrt Kaggle but it's my opinion to share the logs after the final submission." }, { "id": 934159, "postDate": "2020-07-18T09:12:52.643Z", "content": "It’s complicated by the fact that we are the ones writing the code that is run. One could for example do something like\n\nassert(True == False, “private test set detail”)\n\nI mean maybe Kaggle could engineer some python interpreter which encrypts variables but that seems like a whole lot of work for something which probably has other loopholes.
We have to deal with the situation as is and do our best to troubleshoot “blindly”.
", "rawMarkdown": "It’s complicated by the fact that we are the ones writing the code that is run. One could for example do something like\n```\nassert(True == False, “private test set detail”)\n```\nI mean maybe Kaggle could engineer some python interpreter which encrypts variables but that seems like a whole lot of work for something which probably has other loopholes.\n\nWe have to deal with the situation as is and do our best to troubleshoot “blindly”.", "votes": 2 }, { "id": 934162, "postDate": "2020-07-18T09:15:06.067Z", "content": "Yes, I also think so. We have to deal with it.
", "rawMarkdown": "Yes, I also think so. We have to deal with it." }, { "id": 934679, "postDate": "2020-07-18T16:24:29.733Z", "content": "okay. I have tried with the test set to ensure the output file has empty string when wheat is not detected. I'm still getting error for test set.
", "rawMarkdown": "okay. I have tried with the test set to ensure the output file has empty string when wheat is not detected. I'm still getting error for test set." }, { "id": 934781, "postDate": "2020-07-18T18:11:17.670Z", "content": "That file seems to have one issue, you have “.jpg” included in the image_id. Also make sure that your xmin and ymin are between 0 and 1023 and your height/width is non-zero
", "rawMarkdown": "That file seems to have one issue, you have “.jpg” included in the image_id. Also make sure that your xmin and ymin are between 0 and 1023 and your height/width is non-zero" }, { "id": 935026, "postDate": "2020-07-19T03:13:32.420Z", "content": "thanks Matthew, i have also tried without .jpg. I will double check the values as well.
", "rawMarkdown": "thanks Matthew, i have also tried without .jpg. I will double check the values as well." } ] }, { "id": 933523, "postDate": "2020-07-17T18:41:21.930Z", "content": "Not sure but try to recheck your submission file, it may have an extra column for index. Also, you can export your CSV file to you system and then submit it. So, take your CSV then jus do\n\nDataFrame_name.to_csv(\"submission\", index=False)\n\nNow you could upload that file.\nNot sure but hope it helps.
We cannot submit the CSV file from our system for this competition.
", "rawMarkdown": "We cannot submit the CSV file from our system for this competition." } ] }, { "id": 933028, "postDate": "2020-07-17T12:38:05.637Z", "content": "Have you tried to run your notebook on images which don't have any wheat?\nI created a sample dataset (https://www.kaggle.com/vudangthinh/gwd-test), which includes images without wheats.\nYou can test your notebook on this sample dataset to check your submission file.
\n\nIn addition, you should check other threads, there are many people having the same problem.
", "rawMarkdown": "Have you tried to run your notebook on images which don't have any wheat?\nI created a sample dataset ([https://www.kaggle.com/vudangthinh/gwd-test](https://www.kaggle.com/vudangthinh/gwd-test)), which includes images without wheats.\nYou can test your notebook on this sample dataset to check your submission file.\n\nIn addition, you should check other threads, there are many people having the same problem.", "replies": [ { "id": 934681, "postDate": "2020-07-18T16:25:41.090Z", "content": "thanks, i have tried with you sample dataset to ensure empty string are present in output. I have gone through other threads already.
", "rawMarkdown": "thanks, i have tried with you sample dataset to ensure empty string are present in output. I have gone through other threads already." }, { "id": 938019, "postDate": "2020-07-21T09:54:37.213Z", "content": "Hello Bingi.
\n\nDid you fix the submission error? I am having exactly the same. I don't understand what is that some people saying that there are more that 10 files in the test folder. I can only see 10. I also created the sumbmission file base reading everything in the test folder and I only get 10 images.
\n\nI cannot find on the overview neither the data section anything about the hidden images. If you had anything, please let me know. Thanks.
", "rawMarkdown": "Hello Bingi.\n\nDid you fix the submission error? I am having exactly the same. I don't understand what is that some people saying that there are more that 10 files in the test folder. I can only see 10. I also created the sumbmission file base reading everything in the test folder and I only get 10 images. \n\nI cannot find on the overview neither the data section anything about the hidden images. If you had anything, please let me know. Thanks." }, { "id": 938071, "postDate": "2020-07-21T10:26:06.280Z", "content": "@luisvivas , 10 images you are able to see because they are public test images and they are used when you save and commit your notebook. Once your submission.csv file is created then you submit it. At this point your notebook rerun on private hidden test images. As per this challenge overview section- The test data includes about 1,000 images from Australia, Japan, and China.
", "rawMarkdown": "@luisvivas , 10 images you are able to see because they are public test images and they are used when you save and commit your notebook. Once your submission.csv file is created then you submit it. At this point your notebook rerun on private hidden test images. As per this challenge overview section- The test data includes about 1,000 images from Australia, Japan, and China.", "votes": 1 }, { "id": 938084, "postDate": "2020-07-21T10:36:35.363Z", "content": "Thanks @iamprateek now I think what my error is, I have to submit using the same kernel and predcits the test, I was using a different kernel just to submit to make it easier to disable the internet. Regards.
", "rawMarkdown": "Thanks @iamprateek now I think what my error is, I have to submit using the same kernel and predcits the test, I was using a different kernel just to submit to make it easier to disable the internet. Regards.", "votes": 1 }, { "id": 938085, "postDate": "2020-07-21T10:37:54.020Z", "content": "No I could not. I have tried all the suggestions from this page with no luck https://www.kaggle.com/c/global-wheat-detection/discussion/149871
\n\nI'm now trying to find a working inference notebook and keep adding my logic until it breaks. I will post here once I understand the problem.
", "rawMarkdown": "No I could not. I have tried all the suggestions from this page with no luck https://www.kaggle.com/c/global-wheat-detection/discussion/149871\n\nI'm now trying to find a working inference notebook and keep adding my logic until it breaks. I will post here once I understand the problem.", "votes": 1 }, { "id": 938093, "postDate": "2020-07-21T10:42:27.003Z", "content": "My way to resolve the submission error is-
\nCopy 1000 images from the train folder to the test folder and make predictions on those in my local system.
\nFor reproducing the issue I suggest you to do the same.
Hi @iamprateek. But this will create troubles since those images are not going to be the same as the real test set. Well, maybe just to get rid of the error. I see. This looks more complicated that creating the model.
", "rawMarkdown": "Hi @iamprateek. But this will create troubles since those images are not going to be the same as the real test set. Well, maybe just to get rid of the error. I see. This looks more complicated that creating the model. ", "votes": 1 }, { "id": 938149, "postDate": "2020-07-21T11:24:02.813Z", "content": "Yes, you are right @luisvivas ! We don't have any idea about hidden test images. That's why put some different size images and also put a .txt file there. Wrap your make prediction code inside the try block and in case of exception return an empty string. Do this test in your local machine. It will help you to avoid submission error.
", "rawMarkdown": "Yes, you are right @luisvivas ! We don't have any idea about hidden test images. That's why put some different size images and also put a .txt file there. Wrap your make prediction code inside the try block and in case of exception return an empty string. Do this test in your local machine. It will help you to avoid submission error.", "votes": 2 } ] }, { "id": 933000, "postDate": "2020-07-17T12:16:06.893Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 942923, "author_name": "Bingi", "author_url": "", "post_date": "2020-07-24T04:22:23.720000", "content": "I could fix my problem at last, just posting it here hoping it might help others.\nThere were two things causing the problem:\n1. I was doing \"Quick Save\" most of the times while saving the versions while the instructions say that we have to do \"Save and Run all\"\n2. I was adding RGB values of all test images into a list to visualise the object detection using matplotlib, as the public test images were only 10, I never got an error when ran on 10 images but there were 1000s of images it failed as CUDA / CPU was not sufficient. So its worth a while to examine all variables being used and ensure they do not cause memory crashes
\n\nOne other thing I tried and was not sure if it made any difference was - I has my own datasets for offline packages and pretrained weights which were private before and I made them to public.
\n\nThanks for all your suggestions. Sorry if any of my \"cause of issue\" sound dumb to you :)
", "votes": 1, "replies": [] }, { "id": 933377, "author_name": "Matt", "author_url": "", "post_date": "2020-07-17T17:17:28.747000", "content": "Please try the suggestions here first: Submission scoring error/submission CSV not found
", "votes": 2, "replies": [] }, { "id": 941040, "author_name": "YummyCocopalm", "author_url": "", "post_date": "2020-07-23T06:07:25.887000", "content": "I think the data in submission stage have only test folder. If you are using some codes like read_csv('sample.csv'), it can raise the issue. I'm not sure about this, but it worked for my case.
", "votes": 0, "replies": [] }, { "id": 937553, "author_name": "Bingi", "author_url": "", "post_date": "2020-07-21T04:38:26.700000", "content": "By now, I have tried all the things I can and I'm running out of ideas. Few things I have tried:
\n\nIt is really frustrating now :) any help would be really appreciated.
", "votes": 0, "replies": [] }, { "id": 934042, "author_name": "IRONSIGHT", "author_url": "", "post_date": "2020-07-18T07:47:48.823000", "content": "did you put empty string where nothing is detected??
", "votes": 0, "replies": [ { "id": 934066, "author_name": "Bingi", "author_url": "", "post_date": "2020-07-18T08:02:05.253000", "content": "my test results always had something detected.
", "votes": 0, "replies": [] }, { "id": 934075, "author_name": "Matt", "author_url": "", "post_date": "2020-07-18T08:04:09.053000", "content": "You wouldn't be able to see that for the whole test set. When you submit, your kernel is rerun with the full test set, not just the 10 public images.
", "votes": 0, "replies": [] }, { "id": 934115, "author_name": "Prateek Gupta", "author_url": "", "post_date": "2020-07-18T08:41:26.047000", "content": "@matthewmasters can we see any log file generated after running the submission on the whole test set? It will be very helpful to know what is the exact issue causing the 'Submission CSV Not Found' error.
", "votes": 0, "replies": [] }, { "id": 934125, "author_name": "Matt", "author_url": "", "post_date": "2020-07-18T08:48:41.690000", "content": "I don't believe so. I think a log is problematic as it could be used to leak private test data. There is likely some code in your notebook that throws an error when run on the full set. My suggestion is to simplify your notebook as much as possible to isolate the issue or base your submission notebook off one of the public inference notebooks that works.
", "votes": 0, "replies": [] }, { "id": 934133, "author_name": "Prateek Gupta", "author_url": "", "post_date": "2020-07-18T08:52:33.103000", "content": "Logs are always helpful in debugging and we can encrypt sensitive information for not disclosing. If we do not face any issue with public data while submitting then how can we trace and diagnose the error in test data? Host can hide private test details and can show errors in the logs.
", "votes": 0, "replies": [] }, { "id": 934140, "author_name": "Matt", "author_url": "", "post_date": "2020-07-18T08:57:19.710000", "content": "I agree they would be helpful. But unfortunately I don't think it's something Kaggle will do, and for good reason. What I mean by leaking is that someone could cheat by outputting information about the private test data in the logs that would help them in their submission. This would be hard to protect against from Kaggle's side.
", "votes": 2, "replies": [] }, { "id": 934148, "author_name": "Prateek Gupta", "author_url": "", "post_date": "2020-07-18T09:04:01.040000", "content": "If the host can put read-only access to the logs file then maybe no one can alter this and they can show only errors in the logs, rest other things they can encrypt. Think about the username/password or credit card details- they all are in an application log but in encrypted form. In spite of security risk developers maintain the logs so that they can easily solve the issue in production. Maybe I am not right wrt Kaggle but it's my opinion to share the logs after the final submission.
", "votes": 0, "replies": [] }, { "id": 934159, "author_name": "Matt", "author_url": "", "post_date": "2020-07-18T09:12:52.643000", "content": "It’s complicated by the fact that we are the ones writing the code that is run. One could for example do something like\n\nassert(True == False, “private test set detail”)\n\nI mean maybe Kaggle could engineer some python interpreter which encrypts variables but that seems like a whole lot of work for something which probably has other loopholes.
We have to deal with the situation as is and do our best to troubleshoot “blindly”.
", "votes": 2, "replies": [] }, { "id": 934162, "author_name": "Prateek Gupta", "author_url": "", "post_date": "2020-07-18T09:15:06.067000", "content": "Yes, I also think so. We have to deal with it.
", "votes": 0, "replies": [] }, { "id": 934679, "author_name": "Bingi", "author_url": "", "post_date": "2020-07-18T16:24:29.733000", "content": "okay. I have tried with the test set to ensure the output file has empty string when wheat is not detected. I'm still getting error for test set.
", "votes": 0, "replies": [] }, { "id": 934781, "author_name": "Matt", "author_url": "", "post_date": "2020-07-18T18:11:17.670000", "content": "That file seems to have one issue, you have “.jpg” included in the image_id. Also make sure that your xmin and ymin are between 0 and 1023 and your height/width is non-zero
", "votes": 0, "replies": [] }, { "id": 935026, "author_name": "Bingi", "author_url": "", "post_date": "2020-07-19T03:13:32.420000", "content": "thanks Matthew, i have also tried without .jpg. I will double check the values as well.
", "votes": 0, "replies": [] } ] }, { "id": 933523, "author_name": "Abhishek Prajapat", "author_url": "", "post_date": "2020-07-17T18:41:21.930000", "content": "Not sure but try to recheck your submission file, it may have an extra column for index. Also, you can export your CSV file to you system and then submit it. So, take your CSV then jus do\n\nDataFrame_name.to_csv(\"submission\", index=False)\n\nNow you could upload that file.\nNot sure but hope it helps.
We cannot submit the CSV file from our system for this competition.
", "votes": 0, "replies": [] } ] }, { "id": 933028, "author_name": "ThinhVu", "author_url": "", "post_date": "2020-07-17T12:38:05.637000", "content": "Have you tried to run your notebook on images which don't have any wheat?\nI created a sample dataset (https://www.kaggle.com/vudangthinh/gwd-test), which includes images without wheats.\nYou can test your notebook on this sample dataset to check your submission file.
\n\nIn addition, you should check other threads, there are many people having the same problem.
", "votes": 0, "replies": [ { "id": 934681, "author_name": "Bingi", "author_url": "", "post_date": "2020-07-18T16:25:41.090000", "content": "thanks, i have tried with you sample dataset to ensure empty string are present in output. I have gone through other threads already.
", "votes": 0, "replies": [] }, { "id": 938019, "author_name": "Luis Vivas", "author_url": "", "post_date": "2020-07-21T09:54:37.213000", "content": "Hello Bingi.
\n\nDid you fix the submission error? I am having exactly the same. I don't understand what is that some people saying that there are more that 10 files in the test folder. I can only see 10. I also created the sumbmission file base reading everything in the test folder and I only get 10 images.
\n\nI cannot find on the overview neither the data section anything about the hidden images. If you had anything, please let me know. Thanks.
", "votes": 0, "replies": [] }, { "id": 938071, "author_name": "Prateek Gupta", "author_url": "", "post_date": "2020-07-21T10:26:06.280000", "content": "@luisvivas , 10 images you are able to see because they are public test images and they are used when you save and commit your notebook. Once your submission.csv file is created then you submit it. At this point your notebook rerun on private hidden test images. As per this challenge overview section- The test data includes about 1,000 images from Australia, Japan, and China.
", "votes": 1, "replies": [] }, { "id": 938084, "author_name": "Luis Vivas", "author_url": "", "post_date": "2020-07-21T10:36:35.363000", "content": "Thanks @iamprateek now I think what my error is, I have to submit using the same kernel and predcits the test, I was using a different kernel just to submit to make it easier to disable the internet. Regards.
", "votes": 1, "replies": [] }, { "id": 938085, "author_name": "Bingi", "author_url": "", "post_date": "2020-07-21T10:37:54.020000", "content": "No I could not. I have tried all the suggestions from this page with no luck https://www.kaggle.com/c/global-wheat-detection/discussion/149871
\n\nI'm now trying to find a working inference notebook and keep adding my logic until it breaks. I will post here once I understand the problem.
", "votes": 1, "replies": [] }, { "id": 938093, "author_name": "Prateek Gupta", "author_url": "", "post_date": "2020-07-21T10:42:27.003000", "content": "My way to resolve the submission error is-
\nCopy 1000 images from the train folder to the test folder and make predictions on those in my local system.
\nFor reproducing the issue I suggest you to do the same.
Hi @iamprateek. But this will create troubles since those images are not going to be the same as the real test set. Well, maybe just to get rid of the error. I see. This looks more complicated that creating the model.
", "votes": 1, "replies": [] }, { "id": 938149, "author_name": "Prateek Gupta", "author_url": "", "post_date": "2020-07-21T11:24:02.813000", "content": "Yes, you are right @luisvivas ! We don't have any idea about hidden test images. That's why put some different size images and also put a .txt file there. Wrap your make prediction code inside the try block and in case of exception return an empty string. Do this test in your local machine. It will help you to avoid submission error.
", "votes": 2, "replies": [] } ] }, { "id": 933000, "author_name": "", "author_url": "", "post_date": "2020-07-17T12:16:06.893000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "932840": "Hi All,\n\nI'm facing issue while submitting, I receive \"Submission scoring error\". I have an inference notebook for submitting predictions. So far I have checked the below:\n1. Internet on inference notebook is turned off\n2. Tried with GPU on / off\n3. Submission file has 10 rows apart from the headers\n4. Submission file has 2 columns (sample entry as below)\n5. There is an empty line at the end.\n\nAny guidance / help would be appreciated. I have been struggling since a day\n\nSample submission entry:\nimage_id,PredictionString\naac893a91,0.9916 614 913 82 110 0.9883 74 0 93 163\n51f1be19e,0.9780 282 473 128 118 0.9721 0 376 57 107 0.9664 768 884 152 101\nf5a1f0358,0.9701 152 246 79 96 0.9665 540 269 110 118\n796707dd7,0.9506 0 451 69 76 0.9462 894 331 106 95 0.9275 707 832 100 97\n51b3e36ab,0.9909 0 906 82 117 0.9889 417 930 89 93 0.9888 236 640 90 167\n348a992bb,0.9929 1 317 119 101 0.9873 0 471 137 100\ncc3532ff6,0.9869 0 404 151 110 0.9854 768 818 173 171\n2fd875eaa,0.9924 942 646 80 105 0.9905 103 587 143 86 0.9897 456 490 84 143\ncb8d261a3,0.9661 24 858 79 148 0.9402 751 710 88 91\n53f253011,0.9877 930 199 93 144 0.9807 910 743 74 88\n\nCheers,\nBingi", "942923": "I could fix my problem at last, just posting it here hoping it might help others.\nThere were two things causing the problem:\n1. I was doing \"Quick Save\" most of the times while saving the versions while the instructions say that we have to do \"Save and Run all\"\n2. I was adding RGB values of all test images into a list to visualise the object detection using matplotlib, as the public test images were only 10, I never got an error when ran on 10 images but there were 1000s of images it failed as CUDA / CPU was not sufficient. So its worth a while to examine all variables being used and ensure they do not cause memory crashes\n\nOne other thing I tried and was not sure if it made any difference was - I has my own datasets for offline packages and pretrained weights which were private before and I made them to public.\n\nThanks for all your suggestions. Sorry if any of my \"cause of issue\" sound dumb to you :)", "933377": "Please try the suggestions here first: [Submission scoring error/submission CSV not found](https://www.kaggle.com/c/global-wheat-detection/discussion/149871)", "941040": "I think the data in submission stage have only test folder. If you are using some codes like read_csv('sample.csv'), it can raise the issue. I'm not sure about this, but it worked for my case.", "937553": "By now, I have tried all the things I can and I'm running out of ideas. Few things I have tried:\n\n1. Took a working inference notebook - https://www.kaggle.com/shonenkov/inference-efficientdet and replaced the logic of the model to use mine and left the remaining part of the code intact. Still same issue.\n2. Ensure the co-ordinates are between 1 and 1023. No luck.\n3. Added try: except: while generating co-ordinates and formatting the csv. Tested the same by triggering some error and could see the csv file to have empty strings whenver error was occured. Still no luck.\n\nIt is really frustrating now :) any help would be really appreciated.", "934042": "did you put empty string where nothing is detected??", "933523": "Not sure but try to recheck your submission file, it may have an extra column for index. Also, you can export your CSV file to you system and then submit it. So, take your CSV then jus do\n```\nDataFrame_name.to_csv(\"submission\", index=False)\n```\nNow you could upload that file.\nNot sure but hope it helps.", "933028": "Have you tried to run your notebook on images which don't have any wheat?\nI created a sample dataset ([https://www.kaggle.com/vudangthinh/gwd-test](https://www.kaggle.com/vudangthinh/gwd-test)), which includes images without wheats.\nYou can test your notebook on this sample dataset to check your submission file.\n\nIn addition, you should check other threads, there are many people having the same problem.", "933000": "" } }