Hi All,
\nI'm trying to submit predictions thru my kernel. The kernel runs in about 5 minutes and without error. I can see my submission.csv file within the Output section of my kernel. However, I keep getting the following error: Submission CSV Not Found on the My Submissions page
\nI've found a few other links with this same topic, but they seem to deal with memory overflow or image analysis models:
\nhttps://www.kaggle.com/c/aptos2019-blindness-detection/discussion/98374
\nhttps://www.kaggle.com/questions-and-answers/110395
Does anyone have any guidance on what else this could be or what I can try? Thanks in advance!
", "messages": [ { "id": 1268020, "postDate": "2021-04-09T04:18:41.607Z", "content": "@jaroberti \"Submission CSV Not Found\" means that your kernel encountered an error before writing results.
\nYour kernel runs in 5-mins on 4 test files, but the actual submission is against 8000 test files.
\nDebug by simulating \"test\" data:
Thank you for the response @sirishks. Unfortunately I'm a bit confused by your response, but this is most likely because I do not have a lot of experience with Kaggle ☹️. Some of the public kernels I'm seeing for this competition have run times of a few minutes and seem to use only the 4 test files. I understand that the true submission will be against thousands of files, but I'm just trying to get my kernel to run a baseline with these 4 test files, much like the public kernels.
\nMy log has no errors, but there does seem to be a discrepancy between my log and those of the public kernels. For instance, all the public kernels I'm seeing contain these lines at the end of their respective logs:
\n [NbConvertApp] Converting notebook _notebook__.ipynb to html\n\n [NbConvertApp] Support files will be in _results_files\n\n [NbConvertApp] Making directory _results__files\n\n [NbConvertApp] Making directory _results__files\n\n [NbConvertApp] Writing 832525 bytes to _results_.html\nHowever, my log only shows this:
\n [NbConvertApp] Converting notebook __notebook__.ipynb to html\n\n [NbConvertApp] Writing 304501 bytes to __results__.html\nCould this be the issue or an additional issue? Thanks very much for your time and help on this. I really appreciate it.
", "rawMarkdown": "Thank you for the response @sirishks. Unfortunately I'm a bit confused by your response, but this is most likely because I do not have a lot of experience with Kaggle ☹️. Some of the public kernels I'm seeing for this competition have run times of a few minutes and seem to use only the 4 test files. I understand that the true submission will be against thousands of files, but I'm just trying to get my kernel to run a baseline with these 4 test files, much like the public kernels.\n\nMy log has no errors, but there does seem to be a discrepancy between my log and those of the public kernels. For instance, all the public kernels I'm seeing contain these lines at the end of their respective logs:\n\n [NbConvertApp] Converting notebook _notebook__.ipynb to html\n\n [NbConvertApp] Support files will be in _results_files\n\n [NbConvertApp] Making directory _results__files\n\n [NbConvertApp] Making directory _results__files\n\n [NbConvertApp] Writing 832525 bytes to _results_.html\n\nHowever, my log only shows this:\n\n [NbConvertApp] Converting notebook __notebook__.ipynb to html\n\n [NbConvertApp] Writing 304501 bytes to __results__.html\n\nCould this be the issue or an additional issue? Thanks very much for your time and help on this. I really appreciate it." } ] }, { "id": 1267897, "postDate": "2021-04-08T23:35:35.563Z", "content": "Hi All,
\nI'm trying to submit predictions thru my kernel. The kernel runs in about 5 minutes and without error. I can see my submission.csv file within the Output section of my kernel. However, I keep getting the following error: Submission CSV Not Found on the My Submissions page
\nI've found a few other links with this same topic, but they seem to deal with memory overflow or image analysis models:
\nhttps://www.kaggle.com/c/aptos2019-blindness-detection/discussion/98374
\nhttps://www.kaggle.com/questions-and-answers/110395
Does anyone have any guidance on what else this could be or what I can try? Thanks in advance!
", "rawMarkdown": "Hi All,\n\nI'm trying to submit predictions thru my kernel. The kernel runs in about 5 minutes and without error. I can see my submission.csv file within the Output section of my kernel. However, I keep getting the following error: *Submission CSV Not Found* on the *My Submissions* page\n\nI've found a few other links with this same topic, but they seem to deal with memory overflow or image analysis models:\nhttps://www.kaggle.com/c/aptos2019-blindness-detection/discussion/98374\nhttps://www.kaggle.com/questions-and-answers/110395\n\nDoes anyone have any guidance on what else this could be or what I can try? Thanks in advance!", "votes": 3 }, { "id": 1283525, "postDate": "2021-04-25T02:38:48.063Z", "content": "I finally figured it out. I spent days trying to figure this one out.
\nThe problem is that we only create a submission file with 4 documents. This is not enough to catch everything that could go wrong with your submission. If you're having this problem I recommend that you create a submission file with the 15.000 documents from the training set. You'll definitely encounter a few issues and you'll learn how to handle them.
", "rawMarkdown": "I finally figured it out. I spent days trying to figure this one out.\n\nThe problem is that we only create a submission file with 4 documents. This is not enough to catch everything that could go wrong with your submission. If you're having this problem I recommend that you create a submission file with the 15.000 documents from the training set. You'll definitely encounter a few issues and you'll learn how to handle them.", "votes": 1 }, { "id": 1276661, "postDate": "2021-04-17T20:00:53.320Z", "content": "I had the same issue. In my case, it was OOM- resource exhaustion error while calling model.predict() on entire tests set. Converting test set into batches solved my problem. Hope that helps!
", "rawMarkdown": "I had the same issue. In my case, it was OOM- resource exhaustion error while calling model.predict() on entire tests set. Converting test set into batches solved my problem. Hope that helps!", "votes": 2, "replies": [ { "id": 1276706, "postDate": "2021-04-17T21:38:20.500Z", "content": "Thanks. I am not using the same tools (I have no idea about model.predict). But I think I see what you mean. It seems like we have to be careful with memory management during the submission. I bet one of those publications must be extremely long.
\nThus far I thought it was a problem with the format of my file.
\nI'm going to see what I can do with my next 5 shots.
I'm having the same problem. No idea how to fix this.
", "rawMarkdown": "I'm having the same problem. No idea how to fix this." }, { "id": 1269920, "postDate": "2021-04-11T04:33:42.240Z", "content": "Hi @jaroberti, I had this same issue-- not sure if you figured it out yet, but I'll post how I fixed it.
\nI had hard-coded the 4 test file ids in my code for when running predictions. So when my code ran, it would go out and read just those 4 files. However, the error we got is really \"submission file missing, or not the right length\". I think their submission workers want to make sure that every test file has a row in the final submission csv.
\nI fixed it by reading in the sample_submission.csv file, and loading the ids from there instead of hard-coding them. You'll also notice that the run time when scoring your notebook will be wayyyy longer because now you are running 8,000 files instead of just 4. If you want to validate your code can actually submit, you can make your prediction loop run through a few then quit-- but make sure when you actually output the submission.csv that there is a row for every file id in the sample_submission.csv otherwise it won't pass their checks.
\n(it's also possible from Sirish's comment, that the 4 test ids are not present when they are actually scoring the model, and so when my code went to read that json file it errored and caused the \"Submission CSV Not Found\" error)
", "rawMarkdown": "Hi @jaroberti, I had this same issue-- not sure if you figured it out yet, but I'll post how I fixed it.\n\nI had hard-coded the 4 test file ids in my code for when running predictions. So when my code ran, it would go out and read just those 4 files. However, the error we got is really \"submission file missing, or not the right length\". I think their submission workers want to make sure that every test file has a row in the final submission csv. \n\nI fixed it by reading in the sample_submission.csv file, and loading the ids from there instead of hard-coding them. You'll also notice that the run time when scoring your notebook will be wayyyy longer because now you are running 8,000 files instead of just 4. If you want to validate your code can actually submit, you can make your prediction loop run through a few then quit-- but make sure when you actually output the submission.csv that there is a row for every file id in the sample_submission.csv otherwise it won't pass their checks.\n\n(it's also possible from Sirish's comment, that the 4 test ids are not present when they are actually scoring the model, and so when my code went to read that json file it errored and caused the \"Submission CSV Not Found\" error)", "replies": [ { "id": 1270528, "postDate": "2021-04-11T18:28:39.393Z", "content": "Hi @zaynesprague , I appreciate you taking the time to answer this. I want to preface my responses with: this is the first Kaggle competition I've competed in where you need to submit a notebook kernel. There's clearly a bit of a learning curve for me.
\nThe interesting thing is that I'm not hardcoding the sample IDs. I'm reading in the sample_submission.csv file and then running my model on the 4 respective test files whose IDs are in the sample_submission.csv file. I'm then outputting a 'submission.csv' file in the correct format. I'm also reading in all the training data too. My notebook still runs in a few minutes. I'm now receiving a different error than the \"Submission CVS not found\". The new error is: \"Error !\" There's an info icon next to it and when I click that it says: \"No additional details provided for this error.\"
\nI'm literally just trying to get a baseline score here before diving into a full model, so I was trying to make the notebook run fast and meet the minimum requirements. But, from what you and @sirishks are saying, it sounds like I also need to find mentions of datasets within the training data set as well. Am I reading that correctly? If so, that definitely makes sense because that's how you build a model, but do I also have to output a file with training set IDs and applicable dataset mentions in addition to the 4 test IDs?
\nIf the test IDs aren't present when they score the model then what are they scoring on? Are they taking a random sample of the training dataset and using that as the 'test' data?
", "rawMarkdown": "Hi @zaynesprague , I appreciate you taking the time to answer this. I want to preface my responses with: this is the first Kaggle competition I've competed in where you need to submit a notebook kernel. There's clearly a bit of a learning curve for me.\n\nThe interesting thing is that I'm not hardcoding the sample IDs. I'm reading in the sample_submission.csv file and then running my model on the 4 respective test files whose IDs are in the sample_submission.csv file. I'm then outputting a 'submission.csv' file in the correct format. I'm also reading in all the training data too. My notebook still runs in a few minutes. I'm now receiving a different error than the \"Submission CVS not found\". The new error is: \"Error !\" There's an info icon next to it and when I click that it says: \"No additional details provided for this error.\" \n\nI'm literally just trying to get a baseline score here before diving into a full model, so I was trying to make the notebook run fast and meet the minimum requirements. But, from what you and @sirishks are saying, it sounds like I also need to find mentions of datasets within the training data set as well. Am I reading that correctly? If so, that definitely makes sense because that's how you build a model, but do I also have to output a file with training set IDs and applicable dataset mentions in addition to the 4 test IDs? \n\nIf the test IDs aren't present when they score the model then what are they scoring on? Are they taking a random sample of the training dataset and using that as the 'test' data? " }, { "id": 1270628, "postDate": "2021-04-11T20:07:30.137Z", "content": "No worries-- this is my first competition requiring a notebook kernel too so I definitely agree on the learning curve.
\nI don't think you have to run through the training data to submit a notebook and get a score (though your score would probably be very low if you didn't look at the training data haha). You do need to create a script that will read in all the test ids from the sample_submissions.csv, and then for each one of those ids in that csv you need a row in the \"submission.csv\" that has a predictionString for it.
\nSo for when you run it, there will be 4 ids in the submission.csv, but when the backend runs your notebook for scoring it'll switch out that sample_submission.csv with one that has 8,000 rows-- so naturally your notebook needs to be able to handle those 8,000 ids and write them out into your \"submission.csv\".
\nSo I think a bare minimum notebook that would get a score of 0 but technically not error would be
\n!cp ../input/coleridgeinitiative-show-us-the-data/sample_submission.csv submission.csv
\n(I haven't confirmed that though)
Also, it might help to publish the notebook somewhere so we can view the code and maybe help diagnose it?
", "rawMarkdown": "No worries-- this is my first competition requiring a notebook kernel too so I definitely agree on the learning curve. \n\nI don't think you have to run through the training data to submit a notebook and get a score (though your score would probably be very low if you didn't look at the training data haha). You do need to create a script that will read in all the test ids from the sample_submissions.csv, and then for each one of those ids in that csv you need a row in the \"submission.csv\" that has a predictionString for it. \n\nSo for when you run it, there will be 4 ids in the submission.csv, but when the backend runs your notebook for scoring it'll switch out that sample_submission.csv with one that has 8,000 rows-- so naturally your notebook needs to be able to handle those 8,000 ids and write them out into your \"submission.csv\". \n\nSo I think a bare minimum notebook that would get a score of 0 but technically not error would be\n`!cp ../input/coleridgeinitiative-show-us-the-data/sample_submission.csv submission.csv`\n(I haven't confirmed that though)\n\nAlso, it might help to publish the notebook somewhere so we can view the code and maybe help diagnose it? " } ] }, { "id": 1268066, "postDate": "2021-04-09T05:31:57.113Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1268020, "author_name": "Sirish Somanchi", "author_url": "", "post_date": "2021-04-09T04:18:41.607000", "content": "@jaroberti \"Submission CSV Not Found\" means that your kernel encountered an error before writing results.
\nYour kernel runs in 5-mins on 4 test files, but the actual submission is against 8000 test files.
\nDebug by simulating \"test\" data:
Thank you for the response @sirishks. Unfortunately I'm a bit confused by your response, but this is most likely because I do not have a lot of experience with Kaggle ☹️. Some of the public kernels I'm seeing for this competition have run times of a few minutes and seem to use only the 4 test files. I understand that the true submission will be against thousands of files, but I'm just trying to get my kernel to run a baseline with these 4 test files, much like the public kernels.
\nMy log has no errors, but there does seem to be a discrepancy between my log and those of the public kernels. For instance, all the public kernels I'm seeing contain these lines at the end of their respective logs:
\n [NbConvertApp] Converting notebook _notebook__.ipynb to html\n\n [NbConvertApp] Support files will be in _results_files\n\n [NbConvertApp] Making directory _results__files\n\n [NbConvertApp] Making directory _results__files\n\n [NbConvertApp] Writing 832525 bytes to _results_.html\nHowever, my log only shows this:
\n [NbConvertApp] Converting notebook __notebook__.ipynb to html\n\n [NbConvertApp] Writing 304501 bytes to __results__.html\nCould this be the issue or an additional issue? Thanks very much for your time and help on this. I really appreciate it.
", "votes": 0, "replies": [] } ] }, { "id": 1283525, "author_name": "Wilmer E. Henao", "author_url": "", "post_date": "2021-04-25T02:38:48.063000", "content": "I finally figured it out. I spent days trying to figure this one out.
\nThe problem is that we only create a submission file with 4 documents. This is not enough to catch everything that could go wrong with your submission. If you're having this problem I recommend that you create a submission file with the 15.000 documents from the training set. You'll definitely encounter a few issues and you'll learn how to handle them.
", "votes": 1, "replies": [] }, { "id": 1276661, "author_name": "MananG", "author_url": "", "post_date": "2021-04-17T20:00:53.320000", "content": "I had the same issue. In my case, it was OOM- resource exhaustion error while calling model.predict() on entire tests set. Converting test set into batches solved my problem. Hope that helps!
", "votes": 2, "replies": [ { "id": 1276706, "author_name": "Wilmer E. Henao", "author_url": "", "post_date": "2021-04-17T21:38:20.500000", "content": "Thanks. I am not using the same tools (I have no idea about model.predict). But I think I see what you mean. It seems like we have to be careful with memory management during the submission. I bet one of those publications must be extremely long.
\nThus far I thought it was a problem with the format of my file.
\nI'm going to see what I can do with my next 5 shots.
I'm having the same problem. No idea how to fix this.
", "votes": 0, "replies": [] }, { "id": 1269920, "author_name": "Zayne Sprague", "author_url": "", "post_date": "2021-04-11T04:33:42.240000", "content": "Hi @jaroberti, I had this same issue-- not sure if you figured it out yet, but I'll post how I fixed it.
\nI had hard-coded the 4 test file ids in my code for when running predictions. So when my code ran, it would go out and read just those 4 files. However, the error we got is really \"submission file missing, or not the right length\". I think their submission workers want to make sure that every test file has a row in the final submission csv.
\nI fixed it by reading in the sample_submission.csv file, and loading the ids from there instead of hard-coding them. You'll also notice that the run time when scoring your notebook will be wayyyy longer because now you are running 8,000 files instead of just 4. If you want to validate your code can actually submit, you can make your prediction loop run through a few then quit-- but make sure when you actually output the submission.csv that there is a row for every file id in the sample_submission.csv otherwise it won't pass their checks.
\n(it's also possible from Sirish's comment, that the 4 test ids are not present when they are actually scoring the model, and so when my code went to read that json file it errored and caused the \"Submission CSV Not Found\" error)
", "votes": 0, "replies": [ { "id": 1270528, "author_name": "Rolocado", "author_url": "", "post_date": "2021-04-11T18:28:39.393000", "content": "Hi @zaynesprague , I appreciate you taking the time to answer this. I want to preface my responses with: this is the first Kaggle competition I've competed in where you need to submit a notebook kernel. There's clearly a bit of a learning curve for me.
\nThe interesting thing is that I'm not hardcoding the sample IDs. I'm reading in the sample_submission.csv file and then running my model on the 4 respective test files whose IDs are in the sample_submission.csv file. I'm then outputting a 'submission.csv' file in the correct format. I'm also reading in all the training data too. My notebook still runs in a few minutes. I'm now receiving a different error than the \"Submission CVS not found\". The new error is: \"Error !\" There's an info icon next to it and when I click that it says: \"No additional details provided for this error.\"
\nI'm literally just trying to get a baseline score here before diving into a full model, so I was trying to make the notebook run fast and meet the minimum requirements. But, from what you and @sirishks are saying, it sounds like I also need to find mentions of datasets within the training data set as well. Am I reading that correctly? If so, that definitely makes sense because that's how you build a model, but do I also have to output a file with training set IDs and applicable dataset mentions in addition to the 4 test IDs?
\nIf the test IDs aren't present when they score the model then what are they scoring on? Are they taking a random sample of the training dataset and using that as the 'test' data?
", "votes": 0, "replies": [] }, { "id": 1270628, "author_name": "Zayne Sprague", "author_url": "", "post_date": "2021-04-11T20:07:30.137000", "content": "No worries-- this is my first competition requiring a notebook kernel too so I definitely agree on the learning curve.
\nI don't think you have to run through the training data to submit a notebook and get a score (though your score would probably be very low if you didn't look at the training data haha). You do need to create a script that will read in all the test ids from the sample_submissions.csv, and then for each one of those ids in that csv you need a row in the \"submission.csv\" that has a predictionString for it.
\nSo for when you run it, there will be 4 ids in the submission.csv, but when the backend runs your notebook for scoring it'll switch out that sample_submission.csv with one that has 8,000 rows-- so naturally your notebook needs to be able to handle those 8,000 ids and write them out into your \"submission.csv\".
\nSo I think a bare minimum notebook that would get a score of 0 but technically not error would be
\n!cp ../input/coleridgeinitiative-show-us-the-data/sample_submission.csv submission.csv
\n(I haven't confirmed that though)
Also, it might help to publish the notebook somewhere so we can view the code and maybe help diagnose it?
", "votes": 0, "replies": [] } ] }, { "id": 1268066, "author_name": "", "author_url": "", "post_date": "2021-04-09T05:31:57.113000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1268020": "@jaroberti \"*Submission CSV Not Found*\" means that your kernel encountered an error before writing results.\n\nYour kernel runs in 5-mins on 4 test files, but the actual submission is against 8000 test files.\nDebug by simulating \"test\" data:\n- Run against train[:8000] and check RAM usage and time taken on a larger input\n- Create a file with random text, not containing known dataset names and check if your code returns \"empty\" results or throws errors", "1267897": "Hi All,\n\nI'm trying to submit predictions thru my kernel. The kernel runs in about 5 minutes and without error. I can see my submission.csv file within the Output section of my kernel. However, I keep getting the following error: *Submission CSV Not Found* on the *My Submissions* page\n\nI've found a few other links with this same topic, but they seem to deal with memory overflow or image analysis models:\nhttps://www.kaggle.com/c/aptos2019-blindness-detection/discussion/98374\nhttps://www.kaggle.com/questions-and-answers/110395\n\nDoes anyone have any guidance on what else this could be or what I can try? Thanks in advance!", "1283525": "I finally figured it out. I spent days trying to figure this one out.\n\nThe problem is that we only create a submission file with 4 documents. This is not enough to catch everything that could go wrong with your submission. If you're having this problem I recommend that you create a submission file with the 15.000 documents from the training set. You'll definitely encounter a few issues and you'll learn how to handle them.", "1276661": "I had the same issue. In my case, it was OOM- resource exhaustion error while calling model.predict() on entire tests set. Converting test set into batches solved my problem. Hope that helps!", "1271721": "I'm having the same problem. No idea how to fix this.", "1269920": "Hi @jaroberti, I had this same issue-- not sure if you figured it out yet, but I'll post how I fixed it.\n\nI had hard-coded the 4 test file ids in my code for when running predictions. So when my code ran, it would go out and read just those 4 files. However, the error we got is really \"submission file missing, or not the right length\". I think their submission workers want to make sure that every test file has a row in the final submission csv. \n\nI fixed it by reading in the sample_submission.csv file, and loading the ids from there instead of hard-coding them. You'll also notice that the run time when scoring your notebook will be wayyyy longer because now you are running 8,000 files instead of just 4. If you want to validate your code can actually submit, you can make your prediction loop run through a few then quit-- but make sure when you actually output the submission.csv that there is a row for every file id in the sample_submission.csv otherwise it won't pass their checks.\n\n(it's also possible from Sirish's comment, that the 4 test ids are not present when they are actually scoring the model, and so when my code went to read that json file it errored and caused the \"Submission CSV Not Found\" error)", "1268066": "" } }