![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2145%2F8fcb89642381ab5d795879f2c78c9de9%2Fpending.png?generation=1761590716246014&alt=media\")
I made two submissions in quick succession via the CLI (i.e. I didn't wait for the first one to be scored before submitting the second). The first submission was scored in a couple of minutes, but the second is still pending for an hour now… Can I get some help?
", "messages": [ { "id": 3307643, "postDate": "2025-10-27T13:31:28.103Z", "content": "I made two submissions in quick succession via the CLI (i.e. I didn't wait for the first one to be scored before submitting the second). The first submission was scored in a couple of minutes, but the second is still pending for an hour now… Can I get some help?
", "rawMarkdown": "I made two submissions in quick succession via the CLI (i.e. I didn't wait for the first one to be scored before submitting the second). The first submission was scored in a couple of minutes, but the second is still pending for an hour now... Can I get some help?", "votes": 12 }, { "id": 3307774, "postDate": "2025-10-27T18:47:01.207Z", "content": "My submission is still pending after 6 hours, but I have some good news: additional submissions that I made afterwards were scored quickly, despite the earlier pending submission.
\nIn addition, my latest submission (that scored quickly) is the same as the earlier one (that is still pending). So the problem does not seem to be with the submission file itself.
\n
Finally:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2145%2F7c286697a66b72c517973b04c59e8dc2%2Ftimeout.png?generation=1761624661155449&alt=media\")
Now I remember in CAFA5 we had a similar issue… https://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/403258 but the error was triggered quite soon, it wasn't a timeout…
", "rawMarkdown": "Now I remember in CAFA5 we had a similar issue... https://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/403258 but the error was triggered quite soon, it wasn't a timeout...", "votes": 1, "replies": [ { "id": 3307756, "postDate": "2025-10-27T17:37:01.487Z", "content": "Is not for the submission lost. We are still early in the competition. But I'm pretty sure the exact same output should not succed just sometimes.
", "rawMarkdown": "Is not for the submission lost. We are still early in the competition. But I'm pretty sure the exact same output should not succed just sometimes.", "votes": 1, "replies": [ { "id": 3307995, "postDate": "2025-10-28T10:40:10.733Z", "content": "I've seen this in other competitions too. Just occasionally, a submission seems to get stuck until it eventually times out or errors, even though another apparently identical one runs quickly and succeeds. In my experience, it's rare and unpredictable.
", "rawMarkdown": "I've seen this in other competitions too. Just occasionally, a submission seems to get stuck until it eventually times out or errors, even though another apparently identical one runs quickly and succeeds. In my experience, it's rare and unpredictable.", "votes": 1 } ] } ] }, { "id": 3307745, "postDate": "2025-10-27T17:16:07.067Z", "content": "@ahphan @martynaplomecka I've just got an Timeout exceeded error with this public notebook https://www.kaggle.com/code/mafiosoquasar/cafa-6-protein-weighted-ensemble so I think the issue must be on scoring process side.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8722753%2Fb6163235b0fa29e4b14e23b107abc0ee%2Ftimeout.jpg?generation=1761585711703349&alt=media\")
Agree, it looks like that scoring takes ages and then fails for files of the same size…
", "rawMarkdown": "Agree, it looks like that scoring takes ages and then fails for files of the same size...", "votes": 1, "replies": [ { "id": 3307669, "postDate": "2025-10-27T14:24:50.327Z", "content": "Please see the answer above
", "rawMarkdown": "Please see the answer above", "replies": [ { "id": 3308494, "postDate": "2025-10-29T15:02:02.020Z", "content": "Thank you, @martynaplomecka, but I wonder if you could give it a second look! I'm getting a timeout every 5/10 submissions. I'm also getting a new out-of-memory error, even though comparable file sizes were not a problem before.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F521899%2F793261009a5ecfe6bfb80a9b8d1722fd%2FScreenshot%202025-10-29%20at%2014.57.16.png?generation=1761750080440854&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F521899%2F6d8ed517a539d7a581ed85d40773e98e%2FScreenshot%202025-10-29%20at%2014.54.07.png?generation=1761749806549081&alt=media\")
Hi @mtinti!\nWe’re monitoring the situation closely and have tested the platform again by submitting files of various sizes and resubmitting the notebooks that users had flagged. We’ve also observed a large number of successful submissions in the past few hours.
", "rawMarkdown": "Hi @mtinti!\nWe’re monitoring the situation closely and have tested the platform again by submitting files of various sizes and resubmitting the notebooks that users had flagged. We’ve also observed a large number of successful submissions in the past few hours. ", "votes": 1 }, { "id": 3309225, "postDate": "2025-10-31T08:45:18.823Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3310769, "postDate": "2025-11-03T16:06:42.463Z", "content": "hi, even through the check, no id not in the test set, no dupelite, no outscope score, it also failed one time in our situation
", "rawMarkdown": "hi, even through the check, no id not in the test set, no dupelite, no outscope score, it also failed one time in our situation" }, { "id": 3312482, "postDate": "2025-11-07T09:58:14.263Z", "content": "Hello, madam, this situation seem to appeared twice in my 3 submission and appeared twice in 3 days. They could also go pass the script of the code domain shared by a people. And this greatly disturb my test on the online public lb, I sincerely you could moniter my and other people' submission to find out where the problem comes out and solve it.
\nseed,
", "rawMarkdown": "Hello, madam, this situation seem to appeared twice in my 3 submission and appeared twice in 3 days. They could also go pass the script of the code domain shared by a people. And this greatly disturb my test on the online public lb, I sincerely you could moniter my and other people' submission to find out where the problem comes out and solve it.\n\n\nseed,", "votes": 1 }, { "id": 3312742, "postDate": "2025-11-07T20:55:37.393Z", "content": "Experiencing similar behavior. I submitted the same submission file twice. The first one has been scoring for 9 hours (expecting it to timeout), whereas second scored in ~1 hour.
", "rawMarkdown": "Experiencing similar behavior. I submitted the same submission file twice. The first one has been scoring for 9 hours (expecting it to timeout), whereas second scored in ~1 hour. " } ] } ] }, { "id": 3307648, "postDate": "2025-10-27T13:37:12.660Z", "content": "Hi, the scoring can take up to an hour if your file is close to 8GB in size. How large is your submission file?
", "rawMarkdown": "Hi, the scoring can take up to an hour if your file is close to 8GB in size. How large is your submission file?", "votes": 1, "replies": [ { "id": 3307651, "postDate": "2025-10-27T13:43:30.960Z", "content": "About 100 MB. In fact, this second submission (the one that is pending) was smaller than the first one.
", "rawMarkdown": "About 100 MB. In fact, this second submission (the one that is pending) was smaller than the first one.", "replies": [ { "id": 3307654, "postDate": "2025-10-27T13:47:43.650Z", "content": "@martynaplomecka can you look into this to see what's wrong
", "rawMarkdown": "@martynaplomecka can you look into this to see what's wrong" }, { "id": 3307668, "postDate": "2025-10-27T14:24:34.283Z", "content": "I just tested the evaluation with a file of approximately 45 MB and the scoring took less than two minutes. I also see that other people are submitting their predictions, so the issue doesn’t appear to be on the platform side
", "rawMarkdown": "I just tested the evaluation with a file of approximately 45 MB and the scoring took less than two minutes. I also see that other people are submitting their predictions, so the issue doesn’t appear to be on the platform side" }, { "id": 3307672, "postDate": "2025-10-27T14:34:48.127Z", "content": "Two minutes was my experience in previous submissions too. Should I risk making new submissions when the last one is still pending?
", "rawMarkdown": "Two minutes was my experience in previous submissions too. Should I risk making new submissions when the last one is still pending?" }, { "id": 3343397, "postDate": "2025-11-21T16:11:48.540Z", "content": "it happened twice again during two days
", "rawMarkdown": "it happened twice again during two days" } ] }, { "id": 3307685, "postDate": "2025-10-27T15:14:07.547Z", "content": "Would it be possible to cancel my submission that is pending? This way, I could try to use my remaining submissions for the day.
", "rawMarkdown": "Would it be possible to cancel my submission that is pending? This way, I could try to use my remaining submissions for the day.\n" } ] }, { "id": 3390530, "postDate": "2026-01-13T10:53:56.673Z", "content": "this happened again, how to solve this problem? I tried both tsv submission and notebook submission
", "rawMarkdown": "this happened again, how to solve this problem? I tried both tsv submission and notebook submission" }, { "id": 3334524, "postDate": "2025-11-17T15:05:20.297Z", "content": "I have now made 59 submissions, and have lost 4 of them to this unexpected timeout behaviour. In each case, the identical submission file scored successfully in a few minutes on the next attempt. That's about a 7% failure rate (but my small sample naturally gives only an approximate estimate of the true underlying rate).
\nAt presnt, it's annoying, but not deal-breaking. There are plenty of submissions available in this competition, and I will remember not to rely on making 5 essential submissions on the last day!
", "rawMarkdown": "I have now made 59 submissions, and have lost 4 of them to this unexpected timeout behaviour. In each case, the identical submission file scored successfully in a few minutes on the next attempt. That's about a 7% failure rate (but my small sample naturally gives only an approximate estimate of the true underlying rate).\n\nAt presnt, it's annoying, but not deal-breaking. There are plenty of submissions available in this competition, and I will remember not to rely on making 5 essential submissions on the last day!", "replies": [ { "id": 3334566, "postDate": "2025-11-17T15:15:57.760Z", "content": "How big are the submissions? Consider that more than 3 digits are totally unnecessary. And not decided yet for private LB could be just 2 digits. I've read that big files could lead to random fails easyly.
", "rawMarkdown": "How big are the submissions? Consider that more than 3 digits are totally unnecessary. And not decided yet for private LB could be just 2 digits. I've read that big files could lead to random fails easyly.", "replies": [ { "id": 3334689, "postDate": "2025-11-17T15:56:47.157Z", "content": "We're talking about a small (for this competition) file of 244 MB; files 8 times larger than that have scored successfully for me in around 10-20 minutes. Your point about 3dp is indeed correct, that's the requisite precision for this competition and would allow the final private score to be evaluated at 0.001 increments if the organisers choose to go to a finer granularity than the current 0.01. However, there's no reason to think this is related to the failures I observed, all of those files scored successfully in ~10 minutes on the first resubmission of the same tsv prediction file.
", "rawMarkdown": "We're talking about a small (for this competition) file of 244 MB; files 8 times larger than that have scored successfully for me in around 10-20 minutes. Your point about 3dp is indeed correct, that's the requisite precision for this competition and would allow the final private score to be evaluated at 0.001 increments if the organisers choose to go to a finer granularity than the current 0.01. However, there's no reason to think this is related to the failures I observed, all of those files scored successfully in ~10 minutes on the first resubmission of the same tsv prediction file.", "votes": 1 } ] }, { "id": 3334794, "postDate": "2025-11-17T16:32:15.833Z", "content": "Yes, the issue still appear somtimes
", "rawMarkdown": "Yes, the issue still appear somtimes", "votes": 1 } ] }, { "id": 3309916, "postDate": "2025-11-01T17:50:40.903Z", "content": "For example, can we double-check what happens if a submission file does not contain protein IDs used for the scoring?
", "rawMarkdown": "For example, can we double-check what happens if a submission file does not contain protein IDs used for the scoring? ", "replies": [ { "id": 3310768, "postDate": "2025-11-03T16:05:24.173Z", "content": "even through the check, it also failed one time in our situation
", "rawMarkdown": "even through the check, it also failed one time in our situation" } ] } ], "comments": [ { "id": 3307774, "author_name": "Diogo R. Ferreira", "author_url": "", "post_date": "2025-10-27T18:47:01.207000", "content": "My submission is still pending after 6 hours, but I have some good news: additional submissions that I made afterwards were scored quickly, despite the earlier pending submission.
\nIn addition, my latest submission (that scored quickly) is the same as the earlier one (that is still pending). So the problem does not seem to be with the submission file itself.
\n
Finally:
\n
Now I remember in CAFA5 we had a similar issue… https://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/403258 but the error was triggered quite soon, it wasn't a timeout…
", "votes": 1, "replies": [ { "id": 3307756, "author_name": "Ángel Jacinto Sánchez Ruiz", "author_url": "", "post_date": "2025-10-27T17:37:01.487000", "content": "Is not for the submission lost. We are still early in the competition. But I'm pretty sure the exact same output should not succed just sometimes.
", "votes": 1, "replies": [ { "id": 3307995, "author_name": "John Mitchell", "author_url": "", "post_date": "2025-10-28T10:40:10.733000", "content": "I've seen this in other competitions too. Just occasionally, a submission seems to get stuck until it eventually times out or errors, even though another apparently identical one runs quickly and succeeds. In my experience, it's rare and unpredictable.
", "votes": 1, "replies": [] } ] } ] }, { "id": 3307745, "author_name": "Ángel Jacinto Sánchez Ruiz", "author_url": "", "post_date": "2025-10-27T17:16:07.067000", "content": "@ahphan @martynaplomecka I've just got an Timeout exceeded error with this public notebook https://www.kaggle.com/code/mafiosoquasar/cafa-6-protein-weighted-ensemble so I think the issue must be on scoring process side.
\n
Agree, it looks like that scoring takes ages and then fails for files of the same size…
", "votes": 1, "replies": [ { "id": 3307669, "author_name": "MartynaPlomecka", "author_url": "", "post_date": "2025-10-27T14:24:50.327000", "content": "Please see the answer above
", "votes": 0, "replies": [ { "id": 3308494, "author_name": "MT", "author_url": "", "post_date": "2025-10-29T15:02:02.020000", "content": "Thank you, @martynaplomecka, but I wonder if you could give it a second look! I'm getting a timeout every 5/10 submissions. I'm also getting a new out-of-memory error, even though comparable file sizes were not a problem before.
\n
Hi @mtinti!\nWe’re monitoring the situation closely and have tested the platform again by submitting files of various sizes and resubmitting the notebooks that users had flagged. We’ve also observed a large number of successful submissions in the past few hours.
", "votes": 1, "replies": [] }, { "id": 3309225, "author_name": "", "author_url": "", "post_date": "2025-10-31T08:45:18.823000", "content": "", "votes": 0, "replies": [] }, { "id": 3310769, "author_name": "common-people", "author_url": "", "post_date": "2025-11-03T16:06:42.463000", "content": "hi, even through the check, no id not in the test set, no dupelite, no outscope score, it also failed one time in our situation
", "votes": 0, "replies": [] }, { "id": 3312482, "author_name": "common-people", "author_url": "", "post_date": "2025-11-07T09:58:14.263000", "content": "Hello, madam, this situation seem to appeared twice in my 3 submission and appeared twice in 3 days. They could also go pass the script of the code domain shared by a people. And this greatly disturb my test on the online public lb, I sincerely you could moniter my and other people' submission to find out where the problem comes out and solve it.
\nseed,
", "votes": 1, "replies": [] }, { "id": 3312742, "author_name": "Matt graham", "author_url": "", "post_date": "2025-11-07T20:55:37.393000", "content": "Experiencing similar behavior. I submitted the same submission file twice. The first one has been scoring for 9 hours (expecting it to timeout), whereas second scored in ~1 hour.
", "votes": 0, "replies": [] } ] } ] }, { "id": 3307648, "author_name": "An Phan", "author_url": "", "post_date": "2025-10-27T13:37:12.660000", "content": "Hi, the scoring can take up to an hour if your file is close to 8GB in size. How large is your submission file?
", "votes": 1, "replies": [ { "id": 3307651, "author_name": "Diogo R. Ferreira", "author_url": "", "post_date": "2025-10-27T13:43:30.960000", "content": "About 100 MB. In fact, this second submission (the one that is pending) was smaller than the first one.
", "votes": 0, "replies": [ { "id": 3307654, "author_name": "An Phan", "author_url": "", "post_date": "2025-10-27T13:47:43.650000", "content": "@martynaplomecka can you look into this to see what's wrong
", "votes": 0, "replies": [] }, { "id": 3307668, "author_name": "MartynaPlomecka", "author_url": "", "post_date": "2025-10-27T14:24:34.283000", "content": "I just tested the evaluation with a file of approximately 45 MB and the scoring took less than two minutes. I also see that other people are submitting their predictions, so the issue doesn’t appear to be on the platform side
", "votes": 0, "replies": [] }, { "id": 3307672, "author_name": "Diogo R. Ferreira", "author_url": "", "post_date": "2025-10-27T14:34:48.127000", "content": "Two minutes was my experience in previous submissions too. Should I risk making new submissions when the last one is still pending?
", "votes": 0, "replies": [] }, { "id": 3343397, "author_name": "common-people", "author_url": "", "post_date": "2025-11-21T16:11:48.540000", "content": "it happened twice again during two days
", "votes": 0, "replies": [] } ] }, { "id": 3307685, "author_name": "Diogo R. Ferreira", "author_url": "", "post_date": "2025-10-27T15:14:07.547000", "content": "Would it be possible to cancel my submission that is pending? This way, I could try to use my remaining submissions for the day.
", "votes": 0, "replies": [] } ] }, { "id": 3390530, "author_name": "shanzhong8", "author_url": "", "post_date": "2026-01-13T10:53:56.673000", "content": "this happened again, how to solve this problem? I tried both tsv submission and notebook submission
", "votes": 0, "replies": [] }, { "id": 3334524, "author_name": "John Mitchell", "author_url": "", "post_date": "2025-11-17T15:05:20.297000", "content": "I have now made 59 submissions, and have lost 4 of them to this unexpected timeout behaviour. In each case, the identical submission file scored successfully in a few minutes on the next attempt. That's about a 7% failure rate (but my small sample naturally gives only an approximate estimate of the true underlying rate).
\nAt presnt, it's annoying, but not deal-breaking. There are plenty of submissions available in this competition, and I will remember not to rely on making 5 essential submissions on the last day!
", "votes": 0, "replies": [ { "id": 3334566, "author_name": "Ángel Jacinto Sánchez Ruiz", "author_url": "", "post_date": "2025-11-17T15:15:57.760000", "content": "How big are the submissions? Consider that more than 3 digits are totally unnecessary. And not decided yet for private LB could be just 2 digits. I've read that big files could lead to random fails easyly.
", "votes": 0, "replies": [ { "id": 3334689, "author_name": "John Mitchell", "author_url": "", "post_date": "2025-11-17T15:56:47.157000", "content": "We're talking about a small (for this competition) file of 244 MB; files 8 times larger than that have scored successfully for me in around 10-20 minutes. Your point about 3dp is indeed correct, that's the requisite precision for this competition and would allow the final private score to be evaluated at 0.001 increments if the organisers choose to go to a finer granularity than the current 0.01. However, there's no reason to think this is related to the failures I observed, all of those files scored successfully in ~10 minutes on the first resubmission of the same tsv prediction file.
", "votes": 1, "replies": [] } ] }, { "id": 3334794, "author_name": "common-people", "author_url": "", "post_date": "2025-11-17T16:32:15.833000", "content": "Yes, the issue still appear somtimes
", "votes": 1, "replies": [] } ] }, { "id": 3309916, "author_name": "MT", "author_url": "", "post_date": "2025-11-01T17:50:40.903000", "content": "For example, can we double-check what happens if a submission file does not contain protein IDs used for the scoring?
", "votes": 0, "replies": [ { "id": 3310768, "author_name": "common-people", "author_url": "", "post_date": "2025-11-03T16:05:24.173000", "content": "even through the check, it also failed one time in our situation
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "3307643": "I made two submissions in quick succession via the CLI (i.e. I didn't wait for the first one to be scored before submitting the second). The first submission was scored in a couple of minutes, but the second is still pending for an hour now... Can I get some help?", "3307774": "My submission is still pending after 6 hours, but I have some good news: additional submissions that I made afterwards were scored quickly, despite the earlier pending submission.\n\nIn addition, my latest submission (that scored quickly) is the same as the earlier one (that is still pending). So the problem does not seem to be with the submission file itself.\n\n", "3307751": "Now I remember in CAFA5 we had a similar issue... https://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/403258 but the error was triggered quite soon, it wasn't a timeout...", "3307745": "@ahphan @martynaplomecka I've just got an Timeout exceeded error with this public notebook https://www.kaggle.com/code/mafiosoquasar/cafa-6-protein-weighted-ensemble so I think the issue must be on scoring process side.\n\n", "3307662": "Agree, it looks like that scoring takes ages and then fails for files of the same size...", "3307648": "Hi, the scoring can take up to an hour if your file is close to 8GB in size. How large is your submission file?", "3390530": "this happened again, how to solve this problem? I tried both tsv submission and notebook submission", "3334524": "I have now made 59 submissions, and have lost 4 of them to this unexpected timeout behaviour. In each case, the identical submission file scored successfully in a few minutes on the next attempt. That's about a 7% failure rate (but my small sample naturally gives only an approximate estimate of the true underlying rate).\n\nAt presnt, it's annoying, but not deal-breaking. There are plenty of submissions available in this competition, and I will remember not to rely on making 5 essential submissions on the last day!", "3309916": "For example, can we double-check what happens if a submission file does not contain protein IDs used for the scoring? " } }