I have done a lot of test submission without using any model and it's Timeout.
\nIn the image below you can see that each iteration on test data is under 0.14s.
\n\nOffline kernel run in 31 s.
\nYou can see also picture of this here!.
@sohier @andrewscherer @yus002 could you please give an explanation for this
", "messages": [ { "id": "1669493", "postDate": "01/30/2022 18:20:01", "content": "I have done a lot of test submission without using any model and it's Timeout.
\nIn the image below you can see that each iteration on test data is under 0.14s.
\n\nOffline kernel run in 31 s.
\nYou can see also picture of this here!.
@sohier @andrewscherer @yus002 could you please give an explanation for this
", "rawMarkdown": "I have done a lot of **test** submission **without using any model** and it's **Timeout**.\n\nIn the image below you can see that each iteration on test data is under 0.14s.\n\n[Image](https://drive.google.com/file/d/1Dfsy8n9BgJcPU5zyUlkee2O0Anuh514B/view?usp=sharing)\n\nOffline kernel run in 31 s.\nYou can see also picture of this [here!](https://drive.google.com/file/d/1WTMLnP9rUJCXYi9IUw6mEhsYS32yUlcG/view?usp=sharing).\n\n@sohier @andrewscherer @yus002 could you please give an explanation for this", "votes": null }, { "id": "1669706", "postDate": "01/30/2022 21:45:03", "content": "Hi, I have no idea. My original estimation is that 140739.13 < x < 145909.09 where x is the number of loops, so as long as each loop takes between 0.222 seconds and 0.23 seconds, the code can run successfully. I also test my own submission which runs for 7 hours. Each loop in it takes about 0.167 seconds.
", "rawMarkdown": "Hi, I have no idea. My original estimation is that 140739.13 < x < 145909.09 where x is the number of loops, so as long as each loop takes between 0.222 seconds and 0.23 seconds, the code can run successfully. I also test my own submission which runs for 7 hours. Each loop in it takes about 0.167 seconds.", "votes": null }, { "id": "1669768", "postDate": "01/30/2022 23:16:32", "content": "I posted about inconsistent run times on notebooks too, it will be a real shame if something like this caused someone to be disqualified in the next 12 weeks.. Some reassurance would be good.
", "rawMarkdown": "I posted about inconsistent run times on notebooks too, it will be a real shame if something like this caused someone to be disqualified in the next 12 weeks.. Some reassurance would be good.", "votes": null }, { "id": "1669959", "postDate": "01/31/2022 05:15:26", "content": "I read somewhere that you need to remove any \"print\" and \"display\" functions when iterating over the test set. Did you include measuring and printing the time taken in your code? Remove that and see what happens
", "rawMarkdown": "I read somewhere that you need to remove any \"print\" and \"display\" functions when iterating over the test set. Did you include measuring and printing the time taken in your code? Remove that and see what happens", "votes": null }, { "id": "1670033", "postDate": "01/31/2022 06:59:28", "content": "Thanks.
\nAfter using this API Emulator I found that run times changes a lot per iteration.
\nAnd I was able to fix it.
hmm yes, I had that.
", "rawMarkdown": "hmm yes, I had that.", "votes": null }, { "id": "1670041", "postDate": "01/31/2022 07:03:34", "content": "Sure, from my experiment the successful kernel can also fail
", "rawMarkdown": "Sure, from my experiment the successful kernel can also fail", "votes": null }, { "id": "1671019", "postDate": "02/01/2022 05:11:54", "content": "yes,even very simple notebooks which take about 10-15 seconds are getting timed out after scoring for 10-15 hours π€π€π€π€
", "rawMarkdown": "yes,even very simple notebooks which take about 10-15 seconds are getting timed out after scoring for 10-15 hours π€π€π€π€", "votes": null }, { "id": "1671044", "postDate": "02/01/2022 05:39:55", "content": "Did you find out what's the issue?
", "rawMarkdown": "Did you find out what's the issue?", "votes": null }, { "id": "1671536", "postDate": "02/01/2022 14:43:19", "content": "Hard to say because I changed a lot of things at the same time.
\nI removed all print functions and change my feature engineering function
Do share the script when the competition ended - I gave out after trying a dozen times of timeout lol;
\nThe competition hosts were not exactly helpful.
yes, I don't know why but my guess is a lot of submissions will fail during rerunsβ¦
", "rawMarkdown": "yes, I don't know why but my guess is a lot of submissions will fail during reruns...", "votes": null }, { "id": "1671761", "postDate": "02/01/2022 17:51:04", "content": "Glad it helped : )
", "rawMarkdown": "Glad it helped : )", "votes": null } ], "comments": [ { "id": 1669706, "author_name": "yus002", "author_url": "", "post_date": "01/30/2022 21:45:03", "content": "Hi, I have no idea. My original estimation is that 140739.13 < x < 145909.09 where x is the number of loops, so as long as each loop takes between 0.222 seconds and 0.23 seconds, the code can run successfully. I also test my own submission which runs for 7 hours. Each loop in it takes about 0.167 seconds.
", "votes": null, "replies": [ { "id": 1670033, "author_name": "amedprof", "author_url": "", "post_date": "01/31/2022 06:59:28", "content": "Thanks.
\nAfter using this API Emulator I found that run times changes a lot per iteration.
\nAnd I was able to fix it.
Glad it helped : )
", "votes": null, "replies": [] } ] }, { "id": 1669768, "author_name": "julianmukaj", "author_url": "", "post_date": "01/30/2022 23:16:32", "content": "I posted about inconsistent run times on notebooks too, it will be a real shame if something like this caused someone to be disqualified in the next 12 weeks.. Some reassurance would be good.
", "votes": null, "replies": [ { "id": 1670041, "author_name": "amedprof", "author_url": "", "post_date": "01/31/2022 07:03:34", "content": "Sure, from my experiment the successful kernel can also fail
", "votes": null, "replies": [] } ] }, { "id": 1669959, "author_name": "nrcjea001", "author_url": "", "post_date": "01/31/2022 05:15:26", "content": "I read somewhere that you need to remove any \"print\" and \"display\" functions when iterating over the test set. Did you include measuring and printing the time taken in your code? Remove that and see what happens
", "votes": null, "replies": [ { "id": 1670035, "author_name": "amedprof", "author_url": "", "post_date": "01/31/2022 07:00:25", "content": "hmm yes, I had that.
", "votes": null, "replies": [] } ] }, { "id": 1671019, "author_name": "saumilagrawal10", "author_url": "", "post_date": "02/01/2022 05:11:54", "content": "yes,even very simple notebooks which take about 10-15 seconds are getting timed out after scoring for 10-15 hours π€π€π€π€
", "votes": null, "replies": [ { "id": 1671568, "author_name": "amedprof", "author_url": "", "post_date": "02/01/2022 15:04:35", "content": "yes, I don't know why but my guess is a lot of submissions will fail during rerunsβ¦
", "votes": null, "replies": [] } ] }, { "id": 1671044, "author_name": "skw1990", "author_url": "", "post_date": "02/01/2022 05:39:55", "content": "Did you find out what's the issue?
", "votes": null, "replies": [ { "id": 1671536, "author_name": "amedprof", "author_url": "", "post_date": "02/01/2022 14:43:19", "content": "Hard to say because I changed a lot of things at the same time.
\nI removed all print functions and change my feature engineering function
Do share the script when the competition ended - I gave out after trying a dozen times of timeout lol;
\nThe competition hosts were not exactly helpful.