If I understood correctly, it is required to submit a notebook which runs in 9 hours on a CPU or in 2 hours with a GPU. What I never really understood about code competitions is whether it is allowed to just train locally on your device, upload predictions in a dataset on Kaggle, and submit a notebook which loads those predictions and saves them.
\nI just want confirmation from someone that this is indeed not allowed?
", "messages": [ { "id": 2310640, "postDate": "2023-06-20T14:15:52.650Z", "content": "If I understood correctly, it is required to submit a notebook which runs in 9 hours on a CPU or in 2 hours with a GPU. What I never really understood about code competitions is whether it is allowed to just train locally on your device, upload predictions in a dataset on Kaggle, and submit a notebook which loads those predictions and saves them.
\nI just want confirmation from someone that this is indeed not allowed?
", "rawMarkdown": "If I understood correctly, it is required to submit a notebook which runs in 9 hours on a CPU or in 2 hours with a GPU. What I never really understood about code competitions is whether it is allowed to just train locally on your device, upload predictions in a dataset on Kaggle, and submit a notebook which loads those predictions and saves them.\n\nI just want confirmation from someone that this is indeed not allowed?", "votes": 5 }, { "id": 2310928, "postDate": "2023-06-20T17:41:47.577Z", "content": "This is not a code competiton, you can train and infer your models locally, this notebook will show you how to submit : https://www.kaggle.com/code/inversion/submitting-to-the-cafa-competition
", "rawMarkdown": "This is not a code competiton, you can train and infer your models locally, this notebook will show you how to submit : https://www.kaggle.com/code/inversion/submitting-to-the-cafa-competition", "votes": 1, "replies": [ { "id": 2310937, "postDate": "2023-06-20T17:49:38.087Z", "content": "Oh. Yes that why I got confused, thanks! Assuming you have a local access to a GPU or via Colab, the most efficient way to submit is then:
\nTrain locally
\nInfer locally
\nPush the the tsv file to a private dataset
\nAdjust the path in your submission notebook
\nSubmit
What do you think?
", "rawMarkdown": "Oh. Yes that why I got confused, thanks! Assuming you have a local access to a GPU or via Colab, the most efficient way to submit is then:\n\nTrain locally\nInfer locally\nPush the the tsv file to a private dataset\nAdjust the path in your submission notebook\nSubmit\n\nWhat do you think? \n", "votes": 1, "replies": [ { "id": 2310942, "postDate": "2023-06-20T17:57:42.043Z", "content": "yes, after creating the dataset, if there are memory issues, you can just copy the submission file to the notebook without opening it
", "rawMarkdown": "yes, after creating the dataset, if there are memory issues, you can just copy the submission file to the notebook without opening it", "votes": 1 }, { "id": 2313792, "postDate": "2023-06-23T00:50:21.727Z", "rawMarkdown": "", "isDeleted": true } ] } ] }, { "id": 2310788, "postDate": "2023-06-20T15:52:25.283Z", "content": "@gzguevara, in code competitions, your model test set is hidden. Using your strategy, you could predict your training observations, but how will your work be valid on the unseen test set?
\nBTW, your allotted time with a GPU notebook is usually 9 hours, this is a special case where you are given 2 hours to run the pipeline.
I see. but what about training a model and then upload it to a kaggle dataset? wouldn't this be a valid submission?
", "rawMarkdown": "I see. but what about training a model and then upload it to a kaggle dataset? wouldn't this be a valid submission?", "replies": [ { "id": 2312267, "postDate": "2023-06-21T19:30:54.120Z", "content": "Of course it is and is what most people do when Kaggle's compute resources are insufficient. This is more common in Computer Vision competitions: P100s GPUs are not that powerful and people usually train models by renting more powerful GPUs in AWS, GCP, JarvisLab, etc. After training you just save the model and upload it to Kaggle and just make an inference code. Training is performed outside Kaggle. Hope this helps @gzguevara
", "rawMarkdown": "Of course it is and is what most people do when Kaggle's compute resources are insufficient. This is more common in Computer Vision competitions: P100s GPUs are not that powerful and people usually train models by renting more powerful GPUs in AWS, GCP, JarvisLab, etc. After training you just save the model and upload it to Kaggle and just make an inference code. Training is performed outside Kaggle. Hope this helps @gzguevara " } ] } ] } ], "comments": [ { "id": 2310928, "author_name": "Ahmed El Fazouani", "author_url": "", "post_date": "2023-06-20T17:41:47.577000", "content": "This is not a code competiton, you can train and infer your models locally, this notebook will show you how to submit : https://www.kaggle.com/code/inversion/submitting-to-the-cafa-competition
", "votes": 1, "replies": [ { "id": 2310937, "author_name": "gzguevara", "author_url": "", "post_date": "2023-06-20T17:49:38.087000", "content": "Oh. Yes that why I got confused, thanks! Assuming you have a local access to a GPU or via Colab, the most efficient way to submit is then:
\nTrain locally
\nInfer locally
\nPush the the tsv file to a private dataset
\nAdjust the path in your submission notebook
\nSubmit
What do you think?
", "votes": 1, "replies": [ { "id": 2310942, "author_name": "Ahmed El Fazouani", "author_url": "", "post_date": "2023-06-20T17:57:42.043000", "content": "yes, after creating the dataset, if there are memory issues, you can just copy the submission file to the notebook without opening it
", "votes": 1, "replies": [] }, { "id": 2313792, "author_name": "", "author_url": "", "post_date": "2023-06-23T00:50:21.727000", "content": "", "votes": 0, "replies": [] } ] } ] }, { "id": 2310788, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2023-06-20T15:52:25.283000", "content": "@gzguevara, in code competitions, your model test set is hidden. Using your strategy, you could predict your training observations, but how will your work be valid on the unseen test set?
\nBTW, your allotted time with a GPU notebook is usually 9 hours, this is a special case where you are given 2 hours to run the pipeline.
I see. but what about training a model and then upload it to a kaggle dataset? wouldn't this be a valid submission?
", "votes": 0, "replies": [ { "id": 2312267, "author_name": "moth", "author_url": "", "post_date": "2023-06-21T19:30:54.120000", "content": "Of course it is and is what most people do when Kaggle's compute resources are insufficient. This is more common in Computer Vision competitions: P100s GPUs are not that powerful and people usually train models by renting more powerful GPUs in AWS, GCP, JarvisLab, etc. After training you just save the model and upload it to Kaggle and just make an inference code. Training is performed outside Kaggle. Hope this helps @gzguevara
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2310640": "If I understood correctly, it is required to submit a notebook which runs in 9 hours on a CPU or in 2 hours with a GPU. What I never really understood about code competitions is whether it is allowed to just train locally on your device, upload predictions in a dataset on Kaggle, and submit a notebook which loads those predictions and saves them.\n\nI just want confirmation from someone that this is indeed not allowed?", "2310928": "This is not a code competiton, you can train and infer your models locally, this notebook will show you how to submit : https://www.kaggle.com/code/inversion/submitting-to-the-cafa-competition", "2310788": "@gzguevara, in code competitions, your model test set is hidden. Using your strategy, you could predict your training observations, but how will your work be valid on the unseen test set? \nBTW, your allotted time with a GPU notebook is usually 9 hours, this is a special case where you are given 2 hours to run the pipeline. " } }