I have done the training of my model on google colab! I want to use the trained weights for inference on kaggle? Is there an (optimal) way to do that?
\n\nI tried to download the weights on my computer then tried to upload on kaggle: not working.\nI tried to save wights on the drive and use the link to upload the weights on kaggle: not working.
\n\nAny suggestions?
", "messages": [ { "id": 925288, "postDate": "2020-07-12T01:47:08.353Z", "content": "I trained a model using tensorflow api on colab. i am able to do inference on colab but not on kaggle .\nCan you help me out which files do I need for inference
", "rawMarkdown": "I trained a model using tensorflow api on colab. i am able to do inference on colab but not on kaggle .\nCan you help me out which files do I need for inference", "votes": 1, "replies": [ { "id": 925311, "postDate": "2020-07-12T02:21:33.817Z", "content": "You can train your models on Colab =>Save & Download weights (the trained model as .pth file for example) => Make an inference kernel on Kaggle (You can check some public Kernels) => Upload weights to kaggle (As a Dataset)=> Add them to your inference kernel (As input)=> Commit => Submit.
", "rawMarkdown": "You can train your models on Colab =>Save & Download weights (the trained model as .pth file for example) => Make an inference kernel on Kaggle (You can check some public Kernels) => Upload weights to kaggle (As a Dataset)=> Add them to your inference kernel (As input)=> Commit => Submit." }, { "id": 925326, "postDate": "2020-07-12T02:40:42.153Z", "content": "Do you happens to know of some Inference kernel for .pb file???
", "rawMarkdown": "Do you happens to know of some Inference kernel for .pb file???" }, { "id": 926162, "postDate": "2020-07-12T14:39:58.967Z", "content": "Check this as an example.
", "rawMarkdown": "Check [this](https://www.kaggle.com/ravirajsinh45/global-wheat-tensorflow-object-detection-api) as an example. " } ] }, { "id": 866267, "postDate": "2020-05-29T08:33:59.743Z", "content": "What I do is use gcs bucket. It is really cheap and costs almost nothing. After training I simply use gcs api to directly send the model to the bucket from colab notebook. \n1. When adding it to Kaggle, I create a dataset and select upload by link.\n2. I start downloading from the bucket to my computer, as soon as the download starts, I pause it, copy the link and paste it into Kaggle dataset creator, start the dataset creation and cancel the download on my computer.\n3. Kaggle downloads the model really fast since it is a direct gcs to gcs copy internally and when it is done, you can add this dataset to your inference notebook.\nThis is a hack which I use because I dont know if direct upload of models to datasets is possible via Kaggle api.\nFor Competitions which are not notebook based, you can directly submit the csv file using the kaggle api from colab.
", "rawMarkdown": "What I do is use gcs bucket. It is really cheap and costs almost nothing. After training I simply use gcs api to directly send the model to the bucket from colab notebook. \n1. When adding it to Kaggle, I create a dataset and select upload by link.\n2. I start downloading from the bucket to my computer, as soon as the download starts, I pause it, copy the link and paste it into Kaggle dataset creator, start the dataset creation and cancel the download on my computer.\n3. Kaggle downloads the model really fast since it is a direct gcs to gcs copy internally and when it is done, you can add this dataset to your inference notebook.\nThis is a hack which I use because I dont know if direct upload of models to datasets is possible via Kaggle api.\nFor Competitions which are not notebook based, you can directly submit the csv file using the kaggle api from colab.", "votes": 2 }, { "id": 865129, "postDate": "2020-05-28T12:11:03.863Z", "content": "I have done the training of my model on google colab! I want to use the trained weights for inference on kaggle? Is there an (optimal) way to do that?
\n\nI tried to download the weights on my computer then tried to upload on kaggle: not working.\nI tried to save wights on the drive and use the link to upload the weights on kaggle: not working.
\n\nAny suggestions?
", "rawMarkdown": "I have done the training of my model on google colab! I want to use the trained weights for inference on kaggle? Is there an (optimal) way to do that? \n\nI tried to download the weights on my computer then tried to upload on kaggle: not working.\nI tried to save wights on the drive and use the link to upload the weights on kaggle: not working.\n\nAny suggestions? ", "votes": 2 }, { "id": 865133, "postDate": "2020-05-28T12:18:57.423Z", "content": "Where do the above approaches fail? Are you unable to save or download files from Colab at all?
", "rawMarkdown": "Where do the above approaches fail? Are you unable to save or download files from Colab at all?", "replies": [ { "id": 865152, "postDate": "2020-05-28T12:34:38.497Z", "content": "Actually, I can save/download files from colab. I am not able to upload the files to Kaggle. It takes me a long time to upload a file from my computer to kaggle. And when I tried to use a shareable link form the drive to create a dataset on kaggle that didn't work as well.
", "rawMarkdown": "Actually, I can save/download files from colab. I am not able to upload the files to Kaggle. It takes me a long time to upload a file from my computer to kaggle. And when I tried to use a shareable link form the drive to create a dataset on kaggle that didn't work as well. " }, { "id": 865205, "postDate": "2020-05-28T13:22:41.170Z", "content": "The only way I am aware of uploading weights is by uploading the weights to the website in the browser. I have average US internet and it takes 5-10 minutes for me to upload weights for a fasterRCNN model with resnet50 backbone.
", "rawMarkdown": "The only way I am aware of uploading weights is by uploading the weights to the website in the browser. I have average US internet and it takes 5-10 minutes for me to upload weights for a fasterRCNN model with resnet50 backbone.", "votes": 1 }, { "id": 865228, "postDate": "2020-05-28T13:43:34.753Z", "content": "@ffares I do my training on Colab Pro, with all my input/output files on my Google Drive. After training runs, I download the weights from Google Drive to my local drive, create a new dataset in Kaggle (via YOUR DATASETS) and upload the weights, and then do the usual 'Add Data' to my Kaggle notebook. So far I have not had any issues. It takes a minute or two to upload a 400MB file to Kaggle on my 25Mbps Internet connection. Obviously, your upload speed will vary.
", "rawMarkdown": "@ffares I do my training on Colab Pro, with all my input/output files on my Google Drive. After training runs, I download the weights from Google Drive to my local drive, create a new dataset in Kaggle (via YOUR DATASETS) and upload the weights, and then do the usual 'Add Data' to my Kaggle notebook. So far I have not had any issues. It takes a minute or two to upload a 400MB file to Kaggle on my 25Mbps Internet connection. Obviously, your upload speed will vary.", "votes": 2 }, { "id": 865325, "postDate": "2020-05-28T14:55:06.630Z", "content": "Okay I'll check my internet and I'll find a way! Thanks.
", "rawMarkdown": "Okay I'll check my internet and I'll find a way! Thanks. " }, { "id": 912951, "postDate": "2020-07-02T21:04:20.630Z", "content": "I started using Colabs Pro as well and it seems to constantly reload and timeout. It may restart here and there but it seems to have issues that I never experienced with free. @msjose are you running High Ram and GPU? I'm using it for YOLOv5x. It's running about 2:20 an epoch and often it times out and I have to hit cancel or reload and cancel and then it might rerun epochs from where it ended but if I go to bed I cannot be there to monitor. That's why I got it.. to run it while I'm doing other things. Just curious if you had any issues or if you changed any settings.
", "rawMarkdown": "I started using Colabs Pro as well and it seems to constantly reload and timeout. It may restart here and there but it seems to have issues that I never experienced with free. @msjose are you running High Ram and GPU? I'm using it for YOLOv5x. It's running about 2:20 an epoch and often it times out and I have to hit cancel or reload and cancel and then it might rerun epochs from where it ended but if I go to bed I cannot be there to monitor. That's why I got it.. to run it while I'm doing other things. Just curious if you had any issues or if you changed any settings. " } ] }, { "id": 865224, "postDate": "2020-05-28T13:39:52.887Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 925288, "author_name": "Bhavesh Jain", "author_url": "", "post_date": "2020-07-12T01:47:08.353000", "content": "I trained a model using tensorflow api on colab. i am able to do inference on colab but not on kaggle .\nCan you help me out which files do I need for inference
", "votes": 1, "replies": [ { "id": 925311, "author_name": "Fares FOURATI", "author_url": "", "post_date": "2020-07-12T02:21:33.817000", "content": "You can train your models on Colab =>Save & Download weights (the trained model as .pth file for example) => Make an inference kernel on Kaggle (You can check some public Kernels) => Upload weights to kaggle (As a Dataset)=> Add them to your inference kernel (As input)=> Commit => Submit.
", "votes": 0, "replies": [] }, { "id": 925326, "author_name": "Bhavesh Jain", "author_url": "", "post_date": "2020-07-12T02:40:42.153000", "content": "Do you happens to know of some Inference kernel for .pb file???
", "votes": 0, "replies": [] }, { "id": 926162, "author_name": "Fares FOURATI", "author_url": "", "post_date": "2020-07-12T14:39:58.967000", "content": "Check this as an example.
", "votes": 0, "replies": [] } ] }, { "id": 866267, "author_name": "Utsav Nandi", "author_url": "", "post_date": "2020-05-29T08:33:59.743000", "content": "What I do is use gcs bucket. It is really cheap and costs almost nothing. After training I simply use gcs api to directly send the model to the bucket from colab notebook. \n1. When adding it to Kaggle, I create a dataset and select upload by link.\n2. I start downloading from the bucket to my computer, as soon as the download starts, I pause it, copy the link and paste it into Kaggle dataset creator, start the dataset creation and cancel the download on my computer.\n3. Kaggle downloads the model really fast since it is a direct gcs to gcs copy internally and when it is done, you can add this dataset to your inference notebook.\nThis is a hack which I use because I dont know if direct upload of models to datasets is possible via Kaggle api.\nFor Competitions which are not notebook based, you can directly submit the csv file using the kaggle api from colab.
", "votes": 2, "replies": [] }, { "id": 865133, "author_name": "Matt", "author_url": "", "post_date": "2020-05-28T12:18:57.423000", "content": "Where do the above approaches fail? Are you unable to save or download files from Colab at all?
", "votes": 0, "replies": [ { "id": 865152, "author_name": "Fares FOURATI", "author_url": "", "post_date": "2020-05-28T12:34:38.497000", "content": "Actually, I can save/download files from colab. I am not able to upload the files to Kaggle. It takes me a long time to upload a file from my computer to kaggle. And when I tried to use a shareable link form the drive to create a dataset on kaggle that didn't work as well.
", "votes": 0, "replies": [] }, { "id": 865205, "author_name": "Matt", "author_url": "", "post_date": "2020-05-28T13:22:41.170000", "content": "The only way I am aware of uploading weights is by uploading the weights to the website in the browser. I have average US internet and it takes 5-10 minutes for me to upload weights for a fasterRCNN model with resnet50 backbone.
", "votes": 1, "replies": [] }, { "id": 865228, "author_name": "MSJose", "author_url": "", "post_date": "2020-05-28T13:43:34.753000", "content": "@ffares I do my training on Colab Pro, with all my input/output files on my Google Drive. After training runs, I download the weights from Google Drive to my local drive, create a new dataset in Kaggle (via YOUR DATASETS) and upload the weights, and then do the usual 'Add Data' to my Kaggle notebook. So far I have not had any issues. It takes a minute or two to upload a 400MB file to Kaggle on my 25Mbps Internet connection. Obviously, your upload speed will vary.
", "votes": 2, "replies": [] }, { "id": 865325, "author_name": "Fares FOURATI", "author_url": "", "post_date": "2020-05-28T14:55:06.630000", "content": "Okay I'll check my internet and I'll find a way! Thanks.
", "votes": 0, "replies": [] }, { "id": 912951, "author_name": "Charlie Craine", "author_url": "", "post_date": "2020-07-02T21:04:20.630000", "content": "I started using Colabs Pro as well and it seems to constantly reload and timeout. It may restart here and there but it seems to have issues that I never experienced with free. @msjose are you running High Ram and GPU? I'm using it for YOLOv5x. It's running about 2:20 an epoch and often it times out and I have to hit cancel or reload and cancel and then it might rerun epochs from where it ended but if I go to bed I cannot be there to monitor. That's why I got it.. to run it while I'm doing other things. Just curious if you had any issues or if you changed any settings.
", "votes": 0, "replies": [] } ] }, { "id": 865224, "author_name": "", "author_url": "", "post_date": "2020-05-28T13:39:52.887000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "925288": "I trained a model using tensorflow api on colab. i am able to do inference on colab but not on kaggle .\nCan you help me out which files do I need for inference", "866267": "What I do is use gcs bucket. It is really cheap and costs almost nothing. After training I simply use gcs api to directly send the model to the bucket from colab notebook. \n1. When adding it to Kaggle, I create a dataset and select upload by link.\n2. I start downloading from the bucket to my computer, as soon as the download starts, I pause it, copy the link and paste it into Kaggle dataset creator, start the dataset creation and cancel the download on my computer.\n3. Kaggle downloads the model really fast since it is a direct gcs to gcs copy internally and when it is done, you can add this dataset to your inference notebook.\nThis is a hack which I use because I dont know if direct upload of models to datasets is possible via Kaggle api.\nFor Competitions which are not notebook based, you can directly submit the csv file using the kaggle api from colab.", "865129": "I have done the training of my model on google colab! I want to use the trained weights for inference on kaggle? Is there an (optimal) way to do that? \n\nI tried to download the weights on my computer then tried to upload on kaggle: not working.\nI tried to save wights on the drive and use the link to upload the weights on kaggle: not working.\n\nAny suggestions? ", "865133": "Where do the above approaches fail? Are you unable to save or download files from Colab at all?", "865224": "" } }