Please @wendykan , could you include the dataset Dog face generation competition MiFID metric input as competition data?
\n\nNow if I want to validate my model and check the public FID, I have to:\n- commit my kernel (it can take hours)\n- upload images.zip\n- run Demo MiFID metric for Dog image generation comp\n- and if it's ok, submit (more hours)
(in addition, we only have 3 submissions per day)
\n\nif I could validate the model and calculate FID locally, in the kernel itself, it would save me a lot of time.
\n\nNow, we can't do it because we need to use Dog face generation competition MiFID metric input as private dataset and therefore we can't submit the kernel.
\n\nWhat do you think? Maybe I'm wrong and I can speed up this process
", "messages": [ { "id": "565734", "postDate": "07/01/2019 10:15:21", "content": "Please @wendykan , could you include the dataset Dog face generation competition MiFID metric input as competition data?
\n\nNow if I want to validate my model and check the public FID, I have to:\n- commit my kernel (it can take hours)\n- upload images.zip\n- run Demo MiFID metric for Dog image generation comp\n- and if it's ok, submit (more hours)
(in addition, we only have 3 submissions per day)
\n\nif I could validate the model and calculate FID locally, in the kernel itself, it would save me a lot of time.
\n\nNow, we can't do it because we need to use Dog face generation competition MiFID metric input as private dataset and therefore we can't submit the kernel.
\n\nWhat do you think? Maybe I'm wrong and I can speed up this process
", "rawMarkdown": "Please @wendykan , could you include the dataset [Dog face generation competition MiFID metric input](https://www.kaggle.com/wendykan/dog-face-generation-competition-kid-metric-input) as competition data?\n\nNow if I want to validate my model and check the ```public FID```, I have to:\n- commit my kernel (it can take hours)\n- upload ```images.zip```\n- run [Demo MiFID metric for Dog image generation comp](https://www.kaggle.com/wendykan/demo-mifid-metric-for-dog-image-generation-comp/notebook)\n- and if it's ok, submit (more hours)\n\n*(in addition, we only have 3 submissions per day)*\n\nif I could validate the model and calculate FID **locally**, in the kernel itself, it would save me a lot of time.\n\nNow, we can't do it because we need to use [Dog face generation competition MiFID metric input](https://www.kaggle.com/wendykan/dog-face-generation-competition-kid-metric-input) as private dataset and therefore we can't submit the kernel.\n\n---\n\nWhat do you think? Maybe I'm wrong and I can speed up this process", "votes": null }, { "id": "565808", "postDate": "07/01/2019 12:07:37", "content": "You can create 2 kernels. One for validation and one for submission.
\n\nThis is saving a lot of time to me. Hope this helps to you :)
", "rawMarkdown": "You can create 2 kernels. One for validation and one for submission.\n\n1. Crete a validation Kernel with the MiFID dataset\n2. Add the GANs code\n3. Add the validation code\n4. Execute the kernel and print the final score.\nIf the score is Ok, copy only the GANs code in the second kernel, execute and submit.\n\nThis is saving a lot of time to me. Hope this helps to you :)", "votes": null }, { "id": "565833", "postDate": "07/01/2019 12:38:05", "content": "I can't follow you. How would you train the GAN without validation? To my understanding a discriminator is always needed. \nThis is my first GAN challenge ever, so please be kind with my stupid questions ;)
", "rawMarkdown": "I can't follow you. How would you train the GAN without validation? To my understanding a discriminator is always needed. \nThis is my first GAN challenge ever, so please be kind with my stupid questions ;)", "votes": null }, { "id": "565834", "postDate": "07/01/2019 12:39:41", "content": "@ilu000 you can check this kernel where I train without validation GAN Introduction
", "rawMarkdown": "ilu000 you can check this kernel where I train without validation [GAN Introduction](https://www.kaggle.com/jesucristo/gan-introduction)", "votes": null }, { "id": "565840", "postDate": "07/01/2019 12:44:33", "content": "Gracias Alvaro! thanks @alvaroma I thought about it, but I can't run so many kernels at the same time :(\nlike:\n- commit 2 gans kernels & 2 the validation kernel for those 2 gan (4 kernels simultaneously) -> if the score it's ok I can submit
\n\nYou save time but it requires 4 kernels ...\nAnyway, thanks for the tip, I'll do that :)
", "rawMarkdown": "Gracias Alvaro! thanks @alvaroma I thought about it, but I can't run so many kernels at the same time :(\nlike:\n- commit 2 gans kernels & 2 the validation kernel for those 2 gan (4 kernels simultaneously) -> if the score it's ok I can submit\n\nYou save time but it requires 4 kernels ...\nAnyway, thanks for the tip, I'll do that :)", "votes": null }, { "id": "565846", "postDate": "07/01/2019 13:00:16", "content": "If we included the LB metric data, then we could score 1.5 on the LB by running the metric backwards. That would be overfitting the LB, but one could generalize to help increase private LB score.
", "rawMarkdown": "If we included the LB metric data, then we could score 1.5 on the LB by running the metric backwards. That would be overfitting the LB, but one could generalize to help increase private LB score.", "votes": null }, { "id": "565850", "postDate": "07/01/2019 13:05:01", "content": "@Nanashi LB and FID are very similar, so you can use the 5 kernels for development and validation.\nUse a kernel for submission only when you see an important improvement.
\n\nOn the other hand. You don't always need to execute full epochs to see improvements. In many cases, executing 10 epochs will show you if that works or not in just ~1 hour.
", "rawMarkdown": "Nanashi LB and FID are very similar, so you can use the 5 kernels for development and validation.\nUse a kernel for submission only when you see an important improvement.\n\nOn the other hand. You don't always need to execute full epochs to see improvements. In many cases, executing 10 epochs will show you if that works or not in just ~1 hour.", "votes": null }, { "id": "565868", "postDate": "07/01/2019 13:31:59", "content": "@ilu000 when I say validation I mean executing the FID metric function to see the final score.\nPlease, see: Demo MiFID metric for Dog image generation comp
", "rawMarkdown": "ilu000 when I say validation I mean executing the FID metric function to see the final score.\nPlease, see: [Demo MiFID metric for Dog image generation comp](https://www.kaggle.com/wendykan/demo-mifid-metric-for-dog-image-generation-comp)", "votes": null }, { "id": "565892", "postDate": "07/01/2019 14:01:16", "content": "Thanks for clarification @alvaroma and @jesucristo \nThat was already my workflow, but i wasn't paying that much attention to validation with FID metric, yet.
", "rawMarkdown": "Thanks for clarification @alvaroma and @jesucristo \nThat was already my workflow, but i wasn't paying that much attention to validation with FID metric, yet.", "votes": null }, { "id": "566153", "postDate": "07/01/2019 21:00:44", "content": "I am a bit sceptical. \nAs these are images, you should be pretty good to identify them as dogs yourself. Providing this score for the kernels, can lead to heavy overfitting and that's not really what we want. The goal is to get realistic dog pictures. For that, human score is best and very unbiased.
\n\nScoring the pictures yourself, is probably the best bet to prevent overfitting public LB in this competition anyway.
", "rawMarkdown": "I am a bit sceptical. \nAs these are images, you should be pretty good to identify them as dogs yourself. Providing this score for the kernels, can lead to heavy overfitting and that's not really what we want. The goal is to get realistic dog pictures. For that, human score is best and very unbiased.\n\nScoring the pictures yourself, is probably the best bet to prevent overfitting public LB in this competition anyway.", "votes": null }, { "id": "568803", "postDate": "07/05/2019 13:10:00", "content": "That's really tedious...
\n\nI'd prefer TA solution and add the MiFID dataset from @wendykan 's kernel to the competition.
\n\nThe private evaluation is supposed to have another MiFID dataset anyway, so I fail to see how it would be an issue for the competition.
", "rawMarkdown": "That's really tedious...\n\nI'd prefer TA solution and add the MiFID dataset from @wendykan 's kernel to the competition.\n\nThe private evaluation is supposed to have another MiFID dataset anyway, so I fail to see how it would be an issue for the competition.", "votes": null }, { "id": "568862", "postDate": "07/05/2019 15:00:20", "content": "I usually make 2 versions of my kernerls. The first one is dirty dev script (I use local validation framework there) and the other one is for submission. It takes a few minutes to copy successfull changes from one to another and prevents unwanted experimental settings from leaking into my submissions occasionally.
", "rawMarkdown": "I usually make 2 versions of my kernerls. The first one is dirty dev script (I use local validation framework there) and the other one is for submission. It takes a few minutes to copy successfull changes from one to another and prevents unwanted experimental settings from leaking into my submissions occasionally.", "votes": null } ], "comments": [ { "id": 565808, "author_name": "alvaroma", "author_url": "", "post_date": "07/01/2019 12:07:37", "content": "You can create 2 kernels. One for validation and one for submission.
\n\nThis is saving a lot of time to me. Hope this helps to you :)
", "votes": null, "replies": [ { "id": 565833, "author_name": "ilu000", "author_url": "", "post_date": "07/01/2019 12:38:05", "content": "I can't follow you. How would you train the GAN without validation? To my understanding a discriminator is always needed. \nThis is my first GAN challenge ever, so please be kind with my stupid questions ;)
", "votes": null, "replies": [] }, { "id": 565834, "author_name": "jesucristo", "author_url": "", "post_date": "07/01/2019 12:39:41", "content": "@ilu000 you can check this kernel where I train without validation GAN Introduction
", "votes": null, "replies": [] }, { "id": 565840, "author_name": "jesucristo", "author_url": "", "post_date": "07/01/2019 12:44:33", "content": "Gracias Alvaro! thanks @alvaroma I thought about it, but I can't run so many kernels at the same time :(\nlike:\n- commit 2 gans kernels & 2 the validation kernel for those 2 gan (4 kernels simultaneously) -> if the score it's ok I can submit
\n\nYou save time but it requires 4 kernels ...\nAnyway, thanks for the tip, I'll do that :)
", "votes": null, "replies": [] }, { "id": 565850, "author_name": "alvaroma", "author_url": "", "post_date": "07/01/2019 13:05:01", "content": "@Nanashi LB and FID are very similar, so you can use the 5 kernels for development and validation.\nUse a kernel for submission only when you see an important improvement.
\n\nOn the other hand. You don't always need to execute full epochs to see improvements. In many cases, executing 10 epochs will show you if that works or not in just ~1 hour.
", "votes": null, "replies": [] }, { "id": 565868, "author_name": "alvaroma", "author_url": "", "post_date": "07/01/2019 13:31:59", "content": "@ilu000 when I say validation I mean executing the FID metric function to see the final score.\nPlease, see: Demo MiFID metric for Dog image generation comp
", "votes": null, "replies": [] }, { "id": 565892, "author_name": "ilu000", "author_url": "", "post_date": "07/01/2019 14:01:16", "content": "Thanks for clarification @alvaroma and @jesucristo \nThat was already my workflow, but i wasn't paying that much attention to validation with FID metric, yet.
", "votes": null, "replies": [] }, { "id": 568803, "author_name": "guillaumedesforges", "author_url": "", "post_date": "07/05/2019 13:10:00", "content": "That's really tedious...
\n\nI'd prefer TA solution and add the MiFID dataset from @wendykan 's kernel to the competition.
\n\nThe private evaluation is supposed to have another MiFID dataset anyway, so I fail to see how it would be an issue for the competition.
", "votes": null, "replies": [] } ] }, { "id": 565846, "author_name": "cdeotte", "author_url": "", "post_date": "07/01/2019 13:00:16", "content": "If we included the LB metric data, then we could score 1.5 on the LB by running the metric backwards. That would be overfitting the LB, but one could generalize to help increase private LB score.
", "votes": null, "replies": [] }, { "id": 566153, "author_name": "ilu000", "author_url": "", "post_date": "07/01/2019 21:00:44", "content": "I am a bit sceptical. \nAs these are images, you should be pretty good to identify them as dogs yourself. Providing this score for the kernels, can lead to heavy overfitting and that's not really what we want. The goal is to get realistic dog pictures. For that, human score is best and very unbiased.
\n\nScoring the pictures yourself, is probably the best bet to prevent overfitting public LB in this competition anyway.
", "votes": null, "replies": [] }, { "id": 568862, "author_name": "dvorobiev", "author_url": "", "post_date": "07/05/2019 15:00:20", "content": "I usually make 2 versions of my kernerls. The first one is dirty dev script (I use local validation framework there) and the other one is for submission. It takes a few minutes to copy successfull changes from one to another and prevents unwanted experimental settings from leaking into my submissions occasionally.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "565734": "Please @wendykan , could you include the dataset [Dog face generation competition MiFID metric input](https://www.kaggle.com/wendykan/dog-face-generation-competition-kid-metric-input) as competition data?\n\nNow if I want to validate my model and check the ```public FID```, I have to:\n- commit my kernel (it can take hours)\n- upload ```images.zip```\n- run [Demo MiFID metric for Dog image generation comp](https://www.kaggle.com/wendykan/demo-mifid-metric-for-dog-image-generation-comp/notebook)\n- and if it's ok, submit (more hours)\n\n*(in addition, we only have 3 submissions per day)*\n\nif I could validate the model and calculate FID **locally**, in the kernel itself, it would save me a lot of time.\n\nNow, we can't do it because we need to use [Dog face generation competition MiFID metric input](https://www.kaggle.com/wendykan/dog-face-generation-competition-kid-metric-input) as private dataset and therefore we can't submit the kernel.\n\n---\n\nWhat do you think? Maybe I'm wrong and I can speed up this process", "565808": "You can create 2 kernels. One for validation and one for submission.\n\n1. Crete a validation Kernel with the MiFID dataset\n2. Add the GANs code\n3. Add the validation code\n4. Execute the kernel and print the final score.\nIf the score is Ok, copy only the GANs code in the second kernel, execute and submit.\n\nThis is saving a lot of time to me. Hope this helps to you :)", "565833": "I can't follow you. How would you train the GAN without validation? To my understanding a discriminator is always needed. \nThis is my first GAN challenge ever, so please be kind with my stupid questions ;)", "565834": "ilu000 you can check this kernel where I train without validation [GAN Introduction](https://www.kaggle.com/jesucristo/gan-introduction)", "565840": "Gracias Alvaro! thanks @alvaroma I thought about it, but I can't run so many kernels at the same time :(\nlike:\n- commit 2 gans kernels & 2 the validation kernel for those 2 gan (4 kernels simultaneously) -> if the score it's ok I can submit\n\nYou save time but it requires 4 kernels ...\nAnyway, thanks for the tip, I'll do that :)", "565846": "If we included the LB metric data, then we could score 1.5 on the LB by running the metric backwards. That would be overfitting the LB, but one could generalize to help increase private LB score.", "565850": "Nanashi LB and FID are very similar, so you can use the 5 kernels for development and validation.\nUse a kernel for submission only when you see an important improvement.\n\nOn the other hand. You don't always need to execute full epochs to see improvements. In many cases, executing 10 epochs will show you if that works or not in just ~1 hour.", "565868": "ilu000 when I say validation I mean executing the FID metric function to see the final score.\nPlease, see: [Demo MiFID metric for Dog image generation comp](https://www.kaggle.com/wendykan/demo-mifid-metric-for-dog-image-generation-comp)", "565892": "Thanks for clarification @alvaroma and @jesucristo \nThat was already my workflow, but i wasn't paying that much attention to validation with FID metric, yet.", "566153": "I am a bit sceptical. \nAs these are images, you should be pretty good to identify them as dogs yourself. Providing this score for the kernels, can lead to heavy overfitting and that's not really what we want. The goal is to get realistic dog pictures. For that, human score is best and very unbiased.\n\nScoring the pictures yourself, is probably the best bet to prevent overfitting public LB in this competition anyway.", "568803": "That's really tedious...\n\nI'd prefer TA solution and add the MiFID dataset from @wendykan 's kernel to the competition.\n\nThe private evaluation is supposed to have another MiFID dataset anyway, so I fail to see how it would be an issue for the competition.", "568862": "I usually make 2 versions of my kernerls. The first one is dirty dev script (I use local validation framework there) and the other one is for submission. It takes a few minutes to copy successfull changes from one to another and prevents unwanted experimental settings from leaking into my submissions occasionally." }, "source": "meta" }