![\"fine](\"http://yosinski.com/static/proj/convnet_transfer_lines_crop.png\" "\\"\\"")
Here, i want to share my tested models log loss with you here:
\n\nFor information about fine tuning please study this paper .
\n\n
Many deep neural networks trained on natural images exhibit a curious phenomenon: they all learn roughly the same Gabor filters and color blobs on the first layer. These features seem to be generic — useful for many datasets and tasks — as opposed to specific — useful for only one dataset and task. By the last layer features must be task specific, which prompts the question: how do features transition from general to specific throughout the network? In this paper, presented at NIPS 2014, we show the manner in which features transition from general to specific, and also uncover a few other interesting results along the way.
\n\nReferences:
\n\n", "messages": [ { "id": "255994", "postDate": "12/10/2017 21:43:26", "content": "Here, i want to share my tested models log loss with you here:
\n\nFor information about fine tuning please study this paper .
\n\n
Many deep neural networks trained on natural images exhibit a curious phenomenon: they all learn roughly the same Gabor filters and color blobs on the first layer. These features seem to be generic — useful for many datasets and tasks — as opposed to specific — useful for only one dataset and task. By the last layer features must be task specific, which prompts the question: how do features transition from general to specific throughout the network? In this paper, presented at NIPS 2014, we show the manner in which features transition from general to specific, and also uncover a few other interesting results along the way.
\n\nReferences:
\n\n", "rawMarkdown": "Here, i want to share my tested models log loss with you here:\n\n 1. densnet169 with 10 epoch +fine tuning => log loss= 0.98114 \n 2. densenet121 last two layer with 5epochs+ fine tuning===> log loss=1.38279\n 3. densenet121 last two layer with 50 epochs+ fine tuning===> log loss=0.72061(best performance)\n 4. densenet 121- epoch-with 10 epoch without fine tuning( expensive in computation and time and this model is very massive for notebook so fine tuning is the great technique in this field) log loss=0.75694\n\nFor information about fine tuning please study this [paper][1] .\n\n\n![fine tuning and transfer learning][2]\n\n [1]: http://yosinski.com/media/papers/Yosinski__2014__NIPS__How_Transferable_with_Supp.pdf\n [2]: http://yosinski.com/static/proj/convnet_transfer_lines_crop.png\n\nMany deep neural networks trained on natural images exhibit a curious phenomenon: they all learn roughly the same Gabor filters and color blobs on the first layer. These features seem to be generic — useful for many datasets and tasks — as opposed to specific — useful for only one dataset and task. By the last layer features must be task specific, which prompts the question: how do features transition from general to specific throughout the network? In this paper, presented at NIPS 2014, we show the manner in which features transition from general to specific, and also uncover a few other interesting results along the way. \n\n----------------------------------------------------------\nReferences:\n\nhttp://yosinski.com/", "votes": null }, { "id": "256070", "postDate": "12/11/2017 03:12:08", "content": "Thank you aydin, that's very efficient for optimization.
", "rawMarkdown": "Thank you aydin, that's very efficient for optimization.", "votes": null }, { "id": "256100", "postDate": "12/11/2017 06:18:52", "content": "your welcome, thanks for your warm feedback!! :))
", "rawMarkdown": "your welcome, thanks for your warm feedback!! :))", "votes": null }, { "id": "258614", "postDate": "12/16/2017 15:48:21", "content": "DenseNet does not work very good in this case. I found Inception results better. However, best I could reach was an accuracy of 91% with log loss of 0.5 after training DenseNet 169 on PyTorch on 7 epochs. But on test data, it is still 0.65.
", "rawMarkdown": "DenseNet does not work very good in this case. I found Inception results better. However, best I could reach was an accuracy of 91% with log loss of 0.5 after training DenseNet 169 on PyTorch on 7 epochs. But on test data, it is still 0.65.", "votes": null }, { "id": "260208", "postDate": "12/19/2017 21:37:04", "content": "Thank you for sharing.
", "rawMarkdown": "Thank you for sharing.", "votes": null }, { "id": "260802", "postDate": "12/20/2017 23:18:30", "content": "Thanks for the insightful resources!
", "rawMarkdown": "Thanks for the insightful resources!", "votes": null }, { "id": "260810", "postDate": "12/21/2017 00:17:59", "content": "your welcome !! :)
", "rawMarkdown": "your welcome !! :)", "votes": null } ], "comments": [ { "id": 256070, "author_name": "andrew98", "author_url": "", "post_date": "12/11/2017 03:12:08", "content": "Thank you aydin, that's very efficient for optimization.
", "votes": null, "replies": [ { "id": 256100, "author_name": "ayanzadeh93", "author_url": "", "post_date": "12/11/2017 06:18:52", "content": "your welcome, thanks for your warm feedback!! :))
", "votes": null, "replies": [] } ] }, { "id": 258614, "author_name": "shridhar743", "author_url": "", "post_date": "12/16/2017 15:48:21", "content": "DenseNet does not work very good in this case. I found Inception results better. However, best I could reach was an accuracy of 91% with log loss of 0.5 after training DenseNet 169 on PyTorch on 7 epochs. But on test data, it is still 0.65.
", "votes": null, "replies": [] }, { "id": 260208, "author_name": "sahandv", "author_url": "", "post_date": "12/19/2017 21:37:04", "content": "Thank you for sharing.
", "votes": null, "replies": [] }, { "id": 260802, "author_name": "johnnymedhanie", "author_url": "", "post_date": "12/20/2017 23:18:30", "content": "Thanks for the insightful resources!
", "votes": null, "replies": [ { "id": 260810, "author_name": "ayanzadeh93", "author_url": "", "post_date": "12/21/2017 00:17:59", "content": "your welcome !! :)
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "255994": "Here, i want to share my tested models log loss with you here:\n\n 1. densnet169 with 10 epoch +fine tuning => log loss= 0.98114 \n 2. densenet121 last two layer with 5epochs+ fine tuning===> log loss=1.38279\n 3. densenet121 last two layer with 50 epochs+ fine tuning===> log loss=0.72061(best performance)\n 4. densenet 121- epoch-with 10 epoch without fine tuning( expensive in computation and time and this model is very massive for notebook so fine tuning is the great technique in this field) log loss=0.75694\n\nFor information about fine tuning please study this [paper][1] .\n\n\n![fine tuning and transfer learning][2]\n\n [1]: http://yosinski.com/media/papers/Yosinski__2014__NIPS__How_Transferable_with_Supp.pdf\n [2]: http://yosinski.com/static/proj/convnet_transfer_lines_crop.png\n\nMany deep neural networks trained on natural images exhibit a curious phenomenon: they all learn roughly the same Gabor filters and color blobs on the first layer. These features seem to be generic — useful for many datasets and tasks — as opposed to specific — useful for only one dataset and task. By the last layer features must be task specific, which prompts the question: how do features transition from general to specific throughout the network? In this paper, presented at NIPS 2014, we show the manner in which features transition from general to specific, and also uncover a few other interesting results along the way. \n\n----------------------------------------------------------\nReferences:\n\nhttp://yosinski.com/", "256070": "Thank you aydin, that's very efficient for optimization.", "256100": "your welcome, thanks for your warm feedback!! :))", "258614": "DenseNet does not work very good in this case. I found Inception results better. However, best I could reach was an accuracy of 91% with log loss of 0.5 after training DenseNet 169 on PyTorch on 7 epochs. But on test data, it is still 0.65.", "260208": "Thank you for sharing.", "260802": "Thanks for the insightful resources!", "260810": "your welcome !! :)" }, "source": "meta" }