Congratulations on the excellent performance of the top teams. Also I would like to thank all participants for their active participation to the competition.
\n\nTo the best of my knowledge, the top three teams represent the two host universities (TUT and TUGraz) as well as a non-member to these institutes. Thus, we will provide a small prize to all members of top three teams. I would like to ask these teams to contact me at heikki.huttunen@tut.fi to agree on the details.
\n\nAlso, I would be excited to hear about the techniques people were using publicly in this thread. I know deep networks are strong in face verification, and in our own experiments we reached 0.996 score with VGG-Face features coupled with another deep net. However, the two top teams are above this score, so what's your secret?
", "messages": [ { "id": "101815", "postDate": "12/17/2015 08:31:53", "content": "Congratulations on the excellent performance of the top teams. Also I would like to thank all participants for their active participation to the competition.
\n\nTo the best of my knowledge, the top three teams represent the two host universities (TUT and TUGraz) as well as a non-member to these institutes. Thus, we will provide a small prize to all members of top three teams. I would like to ask these teams to contact me at heikki.huttunen@tut.fi to agree on the details.
\n\nAlso, I would be excited to hear about the techniques people were using publicly in this thread. I know deep networks are strong in face verification, and in our own experiments we reached 0.996 score with VGG-Face features coupled with another deep net. However, the two top teams are above this score, so what's your secret?
", "rawMarkdown": "Congratulations on the excellent performance of the top teams. Also I would like to thank all participants for their active participation to the competition.\r\n\r\nTo the best of my knowledge, the top three teams represent the two host universities (TUT and TUGraz) as well as a non-member to these institutes. Thus, we will provide a small prize to all members of top three teams. I would like to ask these teams to contact me at heikki.huttunen@tut.fi to agree on the details.\r\n\r\nAlso, I would be excited to hear about the techniques people were using publicly in this thread. I know deep networks are strong in face verification, and in our own experiments we reached 0.996 score with VGG-Face features coupled with another deep net. However, the two top teams are above this score, so what's your secret?", "votes": null }, { "id": "101837", "postDate": "12/17/2015 12:27:35", "content": "Seems like nobody won the competition :D
\n\nMy first submission was using a custom CNN trained on CASIA Web Faces (~0.5-0.4 million images), with a combined softmax and LMNN loss. With this I was able to achieve a score of 0.98.
\n\nLater I've switched to Parkhi's VGG-Face features, since he can train larger networks on larger datasets (2.5 million faces). To normalize faces I'm using the 5 facial landmarks and estimate a similarity transform to a normalized shape. From the normalized images I'm extracting CNN features (without any further computationally expensive sub-crop averaging as proposed in the paper). With just this simple method + Cosine distance I'm able to achieve 0.99952 score. Internally I've split the training set into a validation and training set with disjoint identities and measured TP rate at 0.01 FP. With this solution I'm able to achieve ~0.77 TP at 0.01 FP.
\n\nTo further improve results, I've experimented with LMNN embeddings on top of fixed, l2-normed feature vectors (same as in their paper). Following the paper, I've trained a 1024 dimensional LMNN embedding on top of normalized features. With this submission I've improved to 0.99969. On my internal validation set I'm able to achieve ~0.81 TP at 0.01 FP with this method.
\n\nFinally, I've increased the embedding size to 4096 and added stronger regularization, which helped to improve the validation set accuracy from ~0.81 TP at 0.01 FP to ~0.83 at 0.01 FP, but did not show any improvements on the leaderboard score.
\n\nI've also experimented with JointBayes, but the dimensionality of the last feature layer (4096) was too high to estimate non-singular covariance matrices. With a PCA reduction to 400 dimensions it worked, but produced inferior results to the LMNN method (~0.80 TP at 0.01 FP). I guess it would have been better to compress the dimenionality with LMNN and then throw JointBayes on top of that, but since the larger LMNN embedding did not show any improvement on the public score, I've abandoned this approach.
", "rawMarkdown": "Seems like nobody won the competition :D\r\n\r\nMy first submission was using a custom CNN trained on CASIA Web Faces (~0.5-0.4 million images), with a combined softmax and LMNN loss. With this I was able to achieve a score of 0.98.\r\n\r\nLater I've switched to Parkhi's VGG-Face features, since he can train larger networks on larger datasets (2.5 million faces). To normalize faces I'm using the 5 facial landmarks and estimate a similarity transform to a normalized shape. From the normalized images I'm extracting CNN features (without any further computationally expensive sub-crop averaging as proposed in the paper). With just this simple method + Cosine distance I'm able to achieve 0.99952 score. Internally I've split the training set into a validation and training set with disjoint identities and measured TP rate at 0.01 FP. With this solution I'm able to achieve ~0.77 TP at 0.01 FP.\r\n\r\nTo further improve results, I've experimented with LMNN embeddings on top of fixed, l2-normed feature vectors (same as in their paper). Following the paper, I've trained a 1024 dimensional LMNN embedding on top of normalized features. With this submission I've improved to 0.99969. On my internal validation set I'm able to achieve ~0.81 TP at 0.01 FP with this method.\r\n\r\nFinally, I've increased the embedding size to 4096 and added stronger regularization, which helped to improve the validation set accuracy from ~0.81 TP at 0.01 FP to ~0.83 at 0.01 FP, but did not show any improvements on the leaderboard score. \r\n\r\nI've also experimented with JointBayes, but the dimensionality of the last feature layer (4096) was too high to estimate non-singular covariance matrices. With a PCA reduction to 400 dimensions it worked, but produced inferior results to the LMNN method (~0.80 TP at 0.01 FP). I guess it would have been better to compress the dimenionality with LMNN and then throw JointBayes on top of that, but since the larger LMNN embedding did not show any improvement on the public score, I've abandoned this approach.", "votes": null }, { "id": "101838", "postDate": "12/17/2015 12:28:48", "content": "Ahh, such an interesting competition and i didnt have anytime (also, infrastructure) to do it :(
", "rawMarkdown": "Ahh, such an interesting competition and i didnt have anytime (also, infrastructure) to do it :(", "votes": null }, { "id": "102084", "postDate": "12/19/2015 09:54:02", "content": "It was so lucky for my team to get into the top 3 of the rank, although there is a distance of the score between the top2 and us, I would like to share some experience about this competition from my perspective. There is a Github link that I and my teammates used to communicate our code(Matlab):\ncontest codes.
\n\nSo what we did is to try out some machine learning algorithms that we have learned from the lecture that can be used to this competition, after implementing KNN, Logistic Regression, SVM, MLP, we find the SVM and MLP can get a higher score than KNN.
\n\nWhat I found is to use SVM and MLP algorithms to train, use the deep feature which have 4096 dimensions as inputs, after getting the result from the algorithm, use softmax or euclidean distance to get the final distribution of the probabilities that a pair of photos are the same person.
\n\nFor speeding up the program we used PCA to reduce the 4096 dimensions into 1015 dimensions which is able to keep 99% of the original information in deep features.
\n\nIn SVM, we used the matlab built-in function fitceceo or fitcsvm to train a one versus all model over 271 different persons' photos, this means in the ends of the training there will be 271 classifiers, each classifier is one person against all the other 270 persons. After getting the weights of all the 271 classifiers, get the result by calculate the distance between the two photos to every hyperplane(classifier). It can get the score about 0.944 without passing any other parameters to the SVM.
\n\nIn MLP, we defined a simple feed-forward network with only one hidden layer, and used back propagation to train the network. After getting the weights for each layer, input new photo features and get the result in output layer, what's different from SVM is that we didn't use any sigmoid to activate the output neurons, so each photo will become a 271*1 vector without activating, then get the final result base on the euclidean distance between each pair of photos. After making some cross validation we found that setting the regularization parameter to be 3, and the hidden layer size to be 300 can get the highest accuracy, and the score is 0.964 when using these parameters. A pity of MLP is that we didn't try the result with more than one hidden layer, may be it could be better.
\n\nDuring the validation, we found that changing the value of C in SVM can not make any difference of the accuracy of training, I don't know why, may be find more data can be different, we didn't use a RBF kernel because I thought the dimensions are too high to use it. While in validating with MLP, accuracies are changing with the change of regularization parameter and the hidden layer size.
\n\nAll in all, I think it is an very interesting competition that motivates me a lot to implement what I have learned.
", "rawMarkdown": "It was so lucky for my team to get into the top 3 of the rank, although there is a distance of the score between the top2 and us, I would like to share some experience about this competition from my perspective. There is a Github link that I and my teammates used to communicate our code(Matlab):\r\n[contest codes.][1]\r\n\r\nSo what we did is to try out some machine learning algorithms that we have learned from the lecture that can be used to this competition, after implementing KNN, Logistic Regression, SVM, MLP, we find the SVM and MLP can get a higher score than KNN.\r\n\r\nWhat I found is to use SVM and MLP algorithms to train, use the deep feature which have 4096 dimensions as inputs, after getting the result from the algorithm, use softmax or euclidean distance to get the final distribution of the probabilities that a pair of photos are the same person.\r\n\r\nFor speeding up the program we used PCA to reduce the 4096 dimensions into 1015 dimensions which is able to keep 99% of the original information in deep features.\r\n\r\nIn SVM, we used the matlab built-in function fitceceo or fitcsvm to train a one versus all model over 271 different persons' photos, this means in the ends of the training there will be 271 classifiers, each classifier is one person against all the other 270 persons. After getting the weights of all the 271 classifiers, get the result by calculate the distance between the two photos to every hyperplane(classifier). It can get the score about 0.944 without passing any other parameters to the SVM.\r\n\r\nIn MLP, we defined a simple feed-forward network with only one hidden layer, and used back propagation to train the network. After getting the weights for each layer, input new photo features and get the result in output layer, what's different from SVM is that we didn't use any sigmoid to activate the output neurons, so each photo will become a 271*1 vector without activating, then get the final result base on the euclidean distance between each pair of photos. After making some cross validation we found that setting the regularization parameter to be 3, and the hidden layer size to be 300 can get the highest accuracy, and the score is 0.964 when using these parameters. A pity of MLP is that we didn't try the result with more than one hidden layer, may be it could be better.\r\n\r\nDuring the validation, we found that changing the value of C in SVM can not make any difference of the accuracy of training, I don't know why, may be find more data can be different, we didn't use a RBF kernel because I thought the dimensions are too high to use it. While in validating with MLP, accuracies are changing with the change of regularization parameter and the hidden layer size.\r\n\r\nAll in all, I think it is an very interesting competition that motivates me a lot to implement what I have learned.\r\n\r\n\r\n [1]: https://github.com/chenmengyang/ML-contest-Face-Verification", "votes": null } ], "comments": [ { "id": 101837, "author_name": "naxat88", "author_url": "", "post_date": "12/17/2015 12:27:35", "content": "Seems like nobody won the competition :D
\n\nMy first submission was using a custom CNN trained on CASIA Web Faces (~0.5-0.4 million images), with a combined softmax and LMNN loss. With this I was able to achieve a score of 0.98.
\n\nLater I've switched to Parkhi's VGG-Face features, since he can train larger networks on larger datasets (2.5 million faces). To normalize faces I'm using the 5 facial landmarks and estimate a similarity transform to a normalized shape. From the normalized images I'm extracting CNN features (without any further computationally expensive sub-crop averaging as proposed in the paper). With just this simple method + Cosine distance I'm able to achieve 0.99952 score. Internally I've split the training set into a validation and training set with disjoint identities and measured TP rate at 0.01 FP. With this solution I'm able to achieve ~0.77 TP at 0.01 FP.
\n\nTo further improve results, I've experimented with LMNN embeddings on top of fixed, l2-normed feature vectors (same as in their paper). Following the paper, I've trained a 1024 dimensional LMNN embedding on top of normalized features. With this submission I've improved to 0.99969. On my internal validation set I'm able to achieve ~0.81 TP at 0.01 FP with this method.
\n\nFinally, I've increased the embedding size to 4096 and added stronger regularization, which helped to improve the validation set accuracy from ~0.81 TP at 0.01 FP to ~0.83 at 0.01 FP, but did not show any improvements on the leaderboard score.
\n\nI've also experimented with JointBayes, but the dimensionality of the last feature layer (4096) was too high to estimate non-singular covariance matrices. With a PCA reduction to 400 dimensions it worked, but produced inferior results to the LMNN method (~0.80 TP at 0.01 FP). I guess it would have been better to compress the dimenionality with LMNN and then throw JointBayes on top of that, but since the larger LMNN embedding did not show any improvement on the public score, I've abandoned this approach.
", "votes": null, "replies": [] }, { "id": 101838, "author_name": "abhishek", "author_url": "", "post_date": "12/17/2015 12:28:48", "content": "Ahh, such an interesting competition and i didnt have anytime (also, infrastructure) to do it :(
", "votes": null, "replies": [] }, { "id": 102084, "author_name": "chenmengyang", "author_url": "", "post_date": "12/19/2015 09:54:02", "content": "It was so lucky for my team to get into the top 3 of the rank, although there is a distance of the score between the top2 and us, I would like to share some experience about this competition from my perspective. There is a Github link that I and my teammates used to communicate our code(Matlab):\ncontest codes.
\n\nSo what we did is to try out some machine learning algorithms that we have learned from the lecture that can be used to this competition, after implementing KNN, Logistic Regression, SVM, MLP, we find the SVM and MLP can get a higher score than KNN.
\n\nWhat I found is to use SVM and MLP algorithms to train, use the deep feature which have 4096 dimensions as inputs, after getting the result from the algorithm, use softmax or euclidean distance to get the final distribution of the probabilities that a pair of photos are the same person.
\n\nFor speeding up the program we used PCA to reduce the 4096 dimensions into 1015 dimensions which is able to keep 99% of the original information in deep features.
\n\nIn SVM, we used the matlab built-in function fitceceo or fitcsvm to train a one versus all model over 271 different persons' photos, this means in the ends of the training there will be 271 classifiers, each classifier is one person against all the other 270 persons. After getting the weights of all the 271 classifiers, get the result by calculate the distance between the two photos to every hyperplane(classifier). It can get the score about 0.944 without passing any other parameters to the SVM.
\n\nIn MLP, we defined a simple feed-forward network with only one hidden layer, and used back propagation to train the network. After getting the weights for each layer, input new photo features and get the result in output layer, what's different from SVM is that we didn't use any sigmoid to activate the output neurons, so each photo will become a 271*1 vector without activating, then get the final result base on the euclidean distance between each pair of photos. After making some cross validation we found that setting the regularization parameter to be 3, and the hidden layer size to be 300 can get the highest accuracy, and the score is 0.964 when using these parameters. A pity of MLP is that we didn't try the result with more than one hidden layer, may be it could be better.
\n\nDuring the validation, we found that changing the value of C in SVM can not make any difference of the accuracy of training, I don't know why, may be find more data can be different, we didn't use a RBF kernel because I thought the dimensions are too high to use it. While in validating with MLP, accuracies are changing with the change of regularization parameter and the hidden layer size.
\n\nAll in all, I think it is an very interesting competition that motivates me a lot to implement what I have learned.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "101815": "Congratulations on the excellent performance of the top teams. Also I would like to thank all participants for their active participation to the competition.\r\n\r\nTo the best of my knowledge, the top three teams represent the two host universities (TUT and TUGraz) as well as a non-member to these institutes. Thus, we will provide a small prize to all members of top three teams. I would like to ask these teams to contact me at heikki.huttunen@tut.fi to agree on the details.\r\n\r\nAlso, I would be excited to hear about the techniques people were using publicly in this thread. I know deep networks are strong in face verification, and in our own experiments we reached 0.996 score with VGG-Face features coupled with another deep net. However, the two top teams are above this score, so what's your secret?", "101837": "Seems like nobody won the competition :D\r\n\r\nMy first submission was using a custom CNN trained on CASIA Web Faces (~0.5-0.4 million images), with a combined softmax and LMNN loss. With this I was able to achieve a score of 0.98.\r\n\r\nLater I've switched to Parkhi's VGG-Face features, since he can train larger networks on larger datasets (2.5 million faces). To normalize faces I'm using the 5 facial landmarks and estimate a similarity transform to a normalized shape. From the normalized images I'm extracting CNN features (without any further computationally expensive sub-crop averaging as proposed in the paper). With just this simple method + Cosine distance I'm able to achieve 0.99952 score. Internally I've split the training set into a validation and training set with disjoint identities and measured TP rate at 0.01 FP. With this solution I'm able to achieve ~0.77 TP at 0.01 FP.\r\n\r\nTo further improve results, I've experimented with LMNN embeddings on top of fixed, l2-normed feature vectors (same as in their paper). Following the paper, I've trained a 1024 dimensional LMNN embedding on top of normalized features. With this submission I've improved to 0.99969. On my internal validation set I'm able to achieve ~0.81 TP at 0.01 FP with this method.\r\n\r\nFinally, I've increased the embedding size to 4096 and added stronger regularization, which helped to improve the validation set accuracy from ~0.81 TP at 0.01 FP to ~0.83 at 0.01 FP, but did not show any improvements on the leaderboard score. \r\n\r\nI've also experimented with JointBayes, but the dimensionality of the last feature layer (4096) was too high to estimate non-singular covariance matrices. With a PCA reduction to 400 dimensions it worked, but produced inferior results to the LMNN method (~0.80 TP at 0.01 FP). I guess it would have been better to compress the dimenionality with LMNN and then throw JointBayes on top of that, but since the larger LMNN embedding did not show any improvement on the public score, I've abandoned this approach.", "101838": "Ahh, such an interesting competition and i didnt have anytime (also, infrastructure) to do it :(", "102084": "It was so lucky for my team to get into the top 3 of the rank, although there is a distance of the score between the top2 and us, I would like to share some experience about this competition from my perspective. There is a Github link that I and my teammates used to communicate our code(Matlab):\r\n[contest codes.][1]\r\n\r\nSo what we did is to try out some machine learning algorithms that we have learned from the lecture that can be used to this competition, after implementing KNN, Logistic Regression, SVM, MLP, we find the SVM and MLP can get a higher score than KNN.\r\n\r\nWhat I found is to use SVM and MLP algorithms to train, use the deep feature which have 4096 dimensions as inputs, after getting the result from the algorithm, use softmax or euclidean distance to get the final distribution of the probabilities that a pair of photos are the same person.\r\n\r\nFor speeding up the program we used PCA to reduce the 4096 dimensions into 1015 dimensions which is able to keep 99% of the original information in deep features.\r\n\r\nIn SVM, we used the matlab built-in function fitceceo or fitcsvm to train a one versus all model over 271 different persons' photos, this means in the ends of the training there will be 271 classifiers, each classifier is one person against all the other 270 persons. After getting the weights of all the 271 classifiers, get the result by calculate the distance between the two photos to every hyperplane(classifier). It can get the score about 0.944 without passing any other parameters to the SVM.\r\n\r\nIn MLP, we defined a simple feed-forward network with only one hidden layer, and used back propagation to train the network. After getting the weights for each layer, input new photo features and get the result in output layer, what's different from SVM is that we didn't use any sigmoid to activate the output neurons, so each photo will become a 271*1 vector without activating, then get the final result base on the euclidean distance between each pair of photos. After making some cross validation we found that setting the regularization parameter to be 3, and the hidden layer size to be 300 can get the highest accuracy, and the score is 0.964 when using these parameters. A pity of MLP is that we didn't try the result with more than one hidden layer, may be it could be better.\r\n\r\nDuring the validation, we found that changing the value of C in SVM can not make any difference of the accuracy of training, I don't know why, may be find more data can be different, we didn't use a RBF kernel because I thought the dimensions are too high to use it. While in validating with MLP, accuracies are changing with the change of regularization parameter and the hidden layer size.\r\n\r\nAll in all, I think it is an very interesting competition that motivates me a lot to implement what I have learned.\r\n\r\n\r\n [1]: https://github.com/chenmengyang/ML-contest-Face-Verification" }, "source": "meta" }