![\"\"](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3486/media/face1.png\")
The objective of this task is to predict keypoint positions on face images. This can be used as a building block in several applications, such as:
\r\nDetecing facial keypoints is a very challenging problem. Facial features vary greatly from one individual to another, and even for a single individual, there is a large amount of variation due to 3D pose, size, position, viewing angle, and illumination conditions. Computer vision research has come a long way in addressing these difficulties, but there remain many opportunities for improvement.
\r\nThis getting-started competition provides a benchmark data set and an R tutorial to get you going on analysing face images. Get started with R >>
\r\nThe data set for this competition was graciously provided by Dr. Yoshua Bengio of the University of Montreal. The tutorial was developed by James Petterson.
Submissions are scored on the root mean squared error. RMSE is very common and is a suitable general-purpose error metric. Compared to the Mean Absolute Error, RMSE punishes large errors:
\r\n\\[\\textrm{RMSE} = \\sqrt{\\frac{1}{n} \\sum_{i=1}^{n} (y_i - \\hat{y}_i)^2},\\]
\r\nwhere y hat is the predicted value and y is the original value.
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "data-description", "content": "Each predicted keypoint is specified by an (x,y) real-valued pair in the space of pixel indices. There are 15 keypoints, which represent the following elements of the face:
\r\nleft_eye_center, right_eye_center, left_eye_inner_corner, left_eye_outer_corner, right_eye_inner_corner, right_eye_outer_corner, left_eyebrow_inner_end, left_eyebrow_outer_end, right_eyebrow_inner_end, right_eyebrow_outer_end, nose_tip, mouth_left_corner, mouth_right_corner, mouth_center_top_lip, mouth_center_bottom_lip
\r\nLeft and right here refers to the point of view of the subject.
\r\nIn some examples, some of the target keypoint positions are misssing (encoded as missing entries in the csv, i.e., with nothing between two commas).
\r\nThe input image is given in the last field of the data files, and consists of a list of pixels (ordered by row), as integers in (0,255). The images are 96x96 pixels.
\r\nData files
\r\nPlease download the file submissionFileFormat.csv from the data page. This file lists the points whose locations need to be detected.
\r\nSubmission File Columns
\r\nRowId,ImageId,FeatureName,Location\r\n1,1,left_eye_center_x,?\r\n2,1,left_eye_center_y,?\r\n3,1,right_eye_center_x,?\r\n4,1,right_eye_center_y,?\r\netc...\r\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Getting Started With R", "content": "
In this tutorial we will describe a simple benchmark for this competition, written entirely in R. R is a free software programming language, used widely for statistical computing. It is available for Windows, OS X, Linux and other platforms, and is a favorite amongst Kaggle competitors tools.
\r\nThe goal of the competition is to locate specific keypoints on face images. You should build an algorithm that, given an image of a face, automatically locates where these keypoints are located.
\r\nFirst you'll need to get the data. Download training.zip, test.zip and submissionFileFormat.csv, and uncompress them. The training.csv file has 7049 examples of face images with corresponding keypoint locations. We'll use this data to train our algorithm. The test.csv file has face images only, and will be used to test our algorithm, by determining whether we successfully identified the corresponding keypoint locations.
If you haven't done so yet, install R. You can download it and find installation instructions here.
\r\nNow launch R. You should get a prompt similar to this:
\r\nR version 2.15.1 (2012-06-22) -- \"Roasted Marshmallows\"\r\nCopyright (C) 2012 The R Foundation for Statistical Computing\r\nISBN 3-900051-07-0\r\nPlatform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)\r\n\r\nR is free software and comes with ABSOLUTELY NO WARRANTY.\r\nYou are welcome to redistribute it under certain conditions.\r\nType 'license()' or 'licence()' for distribution details.\r\n\r\n Natural language support but running in an English locale\r\n\r\nR is a collaborative project with many contributors.\r\nType 'contributors()' for more information and\r\n'citation()' on how to cite R or R packages in publications.\r\n\r\nType 'demo()' for some demos, 'help()' for on-line help, or\r\n'help.start()' for an HTML browser interface to help.\r\nType 'q()' to quit R.\r\n\r\n> \r\n\r\n
Let's first create variables to store the path to the files you downloaded:
\r\ndata.dir <- '~/data/kaggle/facial_keypoint_detection/'\r\ntrain.file <- paste0(data.dir, 'training.csv')\r\ntest.file <- paste0(data.dir, 'test.csv')\r\n\r\n
You should change data.dir to point to the location where you saved the files.
We can now instruct R to read in the 2 csv files. R has a very convenient function for that, so this is very simple:
\r\nd.train <- read.csv(train.file, stringsAsFactors=F)\r\n\r\n
This creates a data.frame, a fundamental structure in R. It is essentially a matrix where each column can have a different type.
\r\nWe did not tell R what the data type of each column was, so R analyses the data frame and makes guesses. Usually they are right, but it is a good habit to check!
\r\nstr(d.train)\r\n\r\n'data.frame': 7049 obs. of 31 variables:\r\n $ left_eye_center_x : num 66 64.3 65.1 65.2 66.7 ...\r\n $ left_eye_center_y : num 39 35 34.9 37.3 39.6 ...\r\n $ right_eye_center_x : num 30.2 29.9 30.9 32 32.2 ...\r\n $ right_eye_center_y : num 36.4 33.4 34.9 37.3 38 ...\r\n $ left_eye_inner_corner_x : num 59.6 58.9 59.4 60 58.6 ...\r\n $ left_eye_inner_corner_y : num 39.6 35.3 36.3 39.1 39.6 ...\r\n $ left_eye_outer_corner_x : num 73.1 70.7 71 72.3 72.5 ...\r\n $ left_eye_outer_corner_y : num 40 36.2 36.3 38.4 39.9 ...\r\n $ right_eye_inner_corner_x : num 36.4 36 37.7 37.6 37 ...\r\n $ right_eye_inner_corner_y : num 37.4 34.4 36.3 38.8 39.1 ...\r\n $ right_eye_outer_corner_x : num 23.5 24.5 25 25.3 22.5 ...\r\n $ right_eye_outer_corner_y : num 37.4 33.1 36.6 38 38.3 ...\r\n $ left_eyebrow_inner_end_x : num 57 54 55.7 56.4 57.2 ...\r\n $ left_eyebrow_inner_end_y : num 29 28.3 27.6 30.9 30.7 ...\r\n $ left_eyebrow_outer_end_x : num 80.2 78.6 78.9 77.9 77.8 ...\r\n $ left_eyebrow_outer_end_y : num 32.2 30.4 32.7 31.7 31.7 ...\r\n $ right_eyebrow_inner_end_x: num 40.2 42.7 42.2 41.7 38 ...\r\n $ right_eyebrow_inner_end_y: num 29 26.1 28.1 31 30.9 ...\r\n $ right_eyebrow_outer_end_x: num 16.4 16.9 16.8 20.5 15.9 ...\r\n $ right_eyebrow_outer_end_y: num 29.6 27.1 32.1 29.9 30.7 ...\r\n $ nose_tip_x : num 44.4 48.2 47.6 51.9 43.3 ...\r\n $ nose_tip_y : num 57.1 55.7 53.5 54.2 64.9 ...\r\n $ mouth_left_corner_x : num 61.2 56.4 60.8 65.6 60.7 ...\r\n $ mouth_left_corner_y : num 80 76.4 73 72.7 77.5 ...\r\n $ mouth_right_corner_x : num 28.6 35.1 33.7 37.2 31.2 ...\r\n $ mouth_right_corner_y : num 77.4 76 72.7 74.2 77 ...\r\n $ mouth_center_top_lip_x : num 43.3 46.7 47.3 50.3 45 ...\r\n $ mouth_center_top_lip_y : num 72.9 70.3 70.2 70.1 73.7 ...\r\n $ mouth_center_bottom_lip_x: num 43.1 45.5 47.3 51.6 44.2 ...\r\n $ mouth_center_bottom_lip_y: num 84.5 85.5 78.7 78.3 86.9 ...\r\n $ Image : chr \"238 236 237 238 240 240 239 241 241 243 240 239 231 ...\r\n\r\n
In the output above, R lists the column name, followed by its guess at the column type, and then the first few data values in the column. For example, the first column (or variable) in the data frame d.train is named left_eye_center_x. It contains numeric values, and the first two values are 66 and 64.3.
\r\nIn total, we have 7049 rows, each one with 31 columns. The first 30 columns are keypoint locations, which R correctly identified as numbers. The last one is a string representation of the image, identified as a string. This last column is the reason for the stringsAsFactors argument: if you omit it R might treat this column as a factor (i.e., a category).
By the way, to get help on the syntax of any command in R just prepend a question mark to its name. For example:
\r\n?read.csv\r\n\r\n
will open a full description of the command, and the parameters it expects. You can exit this help quickly too, simply press q.
Back to the data: to get a peek at it you can use the head command, which will display only the top few rows:
head(d.train)\r\n\r\n
Unfortunately the rightmost column is quite long, so the output is not very readable. Let's save that column as another variable, and remove it from d.train:
im.train <- d.train$Image\r\nd.train$Image <- NULL\r\n\r\n
In the first line, we assign a variable im.train the values from d.train$Image. As you can see, R provides us with an easy way to identify the column we want to refer to. d.train is our dataframe, and we want the column called Image. Assigning NULL to a column removes it from the dataframe.
Now let’s try the head command again:
\r\nhead(d.train)\r\n\r\n left_eye_center_x left_eye_center_y right_eye_center_x … \r\n1 66.03356 39.00227 30.22701 …\r\n2 64.33294 34.97008 29.94928 …\r\n3 65.05705 34.90964 30.90379 …\r\n4 65.22574 37.26177 32.02310 …\r\n5 66.72530 39.62126 32.24481 …\r\n6 69.68075 39.96875 29.18355 …\r\n\r\n
As you can see there is one column for each keypoint, and one row for each image.
\r\nNow, let’s take a look at the column we moved to im.train. For each image (i.e. in each row) it contains a long string of numbers, where each number represents the intensity of a pixel in the image. Lets look at the first value in the column:
im.train[1]\r\n\r\n[1] \"238 236 237 238 240 240 239 241 241 243 240…\r\n\r\n
To analyze these further, we convert these strings to integers by splitting them and converting the result to integer:
\r\nas.integer(unlist(strsplit(im.train[1], \" \")))\r\n\r\n[1] 238 236 237 238 240 240 239 241 241 243 240 …\r\n\r\n
strsplit splits the string, unlist simplifies its output to a vector of strings and as.integer converts it to a vector of integers.
That works well, but we need to do it for all images, and not only the first one. We could iterate through each record in im.train and apply the string to integers conversion above. However, sequentially processing this conversion can take some time. We can therefore utilize a multi core approach using the doMC library (linux and osx only - if you are working on windows please check this post for alternatives).
First, we'll need to install the library with this command:
\r\ninstall.packages('doMC')\r\n\r\nAfter selecting the CRAN mirror you want to use the installation should proceed automatically. Next, we need to load the library and register it
\r\nlibrary(doMC)\r\nregisterDoMC()\r\n\r\n
Now we’re ready to implement the parallelization.
\r\nim.train <- foreach(im = im.train, .combine=rbind) %dopar% {\r\n as.integer(unlist(strsplit(im, \" \")))\r\n}\r\n\r\nThe foreach loop will evaluate the inner command for each row in im.train, and combine the results with rbind (combine by rows). %dopar% instructs R to do all evaluations in parallel.
im.train is now a matrix with 7049 rows (one for each image) and 9216 columns (one for each pixel):
str(im.train)\r\n\r\n int [1:7049, 1:9216] 238 219 144 193 147 167 109 178 164 226 ...\r\n - attr(*, \"dimnames\")=List of 2\r\n ..$ : chr [1:7049] \"result.1\" \"result.2\" \"result.3\" \"result.4\" ...\r\n ..$ : NULL\r\n\r\n
Repeat the process for test.csv, as we are going to need it later. Notice that in the test file, we don’t have the first 30 columns with the keypoint locations.
\r\nd.test <- read.csv(test.file, stringsAsFactors=F)\r\nim.test <- foreach(im = d.test$Image, .combine=rbind) %dopar% {\r\n as.integer(unlist(strsplit(im, \" \")))\r\n}\r\nd.test$Image <- NULL\r\n\r\nIt’s a good idea to save the data as a R data file at this point, so you don't have to repeat this process again. We save all four variables into the data.Rd file:
save(d.train, im.train, d.test, im.test, file='data.Rd')\r\n\r\n
We can reload them at any time with the following command:
\r\nload('data.Rd')\r\n\r\nNow that the data is loaded let's start looking at the images. Did you notice how the long string comprised of 9216 integers? That’s because each image is a vector of 96*96 pixels (96*96 = 9216).
\r\nTo visualize each image, we thus need to first convert these 9216 integers into a 96x96 matrix:
\r\nim <- matrix(data=rev(im.train[1,]), nrow=96, ncol=96)\r\n\r\n
im.train[1,] returns the first row of im.train, which corresponds to the first training image. rev reverse the resulting vector to match the interpretation of R's image function (which expects the origin to be in the lower left corner). To visualize the image we use R's image function:
image(1:96, 1:96, im, col=gray((0:255)/255))\r\n\r\n
We can then add some keypoints (from the other 30 columns of the training file) to check if everything is correct so far (here, again, we need to adjust the coordinates for the different origin). Let’s color the coordinates for the eyes and nose:
\r\npoints(96-d.train$nose_tip_x[1], 96-d.train$nose_tip_y[1], col=\"red\")\r\npoints(96-d.train$left_eye_center_x[1], 96-d.train$left_eye_center_y[1], col=\"blue\")\r\npoints(96-d.train$right_eye_center_x[1], 96-d.train$right_eye_center_y[1], col=\"green\")\r\n\r\n
![\"\"](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3486/media/face1_with_keypoints.png\")
Another good check is to see how variable is our data. For example, where are the centers of each nose in the 7049 images? (this takes a while to run):
\r\nfor(i in 1:nrow(d.train)) {\r\n points(96-d.train$nose_tip_x[i], 96-d.train$nose_tip_y[i], col=\"red\")\r\n}\r\n\r\n![\"\"](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3486/media/face1_with_all_nose_keypoints.png\")
Most nose points are concentrated in the central region (as expected), but there are quite a few outliers that deserve further investigation, as they could be labeling errors. Looking at one extreme example we get this:
\r\nidx <- which.max(d.train$nose_tip_x)\r\nim <- matrix(data=rev(im.train[idx,]), nrow=96, ncol=96)\r\nimage(1:96, 1:96, im, col=gray((0:255)/255))\r\npoints(96-d.train$nose_tip_x[idx], 96-d.train$nose_tip_y[idx], col=\"red\")\r\n\r\n
![\"\"](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3486/media/face4264_outlier.png\")
In this case there's no labeling error, but this shows that not all faces are centralized as one might expect.
\r\nThere's much more that could be analyzed, but let's start building our first algorithm.
\r\nOne of the simplest things to try is to compute the mean of the coordinates of each keypoint in the training set and use that as a prediction for all images. This is a very simplistic algorithm, as it completely ignores the images, but we can use it a starting point to build a first submission.
\r\nComputing the mean for each column is straightforward with colMeans (na.rm=T tells colMeans to ignore missing values). We get the following values:
colMeans(d.train, na.rm=T)\r\n\r\n left_eye_center_x left_eye_center_y right_eye_center_x \r\n 66.35902 37.65123 30.30610 \r\n right_eye_center_y left_eye_inner_corner_x left_eye_inner_corner_y \r\n 37.97694 59.15934 37.94475 \r\n left_eye_outer_corner_x left_eye_outer_corner_y right_eye_inner_corner_x \r\n 73.33048 37.70701 36.65261 \r\n right_eye_inner_corner_y right_eye_outer_corner_x right_eye_outer_corner_y \r\n 37.98990 22.38450 38.03350 \r\n left_eyebrow_inner_end_x left_eyebrow_inner_end_y left_eyebrow_outer_end_x \r\n 56.06851 29.33268 79.48283 \r\n left_eyebrow_outer_end_y right_eyebrow_inner_end_x right_eyebrow_inner_end_y \r\n 29.73486 39.32214 29.50300 \r\nright_eyebrow_outer_end_x right_eyebrow_outer_end_y nose_tip_x \r\n 15.87118 30.42817 48.37419 \r\n nose_tip_y mouth_left_corner_x mouth_left_corner_y \r\n 62.71588 63.28574 75.97071 \r\n mouth_right_corner_x mouth_right_corner_y mouth_center_top_lip_x \r\n 32.90040 76.17977 47.97541 \r\n mouth_center_top_lip_y mouth_center_bottom_lip_x mouth_center_bottom_lip_y \r\n 72.91944 48.56947 78.97015 \r\n\r\n
To build a submission file we need to apply these computed coordinates to the test instances:
\r\np <- matrix(data=colMeans(d.train, na.rm=T), nrow=nrow(d.test), ncol=ncol(d.train), byrow=T)\r\ncolnames(p) <- names(d.train)\r\npredictions <- data.frame(ImageId = 1:nrow(d.test), p)\r\nhead(predictions)\r\n\r\n ImageId left_eye_center_x left_eye_center_y right_eye_center_x …\r\n1 1 66.35902 37.65123 30.3061 …\r\n2 2 66.35902 37.65123 30.3061 …\r\n3 3 66.35902 37.65123 30.3061 …\r\n4 4 66.35902 37.65123 30.3061 …\r\n5 5 66.35902 37.65123 30.3061 …\r\n6 6 66.35902 37.65123 30.3061 … \r\n\r\n
The expected submission format has one one keypoint per row, but we can easily get that with the help of the reshape2 library:
\r\ninstall.packages('reshape2')\r\nlibrary(reshape2)\r\nsubmission <- melt(predictions, id.vars=\"ImageId\", variable.name=\"FeatureName\", value.name=\"Location\")\r\nhead(submission)\r\n\r\n ImageId FeatureName Location\r\n1 1 left_eye_center_x 66.35902\r\n2 2 left_eye_center_x 66.35902\r\n3 3 left_eye_center_x 66.35902\r\n4 4 left_eye_center_x 66.35902\r\n5 5 left_eye_center_x 66.35902\r\n6 6 left_eye_center_x 66.35902\r\n\r\nWe then join this with the sample submission file to preserve the same order of entries and save the result:
\r\nexample.submission <- read.csv(paste0(data.dir, 'submissionFileFormat.csv'))\r\nsub.col.names <- names(example.submission)\r\nexample.submission$Location <- NULL\r\nsubmission <- merge(example.submission, submission, all.x=T, sort=F)\r\nsubmission <- submission[, sub.col.names]\r\nwrite.csv(submission, file=\"submission_means.csv\", quote=F, row.names=F)\r\n\r\n
If you submit this file you will get a leaderboard score of 3.96244. Not a very exciting result, but shows us that the submission format is correct.
\r\nThe above method was rather simplistic, and didn’t analyse the images at all. Said another way, we didn’t use the information about the intensity of each pixel to identify the keypoints. Let's try to build an algorithm that makes use of this rich data.
\r\nTo simplify we will first focus on a single keypoint: left_eye_center.
\r\nThe idea is to extract a patch around this keypoint in each image, and average the result. This average_patch can then be used as a mask to search for the keypoint in test images.
\r\nWe start defining some parameters:
\r\ncoord <- \"left_eye_center\"\r\npatch_size <- 10\r\n\r\n
coord is the keypoint we are working on, and patch_size is the number of pixels we are going to extract in each direction around the center of the keypoint. So 10 means we will have a square of 21x21 pixels (10+1+10). This will become more clear with an example:
coord_x <- paste(coord, \"x\", sep=\"_\")\r\ncoord_y <- paste(coord, \"y\", sep=\"_\")\r\npatches <- foreach (i = 1:nrow(d.train), .combine=rbind) %do% {\r\n im <- matrix(data = im.train[i,], nrow=96, ncol=96)\r\n x <- d.train[i, coord_x]\r\n y <- d.train[i, coord_y]\r\n x1 <- (x-patch_size)\r\n x2 <- (x+patch_size)\r\n y1 <- (y-patch_size)\r\n y2 <- (y+patch_size)\r\n if ( (!is.na(x)) && (!is.na(y)) && (x1>=1) && (x2<=96) && (y1>=1) && (y2<=96) )\r\n {\r\n as.vector(im[x1:x2, y1:y2])\r\n }\r\n else\r\n {\r\n NULL\r\n }\r\n}\r\nmean.patch <- matrix(data = colMeans(patches), nrow=2*patch_size+1, ncol=2*patch_size+1)\r\n\r\nThis foreach loop will get each image and:
\r\nx and yx1, y1, x2 and y2is.na) and are inside the imageAll the non-NULL vectors will then be combined with rbind, which concatenates them as rows. The result patches will be a matrix where each row is a patch of an image. We then compute the mean of all images with colMeans, put back in matrix format and store in mean.patch. You can then visualize the result with image:
image(1:21, 1:21, mean.patch[21:1,21:1], col=gray((0:255)/255))\r\n\r\n
![\"\"](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3486/media/left_eye_center_mean_patch.png\")
And it does look like an eye! This is the average left eye computed across our 7049 images.
\r\nNow we can use this average_patch to search for the same keypoint in the test images. First we define another parameter:
\r\nsearch_size <- 2\r\n\r\n
search_size indicates how many pixels we are going to move in each direction when searching for the keypoint. We will center the search on the average keypoint location, and go search_size pixels in each direction:
mean_x <- mean(d.train[, coord_x], na.rm=T)\r\nmean_y <- mean(d.train[, coord_y], na.rm=T)\r\nx1 <- as.integer(mean_x)-search_size\r\nx2 <- as.integer(mean_x)+search_size\r\ny1 <- as.integer(mean_y)-search_size\r\ny2 <- as.integer(mean_y)+search_size\r\n\r\n
In this particular case the search will be from (64,35) to (68,39). We can use expand.grid to build a data frame with all combinations of x's and y's:
params <- expand.grid(x = x1:x2, y = y1:y2)\r\nparams\r\n\r\n x y\r\n1 64 35\r\n2 65 35\r\n3 66 35\r\n4 67 35\r\n5 68 35\r\n6 64 36\r\n7 65 36\r\n8 66 36\r\n9 67 36\r\n10 68 36\r\n11 64 37\r\n12 65 37\r\n13 66 37\r\n14 67 37\r\n15 68 37\r\n16 64 38\r\n17 65 38\r\n18 66 38\r\n19 67 38\r\n20 68 38\r\n21 64 39\r\n22 65 39\r\n23 66 39\r\n24 67 39\r\n25 68 39\r\n\r\n
Given a test image we need to try all these combinations, and see which one best matches the average_patch. We will do that by taking patches of the test images around these points and measuring their correlation with the average_patch. Take the first test image as an example:
\r\nim <- matrix(data = im.test[1,], nrow=96, ncol=96)\r\n\r\nr <- foreach(j = 1:nrow(params), .combine=rbind) %dopar% {\r\n x <- params$x[j]\r\n y <- params$y[j]\r\n p <- im[(x-patch_size):(x+patch_size), (y-patch_size):(y+patch_size)]\r\n score <- cor(as.vector(p), as.vector(mean.patch))\r\n score <- ifelse(is.na(score), 0, score)\r\n data.frame(x, y, score)\r\n}\r\n\r\nInside the for loop, given a coordinate we extract an image patch p and compare it to the average_patch with cor. The ifelse is necessary for the cases where all the image patch pixels have the same intensity, as in this case cor returns NA. The result will look like this:
r\r\n\r\n x y score\r\n1 64 35 0.1017430\r\n2 65 35 0.1198157\r\n3 66 35 0.1376269\r\n4 67 35 0.1351847\r\n5 68 35 0.1119015\r\n6 64 36 0.2769096\r\n7 65 36 0.2884035\r\n8 66 36 0.2923847\r\n9 67 36 0.2741814\r\n10 68 36 0.2333830\r\n11 64 37 0.4410122\r\n12 65 37 0.4560440\r\n13 66 37 0.4520532\r\n14 67 37 0.4189839\r\n15 68 37 0.3632465\r\n16 64 38 0.5559125\r\n17 65 38 0.5715887\r\n18 66 38 0.5675701\r\n19 67 38 0.5317430\r\n20 68 38 0.4711673\r\n21 64 39 0.6115627\r\n22 65 39 0.6131023\r\n23 66 39 0.6063069\r\n24 67 39 0.5794715\r\n25 68 39 0.5276036\r\n\r\n
Now all we need to do is return the coordinate with the highest score:
\r\nbest <- r[which.max(r$score), c(\"x\", \"y\")]\r\nbest\r\n\r\n x y\r\n22 65 39\r\n\r\n
To build a submission the whole procedure has to be repeated for each keypoint and for each test image. We won't explain this in detail here, but you can download a complete solution from here. After downloading it, adjust the location of the data in the top of the file and run the code with
\r\nRscript --vanilla tutorial.R\r\n\r\n
It takes a while to run (about 6 minutes in a quad core laptop), and once finished it will create the file submission_search.csv. If you submit it you should get a leaderboard score of 3.80685. It's a small improvement when compared to the result of the means benchmark (3.96244), but that is often the case when exploring new methods.
Most competitions impose a limit on the number of submissions per day to avoid overfitting to the test data. I common approach to overcome this limitation is to split the training data in two sets: one for training (say, 80% of the data, randomly chosen), and another for testing (the rest). We then train our algorithm using only the first set, and can then use the second one to evaluate the performance without making a submission.
\r\nThis can be easily done by replacing this code
\r\nd.train <- read.csv(train.file, stringsAsFactors=F)\r\nd.test <- read.csv(test.file, stringsAsFactors=F)\r\nim.train <- foreach(im = d.train$Image, .combine=rbind) %dopar% {\r\n as.integer(unlist(strsplit(im, \" \")))\r\n}\r\nim.test <- foreach(im = d.test$Image, .combine=rbind) %dopar% {\r\n as.integer(unlist(strsplit(im, \" \")))\r\n}\r\nd.train$Image <- NULL\r\nd.test$Image <- NULL\r\n\r\nwith this
\r\nd <- read.csv(train.file, stringsAsFactors=F)\r\nim <- foreach(im = d$Image, .combine=rbind) %dopar% {\r\n as.integer(unlist(strsplit(im, \" \")))\r\n}\r\nd$Image <- NULL\r\nset.seed(0)\r\nidxs <- sample(nrow(d), nrow(d)*0.8)\r\nd.train <- d[idxs, ]\r\nd.test <- d[-idxs, ]\r\nim.train <- im[idxs,]\r\nim.test <- im[-idxs,]\r\nrm(\"d\", \"im\")\r\n\r\nset.seed fixes the pseudo random number generator seed, so later on you can reproduce this with exactly the same results if needed. Everything else stays the same. Once you have your predictions
p <- matrix(data=colMeans(d.train, na.rm=T), nrow=nrow(d.test), ncol=ncol(d.train), byrow=T)\r\n\r\n
You can then computed the RMSE on your test set:
\r\nsqrt(mean((d.test-p)^2, na.rm=T))\r\n\r\n[1] 3.758999\r\n\r\n
This is a good proxy of the score you would get on the leaderboard (assuming the data on the supplied train and test sets follow the same distribution), so you can use it to compare different methods without making submissions.
\r\nThe method described here is very simple, and won't take you to the top of the leaderboard, so what can you do next?
\r\nThe literature on Computer Vision is vast and can be intimidating. A good starting point, though, is Viola and Jones' seminal paper on object detection. Their proposed framework works quite well, and has an open source implementation available as part of the opencv project.
\r\nIn opencv's website you can find a user guide for training a new object detector. And if you google for opencv haartraining you'll find other more detailed tutorials, such as this one.
The implementation also comes with pre-trained classifiers, which you can easily try. See, for example, the result of applying the pre-trained eye detector to one of the images:
\r\n![\"\"](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3486/media/opencv_detection.png\")
Good luck with the competition!
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "rules", "content": "\r\n\t\r\nYou cannot sign up to Kaggle from multiple accounts and therefore you cannot submit from multiple accounts.
\r\n\r\n Privately sharing code or data outside of teams is not permitted.\r\n It's okay to share code if made available to all participants on the forums.\r\n
\r\n\r\n Team mergers are allowed and can be performed by the team leader. In order to merge, the combined team must have a total submission count less than or equal to the maximum allowed as of the merge date. The maximum allowed is the number of submissions per day multiplied by the number of days the competition has been running.
\r\n
There is no maximum team size.
\r\nYou may submit a maximum of 5 entries per day.
\r\nYou may select up to 5 final submissions for judging.
\r\nStart Date: 5/7/2013 2:24 PM UTC
\r\nMerger Deadline: None
\r\nEntry Deadline: None
\r\nEnd Date: 1/7/2017 12:00 AM UTC
\r\n\t\r\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Deep Learning Tutorial", "content": "Our friend Daniel Nouri, founder of Natural Vision and top contender on the leaderboard, has written a tutorial blog post on this competition. The tutorial outlines how to use convolutional neural nets to detect facial key points on this competition's dataset. We strongly encourage you to follow Daniel's tutorial, step through the exercises, and learn how to build different network architectures.
\r\nDaniel's blog post features Lasagne, which uses Python and Theano. Before tackling the tutorial, you'll need:
\r\nBelow, we walk you through how to complete both of these prerequisites.
\r\nNot everyone has a GPU - it's primarily used for gaming or heavy 3D visualization. The GPU is orders of magnitude faster than the cpu for math operations (such as matrix multiplication), which is essential for many machine learning algorithms. In this tutorial, most of the training of neural nets needs to be run on a GPU, otherwise it could take hours to train.
\r\nIf you don't have a GPU, you can spin up some Amazon Web Services instances to run this tutorial and pay a small fee for the computing hours.
\r\nLog into your AWS management console and click over to EC2. There are two options for creating your instance:
\r\nThere are quite a few steps to configure an AWS instance to run CUDA-GPU successfully. We have found using these steps very useful. However, it can be a bit cumbersome to run so many steps every time. The author has graciously provided an image (AMI) for his configuration, so you can launch it as a starting point (and skip that tutorial).
\r\n![](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3486/media/Screen%20Shot%202015-01-06%20at%2011.30.55%20AM.png\")
![\"\"](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3486/media/Screen%20Shot%202015-01-06%20at%2011.51.02%20AM.png\")
![\"\"](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3486/media/Screen%20Shot%202015-01-06%20at%2011.51.13%20AM.png\")
![\"\"](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3486/media/Screen%20Shot%202015-01-06%20at%2011.46.24%20AM.png\")
![\"\"](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3486/media/Screen%20Shot%202015-01-06%20at%2011.48.05%20AM.png\")
You should see instruction to ssh into the server similar to this:
\r\nssh -i my-key.pem ubuntu@[DNS]\r\n
and you're connected!
\r\nI have scripted the configuration in this github repo. All you need to do is to clone this repository, and run the setup.sh script.
\r\nPlease pay special attention for downloading the data and putting them on AWS server. You need to copy-paste from your browser's cookie into a file called cookie.txt in order to get the correct data files. The downloading step is included in setup.sh, but you need to copy-paste your cookie.txt file yourself. It is described in the README file in the repository.
\r\nKaggle is not responsible for any charges incurred from running this tutorial.
\r\n__________________________________________________________________________________
\r\n