\r\nOne-shot-learning Gesture Recognition\r\nHumans are capable of recognizing patterns like hand gestures after seeing just ONE example. Can machines do that too? \r\nThis challenge is one of four CHALEARN Gesture Challenge events \r\n | \r\n
\r\n
You will never need a remote controller anymore, you will never need a light switch. Lying in bed in the dark, you will point to the ceiling to turn on the light, you will wave your hand to increase the temperature, you will make a T with your hands to turn on the TV set. You and your loved ones will feel safer at home, in parking lots, in airports: nobody will be watching, but computers will detect distressed people and suspicious activities. Computers will teach you how to effectively use gestures to enhance speech, to communicate with people who do not speak your language, to speak with deaf people, and you will easily learn many other sign languages to comminicate under water, to referee sports, etc. All that thanks to gesture recognition!
\r\n\r\n
This is a challenge on gesture and sign language recognition using a Kinect camera. Kinect is revolutionizing the field of gesture recognition by providing an affordable 3D camera, which records both an RGB image and a depth image (using an infrared sensor). The challenge focuses on hand gestures. Applications include man-machine communication, translating sign languages for the deaf, video surveillance, and computer gaming. Check out some examples.
\r\n\r\n
Every application needs a specialized gesture vocabulary. If we want gesture recognition to become part of everyday life, we need gesture recognition machines, which easily get tailored to new gesture vocabularies. This is why the focus of the challenge is on “one-shot-learning” of gestures, which means learning to recognize new categories of gestures from a single video clip of each gesture. The gestures will be drawn from a small vocabulary of gestures, generally related to a particular task, for instance, hand signals used by divers, finger codes to represent numerals, signals used by referees, or marchalling signals to guide vehicles or aircrafts.
\r\n\r\n
\r\n
This challenge is organized by CHALEARN and is sponsored in part by Microsoft (Kinect for Xbox 360). Other sponsors include Texas Instrument. This effort was initiated by the DARPA Deep Learning program and is supported by the US National Science Foundation (NSF) under grants ECCS 1128436 and ECCS 1128296 , the EU Pascal2 network of excellence. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the sponsors.
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Submission Instructions", "content": "\r\n
\r\n | \r\n
[Note: Disregard WARNINGs you may get, which signal that some predictions on training examples are incorrect. The training examples are NOT taken into account in the score prediction. The warnings just help you check your entry.]
\r\nDuring the the development phase, submissions are open for validation data only. This allows the participants to practice making submissions and compare their results with others.
\r\n\r\n
The final evaluation data (on which the entrants will be judged) will be made available encrypted to registered participants on March 7, 2012. The decryption key will be revealed on April 7, 2012.
\r\n\r\n
In the dataset the batches including validation data are identified as validxx, where xx is the batch number. The batches including final evaluation data (when they are released) will be identified as finalxx. Each batch includes 47 RGB videos (called Mnn.avi) and 47 corresponding Kinect depth image videos (called Knn.avi), both corresponding to the simulataneous recording of the same gestures. Class labels for the first N videos are provided for training in a file ending with the suffix \"_train.csv\". The class labels for the remaining unlabeled videos must be predicted.
\r\n\r\n
Prepare a single submission file with all the predictions in a comma separated text format. We describe the format for final entries:
\r\nExample:
\r\n[training examples omitted]\r\n
valid1_10, 11 2 3\r\n
valid1_11, 6 12\r\n
...\r\n
valid2_9, 4\r\n
...\r\n
valid12_10, 5\r\n
...\r\n
final21_11, 7 9\r\n
final21_4, 7 2 4 3 1\r\n
....\r\n
final40_47, 2 6\r\n
\r\n
Please note:
\r\n\r\n
To simplify the verification process, Kaggle is offering to the participants to voluntarily upload their learning machine during the period March 7, April 6, 2012. Upload your model as a zip archive together with your result file on the Kaggle website: at the bottom of the submission page, you will see Attach a model file and a button \"Choose file\".
\r\nBy submitting their software, the authors grant to the organizers a license to evaluate it for the purpose of verifying the challenge results only. The authors retain all rights to their software. The organizers will keep the software confidential.
\r\n
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Prizes", "content": "
\r\n\r\nPrizes donated by Microsoft\r\n | \r\n
\r\n
The exact terms and conditions are described in the official rules.
\r\n\r\n
| \r\n
\r\n
Demonstration competition at CVPR (June 16-17, 2012)
\r\n\r\n
There will also be at least USD 10,000 in travel awards for deserving participants to attend the CVPR 2012 conference (June 17-21, 2012, Providence, Rhode Island, USA).
\r\n\r\n
\r\nIntellectual Property Licenses\r\nIn addition, Microsoft may award up to USD 200,000 in contracts to top ranking participants who wish to license to Microsoft the technology they developed for the challenge. The exact terms and conditions are described in the official rules. \r\n | \r\n
Getting started
Setup
Tasks of the challenge
Data
Evaluation
Submissions
Rules
Help
\r\n
How do I get started?
There are essentially 2 approches that can be taken for data representation:
There is an excellent book on the subject.
\r\nSome approaches require separating the gesture sequences into isolated gestures first, which is relatively easy in this dataset because the users return their hands to a resting position between gestures. Once you have a vector representation of isolated gestures, to do the \"one-shot-learning\", the simplest method is the nearest neighbor method. But you may also look for the best match between temporal sequences directly without isolating gestures using dynamic time warping.
\r\nWhat is CHALEARN?
CHALEARN is a non-for-profit organization dedicated to organizing challenges in machine learning. Visit our website.
What is the challenge schedule?
This is a tentative schedule for round 1, see the full schedule in the official rules. The registered participants will be notified of changes.
December 7, 2011 Challenge platform opens.
January 7, 2012 (or before) Release of additional development Kinect data.
March 7, 2012 Final evaluation data for round 1 (CVPR 2012) released (encrypted).
April 7, 2012 Round 1 final evaluation data decryption key released.
April 10, 2012 Round 1 final evaluation result deadline.
May 1, 2012 Release of round 1 results to the participants.
May 15, 2012 Round 1 papers due.
June 16-17 2012 CVPR 2012: Live demonstration contest. Award ceremony for the first round of the challenge.
What is the goal of the challenge?
The goal is to devise a gesture recognizer capable of recognizing gestures recorded with a Kinect (TM) camera. The emphasis is on one-shot-learning (learning from a single example of gesture).
Are there prizes and travel grant?
Yes, there will be cash prizes, travel grants, and the (optional) opportunity to license your algorithms to Microsoft. See the Prize section.
Will there be a workshop and proceedings?
Yes, we are planning to have 2 workshops, on at CVPR 2012, Providence, Rhode Island, June 16-17, 2012 and one at ICPR 2012, Tsukuba, Japan, November 11-15, 2012. In addition to the regular conference proceedings publication opportunities, we will invite the top ranking participants to publish a longer paper in a special topic on gesture recognition of the Journal of Machine Learning Research (JMLR). There will be a live demonstration contest at both workshops.
Am I obliged to attend the workshop(s) or publish our method(s) to participate in the challenge?
No.
Can I attend the workshop(s) if I do not participate in the challenge?
Yes. You can even submit papers for presentation on the topics of the workshops.
Can I participate in the live demonstration contest if I do not participate in the challenge?
Yes. We will announce how to register to the demonstration contest in January 2012.
Do I need to register to participate?
Yes. You need to register to Kaggle. We also reccomend that you register to our Google group gesturechallenge.
Can I participate on a subset of the tasks?
No. If you do not submit results on some of the tasks, we will replace your missing entries by empty results. Your score will be pernalized because of that.
Which tasks will count towards the final ranking?
The final evaluation data will be used for the final ranking. It will be released towards the end of the challenge, see the schedule.
Can I participate in one round only?
Yes. There are four events to which you can participate: two rounds of quantittive evaluation (this is round 1) and two live demonstration contests. You can participate to any subset of these events. There are prizes for all the events.
What is a task or a data batch?
We have split the problem into individual tasks limited to a small vocabulary of 8 to 15 gesture \"tokens\". We have 85 different vocabularies. Each task consists of recognizing shorts sequences of 1 to 5 gestures performed by the same user in the same recording conditions. For each task we recorded a data batch of 100 gestures, recording multiple times each gesture token. The users had to follow a script (a prescribed order in which to perform the gestures). Because there may be several gestures in a video, there are only 47 videos in each batch (actually 47 x 2, because we recorded video pairs of RGB and depth data). For the validation and final evaluation data, you get only one labeled video of each gesture token and you then have to predict the labels of all the other videos.
What is one-shot-leanring?
Learning from a single example. For this challenge, for every task, you get only one video pair (RGB and depth data) for every gesture token.
Why should I care about one-shot-learning?
Supplying a lot of training examples is impractical in many applications where recording data and/or labeling data is tedious or expensive. Many consumer applications of gesture recognition will become possible only if we can train systems to recognize new gestures with very few examples, and, in the limit, just one. For example, imagine that you want to replace your TV remote controller and program it with your own gestures.
Why not provide more than one training example?
Sure, we could have done that. We think that one-shot-learning captures more the imagination and may contribute better to advance the state of the art in gesture recognition.
Is one-shot-learning more than template matching?
There is a lot to be explored to do one-shot-learning. You may want to use the development data to learn new representations. You may want to develop data generating models and/or distortion models using the development data to generate more examples from your small training dataset for each task. These are just a few suggestions.
Should I label by hand the unlabeled examples?
No. This is the job of your automatic system.
Can I use the unlabeled examples for training my system if I do not label them by hand?
Yes. We do not forbid that. However, this is one question we will ask you in the fact sheets you will have to fill out to help us analyze the results of the challenge.
Are there batches of data with the same gesture lexicon?
Yes, in the development and validation data, some batches are using the same lexicon. For the final evaluation data fresh lexicons not used in the development and test data willl be used and there will be a different lexicon for each batch.
What applications do you have in mind?
Immediately available if you solve this challenge:
Remote control: At home, replace your TV remote controller by a gesture enabled controller, which you cannot loose and works fine in the dark; turn on your light switch at night; allow hospital patients to control appliances and call for help without buttons; allow surgeons and other professional to control applicances user sterile conditions without touching.
Games: Replace your joy stick with your bare hands. Teach your computer the gestures you prefer to control your games. Play games of skills, reflexes and memory, which require learning new gestures and/or teaching them to your computer.
Gesture communication learning: Learn new gesture vocabularies from the sign language for the deaf, from professional signal vocabularies (referee signals, marshaling signals, diving signals, etc.), and from subconcious body language.
Further down the road if we can do user-independent one-shot-learning of gestures:
Video surveillance: Teach surveillance systems to watch for particular types of gestures (like shop-lifting gestures, aggressions, etc.)
Video retrieval: Look for videos containing a gesture that you perform in front of a camera. This includes dictionary lookup in databases of videos of sign language for the deaf.
Why not address the user-independent case?
We may do it for round 2 if the participants solce the user-dependent case in round 1!
Will you provide more data?
Yes, we will provide over 30,000 gestures, which will be made available before January 7, 2012.
How did you record the data?
We hired people to perform series of recordings following given scripts. The gesture vocabularies are drawn from existing applications. Some examples are provided. We asked the users to return their hand to a resting position between gestures.
How did you compress the data?
The videos are formatted is a quasi-lossless compressed AVI format. The compression was performed with FFMPEG http://ffmpeg.org/, using:
ffmpeg -i original.avi -sameq compressed.avi
Did you normalize the depth data?
Yes. We subtracted the closest point over one data batch and divided by the depth range over one data batch. We then mapped the depth to 8-bit integers.
Can I restore the original depth data?
To some extent. We provide in the READMEfile the normalization factors. Using the K_yy.avi files, the original depth values can be restored (approximately) as follows:
1) Average the R,G, and B values to get a value v.
2) Perform v/255*(MaxDepth-MinDepth)+MinDepth
Example Matlab code:
K=read_movie([data_dir 'devel01/K_1.avi']); MinDepth=801; % minimum distance MaxDepth=1964; % maximum distance Diff=MaxDepth-MinDepth; frame_1=K(1).cdata; restored_frame_1=mean(frame_1, 3)/255*Diff+MinDepth;\r\n
Isn't is a pity that you do not provide the original depth data?
We made this compromise to allow us to compress data better and ship the data as AVI movies that are easy to visualize. We lost almost no precision in depth because the effective number of unique depth values is usually lower than 256 in our data.
Can I use data not provided by the organizers to develop my system?
Yes, you are free to you any data you want. We even provide you with our data collection software to facilitate your task.
How easy will it be to decrypt the final evaluation data before you provide the key?
The encryption is just a deterrant. We will use regular encryption allowed by US export laws. It may be possible to decrypt the file if you are a determied cheaters. Cheaters can also label the data by hand. We will carry out post-challenge verifications of the top ranking participants. Be good sports, don't cheat, this takes out the fun for everybody!
The development data seems harder than the validation data, can that be?
Yes, we provided harder data for development to help the participants develop robust systems.
Do I need to use the development data at all for training?
No.
Why do you hide the identity of the lexicons?
For two reasons:
1) In the validation data and the final evaluation data, each batch corresponds to a different lexicon. However, in development data, we recorded multiple batches with the same lexicon but with different users to provide enough development data. We do not want to confuse the participants into thinking that the task is a multi-user task. We gave the users who recorded the data a lot of autonomy to interpret how the gestures should be performed. Hence, two batches using the same lexicon can be very different.
2) It may be possible to exploit domain information obtained from the knowledge of the lexicon to improve the performances. However, in this challenge, we want the participants to strictly focus on learning from the video examples.
What score do you compute?
\r\nFor each line in the result file, we compute the Levenshtein distance between the labels you return and the true labels. We sum these distances over all the lines and divide by the total number of gestures. See the evaluation page.
\r\nWhat do I see in the public leaderboard?
\r\nThere is only one entry per team in the leaderboard that shows the team's best result/position so far.
Do you take the training examples into account in the computation of the score?
No.
Will the results on the validation sets count for the final ranking?
No. However, you may report these results in your paper submitted to the workshop.
What result do I see on the public learderboard?
The results on unlabeled examples of the validation data.
Why do you conduct post-challenge verifications?
To ensure that the winners did not perform any manual or semi-manual labeling of the test examples.
What will the verification data look like?
For one set of verification data, the data batches of the final evaluation set will be matched with other batches recorded by the same person, under the same conditions, for gestures of the same lexicon. Other verification data will include batches from different lexicons similar in difficulty or the same lexicon but a different user to exclude the possibility of any semi-manual data processing.
Is the post-challenge verification completely bullet-proof?
Not completely, but we will make the verification results available to all the participants before we make them publicly available and will conduct additional verifications, if necessary.
Is there a way of circumventing the alea of the post-challenge verification?
Yes, you may send us your executable code BEFORE we release the final evaluation data. Kaggle will provide a software vault to that end.
How do I create a team?
\r\nIf you are the team leader, ask all you team members to register to Kaggle and ask them for their Kaggle ID. Before they make any submission, go to the team page and add them to your team. If they have already made a submission, you have to contact Kaggle for a \"Team merger request\". Do NOT create other accounts, it is forbidden by Kaggle to have multiple accounts for the same user.
\r\nHow many submissions can I make?
As many as you want. However, we limit you to 5 submissions per day.
Can I use a robot to make submissions?
Robot submissions are not explicitly forbidden. We limit the number of submissions to 5 per day to avoid overloading the Kaggle server and discouraging people to overfit the validation data.
Can I use mixed methods?
We encourage you to use a single unified methodology for all the tasks. However, we acknowledge that some tasks are different in nature and may require adjustments or changes in strategy, so we do not impose that you use strictly the same method on all tasks.
Can I submit human performance results?
No. The organizers are providing that as a benchmark result. you need to devise a computer solution. Even semi-automatic methods are not OK, you need to devise a fully automatic system.
I do not see my results on the leaderboard, what's wrong?
You must check the box in the column \"Selected?\" on the submission page and click \"Submit Selection Changes\".
I am getting WARNINGs, what is wrong with my submission?
A warning like \"WARNING: Ignored row 'valid01_1' was supposed to have value of '11' but '1' was given having a distance of 1\" means that you submitted results on training examples whose lables are different from the truth values. In this example, your predicted twice the value 1 instead of once.
Is participation conditionned on releasing code?
No.
Couldn't people cheat by labeling the validation set?
Yes the participants can label the validation set by hand. We do not consider that cheating. This may be actually detrimental to the participants because they will not get feed-back on their performance on the submission platform to compare themselves to others and run at risk of overfitting. We will provide enough development data that using the validation data for training will be of little use.
How will you prevent people from cheating by labeling the final evaluation set?
We will make it difficult by leaving little time between the release of the decryption key for the final evaluation data and the deadline for submission of results. The top ranking participants will be under scrutiny and will have to cooperate with the organizers to verify that they did not cheat. The preferred method is that you submit executable code BEFORE we disclose the decryption key of the final evaluation data. This will allow us to easily verify the results of the top ranking participants by just running their code on final evaluation data. If some candidate winners did not submit their code, they will have to bring us their system for post-challenge verifications using additional verification data, similar to the final evaluation data. Any significant discrepancy in performance may result in disqualification. The results of the verifications will be published.
Can I use an alias not to reveal my identity?
We require participants to identify themselves by their real name when they register, and you must always provide a valid email so we can communicate with you. But your name will remain confidential, unless you agree to reveal it. Your email will always remain confidential. You may select an alias as the name that will be displayed in the public leaderboard. Do NOT create fake accounts under assumed names or multiple accounts for the same person, this is forbidden by Kaggle and may be a cause of disqualifcation.
What is your privacy policy?
For all information entered on the Kaggle website, see the Kaggle privacy policy. For all information supplied to ChaLearn directly, see the ChaLearn privacy policy.
The rules specify I will have to fill out a fact sheet, do you have information about that fact sheet?
You will have to fill out a multiple choice question form that will be sent to you when the challenge is over. It will include high level questions about the method you used, software and hardware platform. Details or proprietary information may be withheld and the participants retain all intellectual property rights on their methods.
Do I need to let you know what my method is?
Disclosing information about your method is optional. However, to participate to the final ranking, you will have to fill out a fact sheet about your method(s) with basic information. We encourage the participants not only to fill out the fact sheets, but write a paper with more details. Best paper awards will distinguish entries with principled, original, and effective methods, and with a clear demonstration of the advantages of the method via theoretical derivations and well designed experiments.
Will the organizers enter the competition?
The challenge organizers have formed a team called ChaLearnAdmin, which makes entries to stimulate participation. The organizers also enter \"Benchmark\" submissions from time to time. None of those entries are part of the competition. The organizers are excluded from competing in the challenge.
Can a participant give an arbitrary hard time to the organizers?
DISCLAIMER: ALL INFORMATION, SOFTWARE, DOCUMENTATION, AND DATA ARE PROVIDED \"AS-IS\". CHALEARN, KAGGLE, MICROSOFT AND/OR OTHER ORGANIZERS DISCLAIM ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR ANY PARTICULAR PURPOSE, AND THE WARRANTY OF NON-INFRIGEMENT OF ANY THIRD PARTY'S INTELLECTUAL PROPERTY RIGHTS. IN NO EVENT SHALL ANY ORGANIZER BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF SOFTWARE, DOCUMENTS, MATERIALS, PUBLICATIONS, OR INFORMATION MADE AVAILABLE FOR THE CHALLENGE.
Where can I get more information about the other events of the CHALEARN gesture challenge?
Is there code I can use to perform the challenge tasks?
We provide the following tools written in Matlab (R):
Who can I ask for more help?
For all other questions, email events@chalearn.org.
Last updated December 7, 2011.
\r\n
December 7, 2011
\r\nSample data released. See the README file for release notes. You may download all the batches develxx.zip xx=1 to 20 and validxx.zip xx=1 to 20 individually or download the two large archives devel_all_files_combined.7z and valid_all_files_combined.7z. The videos are in a quasi-lossless AVI format. See also our data mirrors if you have download problems.
\r\nJanuary 7. 2012
\r\nEntire dataset released. See the README file for release notes. You may download all the batches develxx, xx=1 to 480 in grouped archives devel01-40, devel41-80 ... devel441-480. The videos are in a quasi-lossless AVI format. We also provide videos compressed in a lossy AVI format.
\r\nMarch 7, 2012
\r\nFinal evaluation data for round 1 available from various mirrors in encrypted zip archives (the decryption key will be made available on April 7). The data are available is lossy compressed AVI format (final-1-20) and in quasi-lossless AVi format (final01, final02, ... final20).
\r\nThe depth normalization information may be downloaded from Depth_info_final.txt
\r\nApril 7, 2012
\r\nThe decyption key for the final evaluation data is:
\r\n4lkc221:
\r\nUse Winzip to decompress/decrypt the data (a free trial version is available from the Winzip website).
\r\n\r\n
\r\n
\r\n
\"ChaLearn Gesture Dataset (CGD2011), ChaLearn, California, 2011\"\r\n
To enter the challenge, please read and accept the rules.
\r\n\r\nHighlights\r\nThe official rules include the rules of four CHALEARN Gesture Challenge events we are organizing. You are presently entering the first round of quantitative evaluation, , which opened December 7, 2011. \r\nPlease read the rules carefully because you may not be eligible for prizes (there are US export regulations, which are in the way). We are sorry about that. If you would like to enter just-for-fun without competing for prizes, please let us know. \r\nDuring the development period, which lasts until April 6, 2012, you can only submit results on validation data (the score on the leaderboard corresponds to validation data results). The final evaluation data will be available encrypted from March 7, 2012. The decryption key will be made available April 7, 2012. You will then have just a few days to upload your final results (the current deadline is April 10, 2012). \r\nIf you want to qualify for prizes, you will have to cooperate with the organizers to reproduce your results, to verify that no \"manual\" data labeling was done. This may include submitting executable code or bringing your fully configured platform for testing to the workshop(s) we are holding so we can carry out tests on additional verification data. \r\nWe made great effort to provide you with good data, samples submissions, and sample code, but we may have made mistakes. Please be nice with us... and we will help you if there are problems. Have fun! \r\n | \r\n
Common terms used in these rules:
\r\nThese are the official rules that govern how the One-Shot-Learning ChaLearn Gesture Challenge contest promotion will operate. This promotion will be simply referred to as the “contest” throughout the rest of these rules and may be abbreviated on our website, in our documentation, and other publications as One-Shot-Learning Gesture Challenge or ChaLearn Gesture Challenge.
\r\nIn these rules, “organizers”, “we,” “our,” and “us” refer to CHALEARN and Microsoft Corporation; \"participant”, “you,” and “yourself” refer to an eligible contest participant.
\r\nSECTION 1 Contest description
\r\nThis is a skill-based contest and chance plays no part in the determination of the winner(s). The goal of the contest is to develop a system capable of recognizing hand gestures, signs, or signals, from video data and 3D sensor data, recorded with a Microsoft Kinect sensor. Occasionally gestures may include head movements or facial expressions. The tasks of the challenge will be “single user”, “small vocabulary” recognition of short continuous sequences of gestures.
\r\n(a) Focus of the Contest: The focus of the challenge is on “one-shot-learning” of gestures, which means learning to recognize gestures from a single example of each gesture category, drawn from a relatively small gesture vocabulary. A gesture vocabulary is a set of unique gestures, generally related to a particular task, for instance, hand signals used by divers, finger codes to represent numbers, signals used by referees, or marshaling signals to guide vehicles or aircrafts.
\r\n(b) The contest will include a quantitative evaluation conducted with test data pre-recorded with a Kinect sensor and a live qualitative evaluation. There will be prizes both for the quantitative and the qualitative evaluation. For the live qualitative evaluation, the participants will present a demonstration of their gesture recognition system using a Kinect sensor, in the context of an application.
\r\n(c) All eligible entries received will be judged using the criteria described below to determine winners.
\r\nSECTION 2 Tentative Contest Schedule
\r\nThe registered participants will be notified by email of any change in the schedule.
\r\nJune 20, 2011 CVPR 2011 Gesture Recognition Workshop: Launching of the Challenge.
\r\nNovember 7, 2011 Demonstrations at HCI 2011 held in conjunction with ICCV 2011.
\r\nRelease of sample data collected with Kinect by the organizers.
\r\nDecember 7, 2011 Challenge platform opens (tentative date).
\r\nJanuary 7, 2012 Release of additional development Kinect data.
\r\nMarch 7, 2012 Final evaluation data for round 1 (CVPR 2012) released (encrypted).
\r\nApril 7, 2012 Round 1 final evaluation data decryption key released.
\r\nApril 7+N, 2012 Round 1 final evaluation result deadline. N=3 to 10 days.
\r\nMay 1, 2012 Release of round 1 results to the participants.
\r\nMay 15, 2012 Round 1 papers due.
\r\nJune 16-17 2012 CVPR 2012: Live qualitative evaluation. Award ceremony for the first round of the challenge.
\r\nAugust 7, 2012 Final evaluation data for round 2 (ICPR 2012) released (encrypted).
\r\nSept. 7, 2012 Round 2 final evaluation data decryption key released.
\r\nSept. 7+N, 2012 Round 2 final evaluation result deadline. N=3 to 10 days.
\r\nOct.1, 2012 Release of round 2 results to contest participants.
\r\nOct.15, 2012 Round 2 papers due.
\r\nNov. 11-15 2012 ICPR 2012: Live qualitative evaluation. Award ceremony for the second round of the challenge.
\r\nSECTION 3 Eligibility
\r\nYou are eligible to enter this contest if you meet the following requirements:
\r\n(a) You are a professional or enthusiast in the field of computer science, software development or a related field; and
\r\n(b) You are NOT a resident of any of the following countries: Cuba, Iran, North Korea, Sudan, and Syria. U.S. export regulations prohibit the export of goods and services to Cuba, Iran, North Korea, Sudan and Syria. Therefore residents of these countries / regions are not eligible to participate; and
\r\n(c) You are not an employee or student of CHALEARN or an employee of Microsoft Corporation or an employee of a Microsoft subsidiary; and
\r\n(d) You are not involved in any part of the administration and execution of this contest; and
\r\n(e) You are not an immediate family (parent, sibling, spouse, or child) or household member of a CHALEARN employee or a Microsoft employee, or a Microsoft subsidiary, or a person involved in any part of the administration and execution of this contest.
\r\nThis contest is void within the geographic area identified above and wherever else prohibited by law.
\r\nIf you choose to submit an entry, but are not qualified to enter the contest, this entry is voluntary, and any entry you submit is governed by the remainder of these contest rules; CHALEARN reserves the right to evaluate it for scientific purposes. If you are not qualified to submit a contest entry and still choose to submit one, under no circumstances will such entries qualify for Microsoft sponsored prizes.
\r\nSECTION 4 Entry
\r\nTo be eligible for judging, an entry must meet the following content/technical requirements:
\r\n(a) There will be no requirement to submit code or disclose algorithms to enter the competition. However, to be part of the final ranking the participants will be asked to fill out a survey (fact sheet) briefly describing their methods. In addition, to enter the qualitative evaluation, the participants will be asked to certify that they use a Kinect sensor and the Microsoft Software Development Kit (SDK). To qualify for prizes, the top ranking participants of the quantitative evaluation will be asked to submit themselves to post-challenge verifications, as detailed in Section 7.
\r\n(b) For each conference, the participants will be free to enter either or both evaluations (quantitative and qualitative), which will take place in sequence (quantitative first):
\r\n(c) Data
\r\n(d) In addition, by submitting your entry into this contest you confirm that, to the best of your knowledge:
\r\nSECTION 5 Potential use of entry
\r\nOther than what is set forth below, we are not claiming any ownership rights to your entry. However, by submitting your entry, you:
\r\n(a) Are granting us an irrevocable, worldwide right and license, in exchange for your opportunity to participate in the contest and potential prize awards, for the duration of the protection of the copyrights to:
\r\nThis license does not extend to methods, algorithms, code or systems used to generate your entry, which will only be requested and examined for verification purpose if you are eligible for a prize and desire to claim it, as explained in Sections 7 and 8 below.
\r\n(b) Agree to sign any necessary documentation that may be required for us and our designees to make use of the rights you granted above;
\r\n(c) Understand and acknowledge that that Microsoft Corporation and other entrants may have developed or commissioned materials similar or identical to your submission and you waive any claims you may have resulting from any similarities to your entry;
\r\n(d) Understand that we cannot control the incoming information you will disclose to our representatives or our co-sponsor’s representatives in the course of entering, or what our representatives will remember about your entry. You also understand that we will not restrict work assignments of representatives or our co-sponsor’s representatives who have had access to your entry. By entering this contest, you agree that use of information in our representatives’ or our co-sponsor’s representatives unaided memories in the development or deployment of our products or services does not create liability for us under this agreement or copyright or trade secret law;
\r\n(e) Understand that you will not receive any compensation or credit for use of your entry, other than what is described in these official rules.
\r\nIf you do not want to grant us these rights to your entry, please do not enter this contest.
\r\nSECTION 6 Submission of entry in the quantitative evaluation
\r\n(a) We have retained the services of Kaggle.com to process entry submissions. The submission of entries is via the web interface available at: http://www.kaggle.com/c/gesturechallenge. Follow the instructions on the website to submit entries.
\r\n(b) The participants will be registered as mutually exclusive teams, following the rules of Kaggle.com. Each team can make a maximum of five( 5) entries per day on validation data during the development period. Each team may submit only one single final entry on final evaluation data in each round (one each for the CVPR 2012 conference and for the ICPR 2012 conference). If several final entries are made by the same team, the last one received before the deadline will be counted as the final entry for purposes of the contest. We are not responsible for entries that we do not receive for any reason, or for entries that we receive but are not decipherable for any reason.
\r\n(c) We will automatically disqualify:
\r\nSECTION 7 Judging the entries
\r\nThe board of CHALEARN will select a panel of judges to judge the entries; all judges will be forbidden to enter the contest and will be are experts in computer vision, pattern recognition or machine learning (more particularly in gesture recognition, or experts in challenge organization). A list of the judges will be made available upon request and will include at least one independent judge not employed by or otherwise affiliated with CHALEARN or Microsoft. The judges will review all eligible entries received and select three (3) winners in each quantitative and qualitative evaluations and in each round (a total of 12 winners) based upon the following criteria:
\r\n(a) Quantitative evaluation:
\r\n(b) Qualitative evaluation:
\r\n(i) Relevance: The demonstration meets the requirements of the contest to provide a good application of gesture recognition using Kinect.
\r\n(ii) Usefulness: The demonstration proposes an application or a game, which addresses a problem of real interest for the Industry or the public at large, in any area of application, including entertainment.
\r\n(iii) Technical/scientific contribution: The demonstration illustrates a gesture recognition algorithm, which is a significant technical/scientific contribution, because it increased recognition accuracy above the state-of-the-art, and/or because if addressed aspects of the problem, which were previously unsolved.
\r\n(iv) Novelty/originality: The application demonstrated is particularly novel or original.
\r\n(v) Implementation: The demonstration is neatly implemented with real time performance and quality graphics.
\r\n2. The grades will be averaged over all criteria and panelists to determine the ranking score.
\r\n3. A selection of top ranking participants will demonstrate their system in front of an audience.
\r\nThe decisions of these judges are final and binding. The distribution of prizes according to the decisions made by the judges will be made within three (3) months after completion of the last round of the contest. If we do not receive a sufficient number of entries meeting the entry requirements, we may, at our discretion based on the above criteria, not award any or all of the contest prizes below. In the event of a tie between any eligible entries, the tie will be broken in the quantitative evaluation by giving preference to the earliest submission, using the time stamp of the submission platform, and in the qualitative evaluation by calling an additional judge who will judge the tied entries based on the criteria listed above.
\r\nSECTION 8 Prizes and Awards; Post Contest Evaluation by Microsoft
\r\n(a) Microsoft is the financial sponsor of this contest. There will be Forty Thousand US Dollars (US$40,000.00) in prizes awarded as incentive prizes to boost contest participation; these prizes will not require participants to enter into an IP agreement with Microsoft, to disclose algorithms, or to deliver code to Microsoft.
\r\n(b) Incentive Prizes
\r\nQuantitative evaluation awards - 1st place: $5,000; 2nd place: $3,000; 3rd place: $2,000
\r\nQualitative evaluation awards - 1st place: $5,000; 2nd place: $3,000; 3rd place: $2,000
\r\nQuantitative evaluation awards - 1st place: $5,000; 2nd place: $3,000; 3rd place: $2,000
\r\nQualitative evaluation awards - 1st place: $5,000; 2nd place: $3,000; 3rd place: $2,000
\r\n(c) Best paper awards:
\r\nCHALEARN will also give a best paper award and a best student paper award, with a small associated cash prize and/or a Microsoft Kinect sensor.
\r\n(d) Travel awards:
\r\nFor each round of the contest, CHALEARN will also distribute travel awards to selected participants in the quantitative evaluation who also wish to participate in the qualitative live evaluation (to partially cover their travel expenses). There will be up to Twenty Thousand US Dollars (US$20,000.00) total in travel awards distributed to participants for all rounds of the contest. The awards will be distributed according to merit and need.
\r\n(e) If for any reason the advertised prize is unavailable, unless to do so would be prohibited by law, we reserve the right to substitute a prize(s) of equal or greater value, as permitted. We will only award one prize per team. If you win multiple times in the quantitative or qualitative evaluation of either round, you will only receive the largest prize for which you are eligible, and no other prizes. We reserve the right to distribute remaining prize monies as additional travel awards or as prizes for the next best ranking participants. If you are selected as a potential winner of this contest:
\r\n(f) Post Contest Evaluation by Microsoft; Potential Award of Intellectual Property Agreement with Microsoft
\r\nAdditionally, after the contest ends and independently of the evaluation conducted according the protocol of Section 7, above, Microsoft will offer contest participants the opportunity to have their entries evaluated for the potential award of an intellectual property agreement with Microsoft, for which Microsoft may offer two agreements of up to One Hundred Thousand US Dollars (US$100,000.00) per agreement, for up to Two Hundred Thousand US Dollars (US$200,000.00) total to contest participants who have entered any of the four evaluations organized in conjunction with either CVPR 2012 or ICPR 2012 (one quantitative and one qualitative evaluation at CVPR 1012 and ICPR 2012). In order to be considered for an agreement with Microsoft, contest participants will need to do the following:
\r\nThere is no guarantee that Microsoft will enter into any agreement with a contest participant, however. Microsoft may enter into both US$100,000.00 agreements for the entire US$200,000.00 amount, or may enter into no agreements as a result of this evaluation.
\r\nSECTION 9 Other Sponsored Events
\r\n(i) a data exchange following strict specifications;
\r\n(ii) submission of results on validation data on a web platform;
\r\n(iii) participation in a milestone event to demonstrate systems under development;
\r\n(iv) submission of a video to demonstrate systems under development.
\r\nSECTION 10 Notification
\r\nIf you are a potential winner, we will notify you by sending a message to the e-mail address listed on your entry within seven days following the determination of winners. If the notification that we send is returned as undeliverable, or you are otherwise unreachable for any reason, we may award the prize to an alternate winner. However, if the potential winner is a resident of Portugal, no alternate winner will be chosen and the prize will revert to the charitable institution designated by the civil governor, as required under Portuguese law.
\r\nWinners who have entered the contest as a team will be responsible to share any prize among their members. The prize will be delivered to the registered team leader. If this person becomes unavailable for any reason, the prize will be delivered to be the authorized account holder of the e-mail address used to make the winning entry.
\r\nIf you are a potential winner, we may require you to sign a declaration of eligibility, use, indemnity and liability/publicity release. If you are a potential winner and are a minor in your place of residence, and we require that your parent or legal guardian will be designated as the winner pursuant to Section 8(4) above, we may require that they sign a declaration of eligibility, use, indemnity and liability/publicity release on your behalf. If you, (or your parent/legal guardian if applicable), do not sign and return these required forms within the time period listed on the winner notification message, we may disqualify you (or the designated parent/legal guardian) and select an alternate selected winner.
\r\nSECTION 11 Method of notification
\r\nWe will post the names of confirmed winners online at http://gesture.chalearn.org after contest decisions are made by the judges. This list will remain posted for one year or will be made available upon request by sending email to events@chalearn.org.
\r\nSECTION 12 Conditions.
\r\nBy entering this contest you agree:
\r\n(a) To abide by these official rules;
\r\n(b) To the extent allowable under applicable law, to release and hold harmless CHALEARN, and Microsoft Corporation, their respective parents, subsidiaries, affiliates, employees and agents from any and all liability or any injury, loss, damage, right, claim or action of any kind arising from or in connection with this contest or any prize won save for residents of the United Kingdom, Chile, Korea, Greece, Brazil, Turkey, Hong Kong, France and Germany with respect to claims resulting from death or personal injury arising from CHALEARN’s and Microsoft Corporation’s negligence, for residents of the United Kingdom with respect to claims resulting from the tort of deceit or any other liabilities that may not be excluded by law, and for residents of Australia in respect of any implied condition or warranty the exclusion of which from these official rules would contravene any statute or cause any part of these official rules to be void;
\r\n(c) That CHALEARN’s decisions will be final and binding on all matters related to this contest; and
\r\n(d) That by accepting a prize, CHALEARN and Microsoft Corporation may use your name and place of residence online and in print, or in any other media, in connection with this contest, without payment or compensation to you. The declaration of eligibility, use, indemnity and liability/publicity release provided to the potential winner (pursuant to clause 9 above) will make reference to obtaining his/her free consent to use his/her name and place of residence. In any case, the lack of such consent does not prevent the winner from receiving the prize.
\r\n(e) This contest will be governed by the laws of the state of Washington, and you consent to the exclusive jurisdiction and venue of the courts of the state of Washington for any disputes arising out of this contest. For residents of Austria only: you may withdraw your submission from this contest within seven days of your entry. If you withdraw within seven days of entry, your submission will be returned to you, and we will not make any use of your submission in the future. However, you will not be eligible to win a prize. If you do not withdraw within seven days of entry, you will be bound by the provisions of these official rules. For residents of the United Kingdom only: the provisions of the contracts (rights of third parties) act 1999 will not apply to this agreement. For residents of New Zealand only: the provisions of the contracts (privity) act of 1982 will not apply to this agreement.For Quebec residents: any litigation respecting the conduct or organization of a publicity contest may be submitted to the Régie des Alcools, des Courses et des Jeux for ruling. Any litigation respecting the awarding of a prize may be submitted to the Régie only for the purpose of helping the parties reach a settlement. For residents of Israel only: this agreement does not entitle third parties to benefits under this agreement as defined in Chapter “D” of the Contracts Act (General Part) – 1973.
\r\nSECTION 13 Unforeseen event
\r\nIf someone cheats, or a virus, bug, catastrophic event, or any other unforeseen or unexpected event that cannot be reasonably anticipated or controlled, (also referred to as force majeure) affects the fairness and / or integrity of this contest, we reserve the right to cancel, change or suspend this contest. This right is reserved whether the event is due to human or technical error. If a solution cannot be found to restore the integrity of the contest, we reserve the right to select winners based on the criteria specified above from among all eligible entries received before we had to cancel, change or suspend the contest subject to obtaining the approval from the Régie des Alcools, des Courses et des Jeux with respect to the province of Quebec.
\r\nComputer “hacking” is unlawful. If you attempt to compromise the integrity or the legitimate operation of this contest by hacking or by cheating or committing fraud in any way, we may seek damages from you to the fullest extent permitted by law. Further, we may ban you from participating in any of our future contests, so please play fairly.
\r\nSECTION 14 Sponsor
\r\nChaLearn is the sponsor of this contest.
\r\n955 Creston Road,
\r\nBerkeley, CA 94708, USA
events@chalearn.org
and
Microsoft Corporation is the co-sponsor of this contest
One Microsoft Way
\r\nRedmond, WA 98052
\r\n(425) 882-8080
\r\nSECTION 15 Privacy
\r\nDuring the development phase of the contest and when they submit their final entries, contest participants do not need to disclose their real identity, but must provide a valid email address where they can be delivered notifications to them regarding the contest. To be eligible for prizes, however, contest participants will need to disclose their real identity to contest organizers. To enter the contest, the participants will need to become users of the Kaggle.com platform. Any profile information stored on this platform can be viewed and edited by the users. After the contest, the participants may cancel their account with Kaggle.com and cease to be users of that platform. All personal information will then be destroyed. Kaggle.com’s privacy policy will apply to contest information submitted by participants on Kaggle.com: http://www.kaggle.com/pages/privacy. Otherwise, CHALEARN’s privacy policy will apply to this contest and to all information that we receive from your entry that we receive directly from you or which you have submitted as part of your contest entry on Kaggle.com. Please read the privacy policy on the contest entry page before accepting the official rules and submitting your entry. Please note that by accepting the official rules you are also accepting the terms of the CHALEARN privacy policy: http://www.chalearn.org/privacy.html.
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Data Description 2", "content": "\r\nDownload the data\r\n\r\n | \r\n
\r\n
We are portraying a single user in front of a fixed Kinect (TM) camera, interacting with a computer by performing gestures:
\r\nWe recorded both an RGB and Depth images (representing the distance of the objects to the camera).
\r\n| RGB image | \r\nDepth image | \r\n
| \r\n | \r\n |
\r\n
\r\n
\r\n
\r\n
Each video contains the recording of 1 to 5 gestures from a vocabulary of 8 to 15 gesture tokens. For instance a gesture vocabulary may consist of the signs to referee volleyball games or the signs to represent small animals in the sign language for the deaf.
\r\nYou need to predict the identity of those gestures, represented by a numeric label (from 1 to 15). The data are divided into data batches, each having a different vocabulary of gestures. So the numeric labels represent different gestures in every batch.
\r\nIn the data used for evaluation (called validation data and final evaluation data), you get one video clip for each gesture token as a training example in every batch. You must predict the labels of the gestures played in the other unlabeled videos.
\r\n\r\n
For each video, you provide an ordered list of labels R corresponding to the recognized gestures. We compare this list to the corresponding list of labels T in the prescribed list of gestures that the user had to play. These are the \"true\" gesture labels (provided that the users did not make mistakes). We compute the so-called Levenshtein distance L(R, T), that is the minimum number of edit operations (substitution, insertion, or deletion) that one has to perform to go from R to T (or vice versa). The Levenhstein distance is also known as \"edit distance\".
\r\nFor example:
\r\nL([1 2 4], [3 2]) = 2
\r\nL([1], [2]) = 1
\r\nL([2 2 2], [2]) = 2
\r\nWe provide the Matab(R) code for the Levenshtein distance in our sample code.
\r\n\r\n
The overall score we compute is the sum of the Levenshtein distances for all the lines of the result file compared to the corresponding lines in the truth value file, divided by the total number of gestures in the truth value file. This score is analogous to an error rate. However, it can exceed one.
\r\nPublic score means the score that appears on the leaderboard during the development period and is based on the validation data.
\r\nFinal score means the score that will be computed on the final evaluation data released at the end of the development period, which will not be revealed until the challenge is over. The final score will be used to rank the participants and determine the prizes.
\r\n\r\n
To verify that the participants complied with the rule that there should be no manual labeling of the test data, the top ranking participants eligible to win prizes will be asked to cooperate with the organizers to reproduce their results.
\r\nPreferred procedure
\r\nAt the end of the development period, from March 7 until April 6, 2012, the participants will be offered by Kaggle the possibility of uploading executable code for a standard platform to a software vault. The organizers will evaluate requests to support particular platforms, but do not commit to support all platforms. The burden of proof will rest on the participants, see our backup procedure. The code will be kept in confidence and used only for verification purpose after the challenge is over. The code submitted will need to be standalone and in particular it will not be allowed to access the Internet. It will need to be capable of training models from the final evaluation data training examples, for each data batch, and making label predictions on the test examples of that batch. Instructions on how to prepare the code will be given in February 2012.
\r\nBackup procedure
\r\nIf for some reason a participant elects not to submit executable code before the April 6, 2012 deadline, he/she will have the option of bringing a full system to the site of the workshop at CVPR 2012, or another location mutually agreed upon, to let the organizers perform a live test. The organizers may also decide to run this backup procedure if, for a technical reason, the executable code provided by the participants cannot be run on their computers. The verification will be carried out using verification data similar to the final evaluation data. Statistically significant discrepancies in performance between the final evaluation data and the verification data may be a cause of disqualification. The results of the verifications will be published by the organizers.
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false } ] }