# Dataset Card for the ASL Fingerspelling Recognition Corpus

### Dataset Summary

The ASL Fingerspelling Recognition Corpus (version 1.0) is a collection of hand and facial landmarks generated by Mediapipe version 0.9.0.1 on videos of phrases, addresses, phone numbers, and urls fingerspelling by over 100 Deaf signers.

### Supported Tasks and Leaderboards

https://www.kaggle.com/competitions/asl-fingerspelling/leaderboard 


### Languages

Fingerspelling as seen in American Sign Language

## Dataset Structure

See https://www.kaggle.com/competitions/asl-fingerspelling/data

### Data Splits

Not applicable.

## Dataset Creation

### Curation Rationale

This dataset includes fingerspelling of letters, numbers, and symbols at real world speeds. This may help move sign language recognition forward, making AI more accessible for the Deaf and Hard of Hearing community.

### Source Data

#### Initial Data Collection and Normalization

Signers who communicate using American Sign Language as their primary language were recruited from across the United States. They were shipped a smartphone with an installed data collection app. The app prompted the signer with the English text to sign. Signers pressed an on-screen button on the phone to begin and end the recording of video during which they fingerspelled the on screen text. The video clip boundaries were adjusted to contain video around the fingerspelling, but this process was not perfectly accurate.

The fingerspelling contains a high degree of co-articulation and even lexicalization of full words.  Meaning, the letter handshapes are modified based on what comes before or after. Many variations of handshapes are used to convey meaning.  Many different body poses, zoom levels, appearances, and accessories are present in the dataset.  Signers could use either their left or right hand to produce fingerspelling, and many even switched between hands in different clips.

The signers were shown text which included capitalization, however, most signers chose not to convey the capitalization within their fingerspelled responses.  Some signers conveyed the capitalizations using several different methods, most commonly with one of three methods.  Using a curled-L handshape (perhaps with motion to indicate the entire word is capitalized) before the letter to be capitalized, by physically placing the capitalized letters higher in space than the lower case letters, or by shaking the letter's handshape a little bit to indicate it is capitalized.  This dataset does not attempt to evaluate the detection of capitalization, and all target phrases are lower case.

#### Who are the source language producers?

Over 100 signers recruited by the Deaf Professional Arts Network provided the sign. They are from many regions across the United States and all use American Sign Language as their primary form of communication. They represent a mix of skin tones and genders.

### Annotations

#### Annotation process

Each video was annotated at creation time by the smartphone app. Videos were filtered, modified and clip boundaries assigned through a largely automatic process which corrected many issues in the underlying raw data.  Incorrect clip boundaries are the most prevalent remaining issue in the data samples. Little judgment was made on the correctness or quality of the fingerspelling itself.

#### Who are the annotators?

Google engineers used a mix of manual and automatic processes to improve the dataset quality.

### Personal and Sensitive Information

The landmark data has been de-identified. Landmark data should not be used to identify or re-identify an individual. Landmark data is not intended to enable any form of identity recognition or store any unique biometric identification.

## Considerations for Using the Data

### Discussion of Biases

While ASL is the most common sign language used in the United States, there are many sign languages, including British Sign Language, Native American Sign Languages, Hawaiian Sign Language, French Sign Language, and Signed Exact English. In addition, there are many regional and cultural accents associated with sign in the United States, including Black Sign Language. This dataset focuses on American Sign Language, but it does not capture a representative sample of all the sign variations that would be commonly understood in conversation.  ASL has a grammar that is very different from English, and fingerspelling is only a small part of how meaning is conveyed in ASL.  A larger number of signers is necessary to better represent skin tones, hand features, and different levels of signing dexterity.
 
## Additional Information

### Dataset Curators

The Deaf Professional Arts Network (DPAN), is a 501(c)(3) non-profit founded in 2006 to make music, entertainment, and media accessible.

### Licensing Information

The dataset is provided by Google under CC-BY

### Citation Information

See the bottom of the Kaggle competition overview page for citation information.

### Contributions

Thanks to the staff at DPAN and their contributors who made this dataset possible.  Also, thanks to the students and faculty at Georgia Tech and NTID who collaborated on related datasets.