{
  "fetched_at": "2026-10-18T14:10:13Z",
  "source_url": "https://www.kaggle.com/competitions/ci-avsr/data",
  "pages": [
    {
      "name": "data-description",
      "content": "For the **test_all.csv**, it includes the test meta data for all three subtasks.\n\n## Files\n*   **train_clean.csv** - the meta data for clean training set\n*   **train_noisy.csv** - the meta data for nosiy training set\n*   **valid_clean.csv** - the meta data for clean validation set\n*   **valid_noisy.csv** - the meta data for nosiy validation set\n*   **test_all.csv** - the meta data for test set\n*   **sample_submission.csv** - a sample submission file in the correct format\n*   **images.zip** - a zip file containing all the raw images data retrieved from the video (25fps)\n*   **audios.zip** - a zip file containing all the raw audio data retrieved from the video (16kHz)\n\nIf you are interested to process the raw video data or evaluating on the subtask mentioned on our paper, you can check our Github repo at: https://github.com/HLTCHKUST/CI-AVSR",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Description",
      "content": "# Overview\nCI-AVSR is a competition about recognizing Cantonese In-car Commands with multimodal data, including audio and video. The CI-AVSR dataset consists of 4,984 samples (8.3 hours of video and audio) of 200 unique in-car commands recorded by 30 native Cantonese speakers. In addition, each sample is augmented by 10 different in-car ambient noises that are commonly heard. The goal of this competition is to predict the ASR system trained on **clean** and/or **in-domain noises** data, and test the system on a combined test set covering **clean**, **in-domain noises**, and **out-of-domain noises** data\n\nMore details can be found in this paper: http://www.lrec-conf.org/proceedings/lrec2022/pdf/2022.lrec-1.731.pdf.\nIf you use this dataset, please consider to cite:\n```\n@InProceedings{dai-EtAl:2022:LREC,\n  author    = {Dai, Wenliang  and  Cahyawijaya, Samuel  and  Yu, Tiezheng  and  Barezi, Elham J.  and  Xu, Peng  and  YIU, Cheuk Tung  and  Frieske, Rita  and  Lovenia, Holy  and  Winata, Genta  and  Chen, Qifeng  and  Ma, Xiaojuan  and  Shi, Bertram  and  Fung, Pascale},\n  title     = {CI-AVSR: A Cantonese Audio-Visual Speech Datasetfor In-car Command Recognition},\n  booktitle      = {Proceedings of the Language Resources and Evaluation Conference},\n  month          = {June},\n  year           = {2022},\n  address        = {Marseille, France},\n  publisher      = {European Language Resources Association},\n  pages     = {6786--6793},\n  url       = {https://aclanthology.org/2022.lrec-1.731}\n}\n```\n\nIn addition to the proposed challenge above, we also introduce an additional challenge to recognize in-car commands covering three scenarios (subtasks):\n\n### Subtask-1: Clean Speech Cantonese Multimodal ASR\n\nIn subtask 1, we simulate an ideal situation for an ASR system where clean video and audio data is provided to participants. The clean multimodal data consists of the clean audio data without any background noise and the video of the front view of the speaker. Participants are invited to build a multimodal Cantonese ASR model using only the **train-clean** and {valid-clean} splits as the training and validation set, respectively. We as organizers will evaluate their trained models on the **test-clean** split. The output of the ASR models is graphemes of the corresponding speech command whose vocabulary consists of 284 written Cantonese characters and 26 lower-cased English alphabets. We will provide the complete vocabulary along with the dataset.\n\n### Subtask-2: In-domain Noisy Speech Cantonese Multimodal ASR\nIn subtask 2, we move closer to a real scenario of the in-car multimodal ASR setting, where the given speech command might be intervened by some noises such as the sounds of the engine combustion, horn, and other ambient noises. In this subtask, we assume that all the ambient noises are known such that additional training data with similar characteristic noises can be generated. Specifically, in subtask 2, we focus on evaluating the model's performance on in-domain (i.e. seen in the training set) data with five different noises, including windscreen wiper, horn, alarm, engine ignition, and door noises. Participants are allowed to train their models on the **train-clean** and **train-noisy (in-domain)** sets, and validate their models on both the **valid-clean** and **valid-noisy (in-domain)** sets. We will score their results on the **test-noisy (in-domain)**.\n\n### Subtask-3: Out-of-domain Noisy Speech Cantonese Multimodal ASR\nIn subtask 3, we move even further to the real scenario of the in-car multimodal ASR setting, where we assume that the characteristics of noises are unknown and only appear during the inference time. More specifically, we focus on evaluating the generalization ability of models to see whether they can still perform well on out-of-domain (i.e. unseen in the training data) noisy data, which includes five various noises (background music, rain, hail, people talking, road ambiance). Participants are allowed to train their models on the **train-clean** and **train-noisy (in-domain)** sets, and validate their models on the **valid-clean** and **valid-noisy (in-domain)** sets. We will evaluate their models on the **test-noisy (out-of-domain)** set.  \n\nIf you are interested in the challenge, you can get the data split on our Github repo at https://github.com/HLTCHKUST/CI-AVSR, and consider citing our shared task proposal as follow:\n```\n@article{Dai2022KaggleCC,\n  title={Kaggle Competition: Cantonese Audio-Visual Speech Recognition for In-car Commands},\n  author={Wenliang Dai and Samuel Cahyawijaya and Tiezheng Yu and Elham J. Barezi and Pascale Fung},\n  journal={ArXiv},\n  year={2022},\n  volume={abs/2207.02663}\n}\n```",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Evaluation",
      "content": "## Evaluation Metrics\nWe will provide a test set for each subtask and the participants are invited to submit the generation results to evaluate their models' performance. Due to Kaggle limitation, we don't use **CER** metric as mentioned on our paper, but instead we use a proportional metric namely **LevenshteinMean** as the evaluation metric. In detail, the **LevenshteinMean** is calculated by taking the average count of the number of character substitution, insertion, and deletion from the prediction compared to the gold label, while the actual **CER** is calculated by taking the total  count of the number of character substitution, insertion, and deletion and dividing them by the total number of characters.\n\n## Submission Format\nThe participants are invited to submit a CSV file named with \"TEAM_NAME.csv\" for evaluation. The format of the CSV file should be the same as \"sample_submission.csv\" in our released data.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "rules",
      "content": "# General Rules\n- Don't cheat!\n- Apply yourself!\n- Have fun :)\n\n# One account per participant\nYou cannot sign up to Kaggle from multiple accounts and therefore you cannot submit from multiple accounts.\n\nNo private sharing outside teams\nPrivately sharing code or data outside of teams is not permitted. It's okay to share code if made available to all participants on the forums.\n\n# Team Mergers\nTeam mergers are allowed and can be performed by the team leader. In order to merge, the combined team must have a total submission count less than or equal to the maximum allowed as of the Team Merger Deadline. The maximum allowed is the number of submissions per day multiplied by the number of days the competition has been running.\n\n# Team Limits\nThere is no limit of team size.\n\n# Submission Limits\nYou may submit a maximum of 5 entries per day.\nYou may select up to 2 final submissions for judging.\n\n# Competition Timeline\nThe competition is started from August 1st 2022 to July 31st 2023.\n\n# Use of External Training / Validation Data\nWe allow the use of any external data sources and the use of any large pre-trained language models",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    }
  ]
}