### **Datasets**: 
There are 4 groups of datasets associated with this competition.

1. **Training Set: This dataset, containing labels for the target, is available for download outside of Kaggle for competitors to build their models.** It is broken up into 50 files, for ease of access and download. Due to its large size, it must be accessed through a GCS bucket which is only made available to participants after accepting the competition’s rules. Please read the rules fully before accessing the dataset, as they contain important details about the dataset’s permitted use. It is expected and encouraged that you train your models outside of Kaggle’s notebooks environment and submit to Kaggle by uploading the trained model as an external data source.
2. **Public Validation Set**: When you commit your Kaggle notebook, the submission file output that is generated will be based on the small set of 400 videos/ids contained within this Public Validation Set. This is available on the Kaggle Data page as `test_videos.zip`
3. **Public Test Set: This dataset is completely withheld and is what Kaggle’s platform computes the public leaderboard against.** When you “Submit to Competition” from the “Output” file of a committed notebook that contains the competition’s dataset, your code will be re-run in the background against this Public Test Set. When the re-run is complete, the score will be posted to the public leaderboard. If the re-run fails, you will see an error reflected in your “My Submissions” page. Unfortunately, we are unable to surface any details about your error, so as to prevent error-probing. You are limited to 2 submissions per day, including submissions which error.
4. **Private Test Set: This dataset is privately held outside of Kaggle’s platform, and is used to compute the private leaderboard.** It contains videos with a similar format and nature as the Training and Public Validation/Test Sets, but are real, organic videos with and without deepfakes. After the competition deadline, Kaggle transfers your 2 final selected submissions’ code to the host. They will re-run your code against this private dataset and return prediction submissions back to Kaggle for computing your final private leaderboard scores.

### **Workflow:** 
This is a walk-through of how to make a submission to this code competition. The workflow is unique to this competition, so please read carefully.

1. **Accept the competition’s rules (Kaggle)**. Read these carefully before accepting, as there are important commitments you are making to the host about the use of their data.
2. **Access & download the Training Set (Google Cloud Storage Bucket)**. This is hosted outside of Kaggle. The link to this dataset is made available only after accepting the competition’s rules. See details under the “Datasets” section above.
3. **Train your models (Your choice - Local Machine, VM)**. You should be training your models outside of Kaggle using local or cloud resources. [AWS is offering promotional credits](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/122254) for those needing additional computational resources to approach this challenge.
4. **Use Kaggle Notebooks (Notebook or Script) to create your submission (Kaggle)**. This is a code competition, so submissions must be made through Kaggle. You can upload any pre-trained or locally-trained models as data sources to your submission notebook. Your submission is subject to the [Code Requirements](https://www.kaggle.com/c/deepfake-detection-challenge/overview/code-requirements) mandated by this competition.
5. **Commit your Notebook/Script (Kaggle)**. Click commit on the notebook to run your code against the Public Validation Set. If successful, the notebook will generate a `submission.csv` file as an “Output”
6. **“Submit to Competition” from the “Output” file created (Kaggle)**. This will make your official submission to the competition. The “Submit to Competition” button will only be available if your notebook meets the requirements specified in [Code Requirements](https://www.kaggle.com/c/deepfake-detection-challenge/overview/code-requirements). Your code will be re-run against the Public Test Set in the background. If successful, it will generate a score on the public leaderboard. You can review your submissions’ status on the “My Submissions” page. Note that this can take longer than the prior commit step due to the size of Public Test Set being larger than the Public Validation Set.
7. **Select 2 final submissions by the Submission Deadline (Kaggle)**. On your “My Submissions” page, your team leader can select the checkbox under “Use for final score” next to the 2 submissions to be re-run against the Private Test set.
8. **Wait and see!** After the submission deadline, Kaggle will transfer your 2 selected submissions’ code to the host. They will re-run your code against their Private Test Set and return prediction submissions back to Kaggle for computing your final private leaderboard scores. For the private test set evaluation, the time constraint for your code’s run time will scale with that private test set's size. Review the [Timeline Page](https://www.kaggle.com/c/deepfake-detection-challenge/overview/timeline) for details of the expected timing for the private leaderboard review.

###**Tips for successful submissions**:

- **Train your models outside of Kaggle.** The training set is intentionally not available as a competition data source directly through Kaggle’s notebooks. It is not practical to train a model on Kaggle within the time constraints imposed.
- **Your code must output a submission.csv that predicts on any set of `test_videos.zip` that it is fed as an input.** We’ve designed the Public Validation/Test Sets to aid in ensuring your code is robust enough for the Private Test Set. The Public Validation Set gives an example of what you can expect your model should handle while working in an interactive notebook session. The Public Test Set allows you to test your code’s performance on a withheld test set. 
- **Ensure your code can be re-run on any unseen test set.** Do not hard-code your submissions based on the public validation set. This will result in failure when your code runs against the public test set. You will not benefit from probing the Public Validation/Test Sets, because your code must perform honestly on the host’s fully Private Test Set.
- **You are limited to 9 hours of GPU compute time.** This constraint is also imposed on the Public Test Set re-run. You should anticipate the Public Test set to be 10 times the size of the Public Validation Set and budget accordingly. The 9 hour compute constraint will be applied to the submission in Kaggle, when evaluated against the Public Test Set. So if you are able to make a successful submission that posts a score to the public leaderboard without an error, then it has met the 9 hour constraint on the Public Test Set. Review the full set of [Code Requirements](https://www.kaggle.com/c/deepfake-detection-challenge/overview/code-requirements).
- **You are still limited to Kaggle’s [global weekly per user GPU time constraints](https://www.kaggle.com/general/108481)**.
- **Don’t make the mistake of creating multiple accounts or private sharing.** Any user who submits from multiple accounts will be removed at the end of the competition, without exception. This includes entire teams where only 1 member has created a multiple account. It also includes teams who are found to have shared privately with other teams or colluded with other teams to make any submissions. It also includes teams who engaged in private sharing prior to forming as a team. We will complete these removals at the end of the competition. If you have knowledge of any teams who have committed rules violations, please submit a claim to [Kaggle Compliance](https://www.kaggle.com/compliance) with your specific evidence.
- **Resolving Submission Errors**: Below is a quick rundown of the some errors that users may receive on submission. Note that we are not in a position to share more granular details of any precise exceptions/errors, as doing so would make the competition vulnerable to error-probing by savvy participants. 
     - **Notebook Timeout**: Your submission notebook exceeded the allowed runtime cap for the competition. Review the competition's [Code Requirements](https://www.kaggle.com/c/deepfake-detection-challenge/overview/code-requirements) for details, and note that the privately held rerun test set could be orders of magnitude larger than the publicly shared validation set.
     - **Submission Scoring Error**: Your notebook generated a `submission.csv` file with incorrect format.  Some examples causing this are having the wrong number of rows or columns, empty values, an incorrect datatype for a value, or invalid submission values from what is expected for the privately rerun test set.
     - **Notebook Threw Exception**: While rerunning your code on the public test set (privately held), your notebook hit an error. 
     - **Submission CSV Not Found**: Your Notebook did not output a `submission.csv` file. Some examples causing this are if your Notebook times out, or if it runs into an error. There is a chance this will be the error returned instead of "Notebook Timeout" depending on error sequencing.
     - **No `Submit to Competition` button on my notebook's output**: This indicates that you have enabled something that is prohibited for the submissions on that competition. Review the competition's [Code Requirements](https://www.kaggle.com/c/deepfake-detection-challenge/overview/code-requirements) for details
     - **Notebook Exceeded Allowed Compute**: This indicates you have violated a [Code Requirement](https://www.kaggle.com/c/deepfake-detection-challenge/overview/code-requirements) constraint during the rerun.
     - **Kaggle Error**: A rare system error. Please try resubmitting to resolve the error and contact Kaggle Support if it persists.



**See the Deepfake Detection Challenge website for answers to more [Frequently Asked Questions](https://deepfakedetectionchallenge.ai/faqs).**