## Model Performance Evaluation

The model's predictions would be evaluated on the basis of **classification accuracy**, which refers to the sum of true positives and true negatives divided by the total number of observations in the dataset.

Learning teams that would get the best model performance on the **private leaderboard** (which contains all of the test data) would get a perfect score on the second case study. Each team would be ranked at the end of the competition, and the score of each team in the second case study would be based from the following imputation table:

| Rank | Case Study Score |
| --- | --- |
| 1 | 100  |
| 2 | 95  |
| 3 | 90 |
| 4 | 85 |
| 5 | 80  |

Throughout the competition, teams would know how their model is performing relative to other learners in the class through the public leaderboard. The public leaderboard only evaluates the model to a random proportion of observations on the test set, so that students would not game the system. The final rankings of all teams would be dependent on the rankings shown on the **private leaderboard**. Hence, student team's rankings may go higher or lower relative to their rank on the **public leaderboard**.

## Submission Format

**For every customer in the dataset**, submission files should contain two columns: `id` and `y`.  `y` refers to the team's predictions for each customer on the test set.

The CSV file should contain a header and have the following format:
```
id, y
1,1
2,0
3,0
4,1
5,0
```

Do take note that the test set has 10297 rows. Make sure that your team is uploading a CSV file containing predictions for 10297 customers.

Students are allowed to upload up to 20 predictions for the test set each day. Once this limit has been breached, the team has to wait for the day to elapse before being able to upload a submission once again. Each submitted predictions would be evaluated on the public leaderboard up until May 17, 2021 (11:59 PM). No more submissions would be accepted by the system beyond this time. The results on the private leaderboard would be up by May 18, 2021 (12:00 NN).