Submissions are evaluated using **multi-class classification accuracy**.

### Metric Definition

Accuracy is calculated as:

```
Accuracy = (Number of Correct Predictions) / (Total Number of Predictions)
```

A prediction is correct only when your predicted `risk_level` **exactly matches** the true risk level. There is no partial credit for "close" predictions.

### Leaderboard Splits

- **Public Leaderboard**: Calculated on approximately **30%** of the test data
- **Private Leaderboard**: Calculated on the remaining **70%** of test data (determines final rankings)

Your final rank is determined by your performance on the **private leaderboard**. The public leaderboard is provided for feedback during the competition but is not used for final scoring.

### Submission Format

For each `id` in the test set, you must predict the `risk_level` (1, 2, 3, or 4). Your submission file should contain a header and have the following format:

```csv
id,risk_level
0,1
1,2
2,1
3,4
4,3
5,2
...
```

#### Format Requirements

- **Header row**: Must include `id,risk_level`
- **23,152 rows**: One prediction for each test sample (plus header)
- **Valid values**: `risk_level` must be 1, 2, 3, or 4 (integers only)
- **Complete predictions**: All test IDs must be present
- **File format**: CSV (comma-separated values)

### Submission Limits

- **5 submissions per day** (24-hour rolling window)
- **Select up to 2 final submissions** for judging before the deadline
- If you don't select submissions, your highest-scoring public leaderboard submissions will be used

### Important Notes

**Select Your Final Submissions**: Make sure to select which 2 submissions you want scored on the private leaderboard. This is critical for your final ranking.

**Public vs Private**: The public leaderboard is only 30% of the data. Models that overfit to the public leaderboard may not perform well on the private leaderboard. Use cross-validation on the training data to assess true model performance.

**Class Distribution**: The training set has balanced classes (495 per class), but the test set distribution may differ. Consider this when developing your models.

### Example Submission Code

```python
import pandas as pd

# Load your predictions
predictions = your_model.predict(test_data)

# Create submission dataframe
submission = pd.DataFrame({
    'id': range(len(predictions)),
    'risk_level': predictions
})

# Save to CSV
submission.to_csv('submission.csv', index=False)
```

### Validation Strategy

Since the public leaderboard only uses 30% of test data, we recommend:

1. **Cross-validation**: Use k-fold cross-validation on the training data
2. **Stratified splits**: Maintain class balance in your validation folds
3. **Multiple metrics**: Track accuracy, F1-score, and confusion matrix
4. **Avoid overfitting**: Don't chase small improvements on public leaderboard

### Tiebreakers

If multiple submissions have identical accuracy (to 6 decimal places):
- The **earlier submission timestamp** will rank higher
- This encourages participants to submit strong solutions early

Good luck!