![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F26097785%2F3fc8cbe1ad3aa7002e71ef05b1d09a4d%2FStats%20(6).jpg?generation=1772058318466051&alt=media)

# 📈 Evaluation & Ranking Framework

---

## 🏆 Primary Metric

Submissions are evaluated using **ROC AUC (Area Under the Receiver Operating Characteristic Curve)**.

### 🔎 What ROC AUC Measures

ROC AUC evaluates a model’s ability to:

* Correctly **rank churn-risk customers higher** than non-risk customers
* Maintain ranking quality **across all possible classification thresholds**

Unlike accuracy, ROC AUC does **not depend on a fixed cutoff** (e.g., 0.5).
Instead, it measures how well the model separates positive and negative classes through probability ordering.

> A higher ROC AUC indicates stronger ranking performance.

---

## 🧮 Leaderboard Structure

### 🟢 Public Leaderboard

* Provides **real-time feedback**
* Based on a **subset of the test dataset**
* Intended for iterative model improvement

### 🔒 Private Leaderboard

* Based on a **hidden evaluation set**
* Determines the **final competition ranking**

> Final standings are based **exclusively** on the Private Leaderboard score.

---

## 🥇 Final Winner Determination

While predictive performance is critical, the final evaluation also considers engineering quality and methodological rigor.

### Evaluation Criteria Beyond ROC AUC:

* **Validation strategy robustness**
  (e.g., proper cross-validation, no leakage, stratification where appropriate)

* **Code organization & reproducibility**
  (clean structure, deterministic results, modular pipeline)

* **Feature engineering quality**
  (meaningful transformations, justified feature construction)

* **Strict absence of data leakage**

---

## ⚖️ Tie-Breaking Policy

In the event of identical ROC AUC scores:

> The winner will be determined based on **engineering rigor and methodological discipline**, not randomness.

---