### Diarization Error Rate (DER)

Submissions are evaluated using **Diarization Error Rate (DER)**:

$$\text{DER} = \frac{\text{False Alarm} + \text{Missed Speech} + \text{Speaker Confusion}}{\text{Total Reference Speech}}$$

| Error Type | What It Means |
|------------|---------------|
| **False Alarm** | Predicted speech where there was silence |
| **Missed Speech** | Missed actual speech (predicted silence) |
| **Speaker Confusion** | Correct speech detection, wrong speaker assigned |

**Score** = 100 × (1 − DER), clipped to 0–100. Lower DER → Higher score.
### DER Worked Example

**Reference**  
- 0–4 s → Speaker A  
- 4–7 s → Speaker B  
- 7–10 s → Silence  

**TOTAL = 7 s**
---



**Hypothesis**
- 0–5 s → Speaker A  
- 5–6 s → Silence  
- 6–10 s → Speaker B  


| Span (s) | Ref | Hyp | Counts |
|---------|-----|-----|--------|
| 0–4 | A | A | Correct (4 s) |
| 4–5 | B | A | CONF = 1 s |
| 5–6 | B | Silence | MISS = 1 s |
| 6–7 | B | B | Correct (1 s) |
| 7–10 | Silence | B | FA = 3 s |


****

Totals:
---
- FA = 3  
- MISS = 1  
- CONF = 1  
- TOTAL = 7  
- TOTAL = 7  

$$
\text{DER} = \frac{3 + 1 + 1}{7} = \frac{5}{7} \approx 0.714 \; (71.4\%)
$$

### Important clarifications (read this!)
- **Speaker IDs do NOT have to match the ground-truth IDs.**  
  Your `speaker_id` values are treated as **labels** (strings). The evaluator automatically finds the **best mapping** between your predicted speakers and the reference speakers to minimize errors.  
  **Example:** if the ground truth uses `"1"` and `"2"`, your system can output `"11"` and `"A"` — that’s fine.

- **Consistency matters.**  
  Even though IDs don’t need to match, you should keep the **same label for the same person across time**. 


### Inference Time Score

Inference efficiency is evaluated using **Real-Time Factor (RTF)**. Faster models receive significantly higher scores.

**RTF Calculation**:
- RTF is computed for every test case per participant
- Final RTF = average RTF across all test cases

**Scoring**:
- Participants are ranked by average RTF (lower = better)
- Percentile ranks map directly to scores:
  - 1st percentile → Score 100
  - 2nd percentile → Score 99
  - Higher percentiles (slower) → proportionally lower scores

This percentile-based scoring rewards meaningful improvements in inference speed.