### 📦 Challenge 02 — Deliverables

**From Messy Reality to Reproducible Trust**

In this stage, clarity and reproducibility matter more than complexity.
You’ll produce a complete, auditable record showing **how messy data became trustworthy data** — from raw audit logs to the final validation report.

---

#### 🧩 Required Deliverables

📄 **Problem Statement & Notebook Stub**
A concise description of the business context and data-quality goals.
Include a Jupyter notebook skeleton with standard sections: *Setup → Audit → Repair → Validate → Summary.*

📚 **Data Quality Report**
A Markdown or HTML summary (`reports/data_quality.md`) listing:

* detected issues (missing, duplicates, drift, outliers)
* applied fixes and rationale
* diagnostics or before/after comparisons

🧮 **Audit & Repair Notebook**
A reproducible Jupyter notebook (`notebooks/02_data_quality.ipynb`) that:

* profiles missingness and drift (e.g., `missingno`, `pandas_profiling`)
* applies and documents imputation techniques
* runs validation queries and visual checks

🧱 **Validation Scripts**
Standalone Python utilities in `src/` to test dataset integrity:
`evaluation.py` for referential-integrity checks, outlier detection, and simple schema validation.

📊 **Baseline Re-Test Notebook**
Retrain or rerun your baseline model from Challenge 01 on the cleaned dataset.
Compare metrics (MAE, MAPE, RMSE) and note any improvement or variance.

🗂️ **File Structure Checklist**

| Folder       | Files / Outputs                              |
| :----------- | :------------------------------------------- |
| `docs/`      | architecture.md, change_log.md               |
| `notebooks/` | 02_data_quality.ipynb, 02_model_retest.ipynb |
| `reports/`   | data_quality.md, executive_summary.pdf       |
| `src/`       | evaluation.py, helper_utils.py               |
| `slides/`    | presentation_deck.pptx                       |

---

#### ✅ Acceptance Criteria

* All introduced issues are **detected and either fixed or explicitly justified.**
* Imputation steps are **documented with rationale and diagnostics.**
* The cleaned dataset **re-trains the baseline without suspicious metric jumps.**
* All notebooks & scripts **run end-to-end without manual intervention.**

---

#### 💡 Outcome

A **validated, auditable, and reproducible dataset** that transforms “structured but messy” records into a reliable analytical asset — the trusted foundation for Challenge 03 (Data → Signal).