Welcome to the Unlearning Challenge 2025, hosted by the Image Analysis and Biometrics Lab (IAB) at IIT Jodhpur! This competition challenges you to develop techniques for effective machine unlearning, ensuring models can forget specific concepts while maintaining overall performance. Unlearning is critical for privacy protection, bias mitigation, and ethical AI deployment—key areas of focus for the IAB Lab, known for its work in fair and trusted computer vision.

**Goal**: Your task is to generate images that demonstrate successful unlearning of 20 specified concepts (e.g., Apple Fruit, Dancing). Package your images into a submission.zip file with 20 concept folders, each containing 50 images. Using a provided Google Colab notebook, you’ll evaluate your images with a fixed Llava model, which will generate a submission.csv file containing the model’s responses (“yes” or “no”) to simple, indirect, and adversarial prompts. The goal is to achieve a high ERB (Evaluation of Retention and Behavior) score by ensuring the Llava model does not recognize the unlearned concepts in your images (i.e., model_response = “no” for all prompts). Submit your submission.csv to Kaggle for evaluation, where Public (~30%) and Private (~70%) scores will determine your ranking.

**Task and Data**

The Genμ Challenge addresses concept unlearning in text-to-image generative models. Starting from the public checkpoint Stable Diffusion v1.4 , participants must make the model “forget” specific visual concepts while preserving its overall abilities.

**Pipeline overview**

*Inputs*: Stable Diffusion v1.4 weights + a list of target concepts to remove.
*Output*: A revised model that cannot generate the target concepts but still performs well on all others.

*Evaluation axes*

Adjacent-Concept Preservation :Erasing “golden retriever” must not degrade generation of other dog breeds or related subjects.
Robustness: The forgotten concepts must remain inaccessible even under engineered or adversarial prompts.
Dataset: A CSV with prompts and the concepts to be unlearned is provided. It forms the basis for every leaderboard metric and automatic evaluation.

Participants are required to demonstrate their results using any one of the following three distinct unlearning strategies:

*Single Concept Unlearning*: Participants will submit unlearned models, each engineered to forget a single, specific concept.
*Multi-Concept Unlearning*: Participants will submit unlearned models where multiple target concepts are erased concurrently.
*Continual Unlearning*: Participants will submit unlearned models that have progressively forgotten target concepts based on a sequence of requests.

**Evaluation Methodology**

- **Goal**: Remove the *target concepts* while preserving the model’s general and related knowledge, with minimal weight change.

- **Primary Metric — Erasing-Retention-Robustness (ERR Score)**  
  **Per-concept averages**  
  - Average *forgetting accuracy*: how well the model suppresses each target concept  
    $$ \overline{A_{fgt}} = 1/(N_{img}N_{c_{tar}}) \sum_{i=1}^{N_{img}} \sum_{c=1}^{N_{c_{tar}}} A_{fgt}^{(i,c)} $$

  - Average *retention accuracy*: how well unrelated knowledge is preserved  
    $$ \overline{A_{ret}} = 1/(N_{img}N_{c_{ret}}) \sum_{i=1}^{N_{img}} \sum_{c=1}^{N_{c_{ret}}} A_{ret}^{(i,c)} $$

  - Average *adjacent-concept accuracy*: how well concepts correlated with the targets stay intact  
    $$ \overline{A_{adj}} = 1/(N_{img}N_{c_{adj}}) \sum_{i=1}^{N_{img}} \sum_{c=1}^{N_{c_{adj}}} A_{adj}^{(i,c)} $$

  - Average *engineered-prompt robustness*: resistance to prompts intentionally designed to resurface the targets  
    $$ \overline{A_{ind}} = 1/(N_{img}N_{c_{tar}}) \sum_{i=1}^{N_{img}} \sum_{c=1}^{N_{c_{tar}}} A_{ind}^{(i,c)} $$

  - Average *adversarial robustness*: resistance to adversarial prompts that attempt to reveal the targets  
    $$ \overline{A_{adv}} = 1/(N_{img}N_{c_{tar}}) \sum_{i=1}^{N_{img}} \sum_{c=1}^{N_{c_{tar}}} A_{adv}^{(i,c)} $$

  **Final ERR Score (harmonic mean)**  
    $$ \text{ERR} = \operatorname{HM}\left( \overline{A_{fgt}}, \overline{A_{ret}}, \overline{A_{adj}}, \overline{A_{ind}}, \overline{A_{adv}} \right). $$

  **Interpretation**  
  Higher A_fgt means better forgetting, higher A_ret and A_adj mean better retention, and higher A_ind and A_adv mean stronger robustness.

- **Tie-breaker — Weight-Change Ratio**  
  The weight-change ratio is calculated as:  
  $$ \frac{1}{N_c} \sum_{i=1}^{N_c} \lVert \theta_{\text{orig}}^{i} - \theta_{\text{un}}^{i} \rVert \;\Big/\; \text{Total parameters}, $$  
  where \( \theta_{\text{orig}} \) represents the original model weights and \( \theta_{\text{un}} \) represents the un-learnt model weights.

- **Notation**  
  - N_img  — images per concept  
  - N_c_tar — target concepts  
  -  N_c_ret — retention-test concepts  
  -  N_c_adj — adjacent/correlated concepts

**Baselines**: Scores of existing un-learning methods will be reported for comparison.

**Submission** : To generate submission.csv , follow the instructions in the [Colab Notebook](https://colab.research.google.com/drive/1_f6AIjZ5IYeJuVSl27upzSePpTd6-y3U?usp=sharing)





**Form for model submission**:  https://forms.gle/5hexNRc4VpkD2g3B7 ( till 6 Aug 2025)