Submissions are evaluated on the following criteria:

|**Criteria (points)**  | **Description** |
| --- | --- |
| **Notebook quality**
(35 points) |  The notebook is clearly written and detailed with:
- Training data
- Hyperparameters
- Prompt
- Training strategy and techniques.
A Gemma2 starter notebook for math reasoning can be found [here](https://github.com/google/tunix/blob/main/examples/grpo_demo.ipynb) ([Kaggle-hosted version](https://www.kaggle.com/code/windmaple/grpo-demo-gemma2-2b)).
Given limited compute on Kaggle TPUs (9 hours per session, 20 hours per week):
- Max output token <1K is fine
- English-only. Multilinguality is not the focus of this hackathon.
- Tool use is not necessary.
- No multimodality
|
| **Model quality for a single Kaggle session**
(45 points) | - The fine tuned model is the direct output from the notebook above and runs on a single Kaggle TPU session (9 hours).
- The generated model checkpoint files must be loadable and runnable via the Gemma2 or Gemma3 modeling code in Tunix on Kaggle.
- Model output should follow this format:
`
model_thinking_trace
`
`
model_answer
`
Evaluation will cover both the reasoning trace and the final answer, and be done via a combination of LLM-as-a-judge and human judgment. Eval user queries are held out during the hackathon and may come from a range of verifiable and non-verifiable domains, including but not limited to:
- Creative writing
- Creative ideation
- Summarization
- Math
- Coding
- Basic science
- Other 
|
| **Video quality**
(20 points) | The video is under 3 minutes, is informative, and of instructional value for developers who want to learn about training reasoning models using Tunix.
It accurately reflects the author’s work in the hackathon and is of high production quality. 
|
| **Model quality across multiple Kaggle sessions**
(optional, 15 points) | This is optional but participants who want to push the envelope can:
- Finetune the model across multiple Kaggle TPU sessions by saving and loading the intermediate checkpoints.
- Use private data.
Participants must explicitly provide a Kaggle model name/ID at the end of the notebook as the submission for this item. If none is provided, 0 point is given for this item.
The model files must be loadable and runnable via the Gemma2 or Gemma3 modeling code in Tunix on Kaggle, and should not be safetensors files.