# **📏 Evaluation**

This competition uses a Custom Exact-Match Metric.

Your submission will be scored based on the Accuracy of your JSON output. To get a point for a row, your JSON must match the ground truth perfectly on:

✅ Total Price (Exact match required)

✅ Drink Name & Size

✅ List of Modifiers (Order does not matter, but content does)

Note: Hallucinated keys or invalid JSON formats will result in a score of 0 for that row.

# **🚀 How to Participate**

Read the Menu: Check menu.md to understand the pricing logic.

Start a Notebook: Load the data and a base model (like Llama-3 or Gemma).

Prompt Engineering: Write a System Prompt that teaches the model how to think step-by-step.

Submit: Generate your submission.csv and climb the leaderboard!

Can you beat models like GPT-5 using a smaller, more efficient model? Good luck!

# **🚫 A Note on Manual Labeling & Data Leakage**

Manual Labeling: Since the test set is 3,500 rows, it is theoretically possible for a human to manually label the data. **Please don't.** This is a benchmark for Artificial Intelligence. If you manually label the data, you have proved nothing other than that you have free time.

Data Contamination: Do not include the test set questions in your training data (fine-tuning). This is considered data leakage.

Verification: Top solutions may be asked to share their inference code to verify the results were generated by a model.