#**☕ BaristaBench: The Agentic State-Tracking Challenge**

"I'll have a Venti Latte, actually make that Decaf, with Oat Milk... oh, and no foam."

We all know LLMs can write poetry and pass the Bar Exam. But can they work the morning rush at a coffee shop without losing money?

BaristaBench is a new community benchmark designed to test Agentic State Tracking, Arithmetic, and Constraint Satisfaction. While the scenario is fun, the problem is technical: can a model maintain a mental "state" of an order, modify it based on messy natural language updates, and output a strictly formatted JSON result with 100% arithmetic precision?

## **🎯 The Goal**
Your task is to build an AI Agent (or prompt-engineer an LLM) that acts as a Point-of-Sale (POS) system.

**🏆 The "Efficiency" Challenge**
While we welcome submissions from massive models (GPT-4, Claude 3.5, Llama-3-70B), the real prestige in this benchmark is doing more with less.

We encourage participants to tag their submissions/notebooks after the competition ends in the Discussion/Code board with:

**🛡️ < 10B Params**: Best score using a model runnable on consumer hardware (e.g., Llama-3-8B, Qwen-7B, Mistral).

**⚡ Open Weight**: Best score using fully open-source models.

**🦖 Godzilla Mode**: Best score regardless of compute/cost.

You will be given a customer's spoken order. These orders are not clean—customers might change their minds, ask for modifiers, cancel items, or speak in a non-linear way.