{ "fetched_at": "2026-10-25T04:54:49Z", "source_url": "https://www.kaggle.com/competitions/challenge-01-spaghetti-schema-2026/data", "pages": [ { "name": "data-description", "content": "## 🧩 Dataset Description\n\nThis dataset provides raw operational data from a fictional grocery retailer, designed for the **Challenge 01 – Spaghetti → Schema** competition.\nYour task is to normalize these “spaghetti-style” flat files into a clean relational schema suitable for analysis, dashboards, and machine learning workflows.\n\n### 📁 Files Included\n\n| File | Description |\n| --------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |\n| **Products_2024.csv** | Product master data including item IDs, names, categories, and supplier information. |\n| **Promotions_2024.csv** | Marketing and discount campaign data including promotion IDs, start/end dates, and participating products. |\n| **Inventory_Snapshots_2024.csv** | Daily or weekly inventory levels by product and store. |\n| **Sales_2024_Q1.csv** – **Sales_2024_Q4.csv** | Transactional sales data by quarter for 2024. Each record includes product, store, date, units sold, and revenue. |\n| **README.txt** | Documentation describing column formats, data relationships, and intended normalization guidance. |\n\n### 🧱 Objective\n\nParticipants must:\n\n1. Analyze the flat files for redundancy and inconsistency.\n2. Design a **normalized schema** (e.g., in SQL or ERD form).\n3. Provide transformation logic or scripts that produce structured tables (e.g., dimension and fact tables).\n\nThis prepares the data for downstream challenges focused on **data integrity, forecasting, and strategy modeling**.\n\n### 🧾 Data Notes\n\n* Data is **synthetic and educational**; it mirrors real-world retail structures but contains no sensitive or private information.\n* Field names and patterns are consistent across quarters for joinability.\n* Time is represented in local store timestamps but can be aggregated at daily or weekly levels.\n\n### ⚖️ License\n\n**CC BY-NC-SA 4.0** — You may share and adapt the dataset for non-commercial educational or research purposes with attribution.\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "rules", "content": "🧩 This is the starting point of the Grocery Retail – Driven by Data series, focusing on turning Excel “spaghetti” into structured, normalized data. It lays the foundation for the data integrity, modeling, and strategy challenges that follow.\n\n🧭 **Rules – Challenge 01 Spaghetti → Schema**\n\nThis is an educational and prototype competition created to explore the process of building real-world data challenges on Kaggle. \nIt is not an official or prize-bearing competition. Participation is open to anyone for learning and collaboration.\n\nThis is the first challenge in the Grocery Retail – Driven by Data series. Like the others, it is educational and open for experimentation. No prizes or formal rankings apply.\n\n1️⃣ **Conduct**\n\nAll participants must follow the [Kaggle Code of Conduct](https://www.kaggle.com/community-guidelines) and [Terms of Service](https://www.kaggle.com/terms). \nBe respectful, cite external sources appropriately, and share constructive feedback with others.\n\n2️⃣ **Data Usage**\n\nThe dataset provided is for educational and demonstration purposes only. \nYou may download, transform, and share derivative work for personal or academic learning, but no commercial use is implied or permitted.\n\n3️⃣ **Collaboration**\n\nTeams and discussion are encouraged. \nPublic sharing of general approaches, code snippets, or schema, transformation, and data-structuring ideas is welcome. \nDo not share direct solutions or private datasets that would invalidate learning goals.\n\n4️⃣ **Submissions**\n\nSince this challenge focuses on schema design and documentation rather than leaderboard accuracy: \nSubmissions may include `.zip` folders, `.ipynb` notebooks, or `.sql` files. \nEvaluation will be manual and qualitative.\n\n5️⃣ **Timeline**\n\nThis challenge is currently in prototype mode and will remain open while materials and data are being refined. \nNo prizes, deadlines, or rankings apply at this stage. \nFuture versions may evolve into formal scored competitions as data complexity increases.\n\n6️⃣ **Acknowledgment**\n\nBy joining this project, you acknowledge that participation is voluntary and intended for educational experimentation and public learning.\n\n---\n\n*This challenge sets the groundwork for Challenge 02 – Errors → Integrity, which focuses on detecting and repairing data issues for trust and reproducibility.*\n\n---\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "abstract", "content": "## 🧩 Spaghetti → Schema\n\n**Challenge 01** is the entry point to the *Grocery – Driven by Data* series and addresses the most fundamental problem in applied data science: before we can model, forecast, or automate, we must be able to *read the system we are working in*.\n\n**Dr. V's Slide Decks :** [The Architecture of Validity](https://starshiptutor.substack.com/p/the-architecture-of-validity), [Challenge 01](https://starshiptutor.substack.com/p/challenge-01-spaghetti-schema-294), [Challenge 02](https://starshiptutor.substack.com/p/challenge-02-errors-integrity), [Challenge 03](https://starshiptutor.substack.com/p/challenge-03-data-signal)
\n**Meet the Avatars :** [Dr. Elena Vasquez](https://starshiptutor.substack.com/p/introducing-dr-elena-vasquez), [Evan Morales](https://starshiptutor.substack.com/p/introducing-evan-morales), [Maya Chen](https://starshiptutor.substack.com/p/introducing-maya-chen), [Daniel Reeve](https://starshiptutor.substack.com/p/introducing-daniel-reeve)
\n**Kaggle Pages :** Challenge 01, [Challenge 02](https://www.kaggle.com/competitions/challenge-02-errors-integrity-2026), [Challenge 03](https://www.kaggle.com/competitions/challenge-03-data-signal-2026)\n\n##📐 Designing Clarity from Spreadsheets\n\n**Dr. Elena Vasquez**, is a fictional Associate Professor of Information Science, who will introduce you to the challenges, while the other avatars will share your struggles in using and understanding them. \n\nChallenge 01 is designed to introduce students and practitioners to data as an operational artifact—shaped by human processes, informal workarounds, and organizational drift. The goal is not to “clean” data in isolation, but to construct **valid structure** inside a system that is already in motion.\n\nParticipants work with a fictional grocery chain, *North West Organics (NWO)*, presented explicitly as a **constraint generator**, not a toy dataset. The raw inputs consist of loosely coordinated spreadsheets with duplicated headers, inconsistent identifiers, drifting categories, and unreliable temporal fields. Rather than fixing symptoms piecemeal, participants must diagnose the system, reverse-engineer its conceptual model, and design a normalized relational schema that makes the data legible.\n\nThe core outcome of Challenge 01 is **legibility, not accuracy**. Through disciplined schema design, normalization, and reproducible ingestion pipelines, participants create a foundation that can be trusted, audited, and extended. This challenge establishes the architectural backbone for every subsequent stage in the series—data quality repair, forecasting, and strategy modeling. Structure comes first; trust follows. Without it, every downstream insight is fragile by design.\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "foundational-rules", "content": "The following Kaggle Competition Foundational Rules (“ Foundational Rules ”) apply to every competition regardless of whether the Sponsor creates competition-specific rules. Any competition-specific rules provided by the Sponsor are in addition to these rules, and in the case of any conflict or inconsistency, these Foundational Rules control and nullify contrary competition-specific rules.\n###GENERAL COMPETITION RULES - BINDING AGREEMENT\n####1. ELIGIBILITY\na. To be eligible to enter the Competition, you must be:\n1. a registered account holder at Kaggle.com; \n2. the older of 18 years old or the age of majority in your jurisdiction of residence (unless otherwise agreed to by Competition Sponsor and appropriate parental/guardian consents have been obtained by Competition Sponsor); \n3. not a resident of Crimea, so-called Donetsk People's Republic (DNR) or Luhansk People's Republic (LNR), Cuba, Iran, Syria, or North Korea; and\n4. not a person or representative of an entity under U.S. export controls or sanctions (see: [https://www.treasury.gov/resourcecenter/sanctions/Programs/Pages/Programs.aspx][1]).\n\nb. Competitions are open to residents of the United States and worldwide, except that if you are a resident of Crimea, so-called Donetsk People's Republic (DNR) or Luhansk People's Republic (LNR), Cuba, Iran, Syria, North Korea, or are subject to U.S. export controls or sanctions, you may not enter the Competition. Other local rules and regulations may apply to you, so please check your local laws to ensure that you are eligible to participate in skills-based competitions. The Competition Host reserves the right to forego or award alternative Prizes where needed to comply with local laws. If a winner is located in a country where prizes cannot be awarded, then they are not eligible to receive a prize.\n\nc. If you are entering as a representative of a company, educational institution or other legal entity, or on behalf of your employer, these rules are binding on you, individually, and the entity you represent or where you are an employee. If you are acting within the scope of your employment, or as an agent of another party, you warrant that such party or your employer has full knowledge of your actions and has consented thereto, including your potential receipt of a Prize. You further warrant that your actions do not violate your employer's or entity's policies and procedures. \n\nd. The Competition Sponsor reserves the right to verify eligibility and to adjudicate on any dispute at any time. If you provide any false information relating to the Competition concerning your identity, residency, mailing address, telephone number, email address, ownership of right, or information required for entering the Competition, you may be immediately disqualified from the Competition.\n\n####2. SPONSOR AND HOSTING PLATFORM\n\na. The Competition is sponsored by Competition Sponsor named above. The Competition is hosted on behalf of Competition Sponsor by Kaggle Inc. (\"Kaggle\"). Kaggle is an independent contractor of Competition Sponsor, and is not a party to this or any agreement between you and Competition Sponsor. You understand that Kaggle has no responsibility with respect to selecting the potential Competition winner(s) or awarding any Prizes. Kaggle will perform certain administrative functions relating to hosting the Competition, and you agree to abide by the provisions relating to Kaggle under these Rules. As a Kaggle.com account holder and user of the Kaggle competition platform, remember you have accepted and are subject to the Kaggle Terms of Service at [www.kaggle.com/terms][2] in addition to these Rules.\n\n####3. COMPETITION PERIOD\na. For the purposes of Prizes, the Competition will run from the Start Date and time to the Final Submission Deadline (such duration the “Competition Period”). The Competition Timeline is subject to change, and Competition Sponsor may introduce additional hurdle deadlines during the Competition Period. Any updated or additional deadlines will be publicized on the Competition Website. It is your responsibility to check the Competition Website regularly to stay informed of any deadline changes. YOU ARE RESPONSIBLE FOR DETERMINING THE CORRESPONDING TIME ZONE IN YOUR LOCATION.\n\n####4. COMPETITION ENTRY\na. NO PURCHASE NECESSARY TO ENTER OR WIN. To enter the Competition, you must register on the Competition Website prior to the Entry Deadline, and follow the instructions for developing and entering your Submission through the Competition Website. Your Submissions must be made in the manner and format, and in compliance with all other requirements, stated on the Competition Website (the \"Requirements\"). Submissions must be received before any Submission deadlines stated on the Competition Website. Submissions not received by the stated deadlines will not be eligible to receive a Prize.\nb. Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.\nc. If the Competition is a multi-stage competition with temporally separate training and/or test data, one or more valid Submissions may be required during each Competition stage in the manner described on the Competition Website in order for the Submissions to be Prize eligible.\nd. Submissions are void if they are in whole or part illegible, incomplete, damaged, altered, counterfeit, obtained through fraud, or late. Competition Sponsor reserves the right to disqualify any entrant who does not follow these Rules, including making a Submission that does not meet the Requirements. \n\n####5. INDIVIDUALS AND TEAMS\na. Individual Account. You may make Submissions only under one, unique Kaggle.com account. You will be disqualified if you make Submissions through more than one Kaggle account, or attempt to falsify an account to act as your proxy. You may submit up to the maximum number of Submissions per day as specified on the Competition Website. \nb. Teams. If permitted under the Competition Website guidelines, multiple individuals may collaborate as a Team; however, you may join or form only one Team. Each Team member must be a single individual with a separate Kaggle account. You must register individually for the Competition before joining a Team. You must confirm your Team membership to make it official by responding to the Team notification message sent to your Kaggle account. Team membership may not exceed the Maximum Team Size stated on the Competition Website.\nc. Team Merger. Teams may request to merge via the Competition Website. Team mergers may be allowed provided that: (i) the combined Team does not exceed the Maximum Team Size; (ii) the number of Submissions made by the merging Teams does not exceed the number of Submissions permissible for one Team at the date of the merger request; (iii) the merger is completed before the earlier of: any merger deadline or the Competition deadline; and (iv) the proposed combined Team otherwise meets all the requirements of these Rules. \nd. Private Sharing. No private sharing outside of Teams. Privately sharing code or data outside of Teams is not permitted. It's okay to share code if made available to all Participants on the forums.\n\n####6. SUBMISSION CODE REQUIREMENTS\na. Private Code Sharing. Unless otherwise specifically permitted under the Competition Website or Competition Specific Rules above, during the Competition Period, you are not allowed to privately share source or executable code developed in connection with or based upon the Competition Data or other source or executable code relevant to the Competition (“Competition Code”). This prohibition includes sharing Competition Code between separate Teams, unless a Team merger occurs. Any such sharing of Competition Code is a breach of these Competition Rules and may result in disqualification.\nb. Public Code Sharing. You are permitted to publicly share Competition Code, provided that such public sharing does not violate the intellectual property rights of any third party. If you do choose to share Competition Code or other such code, you are required to share it on Kaggle.com on the discussion forum or notebooks associated specifically with the Competition for the benefit of all competitors. By so sharing, you are deemed to have licensed the shared code under an Open Source Initiative-approved license (see [www.opensource.org][3]) that in no event limits commercial use of such Competition Code or model containing or depending on such Competition Code.\nc. Use of Open Source. Unless otherwise stated in the Specific Competition Rules above, if open source code is used in the model to generate the Submission, then you must only use open source code licensed under an Open Source Initiative-approved license (see [www.opensource.org][4]) that in no event limits commercial use of such code or model containing or depending on such code.\n\n####7. DETERMINING WINNERS\na. Each Submission will be scored and ranked by the evaluation metric stated on the Competition Website. During the Competition Period, the current ranking will be visible on the Competition Website's Public Leaderboard. The potential winner(s) are determined solely by the leaderboard ranking on the Private Leaderboard, subject to compliance with these Rules. The Public Leaderboard will be based on the public test set and the Private Leaderboard will be based on the private test set.\nb. In the event of a tie, the Submission that was entered first to the Competition will be the winner. In the event a potential winner is disqualified for any reason, the Submission that received the next highest score rank will be chosen as the potential winner.\n\n####8. NOTIFICATION OF WINNERS & DISQUALIFICATION\na. The potential winner(s) will be notified by email. \nb. If a potential winner (i) does not respond to the notification attempt within one (1) week from the first notification attempt or (ii) notifies Kaggle within one week after the Final Submission Deadline that the potential winner does not want to be nominated as a winner or does not want to receive a Prize, then, in each case (i) and (ii) such potential winner will not receive any Prize, and an alternate potential winner will be selected from among all eligible entries received based on the Competition’s judging criteria.\nc. In case (i) and (ii) above Kaggle may disqualify the Participant. However, in case (ii) above, if requested by Kaggle, such potential winner may provide code and documentation to verify the Participant’s compliance with these Rules. If the potential winner provides code and documentation to the satisfaction of Kaggle, the Participant will not be disqualified pursuant to this paragraph.\nd. Competition Sponsor reserves the right to disqualify any Participant from the Competition if the Competition Sponsor reasonably believes that the Participant has attempted to undermine the legitimate operation of the Competition by cheating, deception, or other unfair playing practices or abuses, threatens or harasses any other Participants, Competition Sponsor or Kaggle.\ne. A disqualified Participant may be removed from the Competition leaderboard, at Kaggle's sole discretion. If a Participant is removed from the Competition Leaderboard, additional winning features associated with the Kaggle competition platform, for example Kaggle points or medals, may also not be awarded.\nf. The final leaderboard list will be publicly displayed at Kaggle.com. Determinations of Competition Sponsor are final and binding.\n\n####9. PRIZES\na. Prize(s) are as described on the Competition Website and are only available for winning during the time period described on the Competition Website. The odds of winning any Prize depends on the number of eligible Submissions received during the Competition Period and the skill of the Participants. \nb. All Prizes are subject to Competition Sponsor's review and verification of the Participant’s eligibility and compliance with these Rules, and the compliance of the winning Submissions with the Submissions Requirements. In the event that the Submission demonstrates non-compliance with these Competition Rules, Competition Sponsor may at its discretion take either of the following actions: (i) disqualify the Submission(s); or (ii) require the potential winner to remediate within one week after notice all issues identified in the Submission(s) (including, without limitation, the resolution of license conflicts, the fulfillment of all obligations required by software licenses, and the removal of any software that violates the software restrictions).\nc. A potential winner may decline to be nominated as a Competition winner in accordance with Section 3.8.\nd. Potential winners must return all required Prize acceptance documents within two (2) weeks following notification of such required documents, or such potential winner will be deemed to have forfeited the prize and another potential winner will be selected. Prize(s) will be awarded within approximately thirty (30) days after receipt by Competition Sponsor or Kaggle of the required Prize acceptance documents. Transfer or assignment of a Prize is not allowed. \ne. You are not eligible to receive any Prize if you do not meet the Eligibility requirements in Section 2.7 and Section 3.1 above.\nf. If a Team wins a monetary Prize, the Prize money will be allocated in even shares between the eligible Team members, unless the Team unanimously opts for a different Prize split and notifies Kaggle before Prizes are issued.\n\n####10. TAXES\na. ALL TAXES IMPOSED ON PRIZES ARE THE SOLE RESPONSIBILITY OF THE WINNERS. Payments to potential winners are subject to the express requirement that they submit all documentation requested by Competition Sponsor or Kaggle for compliance with applicable state, federal, local and foreign (including provincial) tax reporting and withholding requirements. Prizes will be net of any taxes that Competition Sponsor is required by law to withhold. If a potential winner fails to provide any required documentation or comply with applicable laws, the Prize may be forfeited and Competition Sponsor may select an alternative potential winner. Any winners who are U.S. residents will receive an IRS Form-1099 in the amount of their Prize.\n\n####11. GENERAL CONDITIONS\na. All federal, state, provincial and local laws and regulations apply.\n\n####12. PUBLICITY\na. You agree that Competition Sponsor, Kaggle and its affiliates may use your name and likeness for advertising and promotional purposes without additional compensation, unless prohibited by law.\n\n####13. PRIVACY\na. You acknowledge and agree that Competition Sponsor and Kaggle may collect, store, share and otherwise use personally identifiable information provided by you during the Kaggle account registration process and the Competition, including but not limited to, name, mailing address, phone number, and email address (“Personal Information”). Kaggle acts as an independent controller with regard to its collection, storage, sharing, and other use of this Personal Information, and will use this Personal Information in accordance with its Privacy Policy <[www.kaggle.com/privacy][6]>, including for administering the Competition. As a Kaggle.com account holder, you have the right to request access to, review, rectification, portability or deletion of any personal data held by Kaggle about you by logging into your account and/or contacting Kaggle Support at <[www.kaggle.com/contact][7]>.\nb. As part of Competition Sponsor performing this contract between you and the Competition Sponsor, Kaggle will transfer your Personal Information to Competition Sponsor, which acts as an independent controller with regard to this Personal Information. As a controller of such Personal Information, Competition Sponsor agrees to comply with all U.S. and foreign data protection obligations with regard to your Personal Information. Kaggle will transfer your Personal Information to Competition Sponsor in the country specified in the Competition Sponsor Address listed above, which may be a country outside the country of your residence. Such country may not have privacy laws and regulations similar to those of the country of your residence.\n\n####14. WARRANTY, INDEMNITY AND RELEASE \na. You warrant that your Submission is your own original work and, as such, you are the sole and exclusive owner and rights holder of the Submission, and you have the right to make the Submission and grant all required licenses. You agree not to make any Submission that: (i) infringes any third party proprietary rights, intellectual property rights, industrial property rights, personal or moral rights or any other rights, including without limitation, copyright, trademark, patent, trade secret, privacy, publicity or confidentiality obligations, or defames any person; or (ii) otherwise violates any applicable U.S. or foreign state or federal law.\nb. To the maximum extent permitted by law, you indemnify and agree to keep indemnified Competition Entities at all times from and against any liability, claims, demands, losses, damages, costs and expenses resulting from any of your acts, defaults or omissions and/or a breach of any warranty set forth herein. To the maximum extent permitted by law, you agree to defend, indemnify and hold harmless the Competition Entities from and against any and all claims, actions, suits or proceedings, as well as any and all losses, liabilities, damages, costs and expenses (including reasonable attorneys fees) arising out of or accruing from: (a) your Submission or other material uploaded or otherwise provided by you that infringes any third party proprietary rights, intellectual property rights, industrial property rights, personal or moral rights or any other rights, including without limitation, copyright, trademark, patent, trade secret, privacy, publicity or confidentiality obligations, or defames any person; (b) any misrepresentation made by you in connection with the Competition; (c) any non-compliance by you with these Rules or any applicable U.S. or foreign state or federal law; (d) claims brought by persons or entities other than the parties to these Rules arising from or related to your involvement with the Competition; and (e) your acceptance, possession, misuse or use of any Prize, or your participation in the Competition and any Competition-related activity.\nc. You hereby release Competition Entities from any liability associated with: (a) any malfunction or other problem with the Competition Website; (b) any error in the collection, processing, or retention of any Submission; or (c) any typographical or other error in the printing, offering or announcement of any Prize or winners.\n\n####15. INTERNET\na. Competition Entities are not responsible for any malfunction of the Competition Website or any late, lost, damaged, misdirected, incomplete, illegible, undeliverable, or destroyed Submissions or entry materials due to system errors, failed, incomplete or garbled computer or other telecommunication transmission malfunctions, hardware or software failures of any kind, lost or unavailable network connections, typographical or system/human errors and failures, technical malfunction(s) of any telephone network or lines, cable connections, satellite transmissions, servers or providers, or computer equipment, traffic congestion on the Internet or at the Competition Website, or any combination thereof, which may limit a Participant’s ability to participate.\n\n####16. RIGHT TO CANCEL, MODIFY OR DISQUALIFY\na. If for any reason the Competition is not capable of running as planned, including infection by computer virus, bugs, tampering, unauthorized intervention, fraud, technical failures, or any other causes which corrupt or affect the administration, security, fairness, integrity, or proper conduct of the Competition, Competition Sponsor reserves the right to cancel, terminate, modify or suspend the Competition. Competition Sponsor further reserves the right to disqualify any Participant who tampers with the submission process or any other part of the Competition or Competition Website. Any attempt by a Participant to deliberately damage any website, including the Competition Website, or undermine the legitimate operation of the Competition is a violation of criminal and civil laws. Should such an attempt be made, Competition Sponsor and Kaggle each reserves the right to seek damages from any such Participant to the fullest extent of the applicable law.\n\n####17. NOT AN OFFER OR CONTRACT OF EMPLOYMENT\na. Under no circumstances will the entry of a Submission, the awarding of a Prize, or anything in these Rules be construed as an offer or contract of employment with Competition Sponsor or any of the Competition Entities. You acknowledge that you have submitted your Submission voluntarily and not in confidence or in trust. You acknowledge that no confidential, fiduciary, agency, employment or other similar relationship is created between you and Competition Sponsor or any of the Competition Entities by your acceptance of these Rules or your entry of your Submission.\n\n####18. DEFINITIONS\na. \"Competition Data\" are the data or datasets available from the Competition Website for the purpose of use in the Competition, including any prototype or executable code provided on the Competition Website. The Competition Data will contain private and public test sets. Which data belongs to which set will not be made available to Participants. \nb. An “Entry” is when a Participant has joined, signed up, or accepted the rules of a competition. Entry is required to make a Submission to a competition.\nc. A “Final Submission” is the Submission selected by the user, or automatically selected by Kaggle in the event not selected by the user, that is/are used for final placement on the competition leaderboard.\nd. A “Participant” or “Participant User” is an individual who participates in a competition by entering the competition and making a Submission.\ne. The “Private Leaderboard” is a ranked display of Participants’ Submission scores against the private test set. The Private Leaderboard determines the final standing in the competition.\nf. The “Public Leaderboard” is a ranked display of Participants’ Submission scores against a representative sample of the test data. This leaderboard is visible throughout the competition.\ng. A “Sponsor” is responsible for hosting the competition, which includes but is not limited to providing the data for the competition, determining winners, and enforcing competition rules.\nh. A “Submission” is anything provided by the Participant to the Sponsor to be evaluated for competition purposes and determine leaderboard position. A Submission may be made as a model, notebook, prediction file, or other format as determined by the Sponsor.\ni. A “Team” is one or more Participants participating together in a Kaggle competition, by officially merging together as a Team within the competition platform.\n\n [1]: https://www.treasury.gov/resource-center/sanctions/Programs/Pages/Programs.aspx\n [2]: http://www.kaggle.com/terms\n [3]: http://www.opensource.org\n [4]: http://www.opensource.org\n [5]: https://www.kaggle.com/WinningModelDocumentationGuidelines\n [6]: http://www.kaggle.com/privacy\n [7]: http://www.kaggle.com/contact\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Series Overview", "content": "🛒 **Series Overview**\n\n**Grocery Retail – Driven by Data** is a four-part open learning and research series that follows a grocery chain’s evolution from spreadsheet chaos to data-driven strategy. \nEach challenge builds on the last—turning messy operations into trusted analytics and predictive insight.\n\nYou’ll move step by step through the modern data journey — from **structure → trust → signal → strategy**:\n\n**1️⃣ Spaghetti → Schema** \n🧩 Untangle inconsistent spreadsheets and design a normalized, analytics-ready database.\n\n**2️⃣ Errors → Integrity** \n🧹 Audit, repair, and validate data quality so models rest on trustworthy foundations.\n👉 [Visit Challenge 02](https://www.kaggle.com/competitions/challenge-02-errors-integrity-2026)\n\n**3️⃣ Data → Signal** \n🌤️ Reveal patterns, seasonality, and growth trends through forecasting and decomposition.\n👉 [Visit Challenge 03](https://www.kaggle.com/competitions/challenge-03-data-signal-2026)\n\n**4️⃣ Shoppers → Strategy** \n🧾 Model loyalty, simulate promotions, and measure ROI for real-world business impact.\n👉 ( Challenge Under Development )\n\n---\n\n### 🧩 The Complete Learning Arc\n\nTogether, these challenges trace a full analytical lifecycle:\n**structure → integrity → insight → impact**\n\n* **Data Engineering** (schemas, ETL, pipelines)\n* **Machine Learning** (forecasting, imputation, uplift models)\n* **Applied Business Intelligence** (ROI simulation, decision support)\nEach challenge stands alone as a reproducible case study, but together they form a living experiment in applied data science—where design, ethics, and machine learning meet the realities of modern retail.\n\nWhether you’re a student, data professional, or researcher, you can join at any stage. \nThe series is designed to grow with you—from your first clean schema to your first production-ready predictive model.\n\nJoin any challenge individually, or follow the complete arc to build a portfolio that bridges:\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "01 – About", "content": "## 🧩 Challenge 01 — About\n\n**From Spreadsheets to Structure**\n\nA regional grocery chain has outgrown its spreadsheet chaos.\nEach department tracks products, stores, and vendors differently — IDs drift, columns change weekly, and no one can answer the same question twice.\n\nYour mission is to turn this “Excel spaghetti” into structure — a single, normalized relational database that connects customers, products, stores, and transactions.\nThe goal: create a trusted foundation ready for analytics, AI, and predictive modeling.\n\n---\n\n### 🧭 Series Context\n\nThis challenge is the **first stage** in the *Grocery Retail – Driven by Data* series — a four-part journey from **structure → trust → signal → strategy**.\nHere, you establish the architectural backbone that every later stage relies on: data quality checks, time-series forecasting, and promotion ROI modeling.\n\n---\n\n### 🎯 What You’ll Build\n\nParticipants will:\n\n* Reverse-engineer a **conceptual data model** from messy spreadsheets.\n* Normalize to **3NF**, balancing relational rigor with analytical usability.\n* Implement the schema in **PostgreSQL** with keys, constraints, and indexes.\n* Build reproducible **ETL pipelines** that clean, type-cast, and load data.\n* Demonstrate **analytic readiness** through summary queries or a baseline model.\n\n---\n\n### 💡 Why It Matters\n\nEvery advanced analytic or machine-learning workflow begins with trusted structure.\nA well-designed schema doesn’t just store data — it defines how the business understands itself.\nBy completing **Challenge 01**, you’ll transform operational chaos into clarity — laying the groundwork for reproducible, insight-driven decision-making across the entire series.\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "01 – Objectives", "content": "## 🎯 Challenge 01 — Objectives\n\n**From Data Chaos to Analytical Clarity**\n\nThe first challenge in the *Grocery Retail – Driven by Data* series transforms scattered spreadsheets into a structured, queryable database.\nParticipants learn how **data modeling, normalization, and reproducible pipelines** turn messy operational data into a foundation for trusted analytics and machine learning.\n\n---\n\n### 🧭 Core Objectives\n\nYou’ll design and implement a schema that connects every part of the retail operation — from stores and vendors to products and transactions.\n\n**Key goals:**\n\n* **Reverse-engineer the data model.** Identify entities (stores, products, vendors, transactions) and define relationships through a conceptual ERD.\n* **Normalize for scalability.** Implement a PostgreSQL schema following **3NF principles** while maintaining analytical flexibility.\n* **Build reliable ETL pipelines.** Create repeatable scripts to clean, type-coerce, and load data while enforcing key constraints and referential integrity.\n* **Demonstrate analytic readiness.** Prove that your schema supports meaningful business queries and simple baseline modeling (e.g., daily sales forecasting).\n* **Document your logic.** Deliver clear data dictionaries, schema diagrams, and concise rationale for every modeling choice.\n\n---\n\n### 💡 Broader Learning Outcomes\n\nThis challenge is designed to develop the habits and mindset of **data architects and applied analysts**:\n\n* Treat data structure as analysis — design schemas that clarify business meaning.\n* Build **transparent, reproducible** ETL processes ready for automation and audit.\n* Connect architecture and analytics — ensuring models rest on **stable, traceable data**.\n* Prepare for **Challenge 02 (Errors → Integrity)**, where the same schema will be tested under real-world data-quality stress.\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "01 – Deliverables", "content": "## 📦 Challenge 01 — Deliverables\n\n**From Messy Files to a Reproducible Data Foundation**\n\nThis challenge emphasizes **clarity, reproducibility, and communication** — the three pillars of trustworthy data science.\nBy the end, participants will produce a fully documented workflow showing how disorganized spreadsheets become a structured, analytics-ready relational database — plus a simple baseline model to validate its usefulness.\n\n---\n\n### 🧩 Required Deliverables\n\n#### 📄 Problem Statement & Notebook Stub\n\nA concise description of the business context, data goals, and success criteria.\nInclude a Jupyter notebook skeleton (`Setup`, `Ingest`, `EDA`, `Modeling`, `Results`) with brief markdown prompts outlining what belongs in each step.\n\n#### 📚 Data Dictionary\n\nA clear, human-readable table listing **every column** across all provided CSVs.\nInclude:\n\n* Data type and valid range\n* Example values\n* Notes on missingness or inconsistent formats\n* Business meaning (what each field actually represents)\n\n#### 🗺️ Entity–Relationship Diagram (ERD)\n\nA conceptual visualization showing entities, keys, and relationships.\nHighlight normalization decisions — e.g., why `Products` and `LineItems` were separated — and justify any compromises between strict 3NF and analytical efficiency.\n\n#### 🧱 Normalized Schema + DDL Scripts\n\nSQL scripts (`ddl.sql`, `indexes.sql`) defining the normalized schema in PostgreSQL.\nDocument naming conventions, key constraints, and indexes optimized for analytical joins (e.g., date, store, product).\n\n#### ⚙️ ETL Pipeline\n\nPython scripts or notebooks that:\n\n* Parse, clean, and load data into PostgreSQL\n* Enforce type coercion and referential integrity\n* Log row counts before/after loads\n* Capture rejected or anomalous rows for audit\n\n#### 📊 Exploratory Data Analysis (EDA)\n\nA notebook visualizing key business dimensions such as category mix, store performance, or temporal trends.\nInclude at least 3–5 visuals (histograms, trends, correlations) and a short written interpretation that connects structure to insight.\n\n#### 🤖 Baseline Model\n\nA simple, defensible predictive model demonstrating analytic readiness — e.g., a regression or time-based forecast of daily store sales.\nReport at least one metric (MAE, MAPE, RMSE) and note any assumptions or data limitations.\n\n#### 🖼️ Presentation Deck\n\nA 6–10 slide summary (PowerPoint or PDF) covering:\n\n* Data pipeline overview\n* ERD and schema design\n* EDA highlights and baseline model results\n* Key takeaways and next steps for **Challenge 02 (Errors → Integrity)**\n\n---\n\n### 🗂️ File Structure Checklist\n\n| Folder | Files / Outputs |\n| :----------- | :----------------------------------------- |\n| `docs/` | `architecture.md`, ERD image, design notes |\n| `sql/` | `ddl.sql`, `indexes.sql` |\n| `scripts/` | `load_data.py`, ETL logs |\n| `notebooks/` | `01_eda.ipynb`, `03_model_baseline.ipynb` |\n| `reports/` | `executive_summary.pdf` |\n| `slides/` | `presentation_deck.pptx` |\n\n---\n\n### ✅ Outcome\n\nA complete, validated, and **well-documented relational data foundation** — the backbone for all future stages:\ndata-quality testing, imputation, forecasting, and ROI modeling.\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "01 – Evaluation", "content": "## ⚖️ Challenge 01 — Evaluation\n\n**Clarity. Reproducibility. Insight.**\n\nThis first challenge focuses on process quality rather than leaderboard competition.\nYour goal isn’t to “win” — it’s to **build a reproducible analytical foundation** that others can trust, reuse, and extend across the series.\n\n---\n\n### 🧩 Evaluation Principles\n\nSubmissions are assessed on five core dimensions that define professional data science craftsmanship:\n\n| Area | What Reviewers Look For |\n| ----------------------------- | -------------------------------------------------------------------------------------------- |\n| **Structure & Normalization** | Logical, well-documented ERD; correct keys, constraints, and naming consistency. |\n| **ETL Integrity** | Clean, repeatable pipeline with type coercion, referential checks, and idempotent execution. |\n| **Analytics Readiness** | Demonstrated ability to query or model against the schema (proof it “works”). |\n| **Baseline Modeling** | A simple, documented predictive example with sound metric and clear reasoning. |\n| **Documentation & Clarity** | Readable notebooks, data dictionary, and concise explanation of design choices. |\n\n---\n\n### 🧮 Scoring Framework\n\nThis challenge is in **open beta**, so evaluation emphasizes learning and collaboration rather than points.\nEach submission receives a qualitative review:\n\n| Rating | Description |\n| ----------------- | ----------------------------------------------------------------------------- |\n| ✅ **Pass** | Complete, reproducible workflow that meets all major requirements. |\n| ⚙️ **Partial** | Solid progress with small issues in documentation, reproducibility, or logic. |\n| 🧭 **Needs Work** | Missing core elements or unclear reasoning — resubmission encouraged. |\n\nThere is no numeric leaderboard — think of this as a **craft-based studio model**.\nFeedback will evolve through discussion threads, peer reviews, and shared improvement examples.\n\n---\n\n### 🏅 Recognition Badges\n\nExceptional submissions may receive informal recognition within the discussion forum:\n\n🥇 **Gold Schema** – Exceptionally clean ERD & ETL; reproducible and elegantly documented\n🥈 **Silver Schema** – Complete, functional workflow with minor improvements possible\n🥉 **Bronze Schema** – Meets core requirements, demonstrates clear learning and iteration\n\nFuture challenges will introduce automated validation tools to check schema integrity and reproducibility.\nFor now, **the community acts as the reviewer**, focusing on quality, clarity, and openness.\n\n---\n\n### 🔮 Looking Ahead\n\nAs the series matures, the evaluation framework will expand to include:\n\n* **Automated reproducibility tests** (schema builds, ETL checks, data validation)\n* **AI-assisted review summaries** highlighting best practices\n* **Peer learning credits** for meaningful reviews and contributions\n\nChallenge 01 establishes the **standard of transparency** that every later challenge builds upon —\nbecause in data science, **trust begins with structure**.\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "01 - Work Plan", "content": "## 🧩 Challenge 01 — Work Plan\n\n**From Spreadsheets to Structure: Step-by-Step**\n\nThis work plan guides participants through the practical stages of **Challenge 01 – Spaghetti → Schema**, the first milestone in the *Grocery Retail – Driven by Data* series.\nIt defines a clear, outcome-based roadmap—from initial data ingest to a validated relational schema and baseline analytical proof.\n\n---\n\n### 🗺️ Workflow Overview\n\n| Phase | Focus | Outcome |\n| :--------------------------------- | :---------------------------------------------------------------------------------------------- | :--------------------------------------------------- |\n| **1. Problem Definition** | Frame the business context, data goals, and evaluation criteria. | Clear problem statement and notebook stub |\n| **2. Data Ingestion** | Read, validate, and profile raw CSVs with explicit types and date parsing. | Clean pandas DataFrames with basic integrity checks |\n| **3. Data Documentation** | Build a complete, human-readable **Data Dictionary**. | Column definitions, types, quirks, and examples |\n| **4. Entity-Relationship Design** | Reverse-engineer tables and relationships from the messy sheets. | Conceptual **ERD** showing PK/FK logic and rationale |\n| **5. Schema & DDL Implementation** | Normalize to 3NF and generate PostgreSQL DDL scripts with constraints and indexes. | Executable schema (ddl.sql + indexes.sql) |\n| **6. ETL Pipeline** | Create repeatable scripts to clean, type-coerce, and load data into the schema. | Automated, idempotent load with audit logs |\n| **7. Exploratory Analysis** | Validate the schema by running descriptive queries and visualizing key dimensions. | 3–5 EDA visuals + short narrative insight |\n| **8. Baseline Model** | Demonstrate analytic readiness with a simple predictive example (e.g., daily-sales regression). | Model notebook + metric (MAE/MAPE/RMSE) |\n| **9. Presentation Summary** | Communicate the workflow and findings. | 6–10 slide summary deck |\n| **10. Submission Package** | Assemble all deliverables in the prescribed folder structure. | Reproducible, well-documented project zip |\n\n---\n\n### 🧰 Practical Notes\n\n* **Environment:** Python 3 + PostgreSQL 15 recommended.\n* **Libraries:** `pandas`, `sqlalchemy`, `matplotlib / seaborn`, `scikit-learn` (for baseline).\n* **Version Control:** Use GitHub or Kaggle Code Versions for traceability.\n* **Reproducibility:** All scripts should be **re-runnable end-to-end** with no manual edits.\n* **Validation Checks:** Row counts ✓ Foreign keys ✓ No duplicate IDs ✓ Type coercion ✓\n\n---\n\n### ✅ Deliverables Checklist\n\n| Category | Files / Outputs |\n| :------------------- | :--------------------------------------------------------------- |\n| **Docs & Design** | `docs/architecture.md`, ERD image, design notes |\n| **SQL Schema** | `sql/ddl.sql`, `sql/indexes.sql` |\n| **ETL Scripts** | `scripts/load_data.py`, ETL logs |\n| **Notebooks** | `01_eda.ipynb`, `03_model_baseline.ipynb` |\n| **Reports & Slides** | `reports/executive_summary.pdf`, `slides/presentation_deck.pptx` |\n\n---\n\n### 🧭 Outcome\n\nBy following this plan, participants will deliver a **fully reproducible relational data foundation** that:\n\n* Integrates scattered retail spreadsheets into one coherent schema.\n* Provides clear documentation and validation evidence.\n* Demonstrates the first analytical bridge between data structure and business insight.\n\nThis milestone sets the stage for **Challenge 02 (Errors → Integrity)**, where the same database will face systematic data-quality tests and imputation tasks.\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "01 - Tagline", "content": "### 🏷️ Challenge 01 — Tagline\n\n**From spreadsheets to structure — building the foundations of predictive retail.**\n\nThis challenge is where chaos becomes clarity.\nIt marks the first step in a larger data-to-insight journey — transforming disconnected spreadsheets into a single, trustworthy database that powers every analytic and machine-learning decision to come.\n\nEvery forecast, every model, every strategic question begins here — with structure you can trust.\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Project Research Context", "content": "## 🧪 Project Research Context\n\n**Why this exists.** *Grocery – Driven by Data* doubles as a living research lab for applied analytics. It follows a realistic journey—**structure → integrity → signal → strategy**—to study how humans and AI collaborate across the full data-science lifecycle.\n\n**What we study.**\n\n* **Data maturity:** How teams evolve from spreadsheet chaos to reproducible, analytics-ready pipelines.\n* **Human–AI collaboration:** Three modes in practice—**AI Tutor** (guidance), **AI Coder** (co-creation), **AI Agent** (delegation)—and how roles, productivity, and quality shift across them.\n* **Reproducibility & rigor:** The impact of clean DDL, versioned notebooks, validation tests, and audit trails on downstream modeling.\n* **Interpretability & ethics:** How transparency, fairness checks, and decision accountability travel with the data and models.\n\n**How we study it.**\n\n* **Open artifacts:** Schemas (DDL), ERDs, ETL scripts, notebooks, validation harnesses, and reports are versioned and comparable across participants.\n* **Comparable tasks:** Each challenge supplies fixed goals, baseline code, and acceptance criteria to enable apples-to-apples evaluation.\n* **Lightweight telemetry (opt-in):** Time-to-completion, error types, and revision patterns help analyze learning curves and collaboration styles.\n* **Mixed methods:** Quantitative metrics (e.g., load success, test pass rates, forecast error) plus qualitative reviews of design rationale and trade-offs.\n\n**What “good” looks like.**\n\n* **Reproducible**: End-to-end runs without manual edits; clear seeds, environments, and data contracts.\n* **Traceable**: Decisions (imputation, features, modeling) documented with rationale and diagnostics.\n* **Responsible**: Bias, leakage, and overfitting checks included by default; results communicated with honest uncertainty.\n\n**Planned outputs.**\n\n* **Benchmarks** for schema integrity, data-quality repair, forecasting robustness, and uplift modeling.\n* **Design patterns** for human–AI teamwork across Tutor/Coder/Agent modes.\n* **Teaching modules** that map directly from the challenges to course labs and workshops.\n* **Promotion playbooks** (later stages) linking model outcomes to ROI, sensitivity, and risk.\n\n**Multi-year arc.**\n\n* **Year 1 – Foundations:** Establish shared datasets, baselines, and validation suites.\n* **Years 2–3 – Collaboration:** Compare human-written vs. AI-assisted pipelines; analyze quality, speed, and interpretability.\n* **Years 4–5 – Autonomy:** Evaluate Agent-style workflows with human oversight, emphasizing governance, explainability, and reliability.\n\n**Invitation.** Students, practitioners, and researchers are welcome to contribute methods, critiques, and case studies. Industry partners can provide anonymized scenarios or constraints to pressure-test real-world applicability.\n\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false } ] }