{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":121150,"databundleVersionId":15061024,"sourceType":"competition"},{"sourceId":14856826,"sourceType":"datasetVersion","datasetId":9480943}],"dockerImageVersionId":31259,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n## 📊 Notebook Markdown: Leaderboard Analysis & Community Trends\n\n### **Purpose of this Analysis**\n\nThis notebook tracks the **Public Leaderboard (LB)** distribution to gain insights into:\n\n1. **Score Distribution:** Identifying \"plateaus\" where many teams have the same score, often indicating a popular public notebook.\n2. **Competitive Gaps:** Visualizing the distance between the Top 10 and the rest of the field.\n3. **Submission Activity:** Correlation between the number of entries and the final public score.\n4. **Community Benchmarking:** Comparing the current LB against the latest high-scoring public kernels to see how much \"originality\" is required to break into the top tier.\n\n---","metadata":{}},{"cell_type":"markdown","source":"# New 2026/02/16 : Public LB migration after the highest public notebook is updating from 35.1 to 35.2\n\nTo analyze the evolution of the leaderboard following the release of a new top public notebook (moving from 35.1 to 35.2), we can compare two snapshots of the leaderboard. This allows us to quantify exactly how much of the community \"migrated\" to the new baseline.\n\n---\n\n## 📊 Notebook Markdown: Benchmark Migration Analysis ()\n\n### **The \"Public Notebook\" Effect**\n\nWhen a new high-scoring notebook is released, we typically observe a **\"Migration Wave.\"** Teams previously clustered around the old benchmark (35.1) or those struggling below it often adopt the new logic, leading to a massive spike at the new benchmark (35.2).","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# 1. Load snapshots\n\ndf_old = pd.read_csv('/kaggle/input/deeppast-leaderboard/leaderboard-2026-02-13.csv')\ndf_new = pd.read_csv('/kaggle/input/deeppast-leaderboard/leaderboard-2026-02-16.csv')\n\ndf_old.columns = ['Rank', 'TeamId','TeamName','LastSubmissionDate', 'Score', 'Entries','Team']\ndf_new.columns = ['Rank', 'TeamId','TeamName','LastSubmissionDate', 'Score', 'Entries','Team']\n\n\n# Configuration\nOLD_BM = 35.1\nNEW_BM = 35.2\nZOOM_RANGE = (34, 36) # Focus area\n\n\nold_in_zone = df_old[(df_old['Score'] >= ZOOM_RANGE[0]) & (df_old['Score'] <= ZOOM_RANGE[1])]\nnew_in_zone = df_new[(df_new['Score'] >= ZOOM_RANGE[0]) & (df_new['Score'] <= ZOOM_RANGE[1])]\n\n# 3. Plotting\nplt.figure(figsize=(14, 7))\n\nsns.histplot(df_old['Score'], bins=1000, color='gray', alpha=0.4, label=f'LB on 2026-02-13', element=\"step\")\nsns.histplot(df_new['Score'], bins=1000, color='#1f77b4', alpha=0.7, label=f'LB on 2026-02-16', element=\"step\")\n\n# Vertical indicators for the Benchmarks\nplt.axvline(OLD_BM, color='orange', linestyle='--', linewidth=1.5, label=f'Old NB ({OLD_BM})')\nplt.axvline(NEW_BM, color='red', linestyle='--', linewidth=2, label=f'New NB ({NEW_BM})')\n\n# ZOOMING IN\nplt.xlim(ZOOM_RANGE) \n\n# Aesthetics\nplt.title(f'Magnified Migration: The {OLD_BM} $\\\\rightarrow$ {NEW_BM} Transition', fontsize=16)\nplt.xlabel('Leaderboard Score (Zoomed)', fontsize=12)\nplt.ylabel('Number of Teams', fontsize=12)\nplt.legend(loc='upper left')\nplt.grid(axis='x', alpha=0.3)\n\nplt.show()\n\n# 4. Statistical Summary of the \"Jump\"\nteams_at_old = len(df_old[(df_old['Score'] >= OLD_BM - 0.001) & (df_old['Score'] <= OLD_BM + 0.001)])\nteams_at_new = len(df_new[(df_new['Score'] >= NEW_BM - 0.001) & (df_new['Score'] <= NEW_BM + 0.001)])\n\nprint(f\"--- High-Resolution Report ---\")\nprint(f\"Teams anchored at {OLD_BM} (Feb 13): {teams_at_old}\")\nprint(f\"Teams anchored at {NEW_BM} (Feb 16): {teams_at_new}\")\nprint(f\"Migration Velocity: {teams_at_new - teams_at_old} net change in benchmark clusters\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-16T10:37:42.809229Z","iopub.execute_input":"2026-02-16T10:37:42.809591Z","iopub.status.idle":"2026-02-16T10:37:43.086164Z","shell.execute_reply.started":"2026-02-16T10:37:42.809560Z","shell.execute_reply":"2026-02-16T10:37:43.085287Z"},"_kg_hide-input":true,"jupyter":{"source_hidden":true}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# 🐍 The \"Entry-to-Score\" Efficiency Plot","metadata":{}},{"cell_type":"code","source":"\n\nactive_zone = df_new[(df_new['Score'] >= 35.0) & (df_new['Score'] <= 35.5)]\n\nplt.figure(figsize=(12, 6))\n\n\nsns.jointplot(data=active_zone, x='Entries', y='Score', \n              kind=\"reg\", color=\"m\", height=7, \n              joint_kws={'line_kws':{'color':'red'}})\n\nplt.suptitle('Submission Efficiency: Cost of the 35.2 Jump', y=1.02)\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-16T10:37:47.941439Z","iopub.execute_input":"2026-02-16T10:37:47.941787Z","iopub.status.idle":"2026-02-16T10:37:48.452879Z","shell.execute_reply.started":"2026-02-16T10:37:47.941758Z","shell.execute_reply":"2026-02-16T10:37:48.452028Z"},"jupyter":{"source_hidden":true}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# 📊 General analysis based on leaderboard 2026/02/13","metadata":{}},{"cell_type":"code","source":"\ndf = pd.read_csv('/kaggle/input/deeppast-leaderboard/leaderboard-2026-02-13.csv')\n\n\ndf.columns = ['Rank', 'TeamId','TeamName','LastSubmissionDate', 'Score', 'Entries','Team']\n\n# Set visual style\nsns.set_theme(style=\"whitegrid\")\nplt.rcParams['figure.figsize'] = (12, 6)\n\n# --- Visualization 1: Score Distribution ---\nplt.figure()\nsns.histplot(df['Score'], bins=50, kde=True, color='skyblue')\nplt.title('Public Leaderboard Score Distribution', fontsize=15)\nplt.xlabel('Score', fontsize=12)\nplt.ylabel('Frequency (Teams)', fontsize=12)\n\n\npublic_best_score = 35.1 \nplt.axvline(public_best_score, color='red', linestyle='--', label=f'Top Public Notebook ({public_best_score})')\nplt.legend()\nplt.show()\n\n# --- Visualization 2: The \"Top 100\" Gap ---\nplt.figure()\ntop_100 = df.head(100)\nsns.lineplot(data=top_100, x='Rank', y='Score', marker='o', markersize=4)\nplt.title('Score Curve for Top 100 Teams', fontsize=15)\nplt.xlabel('Rank', fontsize=12)\nplt.ylabel('Score', fontsize=12)\nplt.gca().invert_xaxis() \nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-16T10:26:58.052015Z","iopub.execute_input":"2026-02-16T10:26:58.052315Z","iopub.status.idle":"2026-02-16T10:26:58.628073Z","shell.execute_reply.started":"2026-02-16T10:26:58.052288Z","shell.execute_reply":"2026-02-16T10:26:58.627273Z"},"jupyter":{"source_hidden":true}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"\n# --- Visualization 3: Entries vs. Score ---\nplt.figure()\nsns.scatterplot(data=df, x='Entries', y='Score', alpha=0.5, hue='Rank', palette='viridis')\nplt.title('Entries vs. Score', fontsize=15)\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-16T10:26:58.629538Z","iopub.execute_input":"2026-02-16T10:26:58.629869Z","iopub.status.idle":"2026-02-16T10:26:58.939607Z","shell.execute_reply.started":"2026-02-16T10:26:58.629835Z","shell.execute_reply":"2026-02-16T10:26:58.938723Z"},"jupyter":{"source_hidden":true}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Shake analysis and the **35.1** benchmark for **February 13, 2026**.\n\n---\n\n## 📉 Shake-up Probability & Benchmark Analysis\n\n### **The \"35.1\" Crowd Density**\n\nIn many Kaggle competitions, a high-scoring public notebook creates a **score plateau**. If thousands of teams are separated by a margin of less than , the final ranking will be extremely volatile when evaluated on the Private Leaderboard. This is known as a **\"Shake-up.\"**\n\n### **Purpose of this Analysis**\n\n* **Calculate Density:** Determine how many teams are \"packed\" within 1% of the public notebook score.\n* **Shake-up Risk:** High density near the benchmark suggests that small variations in private data will cause massive rank shifts.\n* **Gap to Gold:** Visualizing the \"leap\" required to move from the public baseline into the top tier.\n","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# 1. Configuration & Data Loading\nFILE_PATH = '/kaggle/input/deeppast-leaderboard/leaderboard-2026-02-13.csv' \nPUBLIC_NB_SCORE = 35.1  \nDATE_OBSERVED = \"2026-02-13\"\n\ndf = pd.read_csv(FILE_PATH)\ndf = df.sort_values('Score', ascending=False).reset_index(drop=True)\n\n# 2. Calculate Crowd Density (Teams within 0.5% of the Public Score)\nmargin = PUBLIC_NB_SCORE * 0.005\ncrowded_zone = df[(df['Score'] >= PUBLIC_NB_SCORE - margin) & \n                  (df['Score'] <= PUBLIC_NB_SCORE + margin)]\n\ndensity_count = len(crowded_zone)\ntotal_teams = len(df)\nshakeup_risk = \"HIGH\" if (density_count / total_teams) > 0.15 else \"MODERATE\"\n\n# 3. Visualization: The Shake-up Danger Zone\nplt.figure(figsize=(12, 6))\n\n# Main distribution\nsns.kdeplot(df['Score'], fill=True, color='grey', alpha=0.3)\n\n# Highlight the \"Crowded Zone\"\nplt.axvspan(PUBLIC_NB_SCORE - margin, PUBLIC_NB_SCORE + margin, \n            color='red', alpha=0.2, label='Shake-up Danger Zone')\n\nplt.axvline(PUBLIC_NB_SCORE, color='red', linestyle='--', label=f'Public NB ({PUBLIC_NB_SCORE})')\n\n# Aesthetics\nplt.title(f'Leaderboard Density & Shake-up Risk: {DATE_OBSERVED}', fontsize=14)\nplt.xlabel('Score')\nplt.ylabel('Density of Teams')\nplt.legend()\nplt.grid(axis='y', alpha=0.3)\n\nplt.show()\n\n# 4. Print Analysis Report\nprint(f\"--- Shake-up Report: {DATE_OBSERVED} ---\")\nprint(f\"Total Teams on Leaderboard: {total_teams}\")\nprint(f\"Teams in the 'Public NB' Cluster: {density_count}\")\nprint(f\"Cluster Percentage: {(density_count/total_teams)*100:.2f}%\")\nprint(f\"Current Shake-up Risk: {shakeup_risk}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-16T10:26:58.940840Z","iopub.execute_input":"2026-02-16T10:26:58.941192Z","iopub.status.idle":"2026-02-16T10:26:59.190053Z","shell.execute_reply.started":"2026-02-16T10:26:58.941151Z","shell.execute_reply":"2026-02-16T10:26:59.189210Z"},"jupyter":{"source_hidden":true}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n## 💡 Monitoring Community Change\n\nTo track how the community moves over time, keep an eye on the **Score Distribution Histogram**:\n\n* **Sudden Spikes:** If a huge \"tower\" appears at a specific score, it almost always means a new high-performing public notebook was released.\n* **The \"Long Tail\":** A wide spread of scores suggests the competition is difficult and requires custom feature engineering.\n* **Plateauing:** If the top scores haven't moved in days, it might mean the current popular methods have hit a performance ceiling.\n","metadata":{}}]}