{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.11.11"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":104857,"databundleVersionId":12651513,"sourceType":"competition"}],"dockerImageVersionId":31040,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"papermill":{"default_parameters":{},"duration":652.115162,"end_time":"2025-06-14T10:02:37.709067","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2025-06-14T09:51:45.593905","version":"2.6.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"d18589e0-2790-4d31-8062-1f95c8afff9c","cell_type":"markdown","source":"# MotoGP Datathon: Lap Time Prediction \n\nWelcome to this notebook for the MotoGP Datathon, focused on predicting lap times! In this competition, our goal is to leverage various rider, bike, and circuit characteristics to accurately forecast the time it takes for a MotoGP rider to complete a lap.\n\nPredicting lap times is crucial in motorsport for strategy optimization, performance analysis, and understanding competitive dynamics.\n\n---\n\n## 1. Initial Setup and Data Loading \n\nWe begin by importing the necessary libraries for data manipulation and numerical operations. Then, we load the training, testing, and sample submission datasets provided for the competition.\n\n-   `pandas` is indispensable for data handling and analysis.\n-   `numpy` provides powerful numerical computing capabilities, especially useful for array operations.\n\nLet's load the datasets and take a first look at their structure and the initial rows of the training data.","metadata":{}},{"id":"82b02283","cell_type":"code","source":"import pandas as pd\nimport numpy as np\n\ntrain = pd.read_csv('/kaggle/input/burnout-datathon-ieeecsmuj/train.csv')\ntest = pd.read_csv('/kaggle/input/burnout-datathon-ieeecsmuj/test.csv')\nsample = pd.read_csv('/kaggle/input/burnout-datathon-ieeecsmuj/sample_submission.csv')\n\nprint(\"Train shape:\", train.shape)\nprint(\"Test shape:\", test.shape)\ntrain.head()","metadata":{"papermill":{"duration":17.720566,"end_time":"2025-06-14T09:52:09.518768","exception":false,"start_time":"2025-06-14T09:51:51.798202","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2025-06-19T12:46:34.358383Z","iopub.execute_input":"2025-06-19T12:46:34.358588Z","iopub.status.idle":"2025-06-19T12:46:53.674910Z","shell.execute_reply.started":"2025-06-19T12:46:34.358561Z","shell.execute_reply":"2025-06-19T12:46:53.674124Z"}},"outputs":[],"execution_count":null},{"id":"040bf78b-caf9-4106-a6dc-e1c8f23ff483","cell_type":"markdown","source":"## 2. Suppressing Warnings \nTo keep our notebook output clean and focused on essential information, we'll suppress common warnings, such as **DeprecationWarning** and **UserWarning**, which often arise from library updates or specific function usages. This helps improve readability without affecting the core logic.","metadata":{}},{"id":"d732094d-0073-42fe-ac08-61d385259514","cell_type":"code","source":"import warnings\nwarnings.filterwarnings(\"ignore\", category=DeprecationWarning)\nwarnings.filterwarnings(\"ignore\", category=UserWarning)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-19T13:01:26.464779Z","iopub.execute_input":"2025-06-19T13:01:26.465519Z","iopub.status.idle":"2025-06-19T13:01:26.469595Z","shell.execute_reply.started":"2025-06-19T13:01:26.465488Z","shell.execute_reply":"2025-06-19T13:01:26.468894Z"}},"outputs":[],"execution_count":null},{"id":"b5c9166a-8119-4291-9231-09dfe9aa4fa6","cell_type":"markdown","source":"## 3. Exploratory Data Analysis (EDA) \nA quick look at our data to understand its shape and key characteristics.\n\n**2.1 Target Variable (Lap_Time_Seconds)**\nLet's check the distribution of our target variable, Lap_Time_Seconds.\n\n- Observation: (Describe what you see: e.g., \"Mostly normal distribution, slightly skewed towards higher times.\")\n\n\n**2.2 Missing Values**\nIdentifying any missing data is crucial for pre-processing.\n\n- Observation: (State the finding: e.g., \"Some features have nulls, which will need handling,\" or \"Data is largely complete.\")\n\n\n**2.3 Feature Correlation with Lap_Time_Seconds**\nUnderstanding which features strongly influence Lap_Time_Seconds helps in feature selection.\n\n- Observation: (Summarize key correlations: e.g., \"Strong inverse correlation with Speed, positive with Tyre_Wear - intuitive!\")\n","metadata":{}},{"id":"126e7492","cell_type":"code","source":"import seaborn as sns\nimport matplotlib.pyplot as plt\n\n# Target Distribution\nsns.histplot(train['Lap_Time_Seconds'], kde=True, bins=30)\nplt.title(\"Target Distribution - Lap Time (Seconds)\")\nplt.show()\n\n# Null Count\nprint(\"Missing values:\\n\", train.isnull().sum().sort_values(ascending=False).head(10))\n\n\nnumeric_cols = train.select_dtypes(include=['int64', 'float64']).columns\ncorr = train[numeric_cols].corr()\nplt.figure(figsize=(12, 8))\nsns.heatmap(corr[['Lap_Time_Seconds']].sort_values(by='Lap_Time_Seconds', ascending=False), annot=True, cmap='coolwarm')\nplt.title(\"Correlation with Lap_Time_Seconds\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2025-06-19T12:47:35.801469Z","iopub.execute_input":"2025-06-19T12:47:35.801733Z","iopub.status.idle":"2025-06-19T12:47:50.730096Z","shell.execute_reply.started":"2025-06-19T12:47:35.801712Z","shell.execute_reply":"2025-06-19T12:47:50.729266Z"},"papermill":{"duration":16.059946,"end_time":"2025-06-14T09:52:25.582323","exception":false,"start_time":"2025-06-14T09:52:09.522377","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"ccb21d05-6767-415c-b4e9-19676d9a1668","cell_type":"markdown","source":"## 4. Setup & Data Preparation \nHere, we set up our environment by installing necessary library versions, importing core tools, and defining key column names and data paths. This ensures reproducibility and prepares our data for modeling.\n\n**Key Action:** We're explicitly installing specific versions of xgboost and pandas for environment consistency.\n\n**Purpose:** This block is the foundation for our modeling pipeline, ensuring all required modules are available and data is loaded correctly for the next steps.","metadata":{}},{"id":"5cff3c21","cell_type":"code","source":"!pip install --quiet xgboost==2.0.3 pandas==2.2.2\n\nimport pandas as pd, numpy as np, os, xgboost as xgb, time, gc\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import mean_squared_error\n\n# Paths & column names\nDATA_DIR = \"/kaggle/input/burnout-datathon-ieeecsmuj\"   \nTARGET   = \"Lap_Time_Seconds\"                \nID_COL   = \"Unique ID\"                       \n\ntrain_df = pd.read_csv(os.path.join(DATA_DIR, \"train.csv\"))\ntest_df  = pd.read_csv(os.path.join(DATA_DIR, \"test.csv\"))\nprint(\"Train shape:\", train_df.shape, \"  Test shape:\", test_df.shape)","metadata":{"execution":{"iopub.status.busy":"2025-06-19T13:01:27.753636Z","iopub.execute_input":"2025-06-19T13:01:27.754234Z","iopub.status.idle":"2025-06-19T13:01:42.543166Z","shell.execute_reply.started":"2025-06-19T13:01:27.754209Z","shell.execute_reply":"2025-06-19T13:01:42.542341Z"},"papermill":{"duration":0.010876,"end_time":"2025-06-14T09:52:25.598817","exception":false,"start_time":"2025-06-14T09:52:25.587941","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"ce2d3a8d-6695-4d7c-8784-18499ba8ce0f","cell_type":"markdown","source":"## 5. Feature Engineering\nTo enhance our model's predictive power, we're creating a few new features from the existing ones. This process, known as feature engineering, helps capture more complex relationships within the data.\n\n### New Features Created:\n- **secs_per_km:** Derived from *Circuit_Length_km* and *Avg_Speed_kmh*, this feature aims to capture the average time spent per kilometer, which is a direct indicator of speed relative to track length.\n\n- **track_minus_air:** The difference between *Track_Temperature_Celsius* and *Ambient_Temperature_Celsius*. This could be important as tire grip and engine performance can be sensitive to the temperature differential between the track surface and the surrounding air.\n  \n- **is_wet:** A binary (0/1) flag indicating if the *Track_Condition* is \"Wet\". Wet conditions drastically alter lap times, so this explicit feature is crucial.\n\n- **tire_combo:** A categorical feature created by concatenating *Tire_Compound_Front* and *Tire_Compound_Rear*. The combination of front and rear tires is a key strategic choice affecting performance.\n\n\nThese features are expected to provide more direct and relevant information for predicting lap times.","metadata":{}},{"id":"7f6dee4d","cell_type":"code","source":"# Minimal feature engineering\ndef add_features(df):\n    if {\"Circuit_Length_km\", \"Avg_Speed_kmh\"}.issubset(df.columns):\n        df[\"secs_per_km\"] = (df[\"Circuit_Length_km\"] / df[\"Avg_Speed_kmh\"]) * 3600\n    if {\"Track_Temperature_Celsius\", \"Ambient_Temperature_Celsius\"}.issubset(df.columns):\n        df[\"track_minus_air\"] = df[\"Track_Temperature_Celsius\"] - df[\"Ambient_Temperature_Celsius\"]\n    if \"Track_Condition\" in df.columns:\n        df[\"is_wet\"] = df[\"Track_Condition\"].str.contains(\"Wet\", case=False, na=False).astype(int)\n    if {\"Tire_Compound_Front\", \"Tire_Compound_Rear\"}.issubset(df.columns):\n        df[\"tire_combo\"] = df[\"Tire_Compound_Front\"].fillna(\"\") + \"_\" + df[\"Tire_Compound_Rear\"].fillna(\"\")\n    return df\n\ntrain_df = add_features(train_df);  test_df = add_features(test_df)","metadata":{"execution":{"iopub.status.busy":"2025-06-19T13:02:47.741205Z","iopub.execute_input":"2025-06-19T13:02:47.741915Z","iopub.status.idle":"2025-06-19T13:02:49.357190Z","shell.execute_reply.started":"2025-06-19T13:02:47.741884Z","shell.execute_reply":"2025-06-19T13:02:49.356621Z"},"papermill":{"duration":0.082765,"end_time":"2025-06-14T09:52:25.686782","exception":false,"start_time":"2025-06-14T09:52:25.604017","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"990bda32-7efd-4f01-b051-74b18b6aa5ce","cell_type":"markdown","source":"## 6. Feature Transformation: Position\nWe're converting the *position* feature from a numeric type to a categorical string. This ensures that our model treats *position* as distinct categories rather than a continuous numerical value, which is often more appropriate for ranks or discrete placings. The original numeric *position* column is then dropped.\n\n**Why ?** \n\nTreating position as a category (e.g., '1st', '2nd') rather than a number (1, 2) can better represent its true impact, as the difference between 1st and 2nd might not be the same as between 10th and 11th in a purely numerical sense.","metadata":{}},{"id":"f4d19c96","cell_type":"code","source":"# Convert position numeric → categorical string (if present)\nif \"position\" in train_df.columns:\n    train_df[\"Position_cat\"] = train_df[\"position\"].astype(str)\n    test_df[\"Position_cat\"]  = test_df[\"position\"].astype(str)\n    train_df.drop(columns=[\"position\"], inplace=True)\n    test_df.drop(columns=[\"position\"],  inplace=True)","metadata":{"execution":{"iopub.status.busy":"2025-06-19T13:03:18.964739Z","iopub.execute_input":"2025-06-19T13:03:18.965364Z","iopub.status.idle":"2025-06-19T13:03:20.071274Z","shell.execute_reply.started":"2025-06-19T13:03:18.965338Z","shell.execute_reply":"2025-06-19T13:03:20.070455Z"},"papermill":{"duration":4.447552,"end_time":"2025-06-14T09:52:30.139587","exception":false,"start_time":"2025-06-14T09:52:25.692035","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"a09ade5b-858c-4d91-a257-eaafa87a8c4a","cell_type":"markdown","source":"## 7. Feature Type Identification & Missing Value Imputation\nBefore training, we accurately identify categorical and numerical features and handle any missing values. This ensures our data is clean and in the correct format for the model.\n\n**Key Steps:**\n\n- **Categorical/Numerical Split:** Features are separated based on their data types.\n- **Consistency Check:** Only categorical columns present in both train and test sets are kept to avoid issues.\n\n- **NaN Imputation:**\n  \n-- 1. Categorical missing values are filled with the string \"*missing*\".\n\n-- 2. Numerical missing values are imputed using the median value calculated from the training data. This prevents data leakage from the test set.\n\n\n- **Type Casting:** Categorical columns are explicitly cast to the category dtype, which allows XGBoost to efficiently handle them internally.\n  \nThis robust preparation ensures all features are ready for model training.\n\n","metadata":{}},{"id":"bd521256","cell_type":"code","source":"# Identify categorical & numeric columns robustly\ncat_cols = [c for c in train_df.columns\n            if train_df[c].dtype == \"object\" or train_df[c].dtype.name == \"category\"]\nnum_cols = [c for c in train_df.columns if c not in cat_cols + [TARGET]]\n\n# Keep only cats present in both frames\ncat_cols = [c for c in cat_cols if c in test_df.columns]\n\n# Fill NaNs\ntrain_df[cat_cols] = train_df[cat_cols].fillna(\"missing\")\ntest_df[cat_cols]  = test_df[cat_cols].fillna(\"missing\")\nfor col in num_cols:\n    med = train_df[col].median()\n    train_df[col].fillna(med, inplace=True)\n    test_df[col].fillna(med,  inplace=True)\n\n# Cast categorical dtype for XGB native cats\nfor c in cat_cols:\n    train_df[c] = train_df[c].astype(\"category\")\n    test_df[c]  = test_df[c].astype(\"category\")\n\nfeature_cols = cat_cols + num_cols\nprint(f\"Using {len(feature_cols)} features ({len(cat_cols)} categorical).\")","metadata":{"execution":{"iopub.status.busy":"2025-06-19T13:04:34.873690Z","iopub.execute_input":"2025-06-19T13:04:34.874256Z","iopub.status.idle":"2025-06-19T13:04:41.263269Z","shell.execute_reply.started":"2025-06-19T13:04:34.874235Z","shell.execute_reply":"2025-06-19T13:04:41.262472Z"},"papermill":{"duration":1.166455,"end_time":"2025-06-14T09:52:31.312744","exception":false,"start_time":"2025-06-14T09:52:30.146289","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"f1d750c0","cell_type":"code","source":"# Train‑validation split \nX_train, X_val, y_train, y_val = train_test_split(\n    train_df[feature_cols], train_df[TARGET],\n    test_size=0.2, random_state=42\n)\n\ndtrain = xgb.DMatrix(X_train, label=y_train, enable_categorical=True)\ndval   = xgb.DMatrix(X_val,   label=y_val,   enable_categorical=True)\ndtest  = xgb.DMatrix(test_df[feature_cols], enable_categorical=True)","metadata":{"execution":{"iopub.status.busy":"2025-06-19T13:04:59.117656Z","iopub.execute_input":"2025-06-19T13:04:59.118037Z","iopub.status.idle":"2025-06-19T13:05:02.653960Z","shell.execute_reply.started":"2025-06-19T13:04:59.118013Z","shell.execute_reply":"2025-06-19T13:05:02.653390Z"},"papermill":{"duration":114.782641,"end_time":"2025-06-14T09:54:26.101269","exception":false,"start_time":"2025-06-14T09:52:31.318628","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"250a984b-b9df-4364-bea1-a8ebe6cdc09a","cell_type":"markdown","source":"## 8. XGBoost Model Training\nNow, we move to training our predictive model using **XGBoost (eXtreme Gradient Boosting)**. XGBoost is a powerful and efficient machine learning algorithm widely used for structured/tabular data problems. We'll train a regressor to predict Lap_Time_Seconds.\n\n### XGBoost Parameters & Training Strategy:\n\n- **objective & eval_metric:** Set for regression with Root Mean Squared Error (RMSE) as our evaluation metric.\n\n- **Hyperparameters:** A set of tuned parameters (learning_rate, max_depth, subsample, colsample_bytree, min_child_weight, lambda) to control model complexity and prevent overfitting.\n\n- **tree_method: \"gpu_hist\":** Leverages GPU acceleration for significantly faster training, especially on large datasets.\n\n- **enable_categorical:** True: Allows XGBoost to handle category dtypes directly, optimizing performance and encoding.\n\n- **Early Stopping:** We train for a maximum of 5000 boosting rounds but will stop early if the validation RMSE doesn't improve for 200 consecutive rounds, preventing overfitting and saving computation time.\n\n\nThis setup aims to build a robust model that generalizes well to unseen data.","metadata":{}},{"id":"8fe48f01","cell_type":"code","source":"# XGBoost parameters & training \nparams = {\n    \"objective\": \"reg:squarederror\", # Objective for regression tasks (predicting continuous values)\n    \"eval_metric\": \"rmse\",           # Evaluation metric: Root Mean Squared Error\n    \"learning_rate\": 0.05,\n    \"max_depth\": 8,\n    \"subsample\": 0.8,\n    \"colsample_bytree\": 0.8,\n    \"min_child_weight\": 10,\n    \"lambda\": 3.0,\n    \"tree_method\": \"gpu_hist\",       # Use GPU for faster training if available\n    \"enable_categorical\": True,      # Enable native categorical feature handling\n    \"seed\": 42,\n}\n\nstart = time.time()\nmodel = xgb.train(\n    params,\n    dtrain, # Training data (DMatrix)\n    num_boost_round=5000, # Max boosting rounds\n    evals=[(dval, \"val\")], # Validation set for early stopping\n    early_stopping_rounds=200, # Stop if validation RMSE doesn't improve for 200 rounds\n    verbose_eval=250 # Print progress every 250 rounds\n)\n\nprint(f\"Training time: {time.time()-start:.1f} s\")\nprint(\"Best iteration:\", model.best_iteration)\nprint(\"Best val RMSE:\", model.best_score)","metadata":{"execution":{"iopub.status.busy":"2025-06-19T13:05:45.067590Z","iopub.execute_input":"2025-06-19T13:05:45.068167Z","iopub.status.idle":"2025-06-19T13:09:26.796608Z","shell.execute_reply.started":"2025-06-19T13:05:45.068144Z","shell.execute_reply":"2025-06-19T13:09:26.795899Z"},"papermill":{"duration":0.273629,"end_time":"2025-06-14T09:54:26.380266","exception":false,"start_time":"2025-06-14T09:54:26.106637","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"f0b062e2","cell_type":"code","source":"# Predict on test & create submission\ntest_preds = model.predict(dtest, iteration_range=(0, model.best_iteration + 1))\nsubmission = pd.DataFrame({ID_COL: test_df[ID_COL], TARGET: test_preds})\nsubmission.to_csv(\"submission_xgb.csv\", index=False)\nprint(\"submission_xgb.csv saved:\", submission.shape)\nsubmission.head()","metadata":{"execution":{"iopub.status.busy":"2025-06-19T13:09:30.795501Z","iopub.execute_input":"2025-06-19T13:09:30.795771Z","iopub.status.idle":"2025-06-19T13:09:34.991463Z","shell.execute_reply.started":"2025-06-19T13:09:30.795750Z","shell.execute_reply":"2025-06-19T13:09:34.990889Z"},"papermill":{"duration":0.022288,"end_time":"2025-06-14T09:54:26.409167","exception":false,"start_time":"2025-06-14T09:54:26.386879","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"e76ee4fd","cell_type":"code","source":"","metadata":{"papermill":{"duration":0.006236,"end_time":"2025-06-14T10:02:36.783339","exception":false,"start_time":"2025-06-14T10:02:36.777103","status":"completed"},"tags":[]},"outputs":[],"execution_count":null}]}