{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":66912,"databundleVersionId":7438099,"sourceType":"competition"}],"dockerImageVersionId":30635,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 📋Table of Contents\n- [1. INtroduction](#1)\n- [2. 📚 Import Libraries](#2)\n- [3. 📖 Read Dataset](#3)\n- [4. 😅😅Missing Values](#4)\n- [5. EDA](#5)\n- [6. Conclusion conducting of EDA](#6)\n- [7. 🖇🖇📭Data Pipeline](#7)\n- [8. Model Building](#8)\n- [9. 📊Generate Predictions and Create Submission File](#9)","metadata":{}},{"cell_type":"markdown","source":"# <center> <a id=\"1\">INtroduction<center>\n    \n","metadata":{}},{"cell_type":"markdown","source":"**HI, If YOu clicked on this notebook on the insights of the data by EDA jump direct  [here](#6), if you want you can have whole eye on the notebook, you can fork (create a version of yourself) this thing **","metadata":{}},{"cell_type":"markdown","source":"Within the context of Eleme delivery service, at every moment the command platform of “Smart Logistics” sends orders from customers to delivery-men for instant delivery. The decisions being made by the delivery-men are mainly two-fold: \n-   (i) pick-up from a store, \n-   (ii) delivery to a customer. \n\n![https://www.livemint.com/lm-img/img/2023/05/07/1600x900/Food_Delivery_1683478114249_1683478114434.jpg](https://www.livemint.com/lm-img/img/2023/05/07/1600x900/Food_Delivery_1683478114249_1683478114434.jpg)**At any specific moment, a delivery-man may receive new orders assigned to him, while he has some unfinished orders which were previously assigned. In this situation, he needs to decide what to do next (to pick up a new order or to deliver an old one), based on his order status and geographical location.**","metadata":{}},{"cell_type":"markdown","source":"# <center> <a id=\"2\"> 📚 Import Libraries","metadata":{}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.impute import SimpleImputer\nfrom sklearn.preprocessing import StandardScaler, OneHotEncoder, LabelEncoder\nfrom sklearn.compose import ColumnTransformer\nfrom sklearn.model_selection import train_test_split","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2024-01-15T05:50:20.396472Z","iopub.execute_input":"2024-01-15T05:50:20.396786Z","iopub.status.idle":"2024-01-15T05:50:21.252251Z","shell.execute_reply.started":"2024-01-15T05:50:20.396760Z","shell.execute_reply":"2024-01-15T05:50:21.251352Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <center><a id=\"3\"> 📖 Read Dataset","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv('/kaggle/input/ba2023-regression/dataframe_train.csv')\ntest = pd.read_csv('/kaggle/input/ba2023-regression/dataframe_test.csv')","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:50:21.253580Z","iopub.execute_input":"2024-01-15T05:50:21.254022Z","iopub.status.idle":"2024-01-15T05:50:23.790926Z","shell.execute_reply.started":"2024-01-15T05:50:21.253990Z","shell.execute_reply":"2024-01-15T05:50:23.790075Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <center> <a id=\"4\">😅😅Missing Values","metadata":{}},{"cell_type":"code","source":"print(train.isnull().sum())\ntest.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:50:23.792289Z","iopub.execute_input":"2024-01-15T05:50:23.792571Z","iopub.status.idle":"2024-01-15T05:50:23.888357Z","shell.execute_reply.started":"2024-01-15T05:50:23.792542Z","shell.execute_reply":"2024-01-15T05:50:23.887563Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> Wow that's miracle there's no missing values yipee!!, except ofc 'shop_id'\nhaving 4 ..  but can solve it easily","metadata":{}},{"cell_type":"code","source":"print(test['source_tracking_id'].value_counts())\n\n\nyour_column_name = 'source_tracking_id'\n\nmost_frequent_value = test[your_column_name].mode()[0]\ntest[your_column_name] = pd.to_numeric(test[your_column_name].replace({'PICKUP': most_frequent_value, 'ASSIGN': most_frequent_value,\n                                                    'DELIVERY': most_frequent_value}))\n\ntest['source_tracking_id'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:50:36.706722Z","iopub.execute_input":"2024-01-15T05:50:36.707660Z","iopub.status.idle":"2024-01-15T05:50:36.744779Z","shell.execute_reply.started":"2024-01-15T05:50:36.707624Z","shell.execute_reply":"2024-01-15T05:50:36.743987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <center> <a id=\"5\">EDA","metadata":{}},{"cell_type":"code","source":"# Create a deep copy of train and name it data\ndata = train.copy()","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:32:18.964867Z","iopub.execute_input":"2024-01-15T05:32:18.965296Z","iopub.status.idle":"2024-01-15T05:32:19.003834Z","shell.execute_reply.started":"2024-01-15T05:32:18.965260Z","shell.execute_reply":"2024-01-15T05:32:19.002596Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data.describe().T","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:32:19.284870Z","iopub.execute_input":"2024-01-15T05:32:19.285523Z","iopub.status.idle":"2024-01-15T05:32:19.639304Z","shell.execute_reply.started":"2024-01-15T05:32:19.285486Z","shell.execute_reply":"2024-01-15T05:32:19.638196Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data.info()","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:32:19.641472Z","iopub.execute_input":"2024-01-15T05:32:19.642089Z","iopub.status.idle":"2024-01-15T05:32:19.936435Z","shell.execute_reply.started":"2024-01-15T05:32:19.642041Z","shell.execute_reply":"2024-01-15T05:32:19.935175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data.head(4).T","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:32:19.941058Z","iopub.execute_input":"2024-01-15T05:32:19.941460Z","iopub.status.idle":"2024-01-15T05:32:19.959560Z","shell.execute_reply.started":"2024-01-15T05:32:19.941426Z","shell.execute_reply":"2024-01-15T05:32:19.958360Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\ndata.hist(bins=50, figsize=(20,15))\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:32:20.175049Z","iopub.execute_input":"2024-01-15T05:32:20.175843Z","iopub.status.idle":"2024-01-15T05:32:25.839244Z","shell.execute_reply.started":"2024-01-15T05:32:20.175779Z","shell.execute_reply":"2024-01-15T05:32:25.837733Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data.plot(kind=\"scatter\", x=\"target_lng\", y=\"target_lat\", alpha=0.4, label='hour', figsize=(10,7),\n c='hour', cmap=plt.get_cmap(\"jet\"), colorbar=True,\n)\nplt.legend()","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:32:25.841233Z","iopub.execute_input":"2024-01-15T05:32:25.841709Z","iopub.status.idle":"2024-01-15T05:32:54.266996Z","shell.execute_reply.started":"2024-01-15T05:32:25.841653Z","shell.execute_reply":"2024-01-15T05:32:54.265676Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data.plot(kind=\"scatter\", x=\"source_lng\", y=\"source_lat\", alpha=0.4, label='hour', figsize=(10,7),\n c='hour', cmap=plt.get_cmap(\"jet\"), colorbar=True,\n)\nplt.legend()","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:32:54.268289Z","iopub.execute_input":"2024-01-15T05:32:54.268640Z","iopub.status.idle":"2024-01-15T05:33:22.963332Z","shell.execute_reply.started":"2024-01-15T05:32:54.268607Z","shell.execute_reply":"2024-01-15T05:33:22.962093Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data.plot(kind=\"scatter\", x=\"courier_wave_start_lng\", y=\"courier_wave_start_lat\", alpha=0.4, label='hour', figsize=(10,7),\n c='hour', cmap=plt.get_cmap(\"jet\"), colorbar=True,\n)\nplt.legend()","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:33:22.967011Z","iopub.execute_input":"2024-01-15T05:33:22.967407Z","iopub.status.idle":"2024-01-15T05:33:49.859007Z","shell.execute_reply.started":"2024-01-15T05:33:22.967372Z","shell.execute_reply":"2024-01-15T05:33:49.857831Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num = data.select_dtypes(include=['int64', 'float64']).columns\ncol = data.select_dtypes(include=['object']).columns\n\ncorr_matrix = data[num].corr()\ncorr_matrix['expected_use_time'].sort_values(ascending=False)","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:33:49.860338Z","iopub.execute_input":"2024-01-15T05:33:49.860705Z","iopub.status.idle":"2024-01-15T05:33:50.598551Z","shell.execute_reply.started":"2024-01-15T05:33:49.860669Z","shell.execute_reply":"2024-01-15T05:33:50.597251Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Categorical data unique values\nfor i in col:\n    print(f'The unique of length of {i} column is :{len(data[i].unique())}')","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:33:50.600221Z","iopub.execute_input":"2024-01-15T05:33:50.600596Z","iopub.status.idle":"2024-01-15T05:33:50.890101Z","shell.execute_reply.started":"2024-01-15T05:33:50.600563Z","shell.execute_reply":"2024-01-15T05:33:50.888919Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***There now for  columns that we are going to use Label encoder there else face the curse of dimensionality by using Onehotencoder:***\n","metadata":{}},{"cell_type":"code","source":"len(data['tracking_id'].unique())","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:33:50.891698Z","iopub.execute_input":"2024-01-15T05:33:50.892251Z","iopub.status.idle":"2024-01-15T05:33:50.905952Z","shell.execute_reply.started":"2024-01-15T05:33:50.892205Z","shell.execute_reply":"2024-01-15T05:33:50.904591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(data['level'].unique())","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:33:50.907671Z","iopub.execute_input":"2024-01-15T05:33:50.908351Z","iopub.status.idle":"2024-01-15T05:33:50.921018Z","shell.execute_reply.started":"2024-01-15T05:33:50.908290Z","shell.execute_reply":"2024-01-15T05:33:50.919568Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(data['tracking_id'].unique())","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:33:50.922638Z","iopub.execute_input":"2024-01-15T05:33:50.923642Z","iopub.status.idle":"2024-01-15T05:33:50.936781Z","shell.execute_reply.started":"2024-01-15T05:33:50.923601Z","shell.execute_reply":"2024-01-15T05:33:50.935494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(data['wave_index'].unique())","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:33:50.940224Z","iopub.execute_input":"2024-01-15T05:33:50.940641Z","iopub.status.idle":"2024-01-15T05:33:50.952746Z","shell.execute_reply.started":"2024-01-15T05:33:50.940605Z","shell.execute_reply":"2024-01-15T05:33:50.951334Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"*WHy they all have categorical kind of data???*","metadata":{}},{"cell_type":"code","source":"len(data['date'].unique())","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:33:50.957444Z","iopub.execute_input":"2024-01-15T05:33:50.957884Z","iopub.status.idle":"2024-01-15T05:33:50.970442Z","shell.execute_reply.started":"2024-01-15T05:33:50.957845Z","shell.execute_reply":"2024-01-15T05:33:50.969076Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So the data is only for 27 days only ","metadata":{}},{"cell_type":"code","source":"len(data['id'].unique())\n\n# we going to drop id here as every values is diff.\ndata.drop('id', inplace = True, axis = 1)","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:33:50.974798Z","iopub.execute_input":"2024-01-15T05:33:50.975221Z","iopub.status.idle":"2024-01-15T05:33:51.068791Z","shell.execute_reply.started":"2024-01-15T05:33:50.975182Z","shell.execute_reply":"2024-01-15T05:33:51.067401Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <center> <a id=\"6\"> Conclusion conducting of EDA\n\n1. The data mostly no outliners , bcoz they **preprocessed already mentioned by competition organizer and there are also no missing values for that reason**\n2. There are  categorical column that will need to use **LabelEncoder instead of normal OnehotEncoder , bcoz they are just too many of them , to use the OnhotEncoder or else we will face curse of dimensionality**\n3. Each **'Id' Column is unique** so we are dropping it \n4. I found one more feature add as we are dropping one **dis_hour , is our guy for train and test set by feature engineering**\n5. ANd most **numerical id column have categorical kind of data as their values uniqueness is only upto 20 unique columns**\n6. The data **we have is only of 27 days**\n7. The thing is fact that we have 4 strings data inside the numerical column in the test set which I replaced with one value, and replaced with most frequent value ","metadata":{}},{"cell_type":"markdown","source":"# <center> <a id=\"7\">🖇🖇📭Data Pipeline","metadata":{}},{"cell_type":"code","source":"train.drop(['id','action_type'], inplace = True, axis = 1)\ntest.drop('id', inplace =True, axis = 1)\n\ntrain['dis_hour'] = train['grid_distance'] * train['hour']\ntest['dis_hour'] = test['grid_distance'] * test['hour']\n","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:50:48.471064Z","iopub.execute_input":"2024-01-15T05:50:48.471372Z","iopub.status.idle":"2024-01-15T05:50:48.519522Z","shell.execute_reply.started":"2024-01-15T05:50:48.471350Z","shell.execute_reply":"2024-01-15T05:50:48.518699Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y = train['expected_use_time']\nX = train.drop('expected_use_time', axis = 1)","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:50:50.516304Z","iopub.execute_input":"2024-01-15T05:50:50.516619Z","iopub.status.idle":"2024-01-15T05:50:50.550447Z","shell.execute_reply.started":"2024-01-15T05:50:50.516593Z","shell.execute_reply":"2024-01-15T05:50:50.549719Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num = X.select_dtypes(include=['int64', 'float64']).columns\ncol = X.select_dtypes(include=['object']).columns","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:50:58.775827Z","iopub.execute_input":"2024-01-15T05:50:58.776545Z","iopub.status.idle":"2024-01-15T05:50:58.827791Z","shell.execute_reply.started":"2024-01-15T05:50:58.776516Z","shell.execute_reply":"2024-01-15T05:50:58.826665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Preprocessing for numerical data: imputation and scaling\nnumerical_transformer = Pipeline(steps=[\n    ('imputer', SimpleImputer(strategy='median')),\n    ('scaler', StandardScaler())])\n\n# Preprocessing for categorical data: imputation and one-hot encoding\ncategorical_transformer = Pipeline(steps=[\n    ('imputer', SimpleImputer(strategy='most_frequent')),\n    ('onehot', OneHotEncoder(handle_unknown='ignore', drop='first', max_categories = 20))])\n\npreprocessor = ColumnTransformer(\n    transformers=[\n        ('num', numerical_transformer, num),\n        ('cat', categorical_transformer, col)])","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:52:24.370771Z","iopub.execute_input":"2024-01-15T05:52:24.371146Z","iopub.status.idle":"2024-01-15T05:52:24.376366Z","shell.execute_reply.started":"2024-01-15T05:52:24.371119Z","shell.execute_reply":"2024-01-15T05:52:24.375383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <center> <a id = 8>Model Building","metadata":{}},{"cell_type":"code","source":"import optuna\nimport xgboost as xgb\nfrom sklearn.metrics import mean_absolute_error as mae\n\n# Split the data into training and validation sets\nX_train, X_valid, y_train, y_valid = train_test_split(X, y, test_size=0.2, random_state=42)\n\n# # Define the objective function for optimization\n# def objective(trial):\n#     params = {\n#         'objective': 'reg:squarederror',\n#         'booster': 'gbtree',\n#         'eval_metric': 'mae',\n#         'max_depth': trial.suggest_int('max_depth', 3, 10),\n#         'learning_rate': trial.suggest_loguniform('learning_rate', 0.001, 0.1),\n#         'n_estimators': trial.suggest_int('n_estimators', 100, 1000),\n#         'subsample': trial.suggest_float('subsample', 0.5, 1.0),\n#         'colsample_bytree': trial.suggest_float('colsample_bytree', 0.5, 1.0),\n#         'reg_alpha': trial.suggest_float('reg_alpha', 0.0, 1.0),\n#         'reg_lambda': trial.suggest_float('reg_lambda', 0.0, 1.0),\n#         'gamma': trial.suggest_float('gamma', 0.0, 1.0),\n#     }\n#     model = xgb.XGBRegressor(**params)\n#     model.fit(X_train, y_train)\n#     predictions = model.predict(X_valid)\n#     mae = mean_absolute_error(y_valid, predictions)\n#     return mae\n\n# # Create a study object and optimize the objective function\n# study = optuna.create_study(direction='minimize')  # Use 'minimize' since we are dealing with MAE\n# study.optimize(objective, n_trials=1500)\n\n# # Get the best parameters\n# best_params = study.best_params\n# print(\"Best Parameters for XGBoost:\", best_params)\n\n# Train the final model with the best parameters\nfinal_model = xgb.XGBRegressor(**{'objective': 'reg:squarederror','booster': 'gbtree',\n                                'eval_metric': 'mae','max_depth': 10, 'learning_rate': 0.03580576725737922, \n                                'n_estimators': 941, 'subsample': 0.9540139576391355, \n                                'colsample_bytree': 0.6994445410476209, 'reg_alpha': 0.018238556911153413, \n                                'reg_lambda': 0.890724954285552, 'gamma': 0.22590152992571608})\n\nXBG_pipeline = Pipeline(steps=[('preprocessor', preprocessor), ('model', final_model)])\n\nXBG_pipeline.fit(X, y)\n\npred = XBG_pipeline.predict(X_valid)\nloss = mae(y_valid , pred)\nloss\n","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:52:43.355811Z","iopub.execute_input":"2024-01-15T05:52:43.356644Z","iopub.status.idle":"2024-01-15T05:54:38.626981Z","shell.execute_reply.started":"2024-01-15T05:52:43.356615Z","shell.execute_reply":"2024-01-15T05:54:38.626208Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#153.61813026394515 XBG\n#168.6465001258407 LGBM\n# 172.95826607653774 CAT\n# 170.3026608874462 ran","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <center> <a id = 9> 📊Generate Predictions and Create Submission File","metadata":{}},{"cell_type":"code","source":"final_pred = XBG_pipeline.predict(test)","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:55:15.850807Z","iopub.execute_input":"2024-01-15T05:55:15.851146Z","iopub.status.idle":"2024-01-15T05:55:16.581572Z","shell.execute_reply.started":"2024-01-15T05:55:15.851121Z","shell.execute_reply":"2024-01-15T05:55:16.580612Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub = pd.read_csv('/kaggle/input/ba2023-regression/regression_example.csv')\n\nsub['expected_use_time'] = final_pred\n\nsub.to_csv('submission.csv', index = False)","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:55:16.583028Z","iopub.execute_input":"2024-01-15T05:55:16.583279Z","iopub.status.idle":"2024-01-15T05:55:16.656550Z","shell.execute_reply.started":"2024-01-15T05:55:16.583255Z","shell.execute_reply":"2024-01-15T05:55:16.655699Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub.head()","metadata":{"execution":{"iopub.status.busy":"2024-01-15T05:55:17.810410Z","iopub.execute_input":"2024-01-15T05:55:17.810733Z","iopub.status.idle":"2024-01-15T05:55:17.820744Z","shell.execute_reply.started":"2024-01-15T05:55:17.810706Z","shell.execute_reply":"2024-01-15T05:55:17.819976Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"*Plz, don't forget to upvote this , it takes time and work hard to build these things Open Source ....\nThank you , if you did*😊😊","metadata":{}}]}