{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.14","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":5260,"databundleVersionId":37827,"sourceType":"competition"}],"dockerImageVersionId":30786,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Forecasting Inventory Demand Using XGBoost\r\n\r\n## Introduction\r\n\r\nAccurate demand forecasting is pivotal for efficient inventory management, ensuring that products are available to meet consumer needs without overstocking. In this context, leveraging machine learning techniques can significantly enhance the precision of demand predictions. This article delves into a robust pipeline designed to forecast the weekly demand of products at specific stores using the XGBoost algorithm. By systematically preprocessing the data, engineering meaningful features, and training an optimized model, this approach aims to provide reliable demand forecasts based on historical sales transactions in Mexico.\r\n","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2024-11-19T06:57:11.977749Z","iopub.execute_input":"2024-11-19T06:57:11.978700Z","iopub.status.idle":"2024-11-19T06:57:13.351995Z","shell.execute_reply.started":"2024-11-19T06:57:11.978620Z","shell.execute_reply":"2024-11-19T06:57:13.350671Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Methodology\r\n\r\nThe forecasting pipeline is meticulously structured into several key steps, each contributing to the overall effectiveness of the model. Below is a detailed walkthrough of each stage:\r\n\r\n### Step 1: Data Loading and Initial Inspection\r\n\r\nThe process begins by importing essential libraries such as Pandas for data manipulation, NumPy for numerical operations, XGBoost for modeling, and utilities like `gc` and `zipfile` for memory management and handling zipped files, respectively.\r\n\r\nA custom function, `read_zip_csv`, is defined to streamline the reading of CSV files compressed within ZIP archives. This function allows selective column loading, which optimizes memory usage and speeds up data processing.\r\n\r\nThe training and test datasets are loaded using this function, specifically extracting relevant columns:\r\n- **Training Data**: Includes `Semana` (week), `Producto_ID` (product ID), `Cliente_ID` (client ID), and `Demanda_uni_equil` (demand).\r\n- **Test Data**: Comprises `Semana`, `id`, `Cliente_ID`, and `Producto_ID`.\r\n\r\nInitial inspections of the datasets' shapes and sample records are printed to verify successful loading and to understand the data structure.\r\n\r\n### Step 2: Data Filtering and Transformation\r\n\r\nTo focus on meaningful sales data, the training dataset is filtered to include only records from weeks greater than 3. This ensures that the model is trained on consistent and relevant temporal data.\r\n\r\nThe target variable, `Demanda_uni_equil`, undergoes a logarithmic transformation (`np.log1p`) to mitigate skewness and stabilize variance, which are common issues in sales data. This transformation enhances the model's ability to learn and predict demand accurately.\r\n\r\n### Step 3: Data Preparation for Merging\r\n\r\nTo facilitate feature engineering, the training and test datasets are prepared for merging:\r\n- An `id` column is added to the training data with a default value of 0.\r\n- The test data receives a `Demanda_uni_equil` column initialized to 0.\r\n- A `tst` column is introduced to differentiate between training (`0`) and test (`1`) data.\r\n\r\nA combined dataset, `rec_train`, is created by concatenating the week 9 records from the training data with the test data. This unified dataset serves as a foundation for aggregating features that capture historical demand patterns.\r\n\r\n### Step 4: Feature Engineering through Aggregations\r\n\r\nEffective feature engineering is crucial for enhancing model performance. The pipeline computes aggregated features based on historical data from weeks 3 to 8:\r\n\r\n1. **Client-Product Aggregation**:\r\n   - Calculates the mean demand and count of transactions for each client-product pair.\r\n   - These features (`mean_client_prod`, `count_client_prod`) capture individual client purchasing behaviors and product performance.\r\n\r\n2. **Product Aggregation**:\r\n   - Computes the mean demand and transaction count for each product across all clients.\r\n   - Features like `mean_prod` and `count_prod` reflect overall product popularity and demand consistency.\r\n\r\n3. **Client Aggregation**:\r\n   - Determines the mean demand and transaction count for each client.\r\n   - Features `mean_cliente` and `count_cliente` provide insights into individual client demand patterns.\r\n\r\nThese aggregated features are merged back into the combined dataset, enriching the data with valuable historical insights.\r\n\r\n### Step 5: Data Splitting and Model Training\r\n\r\nThe enriched dataset is split back into training and test sets using the `tst` indicator:\r\n- **Training Set**: Contains records with `tst` equal to 0.\r\n- **Test Set**: Comprises records with `tst` equal to 1.\r\n\r\nAn XGBoost model is then trained on the prepared training data:\r\n- The training features (`X_train`) and target (`y_train`) are converted into DMatrix format, which is optimized for XGBoost.\r\n- Model parameters are set to define the complexity and learning behavior:\r\n  - `max_depth`: Controls the maximum depth of trees.\r\n  - `eta`: Learning rate influencing the contribution of each tree.\r\n  - `objective`: Specifies the loss function for regression tasks.\r\n\r\nThe model is trained over 25 boosting rounds, balancing computational efficiency with model performance.\r\n\r\n### Step 6: Making Predictions and Preparing Submission\r\n\r\nPost-training, the model generates predictions on the test set:\r\n- Predictions are initially in the log-transformed scale and are converted back to the original scale using the inverse transformation (`np.expm1`).\r\n- Negative predictions are rectified to zero to ensure non-negative demand values.\r\n\r\nA submission file is created by aligning the predicted demand with the corresponding `id` from the test set. This file is formatted as per competition requirements and saved for submission.\r\n","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport xgboost as xgb\nimport gc\nimport zipfile\n\n# ---------------------------------------------------\n# Step 1: Define a Function to Read Zipped CSV Files\n# ---------------------------------------------------\n\ndef read_zip_csv(zip_path, usecols=None):\n    \"\"\"\n    Reads a CSV file from a zip archive.\n\n    Parameters:\n    - zip_path (str): Path to the zip file.\n    - usecols (list, optional): List of columns to read from the CSV.\n\n    Returns:\n    - pd.DataFrame: DataFrame containing the CSV data.\n    \"\"\"\n    with zipfile.ZipFile(zip_path) as z:\n        # Get the first CSV filename from inside the zip archive\n        csv_filename = z.namelist()[0]\n        with z.open(csv_filename) as f:\n            # Read the CSV file into a DataFrame, selecting only specified columns if provided\n            return pd.read_csv(f, usecols=usecols)\n\n# ---------------------------------------------------\n# Step 2: Load the Training and Test Data\n# ---------------------------------------------------\n\n# Read the training data from the zipped CSV file, selecting relevant columns\ntrain = read_zip_csv('/kaggle/input/grupo-bimbo-inventory-demand/train.csv.zip', \n                    usecols=['Semana', 'Producto_ID', 'Cliente_ID', 'Demanda_uni_equil'])\nprint(\"\\nInitial train data shape:\", train.shape)\nprint(\"\\nSample of initial train data:\")\nprint(train.head())\n\n# Read the test data from the zipped CSV file, selecting relevant columns\ntest = read_zip_csv('/kaggle/input/grupo-bimbo-inventory-demand/test.csv.zip', \n                   usecols=['Semana', 'id', 'Cliente_ID', 'Producto_ID'])\nprint(\"\\nTest data shape:\", test.shape)\nprint(\"\\nSample of test data:\")\nprint(test.head())\n\n# ---------------------------------------------------\n# Step 3: Filter Training Data for Weeks > 3\n# ---------------------------------------------------\n\n# Retain only the rows where 'Semana' (week) is greater than 3 to focus on relevant weeks\ntrain = train[train['Semana'] > 3].copy()\nprint(\"\\nTrain data shape after filtering weeks > 3:\", train.shape)\n\n# ---------------------------------------------------\n# Step 4: Log Transform the Target Variable\n# ---------------------------------------------------\n\n# Display statistics of 'Demanda_uni_equil' before applying log transformation\nprint(\"\\nDemanda_uni_equil statistics before log transform:\")\nprint(train['Demanda_uni_equil'].describe())\n\n# Apply log1p transformation to 'Demanda_uni_equil' to stabilize variance and handle skewness\ntrain['Demanda_uni_equil'] = np.log1p(train['Demanda_uni_equil'])\nprint(\"\\nDemanda_uni_equil statistics after log transform:\")\nprint(train['Demanda_uni_equil'].describe())\n\n# ---------------------------------------------------\n# Step 5: Prepare for Merging Training and Test Data\n# ---------------------------------------------------\n\n# Add an 'id' column to the training data with a default value of 0\ntrain['id'] = 0\n\n# Add a 'Demanda_uni_equil' column to the test data with a default value of 0\ntest['Demanda_uni_equil'] = 0\n\n# Add a 'tst' column to indicate whether a row is from the test set (1) or training set (0)\ntrain['tst'] = 0\ntest['tst'] = 1\n\n# ---------------------------------------------------\n# Step 6: Combine Training Week 9 and Test Data\n# ---------------------------------------------------\n\n# Concatenate the training data for week 9 with the test data to create a combined dataset\nrec_train = pd.concat([train[train['Semana'] == 9], test])\nprint(\"\\nCombined train (week 9) and test shape:\", rec_train.shape)\n\n# Free up memory by deleting the test DataFrame and invoking garbage collection\ndel test\ngc.collect()\n\n# ---------------------------------------------------\n# Step 7: Aggregate Features for Weeks 3-8\n# ---------------------------------------------------\n\n# Aggregate mean demand and count by 'Producto_ID' and 'Cliente_ID' for weeks <= 8\nclient_prod_agg = train[train['Semana'] <= 8].groupby(['Producto_ID', 'Cliente_ID']).agg({\n    'Demanda_uni_equil': ['mean', 'count']\n}).reset_index()\n# Rename the aggregated columns for clarity\nclient_prod_agg.columns = ['Producto_ID', 'Cliente_ID', 'mean_client_prod', 'count_client_prod']\nprint(\"\\nClient-Product aggregation sample:\")\nprint(client_prod_agg.head())\n\n# Merge the aggregated client-product features with the combined training and test data\nrec_train = rec_train.merge(client_prod_agg, \n                           on=['Producto_ID', 'Cliente_ID'], \n                           how='left')\n\n# ---------------------------------------------------\n# Step 8: Aggregate Features by Product\n# ---------------------------------------------------\n\n# Aggregate mean demand and count by 'Producto_ID' for weeks <= 8\nprod_agg = train[train['Semana'] <= 8].groupby('Producto_ID').agg({\n    'Demanda_uni_equil': ['mean', 'count']\n}).reset_index()\n# Rename the aggregated columns for clarity\nprod_agg.columns = ['Producto_ID', 'mean_prod', 'count_prod']\nprint(\"\\nProduct aggregation sample:\")\nprint(prod_agg.head())\n\n# Merge the aggregated product features with the combined training and test data\nrec_train = rec_train.merge(prod_agg, \n                           on='Producto_ID', \n                           how='left')\n\n# ---------------------------------------------------\n# Step 9: Aggregate Features by Client\n# ---------------------------------------------------\n\n# Aggregate mean demand and count by 'Cliente_ID' for weeks <= 8\nclient_agg = train[train['Semana'] <= 8].groupby('Cliente_ID').agg({\n    'Demanda_uni_equil': ['mean', 'count']\n}).reset_index()\n# Rename the aggregated columns for clarity\nclient_agg.columns = ['Cliente_ID', 'mean_cliente', 'count_cliente']\nprint(\"\\nClient aggregation sample:\")\nprint(client_agg.head())\n\n# Merge the aggregated client features with the combined training and test data\nrec_train = rec_train.merge(client_agg, \n                           on='Cliente_ID', \n                           how='left')\n\n# Free up memory by deleting the train DataFrame and invoking garbage collection\ndel train\ngc.collect()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-11-19T07:07:45.988056Z","iopub.execute_input":"2024-11-19T07:07:45.988595Z","iopub.status.idle":"2024-11-19T07:09:54.619298Z","shell.execute_reply.started":"2024-11-19T07:07:45.988549Z","shell.execute_reply":"2024-11-19T07:09:54.618217Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# ---------------------------------------------------\n# Step 10: Prepare Feature Columns for Modeling\n# ---------------------------------------------------\n\n# Define the list of feature columns to be used for training the model\nfeature_cols = ['Semana', 'Producto_ID', 'Cliente_ID', \n                'mean_client_prod', 'count_client_prod',\n                'mean_prod', 'count_prod',\n                'mean_cliente', 'count_cliente']\n\n# ---------------------------------------------------\n# Step 11: Split the Combined Data into Training and Test Sets\n# ---------------------------------------------------\n\n# Create a boolean mask to identify training data (tst == 0)\ntrain_mask = rec_train['tst'] == 0\n\n# Extract the target variable for training\ny_train = rec_train.loc[train_mask, 'Demanda_uni_equil']\n\n# Extract the feature matrix for training\nX_train = rec_train.loc[train_mask, feature_cols]\n\n# Extract the feature matrix for testing\nX_test = rec_train.loc[~train_mask, feature_cols]\n\nprint(\"\\nTraining data shape:\", X_train.shape)\nprint(\"Test data shape:\", X_test.shape)\n\n# ---------------------------------------------------\n# Step 12: Train the XGBoost Model\n# ---------------------------------------------------\n\n# Convert the training data into DMatrix format, which is optimized for XGBoost\ndtrain = xgb.DMatrix(X_train, label=y_train)\n\n# Define the parameters for the XGBoost model\nparams = {\n    'max_depth': 8,                  # Maximum depth of a tree\n    'eta': 0.5,                      # Learning rate\n    'objective': 'reg:squarederror'  # Objective function for regression\n}\n\n# Train the XGBoost model with the specified parameters and number of boosting rounds\nmodel = xgb.train(params, dtrain, num_boost_round=25)\n\n# ---------------------------------------------------\n# Step 13: Make Predictions with the Trained Model\n# ---------------------------------------------------\n\n# Convert the test data into DMatrix format\ndtest = xgb.DMatrix(X_test)\n\n# Predict the log-transformed demand using the trained model\npreds = model.predict(dtest)\nprint(\"\\nPredictions before transformations:\")\nprint(\"Mean:\", np.mean(preds))\nprint(\"Min:\", np.min(preds))\nprint(\"Max:\", np.max(preds))\n\n# ---------------------------------------------------\n# Step 14: Transform Predictions Back to Original Scale\n# ---------------------------------------------------\n\n# Apply the inverse of the log1p transformation to revert predictions to the original scale\npreds = np.expm1(preds)  # Transform back from log scale\n\n# Ensure that all predictions are non-negative\npreds = np.maximum(preds, 0)  # Ensure non-negative predictions\n\nprint(\"\\nPredictions after transformations:\")\nprint(\"Mean:\", np.mean(preds))\nprint(\"Min:\", np.min(preds))\nprint(\"Max:\", np.max(preds))\n\n# ---------------------------------------------------\n# Step 15: Create the Submission File\n# ---------------------------------------------------\n\n# Create a DataFrame for submission with 'id' and the predicted 'Demanda_uni_equil'\nsubmission = pd.DataFrame({\n    'id': rec_train.loc[~train_mask, 'id'].astype(int),\n    'Demanda_uni_equil': preds\n})\n\nprint(\"\\nSubmission sample:\")\nprint(submission.head())\nprint(\"\\nSubmission statistics:\")\nprint(submission['Demanda_uni_equil'].describe())\n\n# Save the submission DataFrame to a CSV file without the index\nsubmission.to_csv('/kaggle/working/submission.csv', index=False)\nprint(\"\\nSubmission saved to 'submission.csv'\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-11-19T07:14:09.483740Z","iopub.execute_input":"2024-11-19T07:14:09.484152Z","iopub.status.idle":"2024-11-19T07:14:52.106142Z","shell.execute_reply.started":"2024-11-19T07:14:09.484121Z","shell.execute_reply":"2024-11-19T07:14:52.105011Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Conclusion\r\n\r\nForecasting product demand is a multifaceted challenge that necessitates meticulous data handling, insightful feature engineering, and robust modeling techniques. This pipeline demonstrates an effective approach using XGBoost, underpinned by strategic data preprocessing and aggregation. By transforming the target variable, engineering aggregated features, and optimizing the model's parameters, the pipeline achieves reliable demand predictions tailored to specific products and clients.\r\n\r\nThe integration of memory management practices ensures that the pipeline remains efficient even when dealing with large datasets. Furthermore, the structured approach to feature engineering captures nuanced demand patterns, enhancing the model's predictive capabilities.\r\n","metadata":{}}]}