{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":94834,"databundleVersionId":11269999,"sourceType":"competition"}],"dockerImageVersionId":30918,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"This Python script provides a basic XGBoost regression solution for the Kaggle assignment. It begins by loading the training and test datasets, then processes categorical features using label encoding to convert them into numerical form. The script selects relevant numeric features while excluding the 'id' column and designating 'performance_score' as the target variable. It splits the data into training and testing sets and trains an XGBoost regressor with a limited number of estimators (25) and a maximum depth of 5 to balance model performance and efficiency. Finally, it evaluates the model using the R-squared (R²) metric, printing the result to indicate how well the model explains the variance in the target variable.","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import LabelEncoder\nfrom xgboost import XGBRegressor\nfrom sklearn.metrics import r2_score\n\n# Load the dataset\ntrain_file_path = \"/kaggle/input/app-of-gen-ai-deep-learning-wustl-spring-2025/train.csv\"\ntest_file_path = \"/kaggle/input/app-of-gen-ai-deep-learning-wustl-spring-2025/test.csv\"\n\ndf = pd.read_csv(train_file_path)\ndf_test = pd.read_csv(test_file_path)\n\n# Drop non-numeric columns except for the target\ndf_numeric = df.select_dtypes(include=['number']).copy()\n\n# Handle categorical columns by encoding them\ncategorical_cols = df.select_dtypes(include=['object']).columns\nlabel_encoders = {}\nfor col in categorical_cols:\n    label_encoders[col] = LabelEncoder()\n    df[col] = label_encoders[col].fit_transform(df[col].astype(str))\n    \n    if col in df_test.columns:\n        df_test[col] = df_test[col].astype(str)\n        df_test[col] = df_test[col].apply(lambda x: x if x in label_encoders[col].classes_ else 'unknown')\n        label_encoders[col].classes_ = np.append(label_encoders[col].classes_, 'unknown')\n        df_test[col] = label_encoders[col].transform(df_test[col])\n\n# Define features and target\nX = df.drop(columns=['id', 'performance_score'])  # Remove ID and target column\ny = df['performance_score']\n\n# Train/test split\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n\n# Train XGBoost model with limited complexity for efficiency\nmodel = XGBRegressor(objective='reg:squarederror', n_estimators=25, max_depth=5, random_state=42)\nmodel.fit(X_train, y_train)\n\n# Predictions\ny_pred = model.predict(X_test)\n\n# Compute R-squared\nr2 = r2_score(y_test, y_pred)\nprint(f\"R-squared: {r2:.4f}\")","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-03-03T04:34:59.029852Z","iopub.execute_input":"2025-03-03T04:34:59.030205Z","iopub.status.idle":"2025-03-03T04:35:01.247723Z","shell.execute_reply.started":"2025-03-03T04:34:59.030179Z","shell.execute_reply":"2025-03-03T04:35:01.246806Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"This code analyzes and visualizes the feature importance of the trained XGBoost model in the Kaggle assignment. It extracts the importance scores assigned by the model to each feature and stores them in a DataFrame, sorting them in descending order to highlight the most influential predictors. The script then generates a horizontal bar chart using Matplotlib to display the feature importance, making it easier to interpret which features contribute most to the model’s predictions. The y-axis is inverted to ensure the most important features appear at the top of the chart.","metadata":{}},{"cell_type":"code","source":"# Feature importance\nimportances = model.feature_importances_\nfeature_importance_df = pd.DataFrame({'Feature': X.columns, 'Importance': importances})\nfeature_importance_df = feature_importance_df.sort_values(by='Importance', ascending=False)\n\n# Display feature importance\nplt.figure(figsize=(10, 6))\nplt.barh(feature_importance_df['Feature'], feature_importance_df['Importance'])\nplt.xlabel(\"Importance\")\nplt.ylabel(\"Feature\")\nplt.title(\"Feature Importance in XGBoost Model\")\nplt.gca().invert_yaxis()\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-03T04:35:01.248844Z","iopub.execute_input":"2025-03-03T04:35:01.249133Z","iopub.status.idle":"2025-03-03T04:35:01.748720Z","shell.execute_reply.started":"2025-03-03T04:35:01.249109Z","shell.execute_reply":"2025-03-03T04:35:01.747432Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"This code prepares the Kaggle submission file for the competition by generating predictions on the test dataset using the trained XGBoost model. It first extracts the 'id' column from the test set to maintain proper identification in the submission. After dropping 'id' from the test features, it applies the trained model to predict performance_score for each test instance. The predictions are then stored in a new DataFrame, ensuring they are paired with the corresponding 'id'. Finally, the script saves the results as a CSV file (\"submission.csv\") in the format required by Kaggle.\n\n## How to Make a Kaggle Submission in the Notebook\n1. Run the notebook to generate submission.csv.\n2. Locate the file in the Kaggle notebook environment under the \"Output\" section.\n3. Go to the Kaggle competition page and navigate to the \"Submit Predictions\" tab.\n4. Upload the submission.csv file and confirm the submission.\n5. Check the leaderboard to see your model's performance ranking.","metadata":{}},{"cell_type":"code","source":"# Prepare Kaggle submission\ntest_ids = df_test['id']\ndf_test = df_test.drop(columns=['id'])\nsubmission_preds = model.predict(df_test)\n\nsubmission = pd.DataFrame({'id': test_ids, 'performance_score': submission_preds})\nsubmission.to_csv(\"submission.csv\", index=False)\nprint(\"Kaggle submission file 'submission.csv' created successfully.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-03T04:35:01.750347Z","iopub.execute_input":"2025-03-03T04:35:01.750750Z","iopub.status.idle":"2025-03-03T04:35:01.776316Z","shell.execute_reply.started":"2025-03-03T04:35:01.750716Z","shell.execute_reply":"2025-03-03T04:35:01.775158Z"}},"outputs":[],"execution_count":null}]}