{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.11","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":105343,"databundleVersionId":12700041,"sourceType":"competition"}],"dockerImageVersionId":31040,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"This Python script provides a baseline XGBoost regression solution for the “Applications of Deep Learning (WUSTL, Summer 2025)” Kaggle competition. It begins by loading the competition’s train.csv and test.csv files from the /kaggle/input/applications-of-deep-learning-wustl-summer-2025/ directory. Categorical features are converted to numeric form via LabelEncoder, and the remaining numeric columns are selected while dropping the id column and setting preservation_score as the target variable. The data is then split into training and validation sets. An XGBRegressor is trained with n_estimators=25 and max_depth=5 to balance model complexity and efficiency. Finally, the model is evaluated on the validation set using Root Mean Squared Error (RMSE), with the resulting score printed to indicate the average prediction error for artifact preservation scores","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import LabelEncoder\nfrom xgboost import XGBRegressor\nfrom sklearn.metrics import mean_squared_error\n\n# Load the dataset\ntrain_file_path = \"/kaggle/input/applications-of-deep-learning-wustl-summer-2025/train.csv\"\ntest_file_path  = \"/kaggle/input/applications-of-deep-learning-wustl-summer-2025/test.csv\"\n\ndf      = pd.read_csv(train_file_path)\ndf_test = pd.read_csv(test_file_path)\n\n# Handle categorical columns by encoding them\ncategorical_cols = df.select_dtypes(include=['object']).columns\nlabel_encoders  = {}\n\nfor col in categorical_cols:\n    le = LabelEncoder()\n    df[col] = le.fit_transform(df[col].astype(str))\n    label_encoders[col] = le\n\n    # apply same encoding (with 'unknown' for unseen) to test set\n    if col in df_test.columns:\n        df_test[col] = df_test[col].astype(str)\n        df_test[col] = df_test[col].apply(lambda x: x if x in le.classes_ else 'unknown')\n        le.classes_ = np.append(le.classes_, 'unknown')\n        df_test[col] = le.transform(df_test[col])\n\n# Define features and target\nX = df.drop(columns=['id', 'preservation_score'])\ny = df['preservation_score']\n\n# Train/test split\nX_train, X_test, y_train, y_test = train_test_split(\n    X, y, test_size=0.2, random_state=42\n)\n\n# Train XGBoost model\nmodel = XGBRegressor(\n    objective='reg:squarederror',\n    n_estimators=25,\n    max_depth=5,\n    random_state=42\n)\nmodel.fit(X_train, y_train)\n\n# Predictions\ny_pred = model.predict(X_test)\n\n# Compute RMSE\nrmse = np.sqrt(mean_squared_error(y_test, y_pred))\nprint(f\"RMSE: {rmse:.4f}\")\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-07-06T20:34:25.202816Z","iopub.execute_input":"2025-07-06T20:34:25.203215Z","iopub.status.idle":"2025-07-06T20:34:26.040747Z","shell.execute_reply.started":"2025-07-06T20:34:25.203177Z","shell.execute_reply":"2025-07-06T20:34:26.039916Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"This code extracts and visualizes the feature‐importance scores from the XGBoost regressor trained for the “Applications of Deep Learning (WUSTL, Summer 2025)” Kaggle competition. It begins by calling model.feature_importances_ on the fitted XGBRegressor to retrieve each input feature’s relative importance in predicting the preservation_score. Those scores are paired with their corresponding feature names in a pandas DataFrame, then sorted in descending order so the most influential predictors rise to the top. Finally, the script uses Matplotlib to plot a horizontal bar chart—annotating each bar with its importance score and inverting the y-axis so the highest‐importance features appear at the top—making it easy to see which variables contribute most to the model’s predictions.","metadata":{}},{"cell_type":"code","source":"# Feature importance\nimportances = model.feature_importances_\nfeature_importance_df = pd.DataFrame({'Feature': X.columns, 'Importance': importances})\nfeature_importance_df = feature_importance_df.sort_values(by='Importance', ascending=False)\n\n# Display feature importance\nplt.figure(figsize=(10, 6))\nplt.barh(feature_importance_df['Feature'], feature_importance_df['Importance'])\nplt.xlabel(\"Importance\")\nplt.ylabel(\"Feature\")\nplt.title(\"Feature Importance in XGBoost Model\")\nplt.gca().invert_yaxis()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-06T20:35:02.362212Z","iopub.execute_input":"2025-07-06T20:35:02.363153Z","iopub.status.idle":"2025-07-06T20:35:03.000477Z","shell.execute_reply.started":"2025-07-06T20:35:02.363121Z","shell.execute_reply":"2025-07-06T20:35:02.999305Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"This code prepares the Kaggle submission file for the competition by generating predictions on the test dataset using the trained XGBoost model. It first extracts the 'id' column from the test set to maintain proper identification in the submission. After dropping 'id' from the test features, it applies the trained model to predict performance_score for each test instance. The predictions are then stored in a new DataFrame, ensuring they are paired with the corresponding 'id'. Finally, the script saves the results as a CSV file (\"submission.csv\") in the format required by Kaggle.\n\n# How to Make a Kaggle Submission in the Notebook\n1. Run the notebook to generate submission.csv.\n2. Locate the file in the Kaggle notebook environment under the \"Output\" section.\n3. Go to the Kaggle competition page and navigate to the \"Submit Predictions\" tab.\n4. Upload the submission.csv file and confirm the submission.\n5. Check the leaderboard to see your model's performance ranking.","metadata":{}},{"cell_type":"code","source":"# Prepare Kaggle submission\ntest_ids = df_test['id']\ndf_test = df_test.drop(columns=['id'])\nsubmission_preds = model.predict(df_test)\n\nsubmission = pd.DataFrame({'id': test_ids, 'preservation_score': submission_preds})\nsubmission.to_csv(\"submission.csv\", index=False)\nprint(\"Kaggle submission file 'submission.csv' created successfully.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-06T20:35:36.931156Z","iopub.execute_input":"2025-07-06T20:35:36.931513Z","iopub.status.idle":"2025-07-06T20:35:36.957551Z","shell.execute_reply.started":"2025-07-06T20:35:36.931491Z","shell.execute_reply":"2025-07-06T20:35:36.956600Z"}},"outputs":[],"execution_count":null}]}