{"metadata":{"kaggle":{"accelerator":"none","dataSources":[{"sourceId":85540,"databundleVersionId":9725394,"sourceType":"competition"}],"dockerImageVersionId":30786,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.14"},"papermill":{"default_parameters":{},"duration":7.830673,"end_time":"2024-11-04T12:36:50.948677","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-11-04T12:36:43.118004","version":"2.6.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Basic XGBoost Solution for Tabular Only\n\n**Generated from this prompt:**\n\n```\nCreate a Kaggle notebook that uses XGBoost to perform a regression prediction on these files:\n\n/kaggle/input/applications-of-deep-learning-wustl-fall-2024/df-test_200x11.csv\n/kaggle/input/applications-of-deep-learning-wustl-fall-2024/df-train_1000x12.csv\n\nThe target is: target_homevalue\n\nhomeid - Unique 5-digit home identifier\nfeat_yrbuilt - Year the home was built\nfeat_nrooms - Number of total rooms\nfeat_nbed - Number of bedrooms\nfeat_nbath - Number of bathrooms\nfeat_sqft - Squared footage of the entire home\nfeat_proptype - Whether the home is a single-family, multi-family, or condo\nfeat_basement - Whether the home has a basement\nfeat_stories - Whether the home has a single or multiple floors\nfeat_solar - Whether the home has solar panels\nfeat_nsales - Number of times sold in the past decade\ntarget_homevalue - Home values (rounded to the nearest dollar)\n```","metadata":{"papermill":{"duration":0.00772,"end_time":"2024-11-04T12:36:45.603805","exception":false,"start_time":"2024-11-04T12:36:45.596085","status":"completed"},"tags":[]}},{"cell_type":"code","source":"!ls /kaggle/input/applications-of-deep-learning-wustl-fall-2024/*.csv","metadata":{"papermill":{"duration":1.030928,"end_time":"2024-11-04T12:36:46.641578","exception":false,"start_time":"2024-11-04T12:36:45.610650","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:28.067361Z","iopub.execute_input":"2024-11-04T12:46:28.067824Z","iopub.status.idle":"2024-11-04T12:46:29.282568Z","shell.execute_reply.started":"2024-11-04T12:46:28.067779Z","shell.execute_reply":"2024-11-04T12:46:29.280839Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Import Libraries","metadata":{"papermill":{"duration":0.00674,"end_time":"2024-11-04T12:36:46.655535","exception":false,"start_time":"2024-11-04T12:36:46.648795","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom sklearn.preprocessing import LabelEncoder\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score\nimport xgboost as xgb","metadata":{"papermill":{"duration":2.592377,"end_time":"2024-11-04T12:36:49.254723","exception":false,"start_time":"2024-11-04T12:36:46.662346","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:29.286111Z","iopub.execute_input":"2024-11-04T12:46:29.286711Z","iopub.status.idle":"2024-11-04T12:46:32.888078Z","shell.execute_reply.started":"2024-11-04T12:46:29.286638Z","shell.execute_reply":"2024-11-04T12:46:32.886455Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load Data","metadata":{"papermill":{"duration":0.006786,"end_time":"2024-11-04T12:36:49.268675","exception":false,"start_time":"2024-11-04T12:36:49.261889","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Load the training and test datasets\ntrain_df = pd.read_csv('/kaggle/input/applications-of-deep-learning-wustl-fall-2024/df-train_1000x12.csv')\ntest_df = pd.read_csv('/kaggle/input/applications-of-deep-learning-wustl-fall-2024/df-test_200x11.csv')","metadata":{"papermill":{"duration":0.031456,"end_time":"2024-11-04T12:36:49.307840","exception":false,"start_time":"2024-11-04T12:36:49.276384","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:32.889837Z","iopub.execute_input":"2024-11-04T12:46:32.890416Z","iopub.status.idle":"2024-11-04T12:46:32.925413Z","shell.execute_reply.started":"2024-11-04T12:46:32.890362Z","shell.execute_reply":"2024-11-04T12:46:32.924068Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Exploration\nLet's take a quick look at the datasets.","metadata":{"papermill":{"duration":0.006765,"end_time":"2024-11-04T12:36:49.321610","exception":false,"start_time":"2024-11-04T12:36:49.314845","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Display first few rows of the training data\ntrain_df.head()","metadata":{"papermill":{"duration":0.031889,"end_time":"2024-11-04T12:36:49.360474","exception":false,"start_time":"2024-11-04T12:36:49.328585","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:32.929077Z","iopub.execute_input":"2024-11-04T12:46:32.929656Z","iopub.status.idle":"2024-11-04T12:46:32.962469Z","shell.execute_reply.started":"2024-11-04T12:46:32.929595Z","shell.execute_reply":"2024-11-04T12:46:32.961245Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Display first few rows of the test data\ntest_df.head()","metadata":{"papermill":{"duration":0.022997,"end_time":"2024-11-04T12:36:49.390863","exception":false,"start_time":"2024-11-04T12:36:49.367866","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:32.964272Z","iopub.execute_input":"2024-11-04T12:46:32.964686Z","iopub.status.idle":"2024-11-04T12:46:32.982844Z","shell.execute_reply.started":"2024-11-04T12:46:32.964646Z","shell.execute_reply":"2024-11-04T12:46:32.981535Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f\"Training data shape: {train_df.shape}\")\nprint(f\"Test data shape: {test_df.shape}\")\n","metadata":{"papermill":{"duration":0.015104,"end_time":"2024-11-04T12:36:49.413258","exception":false,"start_time":"2024-11-04T12:36:49.398154","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:32.985322Z","iopub.execute_input":"2024-11-04T12:46:32.986515Z","iopub.status.idle":"2024-11-04T12:46:32.994717Z","shell.execute_reply.started":"2024-11-04T12:46:32.986452Z","shell.execute_reply":"2024-11-04T12:46:32.993060Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check for missing values in training data\ntrain_df.isnull().sum()","metadata":{"papermill":{"duration":0.020444,"end_time":"2024-11-04T12:36:49.441083","exception":false,"start_time":"2024-11-04T12:36:49.420639","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:32.997744Z","iopub.execute_input":"2024-11-04T12:46:32.999652Z","iopub.status.idle":"2024-11-04T12:46:33.014426Z","shell.execute_reply.started":"2024-11-04T12:46:32.999578Z","shell.execute_reply":"2024-11-04T12:46:33.012943Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check for missing values in test data\ntest_df.isnull().sum()","metadata":{"papermill":{"duration":0.01764,"end_time":"2024-11-04T12:36:49.466170","exception":false,"start_time":"2024-11-04T12:36:49.448530","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:33.015867Z","iopub.execute_input":"2024-11-04T12:46:33.016270Z","iopub.status.idle":"2024-11-04T12:46:33.029370Z","shell.execute_reply.started":"2024-11-04T12:46:33.016228Z","shell.execute_reply":"2024-11-04T12:46:33.027796Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Preprocessing\n## Handle Categorical Variables\n\nThe following features are categorical:\n\n* ```feat_proptype:``` Type of property (single-family, multi-family, condo)\n* ```feat_basement:``` Whether the home has a basement (Yes/No)\n* ```feat_stories:``` Number of floors (Single/Multiple)\n* ```feat_solar:``` Whether the home has solar panels (Yes/No)\n\nWe will encode these categorical variables using LabelEncoder.\n\n","metadata":{"papermill":{"duration":0.007591,"end_time":"2024-11-04T12:36:49.481382","exception":false,"start_time":"2024-11-04T12:36:49.473791","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Create copies to avoid modifying originals\ntrain_data = train_df.copy()\ntest_data = test_df.copy()\n\n# List of categorical columns\ncategorical_cols = ['feat_proptype', 'feat_basement', 'feat_stories', 'feat_solar']\n\n# Combine train and test data for consistent encoding\ncombined_data = pd.concat([train_data[categorical_cols], test_data[categorical_cols]], axis=0)\n\n# Apply Label Encoding\nfor col in categorical_cols:\n    le = LabelEncoder()\n    le.fit(combined_data[col])\n    train_data[col] = le.transform(train_data[col])\n    test_data[col] = le.transform(test_data[col])","metadata":{"papermill":{"duration":0.02811,"end_time":"2024-11-04T12:36:49.517110","exception":false,"start_time":"2024-11-04T12:36:49.489000","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:33.030784Z","iopub.execute_input":"2024-11-04T12:46:33.031239Z","iopub.status.idle":"2024-11-04T12:46:33.056852Z","shell.execute_reply.started":"2024-11-04T12:46:33.031195Z","shell.execute_reply":"2024-11-04T12:46:33.055507Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Feature and Target Separation\nSeparate features and target variable from the training data.\n\n","metadata":{"papermill":{"duration":0.008007,"end_time":"2024-11-04T12:36:49.532837","exception":false,"start_time":"2024-11-04T12:36:49.524830","status":"completed"},"tags":[]}},{"cell_type":"code","source":"X = train_data.drop(['homeid', 'target_homevalue'], axis=1)\ny = train_data['target_homevalue']\nX_test = test_data.drop(['homeid'], axis=1)","metadata":{"papermill":{"duration":0.018977,"end_time":"2024-11-04T12:36:49.559476","exception":false,"start_time":"2024-11-04T12:36:49.540499","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:33.061348Z","iopub.execute_input":"2024-11-04T12:46:33.061833Z","iopub.status.idle":"2024-11-04T12:46:33.076361Z","shell.execute_reply.started":"2024-11-04T12:46:33.061775Z","shell.execute_reply":"2024-11-04T12:46:33.074762Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Train-Test Split\nSplit the training data into training and validation sets.\n","metadata":{"papermill":{"duration":0.00753,"end_time":"2024-11-04T12:36:49.574574","exception":false,"start_time":"2024-11-04T12:36:49.567044","status":"completed"},"tags":[]}},{"cell_type":"code","source":"X_train, X_valid, y_train, y_valid = train_test_split(\n    X, y, test_size=0.2, random_state=42\n)","metadata":{"papermill":{"duration":0.01877,"end_time":"2024-11-04T12:36:49.600999","exception":false,"start_time":"2024-11-04T12:36:49.582229","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:33.079304Z","iopub.execute_input":"2024-11-04T12:46:33.079785Z","iopub.status.idle":"2024-11-04T12:46:33.092214Z","shell.execute_reply.started":"2024-11-04T12:46:33.079740Z","shell.execute_reply":"2024-11-04T12:46:33.090514Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Training\nInitialize the XGBoost regressor and train it on the training set.\n\n","metadata":{"papermill":{"duration":0.007395,"end_time":"2024-11-04T12:36:49.616161","exception":false,"start_time":"2024-11-04T12:36:49.608766","status":"completed"},"tags":[]}},{"cell_type":"code","source":"xgb_reg = xgb.XGBRegressor(\n    objective='reg:squarederror',\n    random_state=42,\n    n_estimators=100,\n    learning_rate=0.1,\n    max_depth=6\n)\n\nxgb_reg.fit(\n    X_train, y_train,\n    eval_set=[(X_valid, y_valid)],\n    verbose=True\n)\n\nprint(\"done\")","metadata":{"papermill":{"duration":0.367207,"end_time":"2024-11-04T12:36:49.991183","exception":false,"start_time":"2024-11-04T12:36:49.623976","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:33.094332Z","iopub.execute_input":"2024-11-04T12:46:33.094815Z","iopub.status.idle":"2024-11-04T12:46:33.585947Z","shell.execute_reply.started":"2024-11-04T12:46:33.094767Z","shell.execute_reply":"2024-11-04T12:46:33.584991Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Evaluation\nEvaluate the model's performance on the validation set.\n\n","metadata":{"papermill":{"duration":0.016029,"end_time":"2024-11-04T12:36:50.024834","exception":false,"start_time":"2024-11-04T12:36:50.008805","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Predict on validation set\ny_valid_pred = xgb_reg.predict(X_valid)\n\n# Calculate evaluation metrics\nmse = mean_squared_error(y_valid, y_valid_pred)\nmae = mean_absolute_error(y_valid, y_valid_pred)\nr2 = r2_score(y_valid, y_valid_pred)\n\nprint(f\"Validation MSE: {mse:.2f}\")\nprint(f\"Validation MAE: {mae:.2f}\")\nprint(f\"Validation R2 Score: {r2:.2f}\")","metadata":{"papermill":{"duration":0.035684,"end_time":"2024-11-04T12:36:50.076641","exception":false,"start_time":"2024-11-04T12:36:50.040957","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:33.587313Z","iopub.execute_input":"2024-11-04T12:46:33.588482Z","iopub.status.idle":"2024-11-04T12:46:33.607318Z","shell.execute_reply.started":"2024-11-04T12:46:33.588429Z","shell.execute_reply":"2024-11-04T12:46:33.606410Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Prediction on Test Data\nRetrain the model on the entire training dataset and make predictions on the test dataset.\n\n","metadata":{"papermill":{"duration":0.015543,"end_time":"2024-11-04T12:36:50.109034","exception":false,"start_time":"2024-11-04T12:36:50.093491","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Retrain on full training data\nxgb_reg.fit(\n    X_train, y_train,\n    eval_set=[(X_valid, y_valid)],\n    verbose=False\n)\n\n# Predict on test data\ny_test_pred = xgb_reg.predict(X_test)","metadata":{"papermill":{"duration":0.129309,"end_time":"2024-11-04T12:36:50.254208","exception":false,"start_time":"2024-11-04T12:36:50.124899","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:33.608636Z","iopub.execute_input":"2024-11-04T12:46:33.609576Z","iopub.status.idle":"2024-11-04T12:46:33.770513Z","shell.execute_reply.started":"2024-11-04T12:46:33.609530Z","shell.execute_reply":"2024-11-04T12:46:33.769439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submission\nCreate a submission file with homeid and the predicted target_homevalue.\n\n","metadata":{"papermill":{"duration":0.016336,"end_time":"2024-11-04T12:36:50.288812","exception":false,"start_time":"2024-11-04T12:36:50.272476","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Prepare the submission dataframe\nsubmission = pd.DataFrame({\n    'homeid': test_data['homeid'],\n    'target_homevalue': y_test_pred\n})\n\n# Save to CSV\nsubmission.to_csv('submission.csv', index=False)","metadata":{"papermill":{"duration":0.028885,"end_time":"2024-11-04T12:36:50.333888","exception":false,"start_time":"2024-11-04T12:36:50.305003","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:33.772071Z","iopub.execute_input":"2024-11-04T12:46:33.772665Z","iopub.status.idle":"2024-11-04T12:46:33.783258Z","shell.execute_reply.started":"2024-11-04T12:46:33.772623Z","shell.execute_reply":"2024-11-04T12:46:33.782139Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{"papermill":{"duration":0.01642,"end_time":"2024-11-04T12:36:50.367100","exception":false,"start_time":"2024-11-04T12:36:50.350680","status":"completed"},"tags":[]}},{"cell_type":"code","source":"submission.head()","metadata":{"papermill":{"duration":0.029182,"end_time":"2024-11-04T12:36:50.413031","exception":false,"start_time":"2024-11-04T12:36:50.383849","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-11-04T12:46:33.785063Z","iopub.execute_input":"2024-11-04T12:46:33.786299Z","iopub.status.idle":"2024-11-04T12:46:33.797947Z","shell.execute_reply.started":"2024-11-04T12:46:33.786256Z","shell.execute_reply":"2024-11-04T12:46:33.796652Z"},"trusted":true},"execution_count":null,"outputs":[]}]}