{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.11","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":105343,"databundleVersionId":12700041,"sourceType":"competition"}],"dockerImageVersionId":31040,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**Multimodal PyTorch Baseline for “Applications of Deep Learning (WUSTL, Summer 2025)”**\n\n1. **Data loading & sampling**  \n   - Reads `train.csv` and `test.csv` from your Kaggle input folder.  \n   - Applies a quick “smoke-test” sampling via `SAMPLE_PERCENT` (e.g. `0.1` for 10% of rows) so you can iterate rapidly before scaling up to the full dataset.  \n\n2. **Tabular preprocessing**  \n   - Numeric columns are standardized with `StandardScaler`.  \n   - Categorical columns are label-encoded on the combined train+test set to handle unseen categories.  \n\n3. **Image handling**  \n   - Loads each JPEG by casting the float `id` → `int` → `\"{id}.jpg\"` and applying standard ImageNet transforms (resize, flip, normalize).  \n\n4. **Model architecture**  \n   - **CNN branch:** Pretrained ResNet-18 (ImageNet weights), dropping the final FC and using it as a feature extractor.  \n   - **Tabular branch:** A small MLP (128→64→ReLU).  \n   - **Head:** Concatenates both feature streams, then runs through 256→1 for regression (preservation score).  \n\n5. **Training loop**  \n   - Uses `MSELoss` + `Adam(lr=1e-4)`.  \n   - Iterates for `num_epochs` (default 5) over the sampled subset.  \n\n6. **Submission cell**  \n   - Switches the dataset to inference mode, runs the model on `df_test`, and writes out a `submission.csv`.  \n\n---\n\n### Performance tips\n\n- **Sample size:** Change `SAMPLE_PERCENT` to control how much of your training data you use during prototyping (e.g. `0.01` or `1.0` for full data).  \n- **Epochs:** Increase `num_epochs` once your pipeline is stable to fully converge on the real dataset.  \n- **Early stopping:** Wrap your training loop with a simple early-stopping criterion (e.g. monitor validation loss and halt when it hasn’t improved for N epochs) to avoid overfitting and save GPU time.  \n","metadata":{}},{"cell_type":"code","source":"# Starter PyTorch Notebook for Applications of Deep Learning Competition\n# Combines a pretrained image CNN with tabular features, handling id-based image lookup.\n\nimport os\nimport pandas as pd\nimport numpy as np\nfrom PIL import Image\n\nimport torch\nimport torch.nn as nn\nfrom torch.utils.data import Dataset, DataLoader\nimport torchvision.transforms as transforms\nimport torchvision.models as models\nfrom sklearn.preprocessing import LabelEncoder, StandardScaler\n\n# Sampling percentage for quick testing\nSAMPLE_PERCENT = 0.05  # use 0.1 for 10% of the training data\nNUM_EPOCHS = 5\n\n# Paths\ndata_dir      = os.path.dirname('../input/applications-of-deep-learning-wustl-summer-2025/train.csv')\ntrain_csv     = os.path.join(data_dir, 'train.csv')\ntest_csv      = os.path.join(data_dir, 'test.csv')\n\n# Load CSVs\ndf_train_full = pd.read_csv(train_csv)\ndf_test        = pd.read_csv(test_csv)\n\n# Sample down the training set for faster iteration\ndf_train = df_train_full.sample(frac=SAMPLE_PERCENT, random_state=42).reset_index(drop=True)\nprint(f\"Sampling {SAMPLE_PERCENT*100}% of training data: {len(df_train)} rows selected from {len(df_train_full)} total.\")\n\n# Identify columns\nid_col      = 'id'\ntarget_col  = 'preservation_score'  # actual target column in dataset\nignore_cols = [id_col, target_col]\n\n# Tabular features\ntabular_cols = [c for c in df_train.columns if c not in ignore_cols]\n\n# Split numeric vs categorical\nnumeric_cols = df_train[tabular_cols].select_dtypes(include=['number']).columns.tolist()\ncat_cols     = df_train[tabular_cols].select_dtypes(include=['object', 'category']).columns.tolist()\n# Keep only features also present in test set\nnumeric_cols = [c for c in numeric_cols if c in df_test.columns]\ncat_cols     = [c for c in cat_cols     if c in df_test.columns]\n\n# Scale numeric features\nscaler = StandardScaler()\nif numeric_cols:\n    df_train[numeric_cols] = scaler.fit_transform(df_train[numeric_cols])\n    df_test[numeric_cols]  = scaler.transform(df_test[numeric_cols])\n\n# Encode categorical features\nencoders = {}\nfor col in cat_cols:\n    le = LabelEncoder()\n    combined = pd.concat([df_train[col].astype(str), df_test[col].astype(str)])\n    le.fit(combined)\n    df_train[col] = le.transform(df_train[col].astype(str))\n    df_test[col]  = le.transform(df_test[col].astype(str))\n    encoders[col] = le\n\n# Final tabular feature list\ntabular_feats = numeric_cols + cat_cols\nnum_tab_feats = len(tabular_feats)\nprint(f\"Using {num_tab_feats} tabular features: {tabular_feats}\")\n\n# Dataset class definition\ntorch.manual_seed(0)\nclass MultiModalDataset(Dataset):\n    def __init__(self, df, data_dir, id_col, transform=None, is_train=True):\n        self.df        = df.reset_index(drop=True)\n        self.data_dir  = data_dir\n        self.id_col    = id_col\n        self.transform = transform\n        self.is_train  = is_train\n        self.tab_data  = torch.tensor(df[tabular_feats].fillna(0).values, dtype=torch.float32)\n        if is_train:\n            self.targets = torch.tensor(df[target_col].values, dtype=torch.float32)\n\n    def __len__(self):\n        return len(self.df)\n\n    def __getitem__(self, idx):\n        row = self.df.iloc[idx]\n        # Construct image filename from id (images are flat in data_dir)\n        img_id   = int(row[self.id_col])\n        img_name = f\"{img_id}.jpg\"\n        img_path = os.path.join(self.data_dir, img_name)\n        img      = Image.open(img_path).convert('RGB')\n        if self.transform:\n            img = self.transform(img)\n\n        tab = self.tab_data[idx]\n        if self.is_train:\n            return img, tab, self.targets[idx]\n        else:\n            return img, tab, img_id\n\n# Image transforms\ntransform = transforms.Compose([\n    transforms.Resize((224,224)),\n    transforms.RandomHorizontalFlip(),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225])\n])\n\n# DataLoader for sampled data\nbatch_size = 32\ndataset    = MultiModalDataset(df_train, data_dir, id_col, transform=transform, is_train=True)\nloader     = DataLoader(dataset, batch_size=batch_size, shuffle=True, num_workers=2)\n\n# Multimodal model definition\nclass MultiModalModel(nn.Module):\n    def __init__(self, num_tab_feats, backbone='resnet18'):\n        super().__init__()\n        cnn = models.resnet18(weights=models.ResNet18_Weights.DEFAULT)\n        feat_dim = cnn.fc.in_features\n        cnn.fc = nn.Identity()\n        self.cnn = cnn\n        self.tab_net = nn.Sequential(\n            nn.Linear(num_tab_feats, 128),\n            nn.ReLU(),\n            nn.Linear(128, 64),\n            nn.ReLU()\n        )\n        self.head = nn.Sequential(\n            nn.Linear(feat_dim + 64, 256),\n            nn.ReLU(),\n            nn.Dropout(0.5),\n            nn.Linear(256, 1)\n        )\n\n    def forward(self, img, tab):\n        img_feat = self.cnn(img)\n        tab_feat = self.tab_net(tab)\n        x = torch.cat([img_feat, tab_feat], dim=1)\n        return self.head(x)\n\n# Training loop on sampled subset\ndevice    = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\nmodel     = MultiModalModel(num_tab_feats).to(device)\noptimizer = torch.optim.Adam(model.parameters(), lr=1e-4)\ncriterion = nn.MSELoss()\n\nfor epoch in range(NUM_EPOCHS):\n    model.train()\n    running_loss = 0\n    for imgs, tabs, ys in loader:\n        imgs, tabs, ys = imgs.to(device), tabs.to(device), ys.unsqueeze(1).to(device)\n        preds = model(imgs, tabs)\n        loss  = criterion(preds, ys)\n        optimizer.zero_grad()\n        loss.backward()\n        optimizer.step()\n        running_loss += loss.item() * imgs.size(0)\n    print(f\"Epoch {epoch+1} Loss: {running_loss/len(dataset):.4f}\")\n\n# Next: validation, checkpointing, and submission\n\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-07T00:14:17.969542Z","iopub.execute_input":"2025-07-07T00:14:17.969924Z","iopub.status.idle":"2025-07-07T00:18:54.791346Z","shell.execute_reply.started":"2025-07-07T00:14:17.969886Z","shell.execute_reply":"2025-07-07T00:18:54.790168Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Submission Generation\n\nThe cell below runs inference over the full test set and produces a Kaggle-ready `submission.csv`. It:\n\n1. Switches the model to evaluation mode  \n2. Builds a DataLoader over `df_test` (using the same preprocessing and transforms)  \n3. Collects each `id` and its predicted preservation score  \n4. Constructs a DataFrame and saves it as `submission.csv`  \n","metadata":{}},{"cell_type":"code","source":"# === Submission Generation ===\n# Prepare test dataset and loader\n# Note: df_test must be preprocessed similarly to df_train (scaled, encoded)\ntest_dataset = MultiModalDataset(df_test, data_dir, id_col, transform=transform, is_train=False)\ntest_loader  = DataLoader(test_dataset, batch_size=batch_size, shuffle=False, num_workers=2)\n\n# Inference on test set\nmodel.eval()\nall_ids   = []\nall_preds = []\nwith torch.no_grad():\n    for imgs, tabs, img_ids in test_loader:\n        imgs, tabs = imgs.to(device), tabs.to(device)\n        outputs = model(imgs, tabs).squeeze(1).cpu().numpy()\n        all_ids.extend(img_ids.tolist())\n        all_preds.extend(outputs.tolist())\n\n# Build submission DataFrame\ndf_submission = pd.DataFrame({\n    id_col: all_ids,\n    target_col: all_preds\n})\n# Save to CSV in Kaggle required format\nsubmission_file = 'submission.csv'\ndf_submission.to_csv(submission_file, index=False)\nprint(f\"Saved submission file: {submission_file}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-07T00:19:05.950446Z","iopub.execute_input":"2025-07-07T00:19:05.951425Z","iopub.status.idle":"2025-07-07T00:20:42.854019Z","shell.execute_reply.started":"2025-07-07T00:19:05.951381Z","shell.execute_reply":"2025-07-07T00:20:42.852845Z"}},"outputs":[],"execution_count":null}]}