{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":64733,"databundleVersionId":8171035,"sourceType":"competition"}],"dockerImageVersionId":30699,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Simple baseline with Sentinel Image Patches — Swin-v2-t + Binary Cross Entropy [0.23555]\n\nThe occurrence of different types of organisms, whether plants or animals, is generally associated with the characteristics of the environment or ecosystem in which they live. This relationship between the presence of species and their habitat is often interdependent and can be affected by various factors, such as climate, which is another modality we provide.\n\nTo demonstrate the performance while using the image data, i.e., Sentinel Image Patches, we provide a straightforward baseline that is based on a slighly modified Swin-v2-t and Binary Cross Entropy. As described above, the satellite patches provide an image-like modality that captures habitats and other aspects of the locality.\n\nConsidering the significant extent for enhancing performance of this baseline, we encourage you to experiment with various techniques, architectures, losses, etc.\n\n#### **Have Fun!**","metadata":{}},{"cell_type":"code","source":"import os\nimport torch\nimport tqdm\nimport numpy as np\nimport pandas as pd\nimport albumentations as A\nimport torchvision.models as models\nimport torchvision.transforms as transforms\nimport torch.nn as nn\nfrom PIL import Image\nfrom torch.utils.data import Dataset, DataLoader\nfrom torch.optim.lr_scheduler import CosineAnnealingLR\nfrom sklearn.metrics import precision_recall_fscore_support","metadata":{"ExecuteTime":{"end_time":"2024-04-30T21:25:07.29831Z","start_time":"2024-04-30T21:25:05.354584Z"},"tags":[],"execution":{"iopub.status.busy":"2024-05-05T23:19:38.383871Z","iopub.execute_input":"2024-05-05T23:19:38.384961Z","iopub.status.idle":"2024-05-05T23:19:46.382344Z","shell.execute_reply.started":"2024-05-05T23:19:38.384920Z","shell.execute_reply":"2024-05-05T23:19:46.381533Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data description\n\nThe Sentinel Image data was acquired through the Sentinel2 satellite program and pre-processed by [Ecodatacube](https://stac.ecodatacube.eu/) to produce raster files scaled to the entire European continent and projected into a unique CRS. We filtered the data in order to pick patches from each spectral band corresponding to a location ((lon, lat) GPS coordinates) and a date matching that of our occurrences', and split them into JPEG files (RGB in 3-channels .jpeg files and NIR in single-channel .jpeg files) with a 128x128 resolution. The images were converted from sentinel uint15 to uint8 by clipping data pixel values over 10000 and applying a gamma correction of 2.5.\n\nThe data can simply be loaded using the following method:\n\n```python\ndef construct_patch_path(output_path, survey_id):\n    \"\"\"Construct the patch file path based on survey_id as './CD/AB/XXXXABCD.jpeg'\"\"\"\n    path = output_path\n    for d in (str(survey_id)[-2:], str(survey_id)[-4:-2]):\n        path = os.path.join(path, d)\n\n    path = os.path.join(path, f\"{survey_id}.jpeg\")\n\n    return path\n```\n\n**References:**\n- *Traceability (lineage): The dataset was produced entirely by mosaicking and seasonally aggregating imagery from the Sentinel-2 Level-2A product (https://sentinels.copernicus.eu/web/sentinel/user-guides/sentinel-2-msi/product-types/level-2a)*\n- *Ecodatacube.eu: Analysis-ready open environmental data cube for Europe (https://doi.org/10.21203/rs.3.rs-2277090/v3)*","metadata":{"execution":{"iopub.status.busy":"2024-05-01T13:30:07.053659Z","iopub.execute_input":"2024-05-01T13:30:07.054038Z","iopub.status.idle":"2024-05-01T13:30:07.058148Z","shell.execute_reply.started":"2024-05-01T13:30:07.054008Z","shell.execute_reply":"2024-05-01T13:30:07.057269Z"}}},{"cell_type":"markdown","source":"## Prepare custom dataset loader\n\nWe have to slightly update the Dataset to provide the relevant data in the appropriate format.","metadata":{}},{"cell_type":"code","source":"transform_albumentations = A.Compose([\n    A.RandomBrightnessContrast(p=0.2),\n    A.ColorJitter(p=0.2),\n    A.OpticalDistortion(p=0.2)\n])\n\ndef construct_patch_path(data_path, survey_id):\n    \"\"\"Construct the patch file path based on plot_id as './CD/AB/XXXXABCD.jpeg'\"\"\"\n    path = data_path\n    for d in (str(survey_id)[-2:], str(survey_id)[-4:-2]):\n        path = os.path.join(path, d)\n\n    path = os.path.join(path, f\"{survey_id}.jpeg\")\n\n    return path\n\nclass TrainDataset(Dataset):\n    def __init__(self, data_dir, metadata, transform=None):\n        self.transform = transform\n        self.data_dir = data_dir\n        self.metadata = metadata\n        self.metadata = self.metadata.dropna(subset=\"speciesId\").reset_index(drop=True)\n        self.metadata['speciesId'] = self.metadata['speciesId'].astype(int)\n        self.label_dict = self.metadata.groupby('surveyId')['speciesId'].apply(list).to_dict()\n        \n        self.metadata = self.metadata.drop_duplicates(subset=\"surveyId\").reset_index(drop=True)\n\n    def __len__(self):\n        return len(self.metadata)\n\n    def __getitem__(self, idx):\n        \n        survey_id = self.metadata.surveyId[idx]\n        species_ids = self.label_dict.get(survey_id, [])  # Get list of species IDs for the survey ID\n        label = torch.zeros(num_classes)  # Initialize label tensor\n        for species_id in species_ids:\n            #label_id = self.species_mapping[species_id]  # Get consecutive integer label\n            label_id = species_id\n            label[label_id] = 1  # Set the corresponding class index to 1 for each species\n        \n        rgb_sample = np.array(Image.open(construct_patch_path(self.data_dir, survey_id)))\n        nir_sample = np.array(Image.open(construct_patch_path(self.data_dir.replace(\"rgb\", \"nir\").replace(\"RGB\", \"NIR\"), survey_id)))\n        \n        \n        rgb_sample = transform_albumentations(image=rgb_sample)\n        nir_sample = transform_albumentations(image=nir_sample)\n        \n        sample = np.concatenate((rgb_sample[\"image\"], nir_sample[\"image\"][...,None]), axis=2)    \n        sample = self.transform(sample)\n\n        return sample, label, survey_id\n    \nclass TestDataset(TrainDataset):\n    def __init__(self, data_dir, metadata, transform=None):\n        self.transform = transform\n        self.data_dir = data_dir\n        self.metadata = metadata\n        \n    def __getitem__(self, idx):\n        \n        survey_id = self.metadata.surveyId[idx]\n        \n        rgb_sample = np.array(Image.open(construct_patch_path(self.data_dir, survey_id)))\n        nir_sample = np.array(Image.open(construct_patch_path(self.data_dir.replace(\"rgb\", \"nir\").replace(\"RGB\", \"NIR\"), survey_id)))\n        \n        rgb_sample = transform_albumentations(image=rgb_sample)\n        nir_sample = transform_albumentations(image=nir_sample)\n        \n        sample = np.concatenate((rgb_sample[\"image\"], nir_sample[\"image\"][...,None]), axis=2)    \n        sample = self.transform(sample)\n\n        return sample, survey_id","metadata":{"ExecuteTime":{"end_time":"2024-04-30T21:25:32.627928Z","start_time":"2024-04-30T21:25:32.612131Z"},"collapsed":false,"jupyter":{"outputs_hidden":false},"tags":[],"execution":{"iopub.status.busy":"2024-05-05T23:19:46.384141Z","iopub.execute_input":"2024-05-05T23:19:46.384767Z","iopub.status.idle":"2024-05-05T23:19:46.403530Z","shell.execute_reply.started":"2024-05-05T23:19:46.384740Z","shell.execute_reply":"2024-05-05T23:19:46.402442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load metadata and prepare data loaders","metadata":{}},{"cell_type":"code","source":"# Dataset and DataLoader\nbatch_size = 64\n\ntransform = transforms.Compose([\n    transforms.ToTensor()\n])\n\n# Load Training metadata\ntrain_data_path = \"/kaggle/input/geolifeclef-2024/PA_Train_SatellitePatches_RGB/pa_train_patches_rgb/\"\ntrain_metadata_path = \"/kaggle/input/geolifeclef-2024/GLC24_PA_metadata_train.csv\"\ntrain_metadata = pd.read_csv(train_metadata_path)\ntrain_dataset = TrainDataset(train_data_path, train_metadata, transform=transform)\ntrain_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True, num_workers=4)\n\n# Load Test metadata\ntest_data_path = \"/kaggle/input/geolifeclef-2024/PA_Test_SatellitePatches_RGB/pa_test_patches_rgb/\"\ntest_metadata_path = \"/kaggle/input/geolifeclef-2024/GLC24_PA_metadata_test.csv\"\ntest_metadata = pd.read_csv(test_metadata_path)\ntest_dataset = TestDataset(test_data_path, test_metadata, transform=transform)\ntest_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False, num_workers=4)","metadata":{"ExecuteTime":{"end_time":"2024-04-30T21:25:34.532017Z","start_time":"2024-04-30T21:25:32.615562Z"},"collapsed":false,"jupyter":{"outputs_hidden":false},"tags":[],"execution":{"iopub.status.busy":"2024-05-05T23:19:46.404728Z","iopub.execute_input":"2024-05-05T23:19:46.405025Z","iopub.status.idle":"2024-05-05T23:19:52.146050Z","shell.execute_reply.started":"2024-05-05T23:19:46.404999Z","shell.execute_reply":"2024-05-05T23:19:52.144951Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Modify pretrained Swin-v2-t model\n\nTo fully use all the R,G,B and NIR channels, we have to modify the input layer of the Swin-v2-t.\nThat is all :)","metadata":{}},{"cell_type":"code","source":"# Check if cuda is available\ndevice = torch.device(\"cpu\")\n\nif torch.cuda.is_available():\n    device = torch.device(\"cuda\")\n    print(\"DEVICE = CUDA\")\n\n# Hyperparameters\nlearning_rate = 0.0002\nnum_epochs = 10\npositive_weigh_factor = 1.0\nnum_classes = 11255 # Number of all unique classes within the PO and PA data.","metadata":{"ExecuteTime":{"end_time":"2024-04-30T21:25:31.014067Z","start_time":"2024-04-30T21:25:31.01006Z"},"collapsed":false,"jupyter":{"outputs_hidden":false},"tags":[],"execution":{"iopub.status.busy":"2024-05-05T23:19:52.148278Z","iopub.execute_input":"2024-05-05T23:19:52.148612Z","iopub.status.idle":"2024-05-05T23:19:52.184365Z","shell.execute_reply.started":"2024-05-05T23:19:52.148585Z","shell.execute_reply":"2024-05-05T23:19:52.183443Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = models.swin_v2_s(weights=\"IMAGENET1K_V1\")\nmodel.features[0][0] = nn.Conv2d(4, 96, kernel_size=(4, 4), stride=(4, 4))\nmodel.head = nn.Linear(in_features=768, out_features=num_classes, bias=True)\nmodel.to(device)\n\noptimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)\nscheduler = CosineAnnealingLR(optimizer, T_max=25, verbose=True)","metadata":{"execution":{"iopub.status.busy":"2024-05-05T23:19:52.185553Z","iopub.execute_input":"2024-05-05T23:19:52.185839Z","iopub.status.idle":"2024-05-05T23:20:06.408126Z","shell.execute_reply.started":"2024-05-05T23:19:52.185813Z","shell.execute_reply":"2024-05-05T23:20:06.407184Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def set_seed(seed):\n    # Set seed for Python's built-in random number generator\n    torch.manual_seed(seed)\n    # Set seed for numpy\n    np.random.seed(seed)\n    # Set seed for CUDA if available\n    if torch.cuda.is_available():\n        torch.cuda.manual_seed_all(seed)\n        # Set cuDNN's random number generator seed for deterministic behavior\n        torch.backends.cudnn.deterministic = True\n        torch.backends.cudnn.benchmark = False\n\nset_seed(69)","metadata":{"tags":[],"execution":{"iopub.status.busy":"2024-05-05T23:20:06.409519Z","iopub.execute_input":"2024-05-05T23:20:06.410272Z","iopub.status.idle":"2024-05-05T23:20:06.416364Z","shell.execute_reply.started":"2024-05-05T23:20:06.410239Z","shell.execute_reply":"2024-05-05T23:20:06.415204Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Training Loop\n\nNothing special, just a standard Pytorch training loop.","metadata":{}},{"cell_type":"code","source":"print(f\"Training for {num_epochs} epochs started.\")\n\nfor epoch in range(num_epochs):\n    model.train()\n    for batch_idx, (data, targets, _) in enumerate(train_loader):\n\n        data = data.to(device)\n        targets = targets.to(device)\n\n        optimizer.zero_grad()\n        outputs = model(data)\n\n        pos_weight = targets*positive_weigh_factor  # All positive weights are equal to 10\n        criterion = torch.nn.BCEWithLogitsLoss(pos_weight=pos_weight)\n        loss = criterion(outputs, targets)\n\n        loss.backward()\n        optimizer.step()\n\n        if batch_idx % 278 == 0:\n            print(f\"Epoch {epoch+1}/{num_epochs}, Batch {batch_idx}/{len(train_loader)}, Loss: {loss.item()}\")\n\n    scheduler.step()\n    print(\"Scheduler:\",scheduler.state_dict())\n\n# Save the trained model\nmodel.eval()\ntorch.save(model.state_dict(), \"resnet18-with-landsat-cubes.pth\")","metadata":{"ExecuteTime":{"start_time":"2024-04-30T21:25:34.536634Z"},"collapsed":false,"is_executing":true,"jupyter":{"outputs_hidden":false},"tags":[],"execution":{"iopub.status.busy":"2024-05-05T23:20:06.417608Z","iopub.execute_input":"2024-05-05T23:20:06.417887Z","iopub.status.idle":"2024-05-06T00:42:02.246466Z","shell.execute_reply.started":"2024-05-05T23:20:06.417863Z","shell.execute_reply":"2024-05-06T00:42:02.245249Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Test Loop\n\nAgain, nothing special, just a standard inference.","metadata":{}},{"cell_type":"code","source":"with torch.no_grad():\n    all_predictions = []\n    surveys = []\n    top_k_indices = None\n    for data, surveyID in tqdm.tqdm(test_loader, total=len(test_loader)):\n\n        data = data.to(device)\n        \n        outputs = model(data)\n        predictions = torch.sigmoid(outputs).cpu().numpy()\n\n        # Sellect top-25 values as predictions\n        top_25 = np.argsort(-predictions, axis=1)[:, :25] \n        if top_k_indices is None:\n            top_k_indices = top_25\n        else:\n            top_k_indices = np.concatenate((top_k_indices, top_25), axis=0)\n\n        surveys.extend(surveyID.cpu().numpy())","metadata":{"collapsed":false,"jupyter":{"outputs_hidden":false},"tags":[],"execution":{"iopub.status.busy":"2024-05-06T00:42:02.248125Z","iopub.execute_input":"2024-05-06T00:42:02.248443Z","iopub.status.idle":"2024-05-06T00:42:36.697795Z","shell.execute_reply.started":"2024-05-06T00:42:02.248411Z","shell.execute_reply":"2024-05-06T00:42:36.696563Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Save prediction file! 🎉🥳🙌🤗","metadata":{}},{"cell_type":"code","source":"data_concatenated = [' '.join(map(str, row)) for row in top_k_indices]\n\npd.DataFrame(\n    {'surveyId': surveys,\n     'predictions': data_concatenated,\n    }).to_csv(\"submission.csv\", index = False)","metadata":{"tags":[],"execution":{"iopub.status.busy":"2024-05-06T00:42:36.699342Z","iopub.execute_input":"2024-05-06T00:42:36.699824Z","iopub.status.idle":"2024-05-06T00:42:36.809697Z","shell.execute_reply.started":"2024-05-06T00:42:36.699764Z","shell.execute_reply":"2024-05-06T00:42:36.808887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}