{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.12"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":91451,"databundleVersionId":11223220,"sourceType":"competition"}],"dockerImageVersionId":30919,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Baseline with WildFusion 🦙 🧬\n\nThis notebook presents an improved baseline for individual identification using the **[MegaDescriptor](https://arxiv.org/pdf/2311.09118)** with **[ALIKED](https://arxiv.org/pdf/2304.03608)** methods, combined using [WildFusion](https://arxiv.org/pdf/2408.12934).\n\n## What is WildFusion?  \nWildFusion is a **feature fusion method** designed to improve the accuracy and robustness of models in individual recognition tasks. Traditional identification models often rely on a single type of feature representation, which may struggle with variations in lighting, angles, and occlusions. WildFusion overcomes these challenges by combining multiple feature extraction techniques, resulting in a more comprehensive and adaptable approach.  ","metadata":{}},{"cell_type":"markdown","source":"## Dependencies instalation\nFor the competition we provide two Python packages for loading and preprocessing of available datasets ([wildlife-datasets](https://github.com/WildlifeDatasets/wildlife-datasets)) and tools / method for animal re-identification ([wildlife-tools](https://github.com/WildlifeDatasets/wildlife-tools)).","metadata":{"execution":{"iopub.status.busy":"2025-02-28T16:18:13.120141Z","iopub.execute_input":"2025-02-28T16:18:13.120497Z","iopub.status.idle":"2025-02-28T16:18:13.1244Z","shell.execute_reply.started":"2025-02-28T16:18:13.120467Z","shell.execute_reply":"2025-02-28T16:18:13.12338Z"}}},{"cell_type":"code","source":"!pip install git+https://github.com/WildlifeDatasets/wildlife-datasets@develop\n!pip install git+https://github.com/WildlifeDatasets/wildlife-tools","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-03-28T19:40:08.072410Z","iopub.execute_input":"2025-03-28T19:40:08.072742Z","iopub.status.idle":"2025-03-28T19:40:47.929671Z","shell.execute_reply.started":"2025-03-28T19:40:08.072711Z","shell.execute_reply":"2025-03-28T19:40:47.928601Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Dependencies import\nWe load all the required packages and then define the function `create_sample_submission`, which converts provided predictions and a submission file for the competition.","metadata":{}},{"cell_type":"code","source":"import os\nimport numpy as np\nimport pandas as pd\nimport timm\nimport torchvision.transforms as T\nfrom wildlife_datasets.datasets import AnimalCLEF2025\nfrom wildlife_tools.features import DeepFeatures\nfrom wildlife_tools.similarity import CosineSimilarity\nfrom wildlife_tools.similarity.wildfusion import SimilarityPipeline, WildFusion\nfrom wildlife_tools.similarity.pairwise.lightglue import MatchLightGlue\nfrom wildlife_tools.features.local import AlikedExtractor\nfrom wildlife_tools.similarity.calibration import IsotonicCalibration\n\ndef create_sample_submission(dataset_query, predictions, file_name='sample_submission.csv'):\n    df = pd.DataFrame({\n        'image_id': dataset_query.metadata['image_id'],\n        'identity': predictions\n    })\n    df.to_csv(file_name, index=False)","metadata":{"execution":{"iopub.status.busy":"2025-03-28T19:43:33.482268Z","iopub.execute_input":"2025-03-28T19:43:33.482590Z","iopub.status.idle":"2025-03-28T19:43:58.661974Z","shell.execute_reply.started":"2025-03-28T19:43:33.482565Z","shell.execute_reply":"2025-03-28T19:43:58.661314Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Inference WildFusion\n\nInstead of training a classifier, we use out of the shelf pretrained models - [MegaDescriptor](https://huggingface.co/BVRA/MegaDescriptor-L-384) and [ALIKED](https://arxiv.org/pdf/2304.03608) - Keypoint and Descriptor Extraction Network. Both MegaDescriptor and ALIKED are used to extract features from all images.\n\n**Note:** _It is highly recommended to use the GPU acceleration._","metadata":{}},{"cell_type":"markdown","source":"We need to specify the `root`, where the data are stored and then two image transformations. \n1. The first transform only resizes the images and is used for visualization.\n2. The second transform also converts it to torch tensor and is used for operations on neural networks.","metadata":{}},{"cell_type":"code","source":"root = '/kaggle/input/animal-clef-2025'\ntransform_display = T.Compose([\n    T.Resize([384, 384]),\n])\ntransform = T.Compose([\n    *transform_display.transforms,\n    T.ToTensor(),\n    T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225))\n])\n\ntransforms_aliked = T.Compose([\n    T.Resize([512, 512]),\n    T.ToTensor()\n])","metadata":{"execution":{"iopub.status.busy":"2025-03-28T19:43:58.662984Z","iopub.execute_input":"2025-03-28T19:43:58.663605Z","iopub.status.idle":"2025-03-28T19:43:58.668341Z","shell.execute_reply.started":"2025-03-28T19:43:58.663580Z","shell.execute_reply":"2025-03-28T19:43:58.667684Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Loading the dataset\ndataset = AnimalCLEF2025(root, load_label=True)\ndataset_database = dataset.get_subset(dataset.metadata['split'] == 'database')\ndataset_query = dataset.get_subset(dataset.metadata['split'] == 'query')\ndataset_calibration = AnimalCLEF2025(root, df=dataset_database.metadata[:100], load_label=True) \n\nn_query = len(dataset_query)","metadata":{"execution":{"iopub.status.busy":"2025-03-28T19:43:58.669699Z","iopub.execute_input":"2025-03-28T19:43:58.669898Z","iopub.status.idle":"2025-03-28T19:44:37.491787Z","shell.execute_reply.started":"2025-03-28T19:43:58.669881Z","shell.execute_reply":"2025-03-28T19:44:37.490813Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Loading the models\nname = 'hf-hub:BVRA/MegaDescriptor-L-384'\nmodel = timm.create_model(name, num_classes=0, pretrained=True)\ndevice = 'cuda'\n\nmatcher_aliked = SimilarityPipeline(\n    matcher = MatchLightGlue(features='aliked', device=device, batch_size=16),\n    extractor = AlikedExtractor(),\n    transform = transforms_aliked,\n    calibration = IsotonicCalibration()\n)\n\nmatcher_mega = SimilarityPipeline(\n    matcher = CosineSimilarity(),\n    extractor = DeepFeatures(model=model, device=device, batch_size=16),\n    transform = transform,\n    calibration = IsotonicCalibration()\n)","metadata":{"_kg_hide-input":false,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-03-28T13:37:50.001557Z","iopub.execute_input":"2025-03-28T13:37:50.001896Z","iopub.status.idle":"2025-03-28T13:38:09.610926Z","shell.execute_reply.started":"2025-03-28T13:37:50.001867Z","shell.execute_reply":"2025-03-28T13:38:09.609990Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Calibrating the WildFusion\nwildfusion = WildFusion(calibrated_pipelines = [matcher_aliked, matcher_mega], priority_pipeline = matcher_mega)\nwildfusion.fit_calibration(dataset_calibration, dataset_calibration)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Compute WildFusion similarity\nsimilarity = wildfusion(dataset_query, dataset_database, B=25)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-28T13:39:24.480952Z","iopub.execute_input":"2025-03-28T13:39:24.481217Z","iopub.status.idle":"2025-03-28T14:10:58.719117Z","shell.execute_reply.started":"2025-03-28T13:39:24.481189Z","shell.execute_reply":"2025-03-28T14:10:58.718087Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"pred_idx = similarity.argsort(axis=1)[:,-1]\npred_scores = similarity[range(n_query), pred_idx]","metadata":{"execution":{"iopub.status.busy":"2025-03-28T14:10:58.720185Z","iopub.execute_input":"2025-03-28T14:10:58.720464Z","iopub.status.idle":"2025-03-28T14:11:00.080627Z","shell.execute_reply.started":"2025-03-28T14:10:58.720433Z","shell.execute_reply":"2025-03-28T14:11:00.079928Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"new_individual = 'new_individual'\nthreshold = 0.6\nlabels = dataset_database.labels_string\npredictions = labels[pred_idx]\npredictions[pred_scores < threshold] = new_individual\ncreate_sample_submission(dataset_query, predictions, file_name='sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2025-03-28T14:11:00.081567Z","iopub.execute_input":"2025-03-28T14:11:00.081856Z","iopub.status.idle":"2025-03-28T14:11:00.097074Z","shell.execute_reply.started":"2025-03-28T14:11:00.081829Z","shell.execute_reply":"2025-03-28T14:11:00.096354Z"},"trusted":true},"outputs":[],"execution_count":null}]}