{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.11.11","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":91451,"databundleVersionId":11223220,"sourceType":"competition"}],"dockerImageVersionId":31041,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n    🫏AnimalCLEF25 @ CVPR-FGVC & LifeCLEF🫏\n</div>","metadata":{}},{"cell_type":"markdown","source":"<img src=\"https://as2.ftcdn.net/v2/jpg/06/59/72/15/1000_F_659721570_qnoEs9OKM0vnj2ja6buse5SwZvSnrhZA.jpg\" alt=\"\">","metadata":{}},{"cell_type":"markdown","source":"The **objective of the** [AnimalCLEF2025](https://www.kaggle.com/competitions/animal-clef-2025/) challenge is to recognize individual animals—specifically lynxes, salamanders, and sea turtles—based on photographs. This notebook provides visual exploration of the dataset and introduces a baseline approach using the advanced re-identification model [MegaDescriptor](https://huggingface.co/BVRA/MegaDescriptor-L-384).\n\nThe dataset is divided into two parts: a *database* set and a *query* set. For each image in the query set, the task is to:\n\n* Determine whether the animal shown is already present in the database.\n* If not, the model should return the prediction `new_individual`.\n* If the animal is found in the database, the model should return the corresponding individual’s ID.\n","metadata":{}},{"cell_type":"markdown","source":"<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n    🙌Installing the Dependencies🙌\n</div>","metadata":{"execution":{"iopub.status.busy":"2025-02-28T16:18:13.120141Z","iopub.execute_input":"2025-02-28T16:18:13.120497Z","iopub.status.idle":"2025-02-28T16:18:13.1244Z","shell.execute_reply.started":"2025-02-28T16:18:13.120467Z","shell.execute_reply":"2025-02-28T16:18:13.12338Z"}}},{"cell_type":"code","source":"!pip install git+https://github.com/WildlifeDatasets/wildlife-datasets@develop\n!pip install git+https://github.com/WildlifeDatasets/wildlife-tools","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-05-16T04:58:49.910598Z","iopub.execute_input":"2025-05-16T04:58:49.910889Z","iopub.status.idle":"2025-05-16T05:00:37.299013Z","shell.execute_reply.started":"2025-05-16T04:58:49.910866Z","shell.execute_reply":"2025-05-16T05:00:37.298086Z"},"trusted":true,"collapsed":true,"jupyter":{"outputs_hidden":true}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"<div class=\"alert alert-block alert-success\" style=\"margin: 20px; padding: 20px; border-radius: 10px; border: 2px solid #4CAF50; background-color: #E6F7E2;\">\n    <b>📂 Dependencies:</b> Successfully installed the recquired dependencies\n</div>","metadata":{}},{"cell_type":"markdown","source":"<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n    📜IMPORTING LIBRARIES📜\n</div>","metadata":{}},{"cell_type":"code","source":"import os\nimport numpy as np\nimport pandas as pd\nimport timm\nimport torchvision.transforms as T\nfrom wildlife_datasets.datasets import AnimalCLEF2025\nfrom wildlife_tools.features import DeepFeatures\nfrom wildlife_tools.similarity import CosineSimilarity\nfrom wildlife_tools.similarity.wildfusion import SimilarityPipeline, WildFusion\nfrom wildlife_tools.similarity.pairwise.lightglue import MatchLightGlue\nfrom wildlife_tools.features.local import AlikedExtractor\nfrom wildlife_tools.similarity.calibration import IsotonicCalibration\n\ndef create_sample_submission(dataset_query, predictions, file_name='sample_submission.csv'):\n    df = pd.DataFrame({\n        'image_id': dataset_query.metadata['image_id'],\n        'identity': predictions\n    })\n    df.to_csv(file_name, index=False)","metadata":{"execution":{"iopub.status.busy":"2025-05-16T05:31:43.561687Z","iopub.execute_input":"2025-05-16T05:31:43.562289Z","iopub.status.idle":"2025-05-16T05:31:43.568177Z","shell.execute_reply.started":"2025-05-16T05:31:43.562265Z","shell.execute_reply":"2025-05-16T05:31:43.567575Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"We need to specify the `root`, where the data are stored and then two image transformations. \n1. The first transform only resizes the images and is used for visualization.\n2. The second transform also converts it to torch tensor and is used for operations on neural networks.","metadata":{}},{"cell_type":"code","source":"root = '/kaggle/input/animal-clef-2025'\ntransform_display = T.Compose([\n    T.Resize([384, 384]),\n    ])\ntransform = T.Compose([\n    *transform_display.transforms,\n    T.ToTensor(),\n    T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225))\n    ])","metadata":{"execution":{"iopub.status.busy":"2025-05-16T05:03:15.061961Z","iopub.execute_input":"2025-05-16T05:03:15.062601Z","iopub.status.idle":"2025-05-16T05:03:15.067661Z","shell.execute_reply.started":"2025-05-16T05:03:15.062577Z","shell.execute_reply":"2025-05-16T05:03:15.066913Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n    🔥VISUALIZING THE DATA🔥\n</div>","metadata":{}},{"cell_type":"markdown","source":"Since `AnimalCLEF2015` is the child class of `datasets.WildlifeDataset` from [wildlife-datasets](https://github.com/WildlifeDatasets/wildlife-datasets/blob/main/wildlife_datasets/datasets/datasets.py), it inherits all its methods and attributes. The following code specify which `transform` we want to use while loading images and that we want to load labels alongside images.","metadata":{}},{"cell_type":"code","source":"dataset = AnimalCLEF2025(root, transform=transform_display, load_label=True)","metadata":{"execution":{"iopub.status.busy":"2025-05-16T05:05:12.709885Z","iopub.execute_input":"2025-05-16T05:05:12.710396Z","iopub.status.idle":"2025-05-16T05:05:52.973101Z","shell.execute_reply.started":"2025-05-16T05:05:12.710368Z","shell.execute_reply":"2025-05-16T05:05:52.972286Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"📌 **Plotting a sample grid** of the data. We can see that all animals in the testing sets are turtles. The column `dataset` states that all the photos come from the SeaTurtleID2022 dataset.","metadata":{}},{"cell_type":"code","source":"dataset.plot_grid()\ndataset.metadata","metadata":{"execution":{"iopub.status.busy":"2025-05-16T05:06:22.103942Z","iopub.execute_input":"2025-05-16T05:06:22.104618Z","iopub.status.idle":"2025-05-16T05:06:23.091637Z","shell.execute_reply.started":"2025-05-16T05:06:22.104597Z","shell.execute_reply":"2025-05-16T05:06:23.090934Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"This is confirmed by showing all datasets from which the data is composed of.","metadata":{}},{"cell_type":"code","source":"dataset.metadata[['dataset', 'split']].value_counts()","metadata":{"execution":{"iopub.status.busy":"2025-05-16T05:06:31.618985Z","iopub.execute_input":"2025-05-16T05:06:31.619623Z","iopub.status.idle":"2025-05-16T05:06:31.633301Z","shell.execute_reply.started":"2025-05-16T05:06:31.619591Z","shell.execute_reply":"2025-05-16T05:06:31.632604Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Now, we can plot the data of any dataset. Here we do it for the SeaTurtleID2022. The images are the same as in the query set. The goal for each image from the query set will be to determine its identity from the database (it will always be in the form `SeaTurtleID2022_t???` or to decide that it is a new individual (it is not present in the database).","metadata":{}},{"cell_type":"code","source":"idx = dataset.metadata['identity'].str.startswith('SeaTurtleID2022')\nidx[idx.isnull()] = False\ndataset.plot_grid(idx=idx);","metadata":{"execution":{"iopub.status.busy":"2025-05-16T05:06:33.943005Z","iopub.execute_input":"2025-05-16T05:06:33.943525Z","iopub.status.idle":"2025-05-16T05:06:34.430620Z","shell.execute_reply.started":"2025-05-16T05:06:33.943502Z","shell.execute_reply":"2025-05-16T05:06:34.430008Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n    🔥INFERENCE WITH MEGA-DESCRIPTOR🔥\n</div>","metadata":{}},{"cell_type":"markdown","source":"Instead of training a classifier, we can just use out of the shelf pretrained models - [MegaDescriptor](https://huggingface.co/BVRA/MegaDescriptor-L-384). We use MegaDescriptor to extract features from all images. \n\n**Note:** _It is highly recommended to use the GPU acceleration._","metadata":{}},{"cell_type":"code","source":"# Loading the dataset\ndataset = AnimalCLEF2025(root, transform=transform, load_label=True)\ndataset_database = dataset.get_subset(dataset.metadata['split'] == 'database')\ndataset_query = dataset.get_subset(dataset.metadata['split'] == 'query')\nn_query = len(dataset_query)","metadata":{"execution":{"iopub.status.busy":"2025-05-16T05:07:30.353529Z","iopub.execute_input":"2025-05-16T05:07:30.354276Z","iopub.status.idle":"2025-05-16T05:07:39.612671Z","shell.execute_reply.started":"2025-05-16T05:07:30.354250Z","shell.execute_reply":"2025-05-16T05:07:39.612118Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#Loading the model\nname = 'hf-hub:BVRA/MegaDescriptor-L-384'\ndevice = 'cuda'\nmodel = timm.create_model(name, num_classes=0, pretrained=True)\nextractor = DeepFeatures(model, device=device, batch_size=32, num_workers=0)\nfeatures_database = extractor(dataset_database)\nfeatures_query = extractor(dataset_query)","metadata":{"_kg_hide-input":false,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-05-16T05:07:39.613834Z","iopub.execute_input":"2025-05-16T05:07:39.614116Z","iopub.status.idle":"2025-05-16T05:23:37.650905Z","shell.execute_reply.started":"2025-05-16T05:07:39.614089Z","shell.execute_reply":"2025-05-16T05:23:37.650108Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Here we extract the similarity between the images. The similarity equals to the cosine similarity between the corresponding feature vectors. Since the cosine similarity reflects the angle between the feature vectors, high similarity means that the feature vectors are close to each other and should depict the same individual.","metadata":{}},{"cell_type":"code","source":"similarity = CosineSimilarity()(features_query, features_database)","metadata":{"execution":{"iopub.status.busy":"2025-05-16T05:25:32.605696Z","iopub.execute_input":"2025-05-16T05:25:32.607011Z","iopub.status.idle":"2025-05-16T05:25:33.014317Z","shell.execute_reply.started":"2025-05-16T05:25:32.606979Z","shell.execute_reply":"2025-05-16T05:25:33.013648Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"For each query image we extract the closest image as predicted by MegaDescriptor. Their indices are stored in `pred_idx` and the similarity score of the top match in `pred_scores`.","metadata":{}},{"cell_type":"code","source":"pred_idx = similarity.argsort(axis=1)[:,-1]\npred_scores = similarity[range(n_query), pred_idx]","metadata":{"execution":{"iopub.status.busy":"2025-05-16T05:25:35.129821Z","iopub.execute_input":"2025-05-16T05:25:35.130351Z","iopub.status.idle":"2025-05-16T05:25:35.983528Z","shell.execute_reply.started":"2025-05-16T05:25:35.130329Z","shell.execute_reply":"2025-05-16T05:25:35.982950Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"This is sufficient to perform predictions as shown below in `predictions`. However, we also need to predict whether the individual is new. We utilize the simplest idea and predict that the individual is new whenever the similarity score of the top prediction is below a certain threshold which we arbirtarily selected as 0.6. We create a submission file, which scores 30.0% on the public leaderboard.","metadata":{}},{"cell_type":"code","source":"new_individual = 'new_individual'\nthreshold = 0.6\nlabels = dataset_database.labels_string\npredictions = labels[pred_idx]\npredictions[pred_scores < threshold] = new_individual\ncreate_sample_submission(dataset_query, predictions, file_name='sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2025-05-16T05:25:37.876219Z","iopub.execute_input":"2025-05-16T05:25:37.876794Z","iopub.status.idle":"2025-05-16T05:25:37.890002Z","shell.execute_reply.started":"2025-05-16T05:25:37.876768Z","shell.execute_reply":"2025-05-16T05:25:37.889309Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"<div style=\"color:white; background-color:#C5A242; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n    🔥SECOND PART OF THIS NOTEBOOK🔥\n</div>","metadata":{}},{"cell_type":"code","source":"root = '/kaggle/input/animal-clef-2025'\ntransform_display = T.Compose([\n    T.Resize([384, 384]),\n])\ntransform = T.Compose([\n    *transform_display.transforms,\n    T.ToTensor(),\n    T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225))\n])\n\ntransforms_aliked = T.Compose([\n    T.Resize([512, 512]),\n    T.ToTensor()\n])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-16T05:32:03.205731Z","iopub.execute_input":"2025-05-16T05:32:03.206040Z","iopub.status.idle":"2025-05-16T05:32:03.211399Z","shell.execute_reply.started":"2025-05-16T05:32:03.206019Z","shell.execute_reply":"2025-05-16T05:32:03.210787Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n    🔥Loading the dataset🔥\n</div>","metadata":{}},{"cell_type":"code","source":"dataset = AnimalCLEF2025(root, load_label=True)\ndataset_database = dataset.get_subset(dataset.metadata['split'] == 'database')\ndataset_query = dataset.get_subset(dataset.metadata['split'] == 'query')\ndataset_calibration = AnimalCLEF2025(root, df=dataset_database.metadata[:100], load_label=True) \n\nn_query = len(dataset_query)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-16T05:32:11.937233Z","iopub.execute_input":"2025-05-16T05:32:11.937530Z","iopub.status.idle":"2025-05-16T05:32:36.377283Z","shell.execute_reply.started":"2025-05-16T05:32:11.937511Z","shell.execute_reply":"2025-05-16T05:32:36.376702Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n    🔥Loading the models🔥\n</div>","metadata":{}},{"cell_type":"code","source":"name = 'hf-hub:BVRA/MegaDescriptor-L-384'\nmodel = timm.create_model(name, num_classes=0, pretrained=True)\ndevice = 'cuda'\n\nmatcher_aliked = SimilarityPipeline(\n    matcher = MatchLightGlue(features='aliked', device=device, batch_size=16),\n    extractor = AlikedExtractor(),\n    transform = transforms_aliked,\n    calibration = IsotonicCalibration()\n)\n\nmatcher_mega = SimilarityPipeline(\n    matcher = CosineSimilarity(),\n    extractor = DeepFeatures(model=model, device=device, batch_size=16),\n    transform = transform,\n    calibration = IsotonicCalibration()\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-16T05:32:49.792203Z","iopub.execute_input":"2025-05-16T05:32:49.792749Z","iopub.status.idle":"2025-05-16T05:32:56.804957Z","shell.execute_reply.started":"2025-05-16T05:32:49.792710Z","shell.execute_reply":"2025-05-16T05:32:56.804254Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n    🔥Calibrating the WildFusion🔥\n</div>","metadata":{}},{"cell_type":"code","source":"wildfusion = WildFusion(calibrated_pipelines = [matcher_aliked, matcher_mega], priority_pipeline = matcher_mega)\nwildfusion.fit_calibration(dataset_calibration, dataset_calibration)\n\n# Compute WildFusion similarity\nsimilarity = wildfusion(dataset_query, dataset_database, B=25)\n\npred_idx = similarity.argsort(axis=1)[:,-1]\npred_scores = similarity[range(n_query), pred_idx]\n\nlabels = dataset_database.labels_string","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-16T05:33:33.166389Z","iopub.execute_input":"2025-05-16T05:33:33.166689Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"for threshold in [0.1, 0.2, 0.3, 0.35, 0.4, 0.5, 0.6]:\n    predictions = labels[pred_idx]\n    predictions[pred_scores < threshold] = 'new_individual'\n    create_sample_submission(dataset_query, predictions, file_name=f'sample_submission_new{int(threshold*100)}.csv')","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}