![\"image\"](\"https://www.kaggle.com/competitions/91451/images/header\")
\nAnimalCLEF25 @ CVPR-FGVC & LifeCLEF: Baseline Evaluation with BAKS and BAUS Scores
","metadata":{}},{"cell_type":"markdown","source":"\nIntroduction
\n\nThis challenge focuses on identifying individual animals from three species: **loggerhead sea turtles (Greece), salamanders (Czech Republic), and Eurasian lynxes (Czech Republic)**. Participants must develop a model that determines whether an animal in an image is **new** (not in the training set) or **known** (with a specific identity).\nTo enhance model performance, competitors can use the **WildlifeReID-10k dataset**, which contains **140,000 images** across various wildlife species.\n\n## Why It Matters\n\nAccurate animal re-identification is vital for **wildlife monitoring, conservation, and research**. However, current models often struggle with overfitting, focusing on background details rather than unique animal features. This competition aims to improve model **generalization and robustness** in real-world scenarios.\n\n## What You'll Learn\n\nParticipants will explore **machine learning techniques** for re-identification, working with **structured datasets** and learning to extract and compare keypoints in images for accurate identification.\n\n## Dataset Overview\n\n- **WildlifeReID-10k:** Large-scale dataset with 10,000 identities and 140,000 images for pre-training.\n- **Test Dataset:** New images of sea turtles, salamanders, and lynxes, split into:\n - **Database:** Labeled images of known individuals.\n - **Query:** Unlabeled images for identification.\n\n[Reference](https://www.kaggle.com/competitions/animal-clef-2025/data)","metadata":{}},{"cell_type":"markdown","source":"#Import Libraries
\n\n- Import necessary libraries like `os`, `pandas`, `numpy`, `matplotlib`, `seaborn`, `PIL`, `cv2`, `sklearn`, `tqdm`, and `random`\n","metadata":{}},{"cell_type":"code","source":"import os\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom PIL import Image\nimport cv2 # Import OpenCV\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import accuracy_score\nfrom tqdm.notebook import tqdm # For progress bars\nimport random\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-21T08:00:01.885651Z","iopub.execute_input":"2025-03-21T08:00:01.886034Z","iopub.status.idle":"2025-03-21T08:00:03.033105Z","shell.execute_reply.started":"2025-03-21T08:00:01.886000Z","shell.execute_reply":"2025-03-21T08:00:03.032010Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#Define Evaluation Metrics
\n\n- Defines a function `calculate_baks_baus` to compute BAKS (Balanced Accuracy on Known Samples) and BAUS (Balanced Accuracy on Unknown Samples). It merges ground truth and prediction dataframes and calculates accuracy for known and unknown samples separately.\n- Defines a `geometric_mean` function to compute the geometric mean of two numbers, used to combine BAKS and BAUS.\n","metadata":{}},{"cell_type":"code","source":"# --- Define Evaluation Metrics ---\ndef calculate_baks_baus(ground_truth, predictions):\n \"\"\"Calculates BAKS and BAUS scores for animal re-identification.\n\n Args:\n ground_truth (pd.DataFrame): DataFrame with 'image_id' and 'identity' columns.\n predictions (pd.DataFrame): DataFrame with 'image_id' and 'predicted_identity' columns.\n\n Returns:\n tuple: (BAKS, BAUS) scores.\n \"\"\"\n\n # Merge ground truth and predictions on 'image_id'\n merged_df = pd.merge(ground_truth, predictions, on='image_id', how='inner')\n merged_df.rename(columns={'identity': 'true_identity'}, inplace=True)\n\n # Identify known and unknown samples\n known_samples = merged_df[merged_df['true_identity'] != 'new_individual']\n unknown_samples = merged_df[merged_df['true_identity'] == 'new_individual']\n\n # Calculate BAKS (Balanced Accuracy on Known Samples)\n if not known_samples.empty:\n # Calculate accuracy for each individual\n individual_accuracies = []\n for individual in known_samples['true_identity'].unique():\n individual_data = known_samples[known_samples['true_identity'] == individual]\n accuracy = accuracy_score(individual_data['true_identity'], individual_data['predicted_identity'])\n individual_accuracies.append(accuracy)\n\n BAKS = np.mean(individual_accuracies)\n else:\n BAKS = 0.0\n\n # Calculate BAUS (Balanced Accuracy on Unknown Samples)\n if not unknown_samples.empty:\n BAUS = accuracy_score(unknown_samples['true_identity'], unknown_samples['predicted_identity'])\n else:\n BAUS = 0.0\n\n return BAKS, BAUS\n\ndef geometric_mean(a, b):\n \"\"\"Calculates the geometric mean of two numbers.\"\"\"\n return np.sqrt(a * b)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-21T08:00:09.141153Z","iopub.execute_input":"2025-03-21T08:00:09.141706Z","iopub.status.idle":"2025-03-21T08:00:09.149410Z","shell.execute_reply.started":"2025-03-21T08:00:09.141664Z","shell.execute_reply":"2025-03-21T08:00:09.148117Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#Define Submission Creation Function
\n- Defines a function `create_submission` to format predictions into a submission-ready pandas DataFrame with \"image_id\" and \"identity\" columns.\n","metadata":{}},{"cell_type":"code","source":"# --- Define Submission Creation Function ---\ndef create_submission(image_ids, predictions):\n \"\"\"Creates a submission DataFrame in the correct format.\n\n Args:\n image_ids (list): List of image IDs.\n predictions (list): List of predictions corresponding to the image IDs.\n\n Returns:\n pd.DataFrame: Submission DataFrame.\n \"\"\"\n submission = pd.DataFrame({'image_id': image_ids, 'identity': predictions})\n return submission\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-21T08:00:14.431793Z","iopub.execute_input":"2025-03-21T08:00:14.432149Z","iopub.status.idle":"2025-03-21T08:00:14.437061Z","shell.execute_reply.started":"2025-03-21T08:00:14.432118Z","shell.execute_reply":"2025-03-21T08:00:14.435825Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#Data Loading and Exploration
\n### Load Data:\n- Specifies paths to the dataset, images, metadata, and sample submission.\n- Loads metadata and sample submission into pandas DataFrames.\n\n### Explore Data:\n- Prints the head and info of the metadata and sample submission DataFrames to understand their structure.\n- Calculates and prints the value counts for the 'species' and 'split' columns in the metadata to see the distribution of categories.\n\n","metadata":{}},{"cell_type":"code","source":"# --- Data Loading and Exploration ---\ndata_path = '/kaggle/input/animal-clef-2025'\nimages_path = os.path.join(data_path, 'images')\nmetadata_path = os.path.join(data_path, 'metadata.csv')\nsample_submission_path = os.path.join(data_path, 'sample_submission.csv')\n\nmetadata_df = pd.read_csv(metadata_path)\nsample_submission_df = pd.read_csv(sample_submission_path)\n\nprint(\"Metadata Head:\")\nmetadata_df.head().style.background_gradient(cmap='plasma')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-21T08:00:19.512178Z","iopub.execute_input":"2025-03-21T08:00:19.512580Z","iopub.status.idle":"2025-03-21T08:00:19.640793Z","shell.execute_reply.started":"2025-03-21T08:00:19.512546Z","shell.execute_reply":"2025-03-21T08:00:19.639726Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(\"\\nSample Submission Head:\")\nsample_submission_df.head().style.background_gradient(cmap='plasma')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-21T08:00:23.627468Z","iopub.execute_input":"2025-03-21T08:00:23.628012Z","iopub.status.idle":"2025-03-21T08:00:23.640420Z","shell.execute_reply.started":"2025-03-21T08:00:23.627977Z","shell.execute_reply":"2025-03-21T08:00:23.639451Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(\"\\nMetadata Info:\")\nmetadata_df.info()\n\nprint(\"\\nValue Counts for 'species' column:\")\nmetadata_df['species'].value_counts()\n\nprint(\"\\nValue Counts for 'split' column:\")\nmetadata_df['split'].value_counts()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-21T08:00:26.707988Z","iopub.execute_input":"2025-03-21T08:00:26.708358Z","iopub.status.idle":"2025-03-21T08:00:26.732192Z","shell.execute_reply.started":"2025-03-21T08:00:26.708329Z","shell.execute_reply":"2025-03-21T08:00:26.731088Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#Visualize Data
\n\n- Creates several plots using `matplotlib` and `seaborn` to visualize data distributions:\n- Distribution of species in the dataset.\n- Distribution of data split (query vs. database).\n- Distribution of image orientation.\n- Time series analysis showing the number of images over years.\n- Distribution of species by split.\n- Distribution of orientation by split.","metadata":{}},{"cell_type":"code","source":"# --- Visualizations ---\n\n# 1. Distribution of Species\nplt.figure(figsize=(10, 6))\nax = sns.countplot(data=metadata_df, x='species', palette=\"viridis\") # Use a nice palette\nplt.title('Distribution of Species', fontsize=16) # Increase title size\nplt.xlabel('Species', fontsize=12) # Label x-axis\nplt.ylabel('Count', fontsize=12) # Label y-axis\nplt.xticks(rotation=45, ha='right') # Rotate x-axis labels for readability\n\n# Add value labels on top of the bars\nfor p in ax.patches:\n ax.annotate(f'{p.get_height()}', (p.get_x() + p.get_width() / 2., p.get_height()),\n ha='center', va='center', fontsize=10, color='black', xytext=(0, 5),\n textcoords='offset points')\n\nplt.tight_layout() # Adjust layout to prevent labels from overlapping\nplt.show()\n\n# 2. Distribution of Data Split (Query vs. Database)\nplt.figure(figsize=(8, 5))\nax = sns.countplot(data=metadata_df, x='split', palette=\"mako\") # Another nice palette\nplt.title('Distribution of Data Split', fontsize=16)\nplt.xlabel('Data Split', fontsize=12)\nplt.ylabel('Count', fontsize=12)\n\n# Add value labels\nfor p in ax.patches:\n ax.annotate(f'{p.get_height()}', (p.get_x() + p.get_width() / 2., p.get_height()),\n ha='center', va='center', fontsize=10, color='black', xytext=(0, 5),\n textcoords='offset points')\nplt.tight_layout()\nplt.show()\n\n# 3. Image Orientation Distribution\nplt.figure(figsize=(8, 6))\nax = sns.countplot(data=metadata_df, x='orientation', palette=\"crest\") # Yet another palette\nplt.title('Image Orientation Distribution', fontsize=16)\nplt.xlabel('Orientation', fontsize=12)\nplt.ylabel('Count', fontsize=12)\n\n# Add value labels\nfor p in ax.patches:\n ax.annotate(f'{p.get_height()}', (p.get_x() + p.get_width() / 2., p.get_height()),\n ha='center', va='center', fontsize=10, color='black', xytext=(0, 5),\n textcoords='offset points')\nplt.tight_layout()\nplt.show()\n\n# 4. Time Series Analysis (Image Dates)\n# Convert 'date' column to datetime objects\nmetadata_df['date'] = pd.to_datetime(metadata_df['date'])\n\n# Extract year and month\nmetadata_df['year'] = metadata_df['date'].dt.year\nmetadata_df['month'] = metadata_df['date'].dt.month\n\n# Plot time series\nplt.figure(figsize=(12, 6))\nsns.lineplot(data=metadata_df.groupby('year').size(), marker='o', color=\"#607c8e\") # Specific color\nplt.title('Number of Images Over Years', fontsize=16)\nplt.xlabel('Year', fontsize=12)\nplt.ylabel('Number of Images', fontsize=12)\nplt.grid(False) # Add grid lines for readability\nplt.tight_layout()\nplt.show()\n\n# 5. Species vs. Split\nplt.figure(figsize=(12, 7))\nax = sns.countplot(data=metadata_df, x='species', hue='split', palette=\"Set2\") # Palette with distinct colors\nplt.title('Species Distribution by Split', fontsize=16)\nplt.xlabel('Species', fontsize=12)\nplt.ylabel('Count', fontsize=12)\nplt.xticks(rotation=45, ha='right')\nplt.legend(title='Data Split', fontsize=10) # Add legend with title\n\n# Add value labels (slightly more complex due to hue)\nfor p in ax.patches:\n ax.annotate(f'{p.get_height()}', (p.get_x() + p.get_width() / 2., p.get_height()),\n ha='center', va='center', fontsize=8, color='black', xytext=(0, 5),\n textcoords='offset points')\nplt.tight_layout()\nplt.show()\n\n# 6. Orientation vs. Split\nplt.figure(figsize=(10, 6))\nax = sns.countplot(data=metadata_df, x='orientation', hue='split', palette=\"Paired\") # Another good palette\nplt.title('Orientation Distribution by Split', fontsize=16)\nplt.xlabel('Orientation', fontsize=12)\nplt.ylabel('Count', fontsize=12)\nplt.legend(title='Data Split', fontsize=10)\n\n# Add value labels\nfor p in ax.patches:\n ax.annotate(f'{p.get_height()}', (p.get_x() + p.get_width() / 2., p.get_height()),\n ha='center', va='center', fontsize=8, color='black', xytext=(0, 5),\n textcoords='offset points')\nplt.tight_layout()\nplt.show()\n","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-03-21T08:00:31.713345Z","iopub.execute_input":"2025-03-21T08:00:31.713706Z","iopub.status.idle":"2025-03-21T08:00:33.404058Z","shell.execute_reply.started":"2025-03-21T08:00:31.713673Z","shell.execute_reply":"2025-03-21T08:00:33.402879Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#Image Visualization
\n- Define a function `display_images` to show sample images per species using `subplots`. Load and display a set of images from each species to get a visual understanding of the data.\n","metadata":{}},{"cell_type":"code","source":"# --- Image Visualization ---\ndef display_images(metadata, num_images=5):\n \"\"\"Displays a few sample images from each species.\"\"\"\n species = metadata['species'].unique()\n num_species = len(species)\n\n # Adjust subplot grid based on number of species\n fig, axes = plt.subplots(num_species, num_images, figsize=(15, 5 * num_species))\n\n # If there's only one species, axes will be a 1D array\n if num_species == 1:\n axes = axes[np.newaxis, :] # Make it a 2D array\n\n fig.tight_layout()\n\n for i, sp in enumerate(species):\n species_data = metadata[metadata['species'] == sp]\n if len(species_data) == 0:\n print(f\"No images found for species: {sp}\")\n continue\n\n # Check if there are enough images for the sample size\n if len(species_data) < num_images:\n num_to_sample = len(species_data)\n else:\n num_to_sample = num_images\n\n try:\n species_sample = species_data.sample(num_to_sample)\n except ValueError as e:\n print(f\"Error sampling images for species {sp}: {e}\")\n continue # Skip to the next species\n\n for j in range(num_to_sample):\n image_id = species_sample.iloc[j]['image_id']\n image_path = os.path.join(data_path, species_sample.iloc[j]['path']) # Use corrected path from metadata\n\n try:\n img = Image.open(image_path)\n ax = axes[i, j]\n ax.imshow(img)\n ax.set_title(f'{sp}')\n ax.axis('off')\n except FileNotFoundError:\n print(f\"Image not found: {image_path}\")\n ax = axes[i, j]\n ax.text(0.5, 0.5, 'Image Not Found', ha='center', va='center')\n ax.axis('off')\n except Exception as e:\n print(f\"Error processing image {image_path}: {e}\")\n ax = axes[i, j]\n ax.text(0.5, 0.5, 'Error Loading Image', ha='center', va='center')\n ax.axis('off')\n plt.show()\n\ndisplay_images(metadata_df, num_images=7) # Increased to 7","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-21T08:00:50.867863Z","iopub.execute_input":"2025-03-21T08:00:50.868213Z","iopub.status.idle":"2025-03-21T08:00:56.767505Z","shell.execute_reply.started":"2025-03-21T08:00:50.868186Z","shell.execute_reply":"2025-03-21T08:00:56.766338Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#Create Baseline Prediction
\n\n- Generates a simple baseline prediction by predicting \"new_individual\" for all images in the sample submission.\n- Creates a submission DataFrame using the `create_submission` function.\n","metadata":{}},{"cell_type":"code","source":"# --- Simple Baseline Prediction ---\n\n# This is a very basic baseline. A real solution would involve image analysis.\n# This example just predicts \"new_individual\" for everything as a starting point.\n\nimage_ids = sample_submission_df['image_id'].tolist()\npredictions = ['new_individual'] * len(image_ids)\n\nbaseline_submission = create_submission(image_ids, predictions)\n\nprint(\"\\nBaseline Submission Head:\")\nbaseline_submission.head()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-21T08:01:29.312965Z","iopub.execute_input":"2025-03-21T08:01:29.313342Z","iopub.status.idle":"2025-03-21T08:01:29.325591Z","shell.execute_reply.started":"2025-03-21T08:01:29.313311Z","shell.execute_reply":"2025-03-21T08:01:29.324521Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#Evaluate Baseline
\n\n- Loads ground truth identities for query images from the metadata.\n- `Calculates BAKS and BAUS` scores using the calculate_baks_baus function, comparing the baseline predictions to the ground truth.\n- Calculates the geometric mean of BAKS and BAUS.\n- Prints the calculated scores.","metadata":{}},{"cell_type":"code","source":"# --- Evaluation ---\n# Load ground truth identities (for query images only)\nground_truth_df = metadata_df[metadata_df['split'] == 'query'][['image_id', 'identity']]\nground_truth_df.rename(columns={'identity': 'true_identity'}, inplace=True)\n\n# Create a dataframe from baseline submission for evaluation\npredictions_df = baseline_submission.copy()\npredictions_df.rename(columns={'identity': 'predicted_identity'}, inplace=True) # Align with evaluate function\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-21T08:01:36.364820Z","iopub.execute_input":"2025-03-21T08:01:36.365244Z","iopub.status.idle":"2025-03-21T08:01:36.374985Z","shell.execute_reply.started":"2025-03-21T08:01:36.365195Z","shell.execute_reply":"2025-03-21T08:01:36.374004Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Calculate BAKS and BAUS\nbaks, baus = calculate_baks_baus(ground_truth_df, predictions_df)\ngeometric_mean_score = geometric_mean(baks, baus)\n\nprint(f\"\\nBAKS: {baks:.4f}\")\nprint(f\"BAUS: {baus:.4f}\")\nprint(f\"Geometric Mean: {geometric_mean_score:.4f}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-21T08:01:40.188145Z","iopub.execute_input":"2025-03-21T08:01:40.188568Z","iopub.status.idle":"2025-03-21T08:01:40.203625Z","shell.execute_reply.started":"2025-03-21T08:01:40.188538Z","shell.execute_reply":"2025-03-21T08:01:40.202489Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#Create and Save Submission File:
\n- Saves the baseline submission DataFrame to a CSV file named \"submission.csv\".\n- Prints the head of submission data","metadata":{}},{"cell_type":"code","source":"# --- Submission ---\nsubmission_filename = 'submission.csv'\nbaseline_submission.to_csv(submission_filename, index=False)\nprint(f\"\\nSubmission file saved to: {submission_filename}\")\nprint(\"\\nSubmission file Head:\")\nbaseline_submission.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-21T08:01:45.367656Z","iopub.execute_input":"2025-03-21T08:01:45.368025Z","iopub.status.idle":"2025-03-21T08:01:45.384903Z","shell.execute_reply.started":"2025-03-21T08:01:45.367996Z","shell.execute_reply":"2025-03-21T08:01:45.383873Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}