{
  "id": 670735,
  "title": "1st Place Solution",
  "url": "/competitions/csiro-biomass/discussion/670735",
  "author_name": "TheoQiu",
  "post_date": "2026-01-29T12:19:36.694000",
  "votes": 137,
  "comment_count": 30,
  "views": 0,
  "content": "<p>Thanks to the organizers for hosting such an intense competition. We were lucky to win and I would like to thank my teammates for that. This was my first full participation in a Kaggle competition and the experience was invaluable to me 💪. Our solutions are as follows.</p>\n<p><br>\n🔵 <strong>CV Strategy</strong>\nDivide into three folds, determined jointly by <strong>the state and sampling date</strong>, and then train a single-fold model.</p>\n<p><br>\n🔵 <strong>Input Data</strong> \nSimilar to most excellent open-source notebooks, our model splits the image into left and right views and passes them through the same backbone to enhance consistency.</p>\n<p><br>\n🔵 <strong>Training Data Augmentation</strong>\nWe used various conventional data augmentation methods, as shown below:</p>\n<pre><code>def get_train_transforms(args):\n    return Compose([\n        HorizontalFlip(p=0.5),\n        VerticalFlip(p=0.5),\n        RandomRotate90(p=0.5),\n        A.GaussNoise(p=0.3),\n        A.RandomBrightnessContrast(\n            brightness_limit=0.2,\n            contrast_limit=0.2,\n            p=0.75\n        ),\n        A.HueSaturationValue(\n            hue_shift_limit=10,\n            sat_shift_limit=20,\n            val_shift_limit=20,\n            p=0.5\n        ),\n        A.CLAHE(clip_limit=2.0, tile_grid_size=(8, 8), p=0.3), \n        A.ColorJitter(\n            brightness=0.2, \n            contrast=0.2, \n            saturation=0.2, \n            hue=0.1, \n            p=0.75\n        ),\n        Normalize(\n            mean=[0.485, 0.456, 0.406],\n            std=[0.229, 0.224, 0.225]\n        ),\n        Resize(args.img_size, args.img_size),\n        ToTensorV2()\n    ])\n</code></pre>\n<p>After the process of splitting the left and right subimages, we apply data enhancement to the subimages independently. In addition, we apply a small size scaling to the images randomly and then complement them with black pixels to simulate the focal length and viewing angle differences of different cameras.</p>\n<pre><code>if random.random() &lt; 0.2:\n    background_image = np.full_like(image, 0)\n    resize_retio = random.uniform(0.85, 1.0)\n    image = cv2.resize(image, None, fx=resize_retio, fy=resize_retio, interpolation=cv2.INTER_CUBIC)\n    h, w, _ = image.shape\n    bg_h, bg_w, _ = background_image.shape\n    top = random.randint(0, bg_h - h)\n    left = random.randint(0, bg_w - w)\n    background_image[top:top + h, left:left + w] = image\n    image = background_image\n</code></pre>\n<p><br>\n🔵 <strong>Backbone Selection</strong>\nWe use DINOv3 as the backbone, which is presumably the approach used by most of the winning teams in this competition. Subsequently, we use a single multi-head self-attention layer to interact with the concatenated features of the left and right sub-images, and then output the fused features through an MLP.</p>\n<p><br>\n🔵 <strong>Head Design</strong>\nWe independently predict the biomass of each species by using <strong>five regression heads</strong>, each of which contains three linear layers. No physical constraints are imposed on them during training, which we believe reduces the model's dependence on the training set.</p>\n<p>Different from most solutions, we have additionally designed five <strong>classification heads</strong>. Given that this task is similar to the crowd counting (CC) problem without a density map, we made a point of reading a lot of work in the field of CC. Referring to the interval division scheme in UEPNet[1], we use the training set to divide seven intervals for each species:</p>\n<pre><code>  BORDERS_DICT = {\n      'Dry_Clover_g': [1.6e-05, 3.9, 10.5353, 20.6523, 37.5911, 71.7865],\n      'Dry_Dead_g': [1.6e-05, 6.1407, 13.1192, 23.277, 38.8581, 83.8407],\n      'Dry_Green_g': [1.6e-05, 13.4232, 27.0782, 45.5236, 79.834, 157.9836],\n      'Dry_Total_g': [1.6e-05, 23.4907, 41.1, 61.1, 96.8288, 185.7],\n      'GDM_g': [1.6e-05, 16.5143, 30.507, 49.5585, 81.0, 157.9836],\n  }\n</code></pre>\n<p>Based on the divided intervals, we add an additional classification head for each species to predict the interval where the biomass lies, which resulted in an improvement of approximately 0.03 in both LB and PB. This is also highly consistent with our CV. Ideally, if the intervals are divided sufficiently small, the model can accurately predict the biomass through interval inference values. (Obviously, this is too difficult for this competition without density maps/segmentation maps.)</p>\n<p><br>\n🔵 <strong>Loss Function</strong>\nThe model we finally submitted uses the common SmoothL1 Loss for regression and Cross-Entropy loss for classification. The loss of each biomass is scaled according to the weights proposed by the competition for the calculation of R^2.</p>\n<pre><code>class WeightedBiomassLoss(nn.Module):\n    def __init__(self, loss_weights_dict):\n        super(WeightedBiomassLoss, self).__init__()\n\n        self.criterion = nn.SmoothL1Loss()\n        self.cls_loss = nn.CrossEntropyLoss(reduction='none')\n        self.cls_weight = 0.3\n        self.weights = loss_weights_dict\n\n    def forward(self, predictions, targets, predictions_cls=None, cls_labels=None):\n        total_cls_loss = None\n        pred_green, pred_dead, pred_clover, pred_gdm, pred_total = predictions\n        if predictions_cls is not None:\n            assert cls_labels is not None\n            pred_green_cls, pred_dead_cls, pred_clover_cls, pred_gdm_cls, pred_total_cls = predictions_cls\n            cls_green_loss = self.cls_loss(pred_green_cls, cls_labels[:, 0])\n            cls_dead_loss = self.cls_loss(pred_dead_cls, cls_labels[:, 1])\n            cls_clover_loss = self.cls_loss(pred_clover_cls, cls_labels[:, 2])\n            cls_gdm_loss = self.cls_loss(pred_gdm_cls, cls_labels[:, 3])\n            cls_total_loss = self.cls_loss(pred_total_cls, cls_labels[:, 4])\n            total_cls_loss = (\n                self.weights['green_loss'] * cls_green_loss.mean() +\n                self.weights['dead_loss'] * cls_dead_loss.mean() +\n                self.weights['clover_loss'] * cls_clover_loss.mean() +\n                self.weights['gdm_loss'] * cls_gdm_loss.mean() +\n                self.weights['total_loss'] * cls_total_loss.mean()\n            )\n\n        true_green = targets[:, 0].unsqueeze(-1) # Shape [batch, 1]\n        true_dead   = targets[:, 1].unsqueeze(-1) # Shape [batch, 1]\n        true_clover = targets[:, 2].unsqueeze(-1) # Shape [batch, 1]\n        true_gdm = targets[:, 3].unsqueeze(-1) # Shape [batch, 1]\n        true_total = targets[:, 4].unsqueeze(-1) # Shape [batch, 1]\n\n        loss_green = self.criterion(pred_green, true_green)\n        loss_dead = self.criterion(pred_dead, true_dead)\n        loss_clover = self.criterion(pred_clover, true_clover)\n        loss_gdm   = self.criterion(pred_gdm, true_gdm)\n        loss_total = self.criterion(pred_total, true_total)\n\n        total_loss = (\n            self.weights['total_loss'] * loss_total +\n            self.weights['gdm_loss'] * loss_gdm +\n            self.weights['green_loss'] * loss_green + \n            self.weights['dead_loss'] * loss_dead +\n            self.weights['clover_loss'] * loss_clover # + \n        )\n        if predictions_cls is not None:\n            total_loss = total_loss + self.cls_weight * total_cls_loss\n        return total_loss, total_cls_loss\n</code></pre>\n<p>There are also some additional experiments that were not ultimately included in the submitted model, but PB has proven that our designs in this process are meaningful. For example, using epsilon-insensitive L1 loss for regression optimization. This observation is based on the uneven distribution of biomass, we hope that <strong>\"large values have large errors and small values have small errors\"</strong>. Therefore, we designed the following loss based on the label, which enabled us to achieve the best single-model result on PB.</p>\n<pre><code>class EpsilonInsensitiveLoss(nn.Module):\n    def __init__(self, eps_point=20, max_eps=5, reduction='mean'):\n        super().__init__()\n        self.eps_point = eps_point\n        self.reduction = reduction\n        self.scale_ratio = 0.1\n        self.max_eps = max_eps\n\n    def make_epsilon(self, target):\n        epsilon = torch.zeros_like(target)\n        epsilon = torch.where(\n            target&lt;=self.eps_point, torch.ones_like(target), epsilon\n        )\n        epsilon = torch.where(\n            target&gt;self.eps_point, target*self.scale_ratio, epsilon\n        )\n        epsilon = torch.clamp(epsilon, max=self.max_eps)\n\n        return epsilon\n\n\n    def forward(self, pred, target):\n        epsilon = self.make_epsilon(target.detach())  \n        abs_diff = torch.abs(pred - target)\n        loss = torch.relu(abs_diff - epsilon)\n\n        if self.reduction == 'mean':\n            return loss.mean()\n        elif self.reduction == 'sum':\n            return loss.sum()\n        return loss\n</code></pre>\n<p><br>\n🔵 <strong>Training Strategy</strong>\nIn the early stage of the competition, we tried various different strategies and finally chose the most effective two-stage training strategy.</p>\n<ul>\n<li><strong>Stage 1:</strong>  Freeze DINO, then train the fusion layers and MLP heads.</li>\n<li><strong>Stage 2:</strong> Fine-tune the entire model.</li>\n</ul>\n<p>Furthermore, under our setup, whether it was LB or CV, we found that the performance of DINO-Large and DINO-Huge was similar. With limited computational resources, the benefit of increasing the input size was better than increasing the model size. Therefore, in the later stages of the competition, our best single model was generally trained with the 1024 pix + large configuration.</p>\n<p><br>\n🔵 <strong>Single model results of DINOv3-Large</strong></p>\n<table>\n<thead>\n<tr>\n<th>exp_name</th>\n<th>input_size (single view)</th>\n<th>LB</th>\n<th>PB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Baseline-large</td>\n<td>512</td>\n<td>0.70</td>\n<td>0.60</td>\n</tr>\n<tr>\n<td>+cls</td>\n<td>512</td>\n<td>0.73</td>\n<td>0.63</td>\n</tr>\n<tr>\n<td>+cls</td>\n<td>1024</td>\n<td>0.74</td>\n<td>0.64</td>\n</tr>\n<tr>\n<td>+aug_scale + cls</td>\n<td>1024</td>\n<td>0.74</td>\n<td>0.65</td>\n</tr>\n<tr>\n<td>+<strong>cls + $\\epsilon$-insensitive loss</strong></td>\n<td><strong>512</strong></td>\n<td><strong>0.71</strong></td>\n<td><strong>0.66</strong></td>\n</tr>\n</tbody>\n</table>\n<p><br>\n🔵 <strong>Test Time</strong>\nOnline training during testing brought us a gain of &gt;0.02.</p>\n<ul>\n<li><strong>Step 1.</strong> We selected four single models that performed well at LB with variability to generate pseudo-labels for the test set.</li>\n<li><strong>Step 2.</strong> Based on the obtained pseudo-labels, we iteratively trained two DINO-Large models. Different from the above training, we used the entire training set and test set for training, no longer performed CV, but instead took the SWA ensemble model of the final multiple epochs. After training with pseudo-labels, we conducted training on the training set for several epochs to prevent the model from overfitting to the pseudo-labels.</li>\n<li><strong>Step 3.</strong> Based on the two DINO-Large models obtained in the previous step, we generated pseudo-labels for the second time, and then trained two DINO-Base models according to the same strategy.</li>\n<li><strong>Step 4.</strong> Fusion of the inference results of the two DINO-Large models and the two DINO-Base models with weights of 0.4 and 0.6 respectively.\nThe submission notebook has been published at <a href=\"https://www.kaggle.com/code/pingfan/lb0-77-pb-0-67-1-debug-infer-two-gpus-v1-d45a08?scriptVersionId=294488905\" target=\"_blank\">Final_Inference_Notebook</a>.</li>\n</ul>\n<p><br>\n🔵 <strong>Post-processing</strong>\nWe used some simple post-processing to slightly improve the scores of LB and PB, as follows:</p>\n<pre><code>out_green = self.head_green(combined)\nout_dead = self.head_dead(combined)\nout_clover = self.head_clover(combined)\nout_gdm = self.head_gdm(combined)\nout_total = self.head_total(combined)\n\nout_clover = out_clover * 0.8\nif out_dead &gt; 20:\n    out_dead *= 1.1\nelif out_dead &lt; 10:\n    out_dead *= 0.9\n\nout_gdm = 0.5*out_gdm+ 0.5*(out_green+out_clover)\n\npred_total1 = out_green + out_clover + out_dead\npred_total2 = out_gdm + out_dead\nout_total = 0.5*out_total + 0.5*pred_total1 + 0.0*pred_total2\n</code></pre>\n<p><br>\n🔵 <strong>Final Competition Score</strong></p>\n<table>\n<thead>\n<tr>\n<th>exp_name</th>\n<th>LB</th>\n<th>PB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>best in PB</td>\n<td>0.77</td>\n<td>0.68</td>\n</tr>\n<tr>\n<td>submitted</td>\n<td>0.77</td>\n<td>0.67</td>\n</tr>\n</tbody>\n</table>\n<p><br>\n🔵 <strong>Ideas that didn't work and Discussions</strong></p>\n<ol>\n<li>Various semi-supervised schemes 😭. Implementing and tuning these methods took up almost the my last few days of the competition, but none of them performed better than direct hard semi-supervised approaches.</li>\n<li>Training one model for one species and then merging five models.</li>\n<li>Introducing physical constraints into training, or training several heads and then calculating the remaining biomass.</li>\n<li>Removing dropout. Given that this competition is essentially a regression task, adding dropout would normally cause inconsistent variance between training and inference. However, dropout performed excellently in our experiments, significantly outperforming its removal.</li>\n<li>Generate additional unlabeled data and external datasets.</li>\n<li>Adding shadows to training images.</li>\n<li>Adding ranking loss. Due to the small amount of training data, we originally wanted to add a ranking loss to force the model to learn to compare biomass in different regions. First, we randomly cropped the training image $x$ to get $x_{crop}$ , and it is certain that $y&gt;y_{crop}$, which gives an internal ranking relationship. Then, we calculated the ranking relationships between images based on labels. This scheme performed well on the CV, once surpassing our previous best model, but got a poor LB, which might be due to overfitting again~</li>\n<li>Replace L1 loss with various other losses.</li>\n<li>Inputs of patches at multiple resolutions ([1×2,2×4,4×8,…]). Although this scheme eventually achieved a decent CV score, it oscillated severely. Facts have also proven that their final performance on LB and PB was not good (especially PB, which got a score of &lt;0.6).</li>\n<li>………..</li>\n</ol>\n<p>🎉🎉🎉\n Last but not least, congratulations to my teammate <a href=\"https://www.kaggle.com/pingfan\" target=\"_blank\">@pingfan</a> , the Kaggle community is about to have a new GM !!!\n🎉🎉🎉 </p>\n<blockquote>\n  <p>[1] Wang, Changan, Qingyu Song, Boshen Zhang, Yabiao Wang, Ying Tai, Xuyi Hu, Chengjie Wang, Jilin Li, Jiayi Ma, and Yang Wu. \"Uniformity in heterogeneity: Diving deep into count interval partition for crowd counting.\" In <em>Proceedings of the IEEE/CVF international conference on computer vision</em>, pp. 3234-3242. 2021.</p>\n</blockquote>",
  "messages": [
    {
      "id": 3398618,
      "postDate": "2026-01-29T12:19:36.693Z",
      "content": "<p>Thanks to the organizers for hosting such an intense competition. We were lucky to win and I would like to thank my teammates for that. This was my first full participation in a Kaggle competition and the experience was invaluable to me 💪. Our solutions are as follows.</p>\n<p><br>\n🔵 <strong>CV Strategy</strong>\nDivide into three folds, determined jointly by <strong>the state and sampling date</strong>, and then train a single-fold model.</p>\n<p><br>\n🔵 <strong>Input Data</strong> \nSimilar to most excellent open-source notebooks, our model splits the image into left and right views and passes them through the same backbone to enhance consistency.</p>\n<p><br>\n🔵 <strong>Training Data Augmentation</strong>\nWe used various conventional data augmentation methods, as shown below:</p>\n<pre><code>def get_train_transforms(args):\n    return Compose([\n        HorizontalFlip(p=0.5),\n        VerticalFlip(p=0.5),\n        RandomRotate90(p=0.5),\n        A.GaussNoise(p=0.3),\n        A.RandomBrightnessContrast(\n            brightness_limit=0.2,\n            contrast_limit=0.2,\n            p=0.75\n        ),\n        A.HueSaturationValue(\n            hue_shift_limit=10,\n            sat_shift_limit=20,\n            val_shift_limit=20,\n            p=0.5\n        ),\n        A.CLAHE(clip_limit=2.0, tile_grid_size=(8, 8), p=0.3), \n        A.ColorJitter(\n            brightness=0.2, \n            contrast=0.2, \n            saturation=0.2, \n            hue=0.1, \n            p=0.75\n        ),\n        Normalize(\n            mean=[0.485, 0.456, 0.406],\n            std=[0.229, 0.224, 0.225]\n        ),\n        Resize(args.img_size, args.img_size),\n        ToTensorV2()\n    ])\n</code></pre>\n<p>After the process of splitting the left and right subimages, we apply data enhancement to the subimages independently. In addition, we apply a small size scaling to the images randomly and then complement them with black pixels to simulate the focal length and viewing angle differences of different cameras.</p>\n<pre><code>if random.random() &lt; 0.2:\n    background_image = np.full_like(image, 0)\n    resize_retio = random.uniform(0.85, 1.0)\n    image = cv2.resize(image, None, fx=resize_retio, fy=resize_retio, interpolation=cv2.INTER_CUBIC)\n    h, w, _ = image.shape\n    bg_h, bg_w, _ = background_image.shape\n    top = random.randint(0, bg_h - h)\n    left = random.randint(0, bg_w - w)\n    background_image[top:top + h, left:left + w] = image\n    image = background_image\n</code></pre>\n<p><br>\n🔵 <strong>Backbone Selection</strong>\nWe use DINOv3 as the backbone, which is presumably the approach used by most of the winning teams in this competition. Subsequently, we use a single multi-head self-attention layer to interact with the concatenated features of the left and right sub-images, and then output the fused features through an MLP.</p>\n<p><br>\n🔵 <strong>Head Design</strong>\nWe independently predict the biomass of each species by using <strong>five regression heads</strong>, each of which contains three linear layers. No physical constraints are imposed on them during training, which we believe reduces the model's dependence on the training set.</p>\n<p>Different from most solutions, we have additionally designed five <strong>classification heads</strong>. Given that this task is similar to the crowd counting (CC) problem without a density map, we made a point of reading a lot of work in the field of CC. Referring to the interval division scheme in UEPNet[1], we use the training set to divide seven intervals for each species:</p>\n<pre><code>  BORDERS_DICT = {\n      'Dry_Clover_g': [1.6e-05, 3.9, 10.5353, 20.6523, 37.5911, 71.7865],\n      'Dry_Dead_g': [1.6e-05, 6.1407, 13.1192, 23.277, 38.8581, 83.8407],\n      'Dry_Green_g': [1.6e-05, 13.4232, 27.0782, 45.5236, 79.834, 157.9836],\n      'Dry_Total_g': [1.6e-05, 23.4907, 41.1, 61.1, 96.8288, 185.7],\n      'GDM_g': [1.6e-05, 16.5143, 30.507, 49.5585, 81.0, 157.9836],\n  }\n</code></pre>\n<p>Based on the divided intervals, we add an additional classification head for each species to predict the interval where the biomass lies, which resulted in an improvement of approximately 0.03 in both LB and PB. This is also highly consistent with our CV. Ideally, if the intervals are divided sufficiently small, the model can accurately predict the biomass through interval inference values. (Obviously, this is too difficult for this competition without density maps/segmentation maps.)</p>\n<p><br>\n🔵 <strong>Loss Function</strong>\nThe model we finally submitted uses the common SmoothL1 Loss for regression and Cross-Entropy loss for classification. The loss of each biomass is scaled according to the weights proposed by the competition for the calculation of R^2.</p>\n<pre><code>class WeightedBiomassLoss(nn.Module):\n    def __init__(self, loss_weights_dict):\n        super(WeightedBiomassLoss, self).__init__()\n\n        self.criterion = nn.SmoothL1Loss()\n        self.cls_loss = nn.CrossEntropyLoss(reduction='none')\n        self.cls_weight = 0.3\n        self.weights = loss_weights_dict\n\n    def forward(self, predictions, targets, predictions_cls=None, cls_labels=None):\n        total_cls_loss = None\n        pred_green, pred_dead, pred_clover, pred_gdm, pred_total = predictions\n        if predictions_cls is not None:\n            assert cls_labels is not None\n            pred_green_cls, pred_dead_cls, pred_clover_cls, pred_gdm_cls, pred_total_cls = predictions_cls\n            cls_green_loss = self.cls_loss(pred_green_cls, cls_labels[:, 0])\n            cls_dead_loss = self.cls_loss(pred_dead_cls, cls_labels[:, 1])\n            cls_clover_loss = self.cls_loss(pred_clover_cls, cls_labels[:, 2])\n            cls_gdm_loss = self.cls_loss(pred_gdm_cls, cls_labels[:, 3])\n            cls_total_loss = self.cls_loss(pred_total_cls, cls_labels[:, 4])\n            total_cls_loss = (\n                self.weights['green_loss'] * cls_green_loss.mean() +\n                self.weights['dead_loss'] * cls_dead_loss.mean() +\n                self.weights['clover_loss'] * cls_clover_loss.mean() +\n                self.weights['gdm_loss'] * cls_gdm_loss.mean() +\n                self.weights['total_loss'] * cls_total_loss.mean()\n            )\n\n        true_green = targets[:, 0].unsqueeze(-1) # Shape [batch, 1]\n        true_dead   = targets[:, 1].unsqueeze(-1) # Shape [batch, 1]\n        true_clover = targets[:, 2].unsqueeze(-1) # Shape [batch, 1]\n        true_gdm = targets[:, 3].unsqueeze(-1) # Shape [batch, 1]\n        true_total = targets[:, 4].unsqueeze(-1) # Shape [batch, 1]\n\n        loss_green = self.criterion(pred_green, true_green)\n        loss_dead = self.criterion(pred_dead, true_dead)\n        loss_clover = self.criterion(pred_clover, true_clover)\n        loss_gdm   = self.criterion(pred_gdm, true_gdm)\n        loss_total = self.criterion(pred_total, true_total)\n\n        total_loss = (\n            self.weights['total_loss'] * loss_total +\n            self.weights['gdm_loss'] * loss_gdm +\n            self.weights['green_loss'] * loss_green + \n            self.weights['dead_loss'] * loss_dead +\n            self.weights['clover_loss'] * loss_clover # + \n        )\n        if predictions_cls is not None:\n            total_loss = total_loss + self.cls_weight * total_cls_loss\n        return total_loss, total_cls_loss\n</code></pre>\n<p>There are also some additional experiments that were not ultimately included in the submitted model, but PB has proven that our designs in this process are meaningful. For example, using epsilon-insensitive L1 loss for regression optimization. This observation is based on the uneven distribution of biomass, we hope that <strong>\"large values have large errors and small values have small errors\"</strong>. Therefore, we designed the following loss based on the label, which enabled us to achieve the best single-model result on PB.</p>\n<pre><code>class EpsilonInsensitiveLoss(nn.Module):\n    def __init__(self, eps_point=20, max_eps=5, reduction='mean'):\n        super().__init__()\n        self.eps_point = eps_point\n        self.reduction = reduction\n        self.scale_ratio = 0.1\n        self.max_eps = max_eps\n\n    def make_epsilon(self, target):\n        epsilon = torch.zeros_like(target)\n        epsilon = torch.where(\n            target&lt;=self.eps_point, torch.ones_like(target), epsilon\n        )\n        epsilon = torch.where(\n            target&gt;self.eps_point, target*self.scale_ratio, epsilon\n        )\n        epsilon = torch.clamp(epsilon, max=self.max_eps)\n\n        return epsilon\n\n\n    def forward(self, pred, target):\n        epsilon = self.make_epsilon(target.detach())  \n        abs_diff = torch.abs(pred - target)\n        loss = torch.relu(abs_diff - epsilon)\n\n        if self.reduction == 'mean':\n            return loss.mean()\n        elif self.reduction == 'sum':\n            return loss.sum()\n        return loss\n</code></pre>\n<p><br>\n🔵 <strong>Training Strategy</strong>\nIn the early stage of the competition, we tried various different strategies and finally chose the most effective two-stage training strategy.</p>\n<ul>\n<li><strong>Stage 1:</strong>  Freeze DINO, then train the fusion layers and MLP heads.</li>\n<li><strong>Stage 2:</strong> Fine-tune the entire model.</li>\n</ul>\n<p>Furthermore, under our setup, whether it was LB or CV, we found that the performance of DINO-Large and DINO-Huge was similar. With limited computational resources, the benefit of increasing the input size was better than increasing the model size. Therefore, in the later stages of the competition, our best single model was generally trained with the 1024 pix + large configuration.</p>\n<p><br>\n🔵 <strong>Single model results of DINOv3-Large</strong></p>\n<table>\n<thead>\n<tr>\n<th>exp_name</th>\n<th>input_size (single view)</th>\n<th>LB</th>\n<th>PB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Baseline-large</td>\n<td>512</td>\n<td>0.70</td>\n<td>0.60</td>\n</tr>\n<tr>\n<td>+cls</td>\n<td>512</td>\n<td>0.73</td>\n<td>0.63</td>\n</tr>\n<tr>\n<td>+cls</td>\n<td>1024</td>\n<td>0.74</td>\n<td>0.64</td>\n</tr>\n<tr>\n<td>+aug_scale + cls</td>\n<td>1024</td>\n<td>0.74</td>\n<td>0.65</td>\n</tr>\n<tr>\n<td>+<strong>cls + $\\epsilon$-insensitive loss</strong></td>\n<td><strong>512</strong></td>\n<td><strong>0.71</strong></td>\n<td><strong>0.66</strong></td>\n</tr>\n</tbody>\n</table>\n<p><br>\n🔵 <strong>Test Time</strong>\nOnline training during testing brought us a gain of &gt;0.02.</p>\n<ul>\n<li><strong>Step 1.</strong> We selected four single models that performed well at LB with variability to generate pseudo-labels for the test set.</li>\n<li><strong>Step 2.</strong> Based on the obtained pseudo-labels, we iteratively trained two DINO-Large models. Different from the above training, we used the entire training set and test set for training, no longer performed CV, but instead took the SWA ensemble model of the final multiple epochs. After training with pseudo-labels, we conducted training on the training set for several epochs to prevent the model from overfitting to the pseudo-labels.</li>\n<li><strong>Step 3.</strong> Based on the two DINO-Large models obtained in the previous step, we generated pseudo-labels for the second time, and then trained two DINO-Base models according to the same strategy.</li>\n<li><strong>Step 4.</strong> Fusion of the inference results of the two DINO-Large models and the two DINO-Base models with weights of 0.4 and 0.6 respectively.\nThe submission notebook has been published at <a href=\"https://www.kaggle.com/code/pingfan/lb0-77-pb-0-67-1-debug-infer-two-gpus-v1-d45a08?scriptVersionId=294488905\" target=\"_blank\">Final_Inference_Notebook</a>.</li>\n</ul>\n<p><br>\n🔵 <strong>Post-processing</strong>\nWe used some simple post-processing to slightly improve the scores of LB and PB, as follows:</p>\n<pre><code>out_green = self.head_green(combined)\nout_dead = self.head_dead(combined)\nout_clover = self.head_clover(combined)\nout_gdm = self.head_gdm(combined)\nout_total = self.head_total(combined)\n\nout_clover = out_clover * 0.8\nif out_dead &gt; 20:\n    out_dead *= 1.1\nelif out_dead &lt; 10:\n    out_dead *= 0.9\n\nout_gdm = 0.5*out_gdm+ 0.5*(out_green+out_clover)\n\npred_total1 = out_green + out_clover + out_dead\npred_total2 = out_gdm + out_dead\nout_total = 0.5*out_total + 0.5*pred_total1 + 0.0*pred_total2\n</code></pre>\n<p><br>\n🔵 <strong>Final Competition Score</strong></p>\n<table>\n<thead>\n<tr>\n<th>exp_name</th>\n<th>LB</th>\n<th>PB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>best in PB</td>\n<td>0.77</td>\n<td>0.68</td>\n</tr>\n<tr>\n<td>submitted</td>\n<td>0.77</td>\n<td>0.67</td>\n</tr>\n</tbody>\n</table>\n<p><br>\n🔵 <strong>Ideas that didn't work and Discussions</strong></p>\n<ol>\n<li>Various semi-supervised schemes 😭. Implementing and tuning these methods took up almost the my last few days of the competition, but none of them performed better than direct hard semi-supervised approaches.</li>\n<li>Training one model for one species and then merging five models.</li>\n<li>Introducing physical constraints into training, or training several heads and then calculating the remaining biomass.</li>\n<li>Removing dropout. Given that this competition is essentially a regression task, adding dropout would normally cause inconsistent variance between training and inference. However, dropout performed excellently in our experiments, significantly outperforming its removal.</li>\n<li>Generate additional unlabeled data and external datasets.</li>\n<li>Adding shadows to training images.</li>\n<li>Adding ranking loss. Due to the small amount of training data, we originally wanted to add a ranking loss to force the model to learn to compare biomass in different regions. First, we randomly cropped the training image $x$ to get $x_{crop}$ , and it is certain that $y&gt;y_{crop}$, which gives an internal ranking relationship. Then, we calculated the ranking relationships between images based on labels. This scheme performed well on the CV, once surpassing our previous best model, but got a poor LB, which might be due to overfitting again~</li>\n<li>Replace L1 loss with various other losses.</li>\n<li>Inputs of patches at multiple resolutions ([1×2,2×4,4×8,…]). Although this scheme eventually achieved a decent CV score, it oscillated severely. Facts have also proven that their final performance on LB and PB was not good (especially PB, which got a score of &lt;0.6).</li>\n<li>………..</li>\n</ol>\n<p>🎉🎉🎉\n Last but not least, congratulations to my teammate <a href=\"https://www.kaggle.com/pingfan\" target=\"_blank\">@pingfan</a> , the Kaggle community is about to have a new GM !!!\n🎉🎉🎉 </p>\n<blockquote>\n  <p>[1] Wang, Changan, Qingyu Song, Boshen Zhang, Yabiao Wang, Ying Tai, Xuyi Hu, Chengjie Wang, Jilin Li, Jiayi Ma, and Yang Wu. \"Uniformity in heterogeneity: Diving deep into count interval partition for crowd counting.\" In <em>Proceedings of the IEEE/CVF international conference on computer vision</em>, pp. 3234-3242. 2021.</p>\n</blockquote>",
      "rawMarkdown": "Thanks to the organizers for hosting such an intense competition. We were lucky to win and I would like to thank my teammates for that. This was my first full participation in a Kaggle competition and the experience was invaluable to me 💪. Our solutions are as follows.\n\n<br>\n🔵 **CV Strategy**\nDivide into three folds, determined jointly by **the state and sampling date**, and then train a single-fold model.\n\n<br>\n🔵 **Input Data** \nSimilar to most excellent open-source notebooks, our model splits the image into left and right views and passes them through the same backbone to enhance consistency.\n\n<br>\n🔵 **Training Data Augmentation**\nWe used various conventional data augmentation methods, as shown below:\n\n```python\ndef get_train_transforms(args):\n    return Compose([\n        HorizontalFlip(p=0.5),\n        VerticalFlip(p=0.5),\n        RandomRotate90(p=0.5),\n        A.GaussNoise(p=0.3),\n        A.RandomBrightnessContrast(\n            brightness_limit=0.2,\n            contrast_limit=0.2,\n            p=0.75\n        ),\n        A.HueSaturationValue(\n            hue_shift_limit=10,\n            sat_shift_limit=20,\n            val_shift_limit=20,\n            p=0.5\n        ),\n        A.CLAHE(clip_limit=2.0, tile_grid_size=(8, 8), p=0.3), \n        A.ColorJitter(\n            brightness=0.2, \n            contrast=0.2, \n            saturation=0.2, \n            hue=0.1, \n            p=0.75\n        ),\n        Normalize(\n            mean=[0.485, 0.456, 0.406],\n            std=[0.229, 0.224, 0.225]\n        ),\n        Resize(args.img_size, args.img_size),\n        ToTensorV2()\n    ])\n\n```\n\nAfter the process of splitting the left and right subimages, we apply data enhancement to the subimages independently. In addition, we apply a small size scaling to the images randomly and then complement them with black pixels to simulate the focal length and viewing angle differences of different cameras.\n\n```python\nif random.random() < 0.2:\n    background_image = np.full_like(image, 0)\n    resize_retio = random.uniform(0.85, 1.0)\n    image = cv2.resize(image, None, fx=resize_retio, fy=resize_retio, interpolation=cv2.INTER_CUBIC)\n    h, w, _ = image.shape\n    bg_h, bg_w, _ = background_image.shape\n    top = random.randint(0, bg_h - h)\n    left = random.randint(0, bg_w - w)\n    background_image[top:top + h, left:left + w] = image\n    image = background_image\n```\n\n<br>\n🔵 **Backbone Selection**\nWe use DINOv3 as the backbone, which is presumably the approach used by most of the winning teams in this competition. Subsequently, we use a single multi-head self-attention layer to interact with the concatenated features of the left and right sub-images, and then output the fused features through an MLP.\n\n<br>\n🔵 **Head Design**\nWe independently predict the biomass of each species by using **five regression heads**, each of which contains three linear layers. No physical constraints are imposed on them during training, which we believe reduces the model's dependence on the training set.\n\nDifferent from most solutions, we have additionally designed five **classification heads**. Given that this task is similar to the crowd counting (CC) problem without a density map, we made a point of reading a lot of work in the field of CC. Referring to the interval division scheme in UEPNet[1], we use the training set to divide seven intervals for each species:\n```python\n  BORDERS_DICT = {\n      'Dry_Clover_g': [1.6e-05, 3.9, 10.5353, 20.6523, 37.5911, 71.7865],\n      'Dry_Dead_g': [1.6e-05, 6.1407, 13.1192, 23.277, 38.8581, 83.8407],\n      'Dry_Green_g': [1.6e-05, 13.4232, 27.0782, 45.5236, 79.834, 157.9836],\n      'Dry_Total_g': [1.6e-05, 23.4907, 41.1, 61.1, 96.8288, 185.7],\n      'GDM_g': [1.6e-05, 16.5143, 30.507, 49.5585, 81.0, 157.9836],\n  }\n```\n\nBased on the divided intervals, we add an additional classification head for each species to predict the interval where the biomass lies, which resulted in an improvement of approximately 0.03 in both LB and PB. This is also highly consistent with our CV. Ideally, if the intervals are divided sufficiently small, the model can accurately predict the biomass through interval inference values. (Obviously, this is too difficult for this competition without density maps/segmentation maps.)\n\n<br>\n🔵 **Loss Function**\nThe model we finally submitted uses the common SmoothL1 Loss for regression and Cross-Entropy loss for classification. The loss of each biomass is scaled according to the weights proposed by the competition for the calculation of R^2.\n\n```python\nclass WeightedBiomassLoss(nn.Module):\n    def __init__(self, loss_weights_dict):\n        super(WeightedBiomassLoss, self).__init__()\n        \n        self.criterion = nn.SmoothL1Loss()\n        self.cls_loss = nn.CrossEntropyLoss(reduction='none')\n        self.cls_weight = 0.3\n        self.weights = loss_weights_dict\n\n    def forward(self, predictions, targets, predictions_cls=None, cls_labels=None):\n        total_cls_loss = None\n        pred_green, pred_dead, pred_clover, pred_gdm, pred_total = predictions\n        if predictions_cls is not None:\n            assert cls_labels is not None\n            pred_green_cls, pred_dead_cls, pred_clover_cls, pred_gdm_cls, pred_total_cls = predictions_cls\n            cls_green_loss = self.cls_loss(pred_green_cls, cls_labels[:, 0])\n            cls_dead_loss = self.cls_loss(pred_dead_cls, cls_labels[:, 1])\n            cls_clover_loss = self.cls_loss(pred_clover_cls, cls_labels[:, 2])\n            cls_gdm_loss = self.cls_loss(pred_gdm_cls, cls_labels[:, 3])\n            cls_total_loss = self.cls_loss(pred_total_cls, cls_labels[:, 4])\n            total_cls_loss = (\n                self.weights['green_loss'] * cls_green_loss.mean() +\n                self.weights['dead_loss'] * cls_dead_loss.mean() +\n                self.weights['clover_loss'] * cls_clover_loss.mean() +\n                self.weights['gdm_loss'] * cls_gdm_loss.mean() +\n                self.weights['total_loss'] * cls_total_loss.mean()\n            )\n        \n        true_green = targets[:, 0].unsqueeze(-1) # Shape [batch, 1]\n        true_dead   = targets[:, 1].unsqueeze(-1) # Shape [batch, 1]\n        true_clover = targets[:, 2].unsqueeze(-1) # Shape [batch, 1]\n        true_gdm = targets[:, 3].unsqueeze(-1) # Shape [batch, 1]\n        true_total = targets[:, 4].unsqueeze(-1) # Shape [batch, 1]\n\n        loss_green = self.criterion(pred_green, true_green)\n        loss_dead = self.criterion(pred_dead, true_dead)\n        loss_clover = self.criterion(pred_clover, true_clover)\n        loss_gdm   = self.criterion(pred_gdm, true_gdm)\n        loss_total = self.criterion(pred_total, true_total)\n        \n        total_loss = (\n            self.weights['total_loss'] * loss_total +\n            self.weights['gdm_loss'] * loss_gdm +\n            self.weights['green_loss'] * loss_green + \n            self.weights['dead_loss'] * loss_dead +\n            self.weights['clover_loss'] * loss_clover # + \n        )\n        if predictions_cls is not None:\n            total_loss = total_loss + self.cls_weight * total_cls_loss\n        return total_loss, total_cls_loss\n\n```\n\nThere are also some additional experiments that were not ultimately included in the submitted model, but PB has proven that our designs in this process are meaningful. For example, using epsilon-insensitive L1 loss for regression optimization. This observation is based on the uneven distribution of biomass, we hope that **\"large values have large errors and small values have small errors\"**. Therefore, we designed the following loss based on the label, which enabled us to achieve the best single-model result on PB.\n\n```python\nclass EpsilonInsensitiveLoss(nn.Module):\n    def __init__(self, eps_point=20, max_eps=5, reduction='mean'):\n        super().__init__()\n        self.eps_point = eps_point\n        self.reduction = reduction\n        self.scale_ratio = 0.1\n        self.max_eps = max_eps\n    \n    def make_epsilon(self, target):\n        epsilon = torch.zeros_like(target)\n        epsilon = torch.where(\n            target<=self.eps_point, torch.ones_like(target), epsilon\n        )\n        epsilon = torch.where(\n            target>self.eps_point, target*self.scale_ratio, epsilon\n        )\n        epsilon = torch.clamp(epsilon, max=self.max_eps)\n    \n        return epsilon\n\n\n    def forward(self, pred, target):\n        epsilon = self.make_epsilon(target.detach())  \n        abs_diff = torch.abs(pred - target)\n        loss = torch.relu(abs_diff - epsilon)\n        \n        if self.reduction == 'mean':\n            return loss.mean()\n        elif self.reduction == 'sum':\n            return loss.sum()\n        return loss\n\n```\n\n<br>\n🔵 **Training Strategy**\nIn the early stage of the competition, we tried various different strategies and finally chose the most effective two-stage training strategy.\n- **Stage 1:**  Freeze DINO, then train the fusion layers and MLP heads.\n- **Stage 2:** Fine-tune the entire model.\n\nFurthermore, under our setup, whether it was LB or CV, we found that the performance of DINO-Large and DINO-Huge was similar. With limited computational resources, the benefit of increasing the input size was better than increasing the model size. Therefore, in the later stages of the competition, our best single model was generally trained with the 1024 pix + large configuration.\n\n<br>\n🔵 **Single model results of DINOv3-Large**\n\n| exp_name                               | input_size (single view) | LB       | PB       |\n| -------------------------------------- | ------------------------ | -------- | -------- |\n| Baseline-large                         | 512                      | 0.70     | 0.60     |\n| +cls                                   | 512                      | 0.73     | 0.63     |\n| +cls                                   | 1024                     | 0.74     | 0.64     |\n| +aug_scale + cls                       | 1024                     | 0.74     | 0.65     |\n| +**cls + $\\epsilon$-insensitive loss** | **512**                  | **0.71** | **0.66** |\n\n<br>\n🔵 **Test Time**\nOnline training during testing brought us a gain of >0.02.\n- **Step 1.** We selected four single models that performed well at LB with variability to generate pseudo-labels for the test set.\n- **Step 2.** Based on the obtained pseudo-labels, we iteratively trained two DINO-Large models. Different from the above training, we used the entire training set and test set for training, no longer performed CV, but instead took the SWA ensemble model of the final multiple epochs. After training with pseudo-labels, we conducted training on the training set for several epochs to prevent the model from overfitting to the pseudo-labels.\n- **Step 3.** Based on the two DINO-Large models obtained in the previous step, we generated pseudo-labels for the second time, and then trained two DINO-Base models according to the same strategy.\n- **Step 4.** Fusion of the inference results of the two DINO-Large models and the two DINO-Base models with weights of 0.4 and 0.6 respectively.\nThe submission notebook has been published at [Final_Inference_Notebook](https://www.kaggle.com/code/pingfan/lb0-77-pb-0-67-1-debug-infer-two-gpus-v1-d45a08?scriptVersionId=294488905).\n\n<br>\n🔵 **Post-processing**\nWe used some simple post-processing to slightly improve the scores of LB and PB, as follows:\n\n```python\nout_green = self.head_green(combined)\nout_dead = self.head_dead(combined)\nout_clover = self.head_clover(combined)\nout_gdm = self.head_gdm(combined)\nout_total = self.head_total(combined)\n\nout_clover = out_clover * 0.8\nif out_dead > 20:\n    out_dead *= 1.1\nelif out_dead < 10:\n    out_dead *= 0.9\n\nout_gdm = 0.5*out_gdm+ 0.5*(out_green+out_clover)\n\npred_total1 = out_green + out_clover + out_dead\npred_total2 = out_gdm + out_dead\nout_total = 0.5*out_total + 0.5*pred_total1 + 0.0*pred_total2\n```\n\n<br>\n🔵 **Final Competition Score**\n\n| exp_name     | LB   | PB   |\n| ------------ | ---- | ---- |\n| best in PB   | 0.77 | 0.68 |\n| submitted | 0.77 | 0.67 |\n\n<br>\n🔵 **Ideas that didn't work and Discussions**\n1. Various semi-supervised schemes 😭. Implementing and tuning these methods took up almost the my last few days of the competition, but none of them performed better than direct hard semi-supervised approaches.\n2. Training one model for one species and then merging five models.\n3. Introducing physical constraints into training, or training several heads and then calculating the remaining biomass.\n4. Removing dropout. Given that this competition is essentially a regression task, adding dropout would normally cause inconsistent variance between training and inference. However, dropout performed excellently in our experiments, significantly outperforming its removal.\n5. Generate additional unlabeled data and external datasets.\n6. Adding shadows to training images.\n7. Adding ranking loss. Due to the small amount of training data, we originally wanted to add a ranking loss to force the model to learn to compare biomass in different regions. First, we randomly cropped the training image $x$ to get $x_{crop}$ , and it is certain that $y>y_{crop}$, which gives an internal ranking relationship. Then, we calculated the ranking relationships between images based on labels. This scheme performed well on the CV, once surpassing our previous best model, but got a poor LB, which might be due to overfitting again~\n8. Replace L1 loss with various other losses.\n9. Inputs of patches at multiple resolutions ([1×2,2×4,4×8,...]). Although this scheme eventually achieved a decent CV score, it oscillated severely. Facts have also proven that their final performance on LB and PB was not good (especially PB, which got a score of <0.6).\n10. ...........\n\n\n🎉🎉🎉\n Last but not least, congratulations to my teammate @pingfan , the Kaggle community is about to have a new GM !!!\n🎉🎉🎉 \n\n> [1] Wang, Changan, Qingyu Song, Boshen Zhang, Yabiao Wang, Ying Tai, Xuyi Hu, Chengjie Wang, Jilin Li, Jiayi Ma, and Yang Wu. \"Uniformity in heterogeneity: Diving deep into count interval partition for crowd counting.\" In *Proceedings of the IEEE/CVF international conference on computer vision*, pp. 3234-3242. 2021.\n\n",
      "votes": 137
    },
    {
      "id": 3398624,
      "postDate": "2026-01-29T12:28:22.747Z",
      "content": "<p>This article is also my first one.🤣</p>",
      "rawMarkdown": "This article is also my first one.🤣",
      "votes": 7,
      "replies": [
        {
          "id": 3398632,
          "postDate": "2026-01-29T12:49:49.120Z",
          "content": "<p>Damn bro, this is a great job! </p>",
          "rawMarkdown": "Damn bro, this is a great job! "
        },
        {
          "id": 3398655,
          "postDate": "2026-01-29T13:27:49.060Z",
          "content": "<p>Congratulations!</p>",
          "rawMarkdown": "Congratulations!"
        },
        {
          "id": 3398688,
          "postDate": "2026-01-29T14:47:35.970Z",
          "content": "<p>congarts super work</p>",
          "rawMarkdown": "congarts super work"
        }
      ]
    },
    {
      "id": 3399288,
      "postDate": "2026-01-30T15:20:30.097Z",
      "content": "<p>I liked this solution: no many ensembles, just logical methods, ideas and working with data - this definitely deserves first place🥇</p>",
      "rawMarkdown": "I liked this solution: no many ensembles, just logical methods, ideas and working with data - this definitely deserves first place🥇",
      "votes": 4
    },
    {
      "id": 3399263,
      "postDate": "2026-01-30T14:23:43.253Z",
      "content": "<p>Really good work and lots of ideas for me to learn from! May I ask, how did you select your training iteration during training? When did you stop training and approx. how many iterations did that take and what was your batch size?</p>",
      "rawMarkdown": "Really good work and lots of ideas for me to learn from! May I ask, how did you select your training iteration during training? When did you stop training and approx. how many iterations did that take and what was your batch size?",
      "votes": 1
    },
    {
      "id": 3398694,
      "postDate": "2026-01-29T14:54:14.130Z",
      "content": "<p>Well done! The GM tricks are very impressive.\nWhy choose the weights arbitrarily instead of using a grid search to optimize the OOF score?</p>",
      "rawMarkdown": "Well done! The GM tricks are very impressive.\nWhy choose the weights arbitrarily instead of using a grid search to optimize the OOF score?",
      "votes": 1,
      "replies": [
        {
          "id": 3399249,
          "postDate": "2026-01-30T13:53:30.590Z",
          "content": "<p>Our method varies greatly between folds, so we did not train multiple folds.</p>",
          "rawMarkdown": "Our method varies greatly between folds, so we did not train multiple folds."
        }
      ]
    },
    {
      "id": 3398663,
      "postDate": "2026-01-29T13:59:31.237Z",
      "content": "<p>Great job guys! getting inspo from crowd counting lmao that's smart AF. </p>",
      "rawMarkdown": "Great job guys! getting inspo from crowd counting lmao that's smart AF. ",
      "votes": 1,
      "replies": [
        {
          "id": 3399243,
          "postDate": "2026-01-30T13:42:28.673Z",
          "content": "<p>😂 CC is indeed very important to us.</p>",
          "rawMarkdown": "😂 CC is indeed very important to us.",
          "votes": 1
        }
      ]
    },
    {
      "id": 3398639,
      "postDate": "2026-01-29T12:53:38.220Z",
      "content": "<p>Code for interval division implemented with reference to the paper.</p>\n<pre><code>def uep_partition(all_counts, m, t0, search_iter=100):\n    counts_sorted = np.sort([d for d in all_counts if d &gt; t0])\n    if len(counts_sorted) == 0:\n        return [t0]\n    L = 0.0\n    H = (counts_sorted[-1] - t0) * len(counts_sorted) \n    final_borders = []\n    for _ in range(search_iter):\n        target_nl = (L + H) / 2.0\n        p = t0\n        n = 0\n        current_borders = [t0]\n        for d_k in counts_sorted:\n            n += 1\n            current_error = (d_k - p) * n\n\n            if current_error &gt; target_nl:\n                p = d_k\n                current_borders.append(p)\n                n = 0\n        if len(current_borders) &gt;= m:\n            L = target_nl\n            final_borders = current_borders\n        else:\n            H = target_nl\n    return sorted(list(set(final_borders)))\n</code></pre>",
      "rawMarkdown": "Code for interval division implemented with reference to the paper.\n```python\ndef uep_partition(all_counts, m, t0, search_iter=100):\n    counts_sorted = np.sort([d for d in all_counts if d > t0])\n    if len(counts_sorted) == 0:\n        return [t0]\n    L = 0.0\n    H = (counts_sorted[-1] - t0) * len(counts_sorted) \n    final_borders = []\n    for _ in range(search_iter):\n        target_nl = (L + H) / 2.0\n        p = t0\n        n = 0\n        current_borders = [t0]\n        for d_k in counts_sorted:\n            n += 1\n            current_error = (d_k - p) * n\n            \n            if current_error > target_nl:\n                p = d_k\n                current_borders.append(p)\n                n = 0\n        if len(current_borders) >= m:\n            L = target_nl\n            final_borders = current_borders\n        else:\n            H = target_nl\n    return sorted(list(set(final_borders)))\n```",
      "votes": 1
    },
    {
      "id": 3398630,
      "postDate": "2026-01-29T12:44:46.870Z",
      "content": "<p>Great job, team! Congratulations! 💯💯</p>\n<p>Q: How did you come to the conclusions about the \"🔵 Post-processing We used some simple post-processing to slightly improve the scores of LB and PB, as follows:\" step. was is done by sweeping some different settings? or rather based on some data assumptions?</p>",
      "rawMarkdown": "Great job, team! Congratulations! 💯💯\n\nQ: How did you come to the conclusions about the \"🔵 Post-processing We used some simple post-processing to slightly improve the scores of LB and PB, as follows:\" step. was is done by sweeping some different settings? or rather based on some data assumptions?",
      "votes": 1,
      "replies": [
        {
          "id": 3398642,
          "postDate": "2026-01-29T12:57:07.180Z",
          "content": "<p>I partly referenced open-source notebooks and partly derived from CV and LB scores.</p>",
          "rawMarkdown": "I partly referenced open-source notebooks and partly derived from CV and LB scores.",
          "votes": 1,
          "replies": [
            {
              "id": 3398644,
              "postDate": "2026-01-29T12:58:20.087Z",
              "content": "<p>i see, tnx 😊</p>",
              "rawMarkdown": "i see, tnx 😊"
            }
          ]
        }
      ]
    },
    {
      "id": 3398845,
      "postDate": "2026-01-29T19:25:12.140Z",
      "content": "<p>Congratulations for the 1st place/Write-up and mostly for the Article too.</p>\n<p>\"Uniformity in heterogeneity: Diving deep into count interval partition for crowd counting\".</p>",
      "rawMarkdown": "Congratulations for the 1st place/Write-up and mostly for the Article too.\n\n\"Uniformity in heterogeneity: Diving deep into count interval partition for crowd counting\"."
    },
    {
      "id": 3419597,
      "postDate": "2026-03-11T05:10:32.250Z",
      "content": "<p>Thank you for the Article ! 🙂🙂👏\nQuick Question : 1. What was ur first step when u started with the competition ? 2. How u Have started strategizing the techniques ?</p>\n<ol>\n<li>can u guide to approach such type of competitions ?</li>\n</ol>\n<p>This would help me for to understand thinking for such type of competitions <a href=\"https://www.kaggle.com/tianhengqiu\" target=\"_blank\">@tianhengqiu</a> </p>",
      "rawMarkdown": "Thank you for the Article ! 🙂🙂👏\nQuick Question : 1. What was ur first step when u started with the competition ? 2. How u Have started strategizing the techniques ?\n3. can u guide to approach such type of competitions ?\n\nThis would help me for to understand thinking for such type of competitions @tianhengqiu "
    },
    {
      "id": 3399754,
      "postDate": "2026-01-31T08:54:24.490Z",
      "content": "<p>Great approach. You proved, no problem is fully a regression or classification problem if you want SOTA performance. Amazing.\nAs the additional decimals got released, can you please share them for your experiments you have mentioned in the write-up? </p>",
      "rawMarkdown": "Great approach. You proved, no problem is fully a regression or classification problem if you want SOTA performance. Amazing.\nAs the additional decimals got released, can you please share them for your experiments you have mentioned in the write-up? "
    },
    {
      "id": 3399218,
      "postDate": "2026-01-30T12:17:32.690Z",
      "content": "<p>what a great writeup! </p>",
      "rawMarkdown": "what a great writeup! "
    },
    {
      "id": 3399047,
      "postDate": "2026-01-30T05:35:58.937Z",
      "content": "<p>Thank you for sharing your solution  and congratulations!</p>",
      "rawMarkdown": "Thank you for sharing your solution  and congratulations!"
    },
    {
      "id": 3398954,
      "postDate": "2026-01-30T00:57:00.207Z",
      "content": "<p>its someone greate</p>",
      "rawMarkdown": "its someone greate"
    },
    {
      "id": 3398929,
      "postDate": "2026-01-29T23:17:11.530Z",
      "content": "<p>Thank you for sharing your solution writeup and congratulations! \nQuick question: is it possible for you guys to share you training code?</p>",
      "rawMarkdown": "Thank you for sharing your solution writeup and congratulations! \nQuick question: is it possible for you guys to share you training code?",
      "replies": [
        {
          "id": 3399244,
          "postDate": "2026-01-30T13:43:50.420Z",
          "content": "<p>You can look at our inference code, which contains almost all of our designs~</p>",
          "rawMarkdown": "You can look at our inference code, which contains almost all of our designs~",
          "replies": [
            {
              "id": 3402738,
              "postDate": "2026-02-06T17:12:37.410Z",
              "content": "<p>where can I find your inference code?</p>",
              "rawMarkdown": "where can I find your inference code?"
            },
            {
              "id": 3410784,
              "postDate": "2026-02-23T16:43:56.220Z",
              "content": "<p>I have made the inference code public. <a href=\"https://www.kaggle.com/code/pingfan/lb0-77-pb-0-67-1-debug-infer-two-gpus-v1-d45a08?scriptVersionId=294488905\" target=\"_blank\">https://www.kaggle.com/code/pingfan/lb0-77-pb-0-67-1-debug-infer-two-gpus-v1-d45a08?scriptVersionId=294488905</a></p>",
              "rawMarkdown": "I have made the inference code public. https://www.kaggle.com/code/pingfan/lb0-77-pb-0-67-1-debug-infer-two-gpus-v1-d45a08?scriptVersionId=294488905"
            }
          ]
        }
      ]
    },
    {
      "id": 3398783,
      "postDate": "2026-01-29T17:15:23.400Z",
      "content": "<p>congratulations your writeup realy informative … may i ask you questions : 1 if you use other backbone selection 2. the pseudo  labeling  in your last approch did you use it for stabilization and  self-distillation   .. thank you so much your approch realy helpping me </p>",
      "rawMarkdown": "congratulations your writeup realy informative ... may i ask you questions : 1 if you use other backbone selection 2. the pseudo  labeling  in your last approch did you use it for stabilization and  self-distillation   .. thank you so much your approch realy helpping me ",
      "replies": [
        {
          "id": 3399242,
          "postDate": "2026-01-30T13:40:25.907Z",
          "content": "<ol>\n<li>I also tried models from the Swin series, and my teammate tried DINOv2. Both of them are not as effective as DINOv3. </li>\n<li>Yes. This is actually a self-distillation strategy.</li>\n</ol>",
          "rawMarkdown": "1. I also tried models from the Swin series, and my teammate tried DINOv2. Both of them are not as effective as DINOv3. \n2. Yes. This is actually a self-distillation strategy."
        }
      ]
    },
    {
      "id": 3398734,
      "postDate": "2026-01-29T16:04:11.210Z",
      "content": "<p>Congratulations! This is great work!</p>",
      "rawMarkdown": "Congratulations! This is great work!"
    },
    {
      "id": 3399337,
      "postDate": "2026-01-30T17:21:39.933Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 3399392,
      "postDate": "2026-01-30T19:25:09.753Z",
      "content": "<p>Congratulations! Thank you for sharing </p>",
      "rawMarkdown": "Congratulations! Thank you for sharing ",
      "votes": 1
    },
    {
      "id": 3399151,
      "postDate": "2026-01-30T09:51:37.260Z",
      "content": "<p>thank you for sharing your solution!</p>",
      "rawMarkdown": "thank you for sharing your solution!",
      "votes": 1
    }
  ],
  "comments": [
    {
      "id": 3398624,
      "author_name": "TheoQiu",
      "author_url": "",
      "post_date": "2026-01-29T12:28:22.747000",
      "content": "<p>This article is also my first one.🤣</p>",
      "votes": 7,
      "replies": [
        {
          "id": 3398632,
          "author_name": "xizaizai",
          "author_url": "",
          "post_date": "2026-01-29T12:49:49.120000",
          "content": "<p>Damn bro, this is a great job! </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 3398655,
          "author_name": "Luciano Dourado",
          "author_url": "",
          "post_date": "2026-01-29T13:27:49.060000",
          "content": "<p>Congratulations!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 3398688,
          "author_name": "Yusuf Sertkaya",
          "author_url": "",
          "post_date": "2026-01-29T14:47:35.970000",
          "content": "<p>congarts super work</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 3399288,
      "author_name": "LevKayumov",
      "author_url": "",
      "post_date": "2026-01-30T15:20:30.097000",
      "content": "<p>I liked this solution: no many ensembles, just logical methods, ideas and working with data - this definitely deserves first place🥇</p>",
      "votes": 4,
      "replies": []
    },
    {
      "id": 3399263,
      "author_name": "clemologe",
      "author_url": "",
      "post_date": "2026-01-30T14:23:43.253000",
      "content": "<p>Really good work and lots of ideas for me to learn from! May I ask, how did you select your training iteration during training? When did you stop training and approx. how many iterations did that take and what was your batch size?</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 3398694,
      "author_name": "EliKal",
      "author_url": "",
      "post_date": "2026-01-29T14:54:14.130000",
      "content": "<p>Well done! The GM tricks are very impressive.\nWhy choose the weights arbitrarily instead of using a grid search to optimize the OOF score?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3399249,
          "author_name": "TheoQiu",
          "author_url": "",
          "post_date": "2026-01-30T13:53:30.590000",
          "content": "<p>Our method varies greatly between folds, so we did not train multiple folds.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 3398663,
      "author_name": "Eddie",
      "author_url": "",
      "post_date": "2026-01-29T13:59:31.237000",
      "content": "<p>Great job guys! getting inspo from crowd counting lmao that's smart AF. </p>",
      "votes": 1,
      "replies": [
        {
          "id": 3399243,
          "author_name": "TheoQiu",
          "author_url": "",
          "post_date": "2026-01-30T13:42:28.673000",
          "content": "<p>😂 CC is indeed very important to us.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 3398639,
      "author_name": "TheoQiu",
      "author_url": "",
      "post_date": "2026-01-29T12:53:38.220000",
      "content": "<p>Code for interval division implemented with reference to the paper.</p>\n<pre><code>def uep_partition(all_counts, m, t0, search_iter=100):\n    counts_sorted = np.sort([d for d in all_counts if d &gt; t0])\n    if len(counts_sorted) == 0:\n        return [t0]\n    L = 0.0\n    H = (counts_sorted[-1] - t0) * len(counts_sorted) \n    final_borders = []\n    for _ in range(search_iter):\n        target_nl = (L + H) / 2.0\n        p = t0\n        n = 0\n        current_borders = [t0]\n        for d_k in counts_sorted:\n            n += 1\n            current_error = (d_k - p) * n\n\n            if current_error &gt; target_nl:\n                p = d_k\n                current_borders.append(p)\n                n = 0\n        if len(current_borders) &gt;= m:\n            L = target_nl\n            final_borders = current_borders\n        else:\n            H = target_nl\n    return sorted(list(set(final_borders)))\n</code></pre>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 3398630,
      "author_name": "V1kstrand",
      "author_url": "",
      "post_date": "2026-01-29T12:44:46.870000",
      "content": "<p>Great job, team! Congratulations! 💯💯</p>\n<p>Q: How did you come to the conclusions about the \"🔵 Post-processing We used some simple post-processing to slightly improve the scores of LB and PB, as follows:\" step. was is done by sweeping some different settings? or rather based on some data assumptions?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3398642,
          "author_name": "TheoQiu",
          "author_url": "",
          "post_date": "2026-01-29T12:57:07.180000",
          "content": "<p>I partly referenced open-source notebooks and partly derived from CV and LB scores.</p>",
          "votes": 1,
          "replies": [
            {
              "id": 3398644,
              "author_name": "V1kstrand",
              "author_url": "",
              "post_date": "2026-01-29T12:58:20.087000",
              "content": "<p>i see, tnx 😊</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3398845,
      "author_name": "Marília Prata",
      "author_url": "",
      "post_date": "2026-01-29T19:25:12.140000",
      "content": "<p>Congratulations for the 1st place/Write-up and mostly for the Article too.</p>\n<p>\"Uniformity in heterogeneity: Diving deep into count interval partition for crowd counting\".</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3419597,
      "author_name": "Sandesh Lavhsetty",
      "author_url": "",
      "post_date": "2026-03-11T05:10:32.250000",
      "content": "<p>Thank you for the Article ! 🙂🙂👏\nQuick Question : 1. What was ur first step when u started with the competition ? 2. How u Have started strategizing the techniques ?</p>\n<ol>\n<li>can u guide to approach such type of competitions ?</li>\n</ol>\n<p>This would help me for to understand thinking for such type of competitions <a href=\"https://www.kaggle.com/tianhengqiu\" target=\"_blank\">@tianhengqiu</a> </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3399754,
      "author_name": "Komil Parmar",
      "author_url": "",
      "post_date": "2026-01-31T08:54:24.490000",
      "content": "<p>Great approach. You proved, no problem is fully a regression or classification problem if you want SOTA performance. Amazing.\nAs the additional decimals got released, can you please share them for your experiments you have mentioned in the write-up? </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3399218,
      "author_name": "Luck is all you need",
      "author_url": "",
      "post_date": "2026-01-30T12:17:32.690000",
      "content": "<p>what a great writeup! </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3399047,
      "author_name": "Roman@Ahmed",
      "author_url": "",
      "post_date": "2026-01-30T05:35:58.937000",
      "content": "<p>Thank you for sharing your solution  and congratulations!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3398954,
      "author_name": "Corinna22",
      "author_url": "",
      "post_date": "2026-01-30T00:57:00.207000",
      "content": "<p>its someone greate</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3398929,
      "author_name": "Natnael Dabs",
      "author_url": "",
      "post_date": "2026-01-29T23:17:11.530000",
      "content": "<p>Thank you for sharing your solution writeup and congratulations! \nQuick question: is it possible for you guys to share you training code?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 3399244,
          "author_name": "TheoQiu",
          "author_url": "",
          "post_date": "2026-01-30T13:43:50.420000",
          "content": "<p>You can look at our inference code, which contains almost all of our designs~</p>",
          "votes": 0,
          "replies": [
            {
              "id": 3402738,
              "author_name": "T.N",
              "author_url": "",
              "post_date": "2026-02-06T17:12:37.410000",
              "content": "<p>where can I find your inference code?</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3410784,
              "author_name": "Baiph",
              "author_url": "",
              "post_date": "2026-02-23T16:43:56.220000",
              "content": "<p>I have made the inference code public. <a href=\"https://www.kaggle.com/code/pingfan/lb0-77-pb-0-67-1-debug-infer-two-gpus-v1-d45a08?scriptVersionId=294488905\" target=\"_blank\">https://www.kaggle.com/code/pingfan/lb0-77-pb-0-67-1-debug-infer-two-gpus-v1-d45a08?scriptVersionId=294488905</a></p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3398783,
      "author_name": "benkerrouche abdelbasset",
      "author_url": "",
      "post_date": "2026-01-29T17:15:23.400000",
      "content": "<p>congratulations your writeup realy informative … may i ask you questions : 1 if you use other backbone selection 2. the pseudo  labeling  in your last approch did you use it for stabilization and  self-distillation   .. thank you so much your approch realy helpping me </p>",
      "votes": 0,
      "replies": [
        {
          "id": 3399242,
          "author_name": "TheoQiu",
          "author_url": "",
          "post_date": "2026-01-30T13:40:25.907000",
          "content": "<ol>\n<li>I also tried models from the Swin series, and my teammate tried DINOv2. Both of them are not as effective as DINOv3. </li>\n<li>Yes. This is actually a self-distillation strategy.</li>\n</ol>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 3398734,
      "author_name": "Kareem",
      "author_url": "",
      "post_date": "2026-01-29T16:04:11.210000",
      "content": "<p>Congratulations! This is great work!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3399337,
      "author_name": "",
      "author_url": "",
      "post_date": "2026-01-30T17:21:39.933000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3399392,
      "author_name": "T41K1",
      "author_url": "",
      "post_date": "2026-01-30T19:25:09.753000",
      "content": "<p>Congratulations! Thank you for sharing </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 3399151,
      "author_name": "Byungsoo Kang",
      "author_url": "",
      "post_date": "2026-01-30T09:51:37.260000",
      "content": "<p>thank you for sharing your solution!</p>",
      "votes": 1,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3398618": "Thanks to the organizers for hosting such an intense competition. We were lucky to win and I would like to thank my teammates for that. This was my first full participation in a Kaggle competition and the experience was invaluable to me 💪. Our solutions are as follows.\n\n<br>\n🔵 **CV Strategy**\nDivide into three folds, determined jointly by **the state and sampling date**, and then train a single-fold model.\n\n<br>\n🔵 **Input Data** \nSimilar to most excellent open-source notebooks, our model splits the image into left and right views and passes them through the same backbone to enhance consistency.\n\n<br>\n🔵 **Training Data Augmentation**\nWe used various conventional data augmentation methods, as shown below:\n\n```python\ndef get_train_transforms(args):\n    return Compose([\n        HorizontalFlip(p=0.5),\n        VerticalFlip(p=0.5),\n        RandomRotate90(p=0.5),\n        A.GaussNoise(p=0.3),\n        A.RandomBrightnessContrast(\n            brightness_limit=0.2,\n            contrast_limit=0.2,\n            p=0.75\n        ),\n        A.HueSaturationValue(\n            hue_shift_limit=10,\n            sat_shift_limit=20,\n            val_shift_limit=20,\n            p=0.5\n        ),\n        A.CLAHE(clip_limit=2.0, tile_grid_size=(8, 8), p=0.3), \n        A.ColorJitter(\n            brightness=0.2, \n            contrast=0.2, \n            saturation=0.2, \n            hue=0.1, \n            p=0.75\n        ),\n        Normalize(\n            mean=[0.485, 0.456, 0.406],\n            std=[0.229, 0.224, 0.225]\n        ),\n        Resize(args.img_size, args.img_size),\n        ToTensorV2()\n    ])\n\n```\n\nAfter the process of splitting the left and right subimages, we apply data enhancement to the subimages independently. In addition, we apply a small size scaling to the images randomly and then complement them with black pixels to simulate the focal length and viewing angle differences of different cameras.\n\n```python\nif random.random() < 0.2:\n    background_image = np.full_like(image, 0)\n    resize_retio = random.uniform(0.85, 1.0)\n    image = cv2.resize(image, None, fx=resize_retio, fy=resize_retio, interpolation=cv2.INTER_CUBIC)\n    h, w, _ = image.shape\n    bg_h, bg_w, _ = background_image.shape\n    top = random.randint(0, bg_h - h)\n    left = random.randint(0, bg_w - w)\n    background_image[top:top + h, left:left + w] = image\n    image = background_image\n```\n\n<br>\n🔵 **Backbone Selection**\nWe use DINOv3 as the backbone, which is presumably the approach used by most of the winning teams in this competition. Subsequently, we use a single multi-head self-attention layer to interact with the concatenated features of the left and right sub-images, and then output the fused features through an MLP.\n\n<br>\n🔵 **Head Design**\nWe independently predict the biomass of each species by using **five regression heads**, each of which contains three linear layers. No physical constraints are imposed on them during training, which we believe reduces the model's dependence on the training set.\n\nDifferent from most solutions, we have additionally designed five **classification heads**. Given that this task is similar to the crowd counting (CC) problem without a density map, we made a point of reading a lot of work in the field of CC. Referring to the interval division scheme in UEPNet[1], we use the training set to divide seven intervals for each species:\n```python\n  BORDERS_DICT = {\n      'Dry_Clover_g': [1.6e-05, 3.9, 10.5353, 20.6523, 37.5911, 71.7865],\n      'Dry_Dead_g': [1.6e-05, 6.1407, 13.1192, 23.277, 38.8581, 83.8407],\n      'Dry_Green_g': [1.6e-05, 13.4232, 27.0782, 45.5236, 79.834, 157.9836],\n      'Dry_Total_g': [1.6e-05, 23.4907, 41.1, 61.1, 96.8288, 185.7],\n      'GDM_g': [1.6e-05, 16.5143, 30.507, 49.5585, 81.0, 157.9836],\n  }\n```\n\nBased on the divided intervals, we add an additional classification head for each species to predict the interval where the biomass lies, which resulted in an improvement of approximately 0.03 in both LB and PB. This is also highly consistent with our CV. Ideally, if the intervals are divided sufficiently small, the model can accurately predict the biomass through interval inference values. (Obviously, this is too difficult for this competition without density maps/segmentation maps.)\n\n<br>\n🔵 **Loss Function**\nThe model we finally submitted uses the common SmoothL1 Loss for regression and Cross-Entropy loss for classification. The loss of each biomass is scaled according to the weights proposed by the competition for the calculation of R^2.\n\n```python\nclass WeightedBiomassLoss(nn.Module):\n    def __init__(self, loss_weights_dict):\n        super(WeightedBiomassLoss, self).__init__()\n        \n        self.criterion = nn.SmoothL1Loss()\n        self.cls_loss = nn.CrossEntropyLoss(reduction='none')\n        self.cls_weight = 0.3\n        self.weights = loss_weights_dict\n\n    def forward(self, predictions, targets, predictions_cls=None, cls_labels=None):\n        total_cls_loss = None\n        pred_green, pred_dead, pred_clover, pred_gdm, pred_total = predictions\n        if predictions_cls is not None:\n            assert cls_labels is not None\n            pred_green_cls, pred_dead_cls, pred_clover_cls, pred_gdm_cls, pred_total_cls = predictions_cls\n            cls_green_loss = self.cls_loss(pred_green_cls, cls_labels[:, 0])\n            cls_dead_loss = self.cls_loss(pred_dead_cls, cls_labels[:, 1])\n            cls_clover_loss = self.cls_loss(pred_clover_cls, cls_labels[:, 2])\n            cls_gdm_loss = self.cls_loss(pred_gdm_cls, cls_labels[:, 3])\n            cls_total_loss = self.cls_loss(pred_total_cls, cls_labels[:, 4])\n            total_cls_loss = (\n                self.weights['green_loss'] * cls_green_loss.mean() +\n                self.weights['dead_loss'] * cls_dead_loss.mean() +\n                self.weights['clover_loss'] * cls_clover_loss.mean() +\n                self.weights['gdm_loss'] * cls_gdm_loss.mean() +\n                self.weights['total_loss'] * cls_total_loss.mean()\n            )\n        \n        true_green = targets[:, 0].unsqueeze(-1) # Shape [batch, 1]\n        true_dead   = targets[:, 1].unsqueeze(-1) # Shape [batch, 1]\n        true_clover = targets[:, 2].unsqueeze(-1) # Shape [batch, 1]\n        true_gdm = targets[:, 3].unsqueeze(-1) # Shape [batch, 1]\n        true_total = targets[:, 4].unsqueeze(-1) # Shape [batch, 1]\n\n        loss_green = self.criterion(pred_green, true_green)\n        loss_dead = self.criterion(pred_dead, true_dead)\n        loss_clover = self.criterion(pred_clover, true_clover)\n        loss_gdm   = self.criterion(pred_gdm, true_gdm)\n        loss_total = self.criterion(pred_total, true_total)\n        \n        total_loss = (\n            self.weights['total_loss'] * loss_total +\n            self.weights['gdm_loss'] * loss_gdm +\n            self.weights['green_loss'] * loss_green + \n            self.weights['dead_loss'] * loss_dead +\n            self.weights['clover_loss'] * loss_clover # + \n        )\n        if predictions_cls is not None:\n            total_loss = total_loss + self.cls_weight * total_cls_loss\n        return total_loss, total_cls_loss\n\n```\n\nThere are also some additional experiments that were not ultimately included in the submitted model, but PB has proven that our designs in this process are meaningful. For example, using epsilon-insensitive L1 loss for regression optimization. This observation is based on the uneven distribution of biomass, we hope that **\"large values have large errors and small values have small errors\"**. Therefore, we designed the following loss based on the label, which enabled us to achieve the best single-model result on PB.\n\n```python\nclass EpsilonInsensitiveLoss(nn.Module):\n    def __init__(self, eps_point=20, max_eps=5, reduction='mean'):\n        super().__init__()\n        self.eps_point = eps_point\n        self.reduction = reduction\n        self.scale_ratio = 0.1\n        self.max_eps = max_eps\n    \n    def make_epsilon(self, target):\n        epsilon = torch.zeros_like(target)\n        epsilon = torch.where(\n            target<=self.eps_point, torch.ones_like(target), epsilon\n        )\n        epsilon = torch.where(\n            target>self.eps_point, target*self.scale_ratio, epsilon\n        )\n        epsilon = torch.clamp(epsilon, max=self.max_eps)\n    \n        return epsilon\n\n\n    def forward(self, pred, target):\n        epsilon = self.make_epsilon(target.detach())  \n        abs_diff = torch.abs(pred - target)\n        loss = torch.relu(abs_diff - epsilon)\n        \n        if self.reduction == 'mean':\n            return loss.mean()\n        elif self.reduction == 'sum':\n            return loss.sum()\n        return loss\n\n```\n\n<br>\n🔵 **Training Strategy**\nIn the early stage of the competition, we tried various different strategies and finally chose the most effective two-stage training strategy.\n- **Stage 1:**  Freeze DINO, then train the fusion layers and MLP heads.\n- **Stage 2:** Fine-tune the entire model.\n\nFurthermore, under our setup, whether it was LB or CV, we found that the performance of DINO-Large and DINO-Huge was similar. With limited computational resources, the benefit of increasing the input size was better than increasing the model size. Therefore, in the later stages of the competition, our best single model was generally trained with the 1024 pix + large configuration.\n\n<br>\n🔵 **Single model results of DINOv3-Large**\n\n| exp_name                               | input_size (single view) | LB       | PB       |\n| -------------------------------------- | ------------------------ | -------- | -------- |\n| Baseline-large                         | 512                      | 0.70     | 0.60     |\n| +cls                                   | 512                      | 0.73     | 0.63     |\n| +cls                                   | 1024                     | 0.74     | 0.64     |\n| +aug_scale + cls                       | 1024                     | 0.74     | 0.65     |\n| +**cls + $\\epsilon$-insensitive loss** | **512**                  | **0.71** | **0.66** |\n\n<br>\n🔵 **Test Time**\nOnline training during testing brought us a gain of >0.02.\n- **Step 1.** We selected four single models that performed well at LB with variability to generate pseudo-labels for the test set.\n- **Step 2.** Based on the obtained pseudo-labels, we iteratively trained two DINO-Large models. Different from the above training, we used the entire training set and test set for training, no longer performed CV, but instead took the SWA ensemble model of the final multiple epochs. After training with pseudo-labels, we conducted training on the training set for several epochs to prevent the model from overfitting to the pseudo-labels.\n- **Step 3.** Based on the two DINO-Large models obtained in the previous step, we generated pseudo-labels for the second time, and then trained two DINO-Base models according to the same strategy.\n- **Step 4.** Fusion of the inference results of the two DINO-Large models and the two DINO-Base models with weights of 0.4 and 0.6 respectively.\nThe submission notebook has been published at [Final_Inference_Notebook](https://www.kaggle.com/code/pingfan/lb0-77-pb-0-67-1-debug-infer-two-gpus-v1-d45a08?scriptVersionId=294488905).\n\n<br>\n🔵 **Post-processing**\nWe used some simple post-processing to slightly improve the scores of LB and PB, as follows:\n\n```python\nout_green = self.head_green(combined)\nout_dead = self.head_dead(combined)\nout_clover = self.head_clover(combined)\nout_gdm = self.head_gdm(combined)\nout_total = self.head_total(combined)\n\nout_clover = out_clover * 0.8\nif out_dead > 20:\n    out_dead *= 1.1\nelif out_dead < 10:\n    out_dead *= 0.9\n\nout_gdm = 0.5*out_gdm+ 0.5*(out_green+out_clover)\n\npred_total1 = out_green + out_clover + out_dead\npred_total2 = out_gdm + out_dead\nout_total = 0.5*out_total + 0.5*pred_total1 + 0.0*pred_total2\n```\n\n<br>\n🔵 **Final Competition Score**\n\n| exp_name     | LB   | PB   |\n| ------------ | ---- | ---- |\n| best in PB   | 0.77 | 0.68 |\n| submitted | 0.77 | 0.67 |\n\n<br>\n🔵 **Ideas that didn't work and Discussions**\n1. Various semi-supervised schemes 😭. Implementing and tuning these methods took up almost the my last few days of the competition, but none of them performed better than direct hard semi-supervised approaches.\n2. Training one model for one species and then merging five models.\n3. Introducing physical constraints into training, or training several heads and then calculating the remaining biomass.\n4. Removing dropout. Given that this competition is essentially a regression task, adding dropout would normally cause inconsistent variance between training and inference. However, dropout performed excellently in our experiments, significantly outperforming its removal.\n5. Generate additional unlabeled data and external datasets.\n6. Adding shadows to training images.\n7. Adding ranking loss. Due to the small amount of training data, we originally wanted to add a ranking loss to force the model to learn to compare biomass in different regions. First, we randomly cropped the training image $x$ to get $x_{crop}$ , and it is certain that $y>y_{crop}$, which gives an internal ranking relationship. Then, we calculated the ranking relationships between images based on labels. This scheme performed well on the CV, once surpassing our previous best model, but got a poor LB, which might be due to overfitting again~\n8. Replace L1 loss with various other losses.\n9. Inputs of patches at multiple resolutions ([1×2,2×4,4×8,...]). Although this scheme eventually achieved a decent CV score, it oscillated severely. Facts have also proven that their final performance on LB and PB was not good (especially PB, which got a score of <0.6).\n10. ...........\n\n\n🎉🎉🎉\n Last but not least, congratulations to my teammate @pingfan , the Kaggle community is about to have a new GM !!!\n🎉🎉🎉 \n\n> [1] Wang, Changan, Qingyu Song, Boshen Zhang, Yabiao Wang, Ying Tai, Xuyi Hu, Chengjie Wang, Jilin Li, Jiayi Ma, and Yang Wu. \"Uniformity in heterogeneity: Diving deep into count interval partition for crowd counting.\" In *Proceedings of the IEEE/CVF international conference on computer vision*, pp. 3234-3242. 2021.\n\n",
    "3398624": "This article is also my first one.🤣",
    "3399288": "I liked this solution: no many ensembles, just logical methods, ideas and working with data - this definitely deserves first place🥇",
    "3399263": "Really good work and lots of ideas for me to learn from! May I ask, how did you select your training iteration during training? When did you stop training and approx. how many iterations did that take and what was your batch size?",
    "3398694": "Well done! The GM tricks are very impressive.\nWhy choose the weights arbitrarily instead of using a grid search to optimize the OOF score?",
    "3398663": "Great job guys! getting inspo from crowd counting lmao that's smart AF. ",
    "3398639": "Code for interval division implemented with reference to the paper.\n```python\ndef uep_partition(all_counts, m, t0, search_iter=100):\n    counts_sorted = np.sort([d for d in all_counts if d > t0])\n    if len(counts_sorted) == 0:\n        return [t0]\n    L = 0.0\n    H = (counts_sorted[-1] - t0) * len(counts_sorted) \n    final_borders = []\n    for _ in range(search_iter):\n        target_nl = (L + H) / 2.0\n        p = t0\n        n = 0\n        current_borders = [t0]\n        for d_k in counts_sorted:\n            n += 1\n            current_error = (d_k - p) * n\n            \n            if current_error > target_nl:\n                p = d_k\n                current_borders.append(p)\n                n = 0\n        if len(current_borders) >= m:\n            L = target_nl\n            final_borders = current_borders\n        else:\n            H = target_nl\n    return sorted(list(set(final_borders)))\n```",
    "3398630": "Great job, team! Congratulations! 💯💯\n\nQ: How did you come to the conclusions about the \"🔵 Post-processing We used some simple post-processing to slightly improve the scores of LB and PB, as follows:\" step. was is done by sweeping some different settings? or rather based on some data assumptions?",
    "3398845": "Congratulations for the 1st place/Write-up and mostly for the Article too.\n\n\"Uniformity in heterogeneity: Diving deep into count interval partition for crowd counting\".",
    "3419597": "Thank you for the Article ! 🙂🙂👏\nQuick Question : 1. What was ur first step when u started with the competition ? 2. How u Have started strategizing the techniques ?\n3. can u guide to approach such type of competitions ?\n\nThis would help me for to understand thinking for such type of competitions @tianhengqiu ",
    "3399754": "Great approach. You proved, no problem is fully a regression or classification problem if you want SOTA performance. Amazing.\nAs the additional decimals got released, can you please share them for your experiments you have mentioned in the write-up? ",
    "3399218": "what a great writeup! ",
    "3399047": "Thank you for sharing your solution  and congratulations!",
    "3398954": "its someone greate",
    "3398929": "Thank you for sharing your solution writeup and congratulations! \nQuick question: is it possible for you guys to share you training code?",
    "3398783": "congratulations your writeup realy informative ... may i ask you questions : 1 if you use other backbone selection 2. the pseudo  labeling  in your last approch did you use it for stabilization and  self-distillation   .. thank you so much your approch realy helpping me ",
    "3398734": "Congratulations! This is great work!",
    "3399337": "",
    "3399392": "Congratulations! Thank you for sharing ",
    "3399151": "thank you for sharing your solution!"
  }
}