# 1st Place Solution (code updated)

First of all, we would like to thank Kaggle and the organizers for hosting such a great competition. And also thanks to
[@hengck23](https://www.kaggle.com/hengck23)
for the amazing posts,
[@junkoda](https://www.kaggle.com/junkoda)
for the metric implementation and all other participants for sharing their experiments.
Overview
Our final submission is an ensemble of two 2.5d convnext tiny unet with 3 channels, and the only differences between these two models are augmentation and number of epochs. Actually the best scored submission is not the selected ensemble but one single model of the ensemble which is 0.835 on private lb.
Data Preparation
We used all training data
including
kidney_1_voi.
Multiview slice (x, y, z)
Normalization: No normalization, just
image = image / 65535.0
Whole slice instead of tiles and all slices resized or cropped to 1536x1536.
Augmentations:
A.Compose([
    A.HorizontalFlip(=0.5),
    A.VerticalFlip(=0.5),
    A.Transpose(=0.5),
    A.Affine(scale={:(0.7, 1.3), :(0.7, 1.3)}, translate_percent={:(0, 0.1), :(0, 0.1)}, rotate=(-30, 30), shear=(-20, 20), =0.5),
    A.RandomBrightnessContrast(=0.4, =0.4, =0.5),
    A.OneOf([
        A.Blur(=3, =0.2),
        A.MedianBlur(=3, =0.2),
    ], =1.0),
    A.OneOf([
        A.ElasticTransform(=1, =50, =10, =1, =0.5),
        A.GridDistortion(=5, =0.1, =1, =0.5)
    ], =0.4),
    A.OneOf([
        A.Resize(1536, 1536, cv2.INTER_LINEAR, =1),
        A.Compose([
            RandomResize(1536, 1536, =0.5, =0.5, =1),
            A.PadIfNeeded(1536, 1536, =, =cv2.BORDER_REPLICATE, =1.0),
            A.RandomCrop(1536, 1536, =1.0)
        ], =1.0),
    ], =1.0),
    A.GaussNoise(=0.05, =0.2),
])
Random 3D rotation to get slices that is not necessarily parallel to axes. The best scored submission used random 3d rotation for augmentation and trained for more epochs.
![](assets/4bb9d8f90b7dcf979f358b50.png)
Modeling & Training
We used unet from SMP with convnext tiny backbone, replaced BatchNorm and ReLU to GroupNorm and GELU and added an extra convolution stem. The input size for all models is 3x1536x1536.
self = nn(
    nn(in_channels, out_channels, , , ),
    (out_channels),
)
For loss function, we used 1.0 focal loss, 1.0 dice loss, 0.01
[boundary loss](https://arxiv.org/abs/1812.07032)
and 1.0 custom loss. The custom loss is inspired by
[@hengck23](https://www.kaggle.com/hengck23)
's
[post](https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/456118#2583472)
and
[@junkoda](https://www.kaggle.com/junkoda)
's
[metric implementation](https://www.kaggle.com/code/junkoda/fast-surface-dice-computation)
.
For optimization, we used AdamW and CosineAnnealingLR from 1e-4 to 0 with warmup. All models were trained for 20 epochs with a batch size of 8 and 4 gradient accumulation steps, except for the model with 3d slice rotation augmentation which was trained for 30 epochs.
Inference
Inference on 3 axes with 8xTTA.
We tried different resize methods for inference. For the best scored submission, all slices are simply resized to 3072x3072; for the selected submission, we used a dynamic scale factor that
(h*scale)*(w*scale)=3200*3200
.
The threshold used for submission is 0.4, and the optimal threshold based on cv and lb is about 0.4~0.5.
torch.compile()
gave about 2x acceleration so that we were able to inference with high resolution and TTAs.
What didn't work
3d models.
External data and pseudo labels.
Transformers.
Stacking more slices (>3) for 2.5d model.
Results
Model
Slice Rotation
Inference size
Public Score
Private Score
1
convnext_tiny
3072
0.889
0.682
2
convnext_tiny
✓
3072
0.888
0.830
3
convnext_tiny
✓
3072
0.867
0.835
4
ensemble(1+2)
-
3200
0.898
0.744(selected)
5
ensemble(1+2)
-
3200(dynamic)
0.895
0.774(selected)
Links
[training code](https://github.com/jing1tian/blood-vessel-segmentation)
inference code
[final submission ensemble 0.774106](https://www.kaggle.com/code/clevert/sennet-1st-place-solution)
[3d rotate single model 0.835346](https://www.kaggle.com/code/clevert/sennet-unet-convnext-3d-rotation)