{
  "id": 360872,
  "title": "A simple approach based on baseline notebooks (6th place solution)",
  "content": "<p>Thanks to the organizers for a very interesting competition, my teammates ( <a href=\"https://www.kaggle.com/yasushiyui\" target=\"_blank\">@yasushiyui</a> and <a href=\"https://www.kaggle.com/kotaiizuka\" target=\"_blank\">@kotaiizuka</a> ) for effort, and all the kagglers who worked hard together.  And special thanks to <a href=\"https://www.kaggle.com/its7171\" target=\"_blank\">@its7171</a>, the author of great baseline notebooks.</p>\n<p>There are already many great top solutions out there, but we would like to share ours as an example of how just a few tweaks to the baseline can increase your score.<br>Please feel free to comment if you notice anything or would like to know more details.</p>\n<h2>Summary</h2>\n<ul>\n<li>Basically, we used the baseline notebook processing pipeline.</li>\n<li>Similar to the Public 1st place solution, three neighboring frames were converted to grayscale and combined in the channel direction for using temporal information. We believe that this idea was probably crucial for our solution to enter the public gold medal zone.</li>\n<li>We applied the techniques usually used in competitions (high resolution images, modification of loss function, custom data augmentation, ensemble, TTA, etc.).</li>\n</ul>\n<h2>Using three neighboring frames</h2>\n<p>The figure below shows the flow of this technique. To avail temporal information and to use the same processing pipeline as for RGB image input, three adjacent images were converted to grayscale and combined in the channel direction. Most experiments used time_stride=1. In one of the final submissions, we ensembled the models trained with time_stride=1, 2, and 3, respectively, to take advantage of the longer temporal information.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1728889%2Fa1d3ffa2d8738db68a0f533ee5f1a39b%2FUsing_three_neighboring_frames.png?generation=1666108969082746&amp;alt=media\" alt=\"\"></p>\n<h2>Parameter settings</h2>\n<ul>\n<li>Training<ul>\n<li>dataset<ul>\n<li>fold0: 3c993bd2_0, 3c993bd2_1</li>\n<li>fold1: 1606b0e6_0, 1606b0e6_1</li>\n<li>fold2: 35bd9041_0, 35bd9041_1</li></ul></li>\n<li>input_size: (3, 720, 1280)</li>\n<li>time_stride: 1, 2, 3</li>\n<li>amp: True</li>\n<li>20 epochs (cooldown_epochs=0)</li>\n<li>evaluation metrics: mAP</li>\n<li>model<ul>\n<li>tf_efficientnet_b0_ap</li>\n<li>tf_efficientnet_b1_ap</li></ul></li>\n<li>loss: 4-class binary cross entropy</li>\n<li>dropout: 0.2</li>\n<li>data augmentation<ul>\n<li>RandomAffine, RandomPerspective, RandomHorizontalFlip, ColorJitter(brightness, contrast), Mixup</li></ul></li></ul></li>\n<li>Averaging checkpoints<ul>\n<li>default settings for avg_checkpoints.py in timm</li></ul></li>\n<li>Inference<ul>\n<li>interpolation method for resize: cv2.INTER_AREA</li>\n<li>crop_pct: 0.875 (default setting for tf_efficientnet_b0_ap)</li>\n<li>3-models ensemble with horizontal flip tta</li>\n<li>inference once every 2 frames for speedup (predictions for frames not inferred are generated by linear interpolation)</li>\n<li>custom data loader to prefetch images behind the GPU inference process</li>\n<li>parameter tunings for window_size and ignore_width in <code>make_sub</code> function</li></ul></li>\n</ul>\n<h2>Scores for each experiment</h2>\n<p>The table below shows the settings and scores of baseline notebooks and our main experiments.<br>\nOur public LB best (0.784) model was not selected for the final submission because it almost exceeded the time limit.</p>\n<table>\n<thead>\n<tr>\n<th>ID</th>\n<th>description</th>\n<th>model</th>\n<th>Val AP</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1</td>\n<td>tito's baseline</td>\n<td>b5_ap</td>\n<td>0.215</td>\n<td>0.249</td>\n</tr>\n<tr>\n<td>2</td>\n<td>+ TTA</td>\n<td>b5_ap</td>\n<td>N/A</td>\n<td>0.255</td>\n</tr>\n<tr>\n<td>3</td>\n<td>+ Fix label error</td>\n<td>b5_ap</td>\n<td>0.242</td>\n<td>0.283</td>\n</tr>\n<tr>\n<td>4</td>\n<td>+ Use three neighboring frames <br>with time_stride=1</td>\n<td>b5_ap</td>\n<td>0.312</td>\n<td>0.425</td>\n</tr>\n<tr>\n<td>5</td>\n<td>+ 960x540 input and inference <br>once every 2 frames <br>(without tta)</td>\n<td>b5_ap</td>\n<td>0.554</td>\n<td>0.563</td>\n</tr>\n<tr>\n<td>6</td>\n<td>+ 4-class bce loss</td>\n<td>b5_ap</td>\n<td>0.609</td>\n<td>0.618</td>\n</tr>\n<tr>\n<td>7</td>\n<td>+ Custom data loader</td>\n<td>b5_ap</td>\n<td>0.625</td>\n<td>0.634</td>\n</tr>\n<tr>\n<td>8</td>\n<td>+ Custom data augmentation</td>\n<td>b5_ap</td>\n<td>0.700</td>\n<td>0.703</td>\n</tr>\n<tr>\n<td>9</td>\n<td>+ Replace b5 model <br>with b0 model and <br>predict with 4-fold ensemble</td>\n<td>b0_ap<br>(4-fold)</td>\n<td>0.669</td>\n<td>0.720</td>\n</tr>\n<tr>\n<td>10</td>\n<td>+ Generate predictions <br>for frames not inferred <br>using linear interpolation</td>\n<td>b0_ap<br>(4-fold)</td>\n<td>0.686</td>\n<td>0.740</td>\n</tr>\n<tr>\n<td>11</td>\n<td>+ 1280x720 input and <br>3-fold ensemble prediction</td>\n<td>b0_ap<br>(3-fold)</td>\n<td>0.722</td>\n<td>0.772</td>\n</tr>\n<tr>\n<td>12</td>\n<td>+ Mixup augmentation</td>\n<td>b0_ap<br>(3-fold)</td>\n<td>0.747</td>\n<td>0.761</td>\n</tr>\n<tr>\n<td>13</td>\n<td>+ Replace one b0 model <br>with b1 model</td>\n<td>b0_ap (2-fold)<br>b1_ap (1-fold)</td>\n<td>0.751</td>\n<td>0.768</td>\n</tr>\n<tr>\n<td>14</td>\n<td>+ Replace three models <br>with 3 full-fit models <br>(each model uses <br>three neighboring frames <br>with time_stride=1, 2, and 3, <br>respectively)</td>\n<td>b0_ap (full-fit, <br>time_stride=1, 2, 3)</td>\n<td>0.753</td>\n<td>0.782</td>\n</tr>\n</tbody>\n</table>\n<h2>Reference</h2>\n<ul>\n<li>Baseline notebooks<ul>\n<li><a href=\"https://www.kaggle.com/code/its7171/dfl-benchmark-training\" target=\"_blank\">DFL benchmark training | Kaggle</a></li>\n<li><a href=\"https://www.kaggle.com/code/its7171/dfl-benchmark-inference\" target=\"_blank\">DFL benchmark inference | Kaggle</a></li></ul></li>\n<li>Fix label erros<ul>\n<li><a href=\"https://www.kaggle.com/code/kmizunoster/dfl-benchmark-training-fix-label-error\" target=\"_blank\">DFL benchmark training + fix label error | Kaggle</a></li>\n<li><a href=\"https://www.kaggle.com/code/kmizunoster/dfl-benchmark-inference-tta-fix-label-error\" target=\"_blank\">DFL benchmark inference + tta + fix label error | Kaggle</a></li></ul></li>\n</ul>",
  "comments": [
    {
      "id": 1997453,
      "author_name": "@kaggleqrdl",
      "author_url": "",
      "post_date": "2022-10-20T22:28:00.273000",
      "content": "<p>One of the best descriptions I've seen on Kaggle, especially the summary table.  A template for all future solution descriptions.</p>",
      "votes": 2,
      "replies": [
        {
          "id": 2001903,
          "author_name": "Leo Black",
          "author_url": "",
          "post_date": "2022-10-24T11:45:07.380000",
          "content": "<p>A template for all future solution descriptions.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1994354,
      "author_name": "arutema47",
      "author_url": "",
      "post_date": "2022-10-19T01:48:38.447000",
      "content": "<p>Great summary! <br>\nI love it when people share the ablation study, helps me understand the approach much better :)</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 1994080,
      "author_name": "Guillermo García Cobo",
      "author_url": "",
      "post_date": "2022-10-18T18:55:57.400000",
      "content": "<p>Thank you for your detailed post <a href=\"https://www.kaggle.com/kmizunoster\" target=\"_blank\">@kmizunoster</a>, specially for the summary table! It has really helped me to understand the evolution of your model. I take good notes for how to nicely iterate in any AI/DL project, which was our main mistake in this competition.</p>\n<p>On the other hand, after checking the table, I would like to know more about points 6 and 7:</p>\n<ul>\n<li>Why did you try changing from CrossEntropy to BCE? I understand that BCE could be useful for multi-label problems, but here a frame should only be classified as one class (so a softmax has more sense than individual sigmoids). Why do you think that it worked better?</li>\n<li>What do you mean by “custom dataloader” and how could the DataLoader change the score?</li>\n</ul>\n<p>Thank you very much once again! Looking forward to your answer.</p>",
      "votes": 2,
      "replies": [
        {
          "id": 1995196,
          "author_name": "kmizunoster",
          "author_url": "",
          "post_date": "2022-10-19T14:11:24.267000",
          "content": "<p><a href=\"https://www.kaggle.com/atmguille\" target=\"_blank\">@atmguille</a> Thank you for your comments.</p>\n<blockquote>\n  <p>Why did you try changing from CrossEntropy to BCE? I understand that BCE could be useful for multi-label problems, but here a frame should only be classified as one class (so a softmax has more sense than individual sigmoids). Why do you think that it worked better?</p>\n</blockquote>\n<p>We tried BCE because we thought BCE would match the competition's evaluation metric better than CE.<br>\nThe metric first calculates the APs for each class, and then averages them to get the averaged AP, so the order of scores within each class is very important.<br>\nCE requires softmax to calcuate scores for each class, which is dependent on the logits of the other classes and may affect the order of scores within each class. On the other hand, BCE only uses logits for its own class, so there is no need to worry about this.</p>\n<blockquote>\n  <p>What do you mean by “custom dataloader” and how could the DataLoader change the score?</p>\n</blockquote>\n<p>The data loader used in the baseline notebook was designed to export all the frames to jpeg files for inference.<br>\nIn order to reduce processing time, we created a custom data loader that can infer each frame captured by cv2.VideoCapture without going through the file.<br>\nAlthough the implementation was not aimed at improving the score, the score increased slightly when the same pre-processing was performed as before.<br>\nPerhaps the conventional method via jpeg files degraded the image quality and had a negative impact on the score.</p>",
          "votes": 3,
          "replies": []
        },
        {
          "id": 1996625,
          "author_name": "Kelvin",
          "author_url": "",
          "post_date": "2022-10-20T10:31:22.277000",
          "content": "<p>Thanks for your answer and I found there was a big different between adding TTA before the softmax layer and after the softmax layer. The latter was much better than the former and I guessed the softmax layer may make a negative effect, but I didn't find a better method to replace it. So, thanks for your great idea and explanation.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 2001906,
      "author_name": "Leo Black",
      "author_url": "",
      "post_date": "2022-10-24T11:46:14.003000",
      "content": "<p>Thx! Would you mind sharing the code?</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2012077,
      "author_name": "Sorous511",
      "author_url": "",
      "post_date": "2022-11-01T04:34:57.903000",
      "content": "<p><a href=\"https://www.kaggle.com/kmizunoster\" target=\"_blank\">@kmizunoster</a> Great solution with description. Would help in many applications too. Thanks ! 👍</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1994756,
      "author_name": "Rahul Pandey",
      "author_url": "",
      "post_date": "2022-10-19T08:09:17.440000",
      "content": "<p>Great share, I will learn and explore your method for some use cases.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1993933,
      "author_name": "Ravi Ramakrishnan",
      "author_url": "",
      "post_date": "2022-10-18T17:06:26.603000",
      "content": "<p>Many thanks for the detailed process description and congrats for the results <a href=\"https://www.kaggle.com/kmizunoster\" target=\"_blank\">@kmizunoster</a> </p>",
      "votes": 0,
      "replies": []
    }
  ],
  "comment_count": 10,
  "messages": [
    {
      "id": 1993876,
      "postDate": "2022-10-18T16:34:31.443Z",
      "content": "<p>Thanks to the organizers for a very interesting competition, my teammates ( <a href=\"https://www.kaggle.com/yasushiyui\" target=\"_blank\">@yasushiyui</a> and <a href=\"https://www.kaggle.com/kotaiizuka\" target=\"_blank\">@kotaiizuka</a> ) for effort, and all the kagglers who worked hard together.  And special thanks to <a href=\"https://www.kaggle.com/its7171\" target=\"_blank\">@its7171</a>, the author of great baseline notebooks.</p>\n<p>There are already many great top solutions out there, but we would like to share ours as an example of how just a few tweaks to the baseline can increase your score.<br>Please feel free to comment if you notice anything or would like to know more details.</p>\n<h2>Summary</h2>\n<ul>\n<li>Basically, we used the baseline notebook processing pipeline.</li>\n<li>Similar to the Public 1st place solution, three neighboring frames were converted to grayscale and combined in the channel direction for using temporal information. We believe that this idea was probably crucial for our solution to enter the public gold medal zone.</li>\n<li>We applied the techniques usually used in competitions (high resolution images, modification of loss function, custom data augmentation, ensemble, TTA, etc.).</li>\n</ul>\n<h2>Using three neighboring frames</h2>\n<p>The figure below shows the flow of this technique. To avail temporal information and to use the same processing pipeline as for RGB image input, three adjacent images were converted to grayscale and combined in the channel direction. Most experiments used time_stride=1. In one of the final submissions, we ensembled the models trained with time_stride=1, 2, and 3, respectively, to take advantage of the longer temporal information.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1728889%2Fa1d3ffa2d8738db68a0f533ee5f1a39b%2FUsing_three_neighboring_frames.png?generation=1666108969082746&amp;alt=media\" alt=\"\"></p>\n<h2>Parameter settings</h2>\n<ul>\n<li>Training<ul>\n<li>dataset<ul>\n<li>fold0: 3c993bd2_0, 3c993bd2_1</li>\n<li>fold1: 1606b0e6_0, 1606b0e6_1</li>\n<li>fold2: 35bd9041_0, 35bd9041_1</li></ul></li>\n<li>input_size: (3, 720, 1280)</li>\n<li>time_stride: 1, 2, 3</li>\n<li>amp: True</li>\n<li>20 epochs (cooldown_epochs=0)</li>\n<li>evaluation metrics: mAP</li>\n<li>model<ul>\n<li>tf_efficientnet_b0_ap</li>\n<li>tf_efficientnet_b1_ap</li></ul></li>\n<li>loss: 4-class binary cross entropy</li>\n<li>dropout: 0.2</li>\n<li>data augmentation<ul>\n<li>RandomAffine, RandomPerspective, RandomHorizontalFlip, ColorJitter(brightness, contrast), Mixup</li></ul></li></ul></li>\n<li>Averaging checkpoints<ul>\n<li>default settings for avg_checkpoints.py in timm</li></ul></li>\n<li>Inference<ul>\n<li>interpolation method for resize: cv2.INTER_AREA</li>\n<li>crop_pct: 0.875 (default setting for tf_efficientnet_b0_ap)</li>\n<li>3-models ensemble with horizontal flip tta</li>\n<li>inference once every 2 frames for speedup (predictions for frames not inferred are generated by linear interpolation)</li>\n<li>custom data loader to prefetch images behind the GPU inference process</li>\n<li>parameter tunings for window_size and ignore_width in <code>make_sub</code> function</li></ul></li>\n</ul>\n<h2>Scores for each experiment</h2>\n<p>The table below shows the settings and scores of baseline notebooks and our main experiments.<br>\nOur public LB best (0.784) model was not selected for the final submission because it almost exceeded the time limit.</p>\n<table>\n<thead>\n<tr>\n<th>ID</th>\n<th>description</th>\n<th>model</th>\n<th>Val AP</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1</td>\n<td>tito's baseline</td>\n<td>b5_ap</td>\n<td>0.215</td>\n<td>0.249</td>\n</tr>\n<tr>\n<td>2</td>\n<td>+ TTA</td>\n<td>b5_ap</td>\n<td>N/A</td>\n<td>0.255</td>\n</tr>\n<tr>\n<td>3</td>\n<td>+ Fix label error</td>\n<td>b5_ap</td>\n<td>0.242</td>\n<td>0.283</td>\n</tr>\n<tr>\n<td>4</td>\n<td>+ Use three neighboring frames <br>with time_stride=1</td>\n<td>b5_ap</td>\n<td>0.312</td>\n<td>0.425</td>\n</tr>\n<tr>\n<td>5</td>\n<td>+ 960x540 input and inference <br>once every 2 frames <br>(without tta)</td>\n<td>b5_ap</td>\n<td>0.554</td>\n<td>0.563</td>\n</tr>\n<tr>\n<td>6</td>\n<td>+ 4-class bce loss</td>\n<td>b5_ap</td>\n<td>0.609</td>\n<td>0.618</td>\n</tr>\n<tr>\n<td>7</td>\n<td>+ Custom data loader</td>\n<td>b5_ap</td>\n<td>0.625</td>\n<td>0.634</td>\n</tr>\n<tr>\n<td>8</td>\n<td>+ Custom data augmentation</td>\n<td>b5_ap</td>\n<td>0.700</td>\n<td>0.703</td>\n</tr>\n<tr>\n<td>9</td>\n<td>+ Replace b5 model <br>with b0 model and <br>predict with 4-fold ensemble</td>\n<td>b0_ap<br>(4-fold)</td>\n<td>0.669</td>\n<td>0.720</td>\n</tr>\n<tr>\n<td>10</td>\n<td>+ Generate predictions <br>for frames not inferred <br>using linear interpolation</td>\n<td>b0_ap<br>(4-fold)</td>\n<td>0.686</td>\n<td>0.740</td>\n</tr>\n<tr>\n<td>11</td>\n<td>+ 1280x720 input and <br>3-fold ensemble prediction</td>\n<td>b0_ap<br>(3-fold)</td>\n<td>0.722</td>\n<td>0.772</td>\n</tr>\n<tr>\n<td>12</td>\n<td>+ Mixup augmentation</td>\n<td>b0_ap<br>(3-fold)</td>\n<td>0.747</td>\n<td>0.761</td>\n</tr>\n<tr>\n<td>13</td>\n<td>+ Replace one b0 model <br>with b1 model</td>\n<td>b0_ap (2-fold)<br>b1_ap (1-fold)</td>\n<td>0.751</td>\n<td>0.768</td>\n</tr>\n<tr>\n<td>14</td>\n<td>+ Replace three models <br>with 3 full-fit models <br>(each model uses <br>three neighboring frames <br>with time_stride=1, 2, and 3, <br>respectively)</td>\n<td>b0_ap (full-fit, <br>time_stride=1, 2, 3)</td>\n<td>0.753</td>\n<td>0.782</td>\n</tr>\n</tbody>\n</table>\n<h2>Reference</h2>\n<ul>\n<li>Baseline notebooks<ul>\n<li><a href=\"https://www.kaggle.com/code/its7171/dfl-benchmark-training\" target=\"_blank\">DFL benchmark training | Kaggle</a></li>\n<li><a href=\"https://www.kaggle.com/code/its7171/dfl-benchmark-inference\" target=\"_blank\">DFL benchmark inference | Kaggle</a></li></ul></li>\n<li>Fix label erros<ul>\n<li><a href=\"https://www.kaggle.com/code/kmizunoster/dfl-benchmark-training-fix-label-error\" target=\"_blank\">DFL benchmark training + fix label error | Kaggle</a></li>\n<li><a href=\"https://www.kaggle.com/code/kmizunoster/dfl-benchmark-inference-tta-fix-label-error\" target=\"_blank\">DFL benchmark inference + tta + fix label error | Kaggle</a></li></ul></li>\n</ul>",
      "rawMarkdown": "Thanks to the organizers for a very interesting competition, my teammates ( @yasushiyui and @kotaiizuka ) for effort, and all the kagglers who worked hard together.  And special thanks to @its7171, the author of great baseline notebooks.\n\nThere are already many great top solutions out there, but we would like to share ours as an example of how just a few tweaks to the baseline can increase your score.<br>Please feel free to comment if you notice anything or would like to know more details.\n\n## Summary\n\n- Basically, we used the baseline notebook processing pipeline.\n- Similar to the Public 1st place solution, three neighboring frames were converted to grayscale and combined in the channel direction for using temporal information. We believe that this idea was probably crucial for our solution to enter the public gold medal zone.\n- We applied the techniques usually used in competitions (high resolution images, modification of loss function, custom data augmentation, ensemble, TTA, etc.).\n\n## Using three neighboring frames\n\nThe figure below shows the flow of this technique. To avail temporal information and to use the same processing pipeline as for RGB image input, three adjacent images were converted to grayscale and combined in the channel direction. Most experiments used time_stride=1. In one of the final submissions, we ensembled the models trained with time_stride=1, 2, and 3, respectively, to take advantage of the longer temporal information.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1728889%2Fa1d3ffa2d8738db68a0f533ee5f1a39b%2FUsing_three_neighboring_frames.png?generation=1666108969082746&alt=media)\n\n## Parameter settings\n\n- Training\n    - dataset\n      - fold0: 3c993bd2_0, 3c993bd2_1\n      - fold1: 1606b0e6_0, 1606b0e6_1\n      - fold2: 35bd9041_0, 35bd9041_1\n    - input_size: (3, 720, 1280)\n    - time_stride: 1, 2, 3\n    - amp: True\n    - 20 epochs (cooldown_epochs=0)\n    - evaluation metrics: mAP\n    - model\n        - tf_efficientnet_b0_ap\n        - tf_efficientnet_b1_ap\n    - loss: 4-class binary cross entropy\n    - dropout: 0.2\n    - data augmentation\n        - RandomAffine, RandomPerspective, RandomHorizontalFlip, ColorJitter(brightness, contrast), Mixup\n- Averaging checkpoints\n    - default settings for avg_checkpoints.py in timm\n- Inference\n    - interpolation method for resize: cv2.INTER_AREA\n    - crop_pct: 0.875 (default setting for tf_efficientnet_b0_ap)\n    - 3-models ensemble with horizontal flip tta\n    - inference once every 2 frames for speedup (predictions for frames not inferred are generated by linear interpolation)\n    - custom data loader to prefetch images behind the GPU inference process\n    - parameter tunings for window_size and ignore_width in `make_sub` function\n\n## Scores for each experiment\n\nThe table below shows the settings and scores of baseline notebooks and our main experiments.\nOur public LB best (0.784) model was not selected for the final submission because it almost exceeded the time limit.\n\n|ID|description|model|Val AP|LB|\n|--|--|--|--|--|\n|1|tito's baseline|b5_ap|0.215|0.249|\n|2|+ TTA|b5_ap|N/A|0.255|\n|3|+ Fix label error|b5_ap|0.242|0.283|\n|4|+ Use three neighboring frames <br>with time_stride=1|b5_ap|0.312|0.425|\n|5|+ 960x540 input and inference <br>once every 2 frames <br>(without tta)|b5_ap|0.554|0.563|\n|6|+ 4-class bce loss|b5_ap|0.609|0.618|\n|7|+ Custom data loader|b5_ap|0.625|0.634|\n|8|+ Custom data augmentation|b5_ap|0.700|0.703|\n|9|+ Replace b5 model <br>with b0 model and <br>predict with 4-fold ensemble|b0_ap<br>(4-fold)|0.669|0.720|\n|10|+ Generate predictions <br>for frames not inferred <br>using linear interpolation|b0_ap<br>(4-fold)|0.686|0.740|\n|11|+ 1280x720 input and <br>3-fold ensemble prediction|b0_ap<br>(3-fold)|0.722|0.772|\n|12|+ Mixup augmentation|b0_ap<br>(3-fold)|0.747|0.761|\n|13|+ Replace one b0 model <br>with b1 model|b0_ap (2-fold)<br>b1_ap (1-fold)|0.751|0.768|\n|14|+ Replace three models <br>with 3 full-fit models <br>(each model uses <br>three neighboring frames <br>with time_stride=1, 2, and 3, <br>respectively)|b0_ap (full-fit, <br>time_stride=1, 2, 3)|0.753|0.782|\n\n## Reference\n- Baseline notebooks\n  - [DFL benchmark training | Kaggle](https://www.kaggle.com/code/its7171/dfl-benchmark-training)\n  - [DFL benchmark inference | Kaggle](https://www.kaggle.com/code/its7171/dfl-benchmark-inference)\n- Fix label erros\n  - [DFL benchmark training + fix label error | Kaggle](https://www.kaggle.com/code/kmizunoster/dfl-benchmark-training-fix-label-error)\n  - [DFL benchmark inference + tta + fix label error | Kaggle](https://www.kaggle.com/code/kmizunoster/dfl-benchmark-inference-tta-fix-label-error)",
      "votes": 29
    },
    {
      "id": 1997453,
      "postDate": "2022-10-20T22:28:00.273Z",
      "content": "<p>One of the best descriptions I've seen on Kaggle, especially the summary table.  A template for all future solution descriptions.</p>",
      "rawMarkdown": "One of the best descriptions I've seen on Kaggle, especially the summary table.  A template for all future solution descriptions.",
      "votes": 2,
      "replies": [
        {
          "id": 2001903,
          "postDate": "2022-10-24T11:45:07.380Z",
          "content": "<p>A template for all future solution descriptions.</p>",
          "rawMarkdown": " A template for all future solution descriptions."
        }
      ]
    },
    {
      "id": 1994354,
      "postDate": "2022-10-19T01:48:38.447Z",
      "content": "<p>Great summary! <br>\nI love it when people share the ablation study, helps me understand the approach much better :)</p>",
      "rawMarkdown": "Great summary! \nI love it when people share the ablation study, helps me understand the approach much better :)",
      "votes": 2
    },
    {
      "id": 1994080,
      "postDate": "2022-10-18T18:55:57.400Z",
      "content": "<p>Thank you for your detailed post <a href=\"https://www.kaggle.com/kmizunoster\" target=\"_blank\">@kmizunoster</a>, specially for the summary table! It has really helped me to understand the evolution of your model. I take good notes for how to nicely iterate in any AI/DL project, which was our main mistake in this competition.</p>\n<p>On the other hand, after checking the table, I would like to know more about points 6 and 7:</p>\n<ul>\n<li>Why did you try changing from CrossEntropy to BCE? I understand that BCE could be useful for multi-label problems, but here a frame should only be classified as one class (so a softmax has more sense than individual sigmoids). Why do you think that it worked better?</li>\n<li>What do you mean by “custom dataloader” and how could the DataLoader change the score?</li>\n</ul>\n<p>Thank you very much once again! Looking forward to your answer.</p>",
      "rawMarkdown": "Thank you for your detailed post @kmizunoster, specially for the summary table! It has really helped me to understand the evolution of your model. I take good notes for how to nicely iterate in any AI/DL project, which was our main mistake in this competition.\n\nOn the other hand, after checking the table, I would like to know more about points 6 and 7:\n* Why did you try changing from CrossEntropy to BCE? I understand that BCE could be useful for multi-label problems, but here a frame should only be classified as one class (so a softmax has more sense than individual sigmoids). Why do you think that it worked better?\n* What do you mean by “custom dataloader” and how could the DataLoader change the score?\n\nThank you very much once again! Looking forward to your answer.",
      "votes": 2,
      "replies": [
        {
          "id": 1995196,
          "postDate": "2022-10-19T14:11:24.267Z",
          "content": "<p><a href=\"https://www.kaggle.com/atmguille\" target=\"_blank\">@atmguille</a> Thank you for your comments.</p>\n<blockquote>\n  <p>Why did you try changing from CrossEntropy to BCE? I understand that BCE could be useful for multi-label problems, but here a frame should only be classified as one class (so a softmax has more sense than individual sigmoids). Why do you think that it worked better?</p>\n</blockquote>\n<p>We tried BCE because we thought BCE would match the competition's evaluation metric better than CE.<br>\nThe metric first calculates the APs for each class, and then averages them to get the averaged AP, so the order of scores within each class is very important.<br>\nCE requires softmax to calcuate scores for each class, which is dependent on the logits of the other classes and may affect the order of scores within each class. On the other hand, BCE only uses logits for its own class, so there is no need to worry about this.</p>\n<blockquote>\n  <p>What do you mean by “custom dataloader” and how could the DataLoader change the score?</p>\n</blockquote>\n<p>The data loader used in the baseline notebook was designed to export all the frames to jpeg files for inference.<br>\nIn order to reduce processing time, we created a custom data loader that can infer each frame captured by cv2.VideoCapture without going through the file.<br>\nAlthough the implementation was not aimed at improving the score, the score increased slightly when the same pre-processing was performed as before.<br>\nPerhaps the conventional method via jpeg files degraded the image quality and had a negative impact on the score.</p>",
          "rawMarkdown": "@atmguille Thank you for your comments.\n\n> Why did you try changing from CrossEntropy to BCE? I understand that BCE could be useful for multi-label problems, but here a frame should only be classified as one class (so a softmax has more sense than individual sigmoids). Why do you think that it worked better?\n\nWe tried BCE because we thought BCE would match the competition's evaluation metric better than CE.\nThe metric first calculates the APs for each class, and then averages them to get the averaged AP, so the order of scores within each class is very important.\nCE requires softmax to calcuate scores for each class, which is dependent on the logits of the other classes and may affect the order of scores within each class. On the other hand, BCE only uses logits for its own class, so there is no need to worry about this.\n\n> What do you mean by “custom dataloader” and how could the DataLoader change the score?\n\nThe data loader used in the baseline notebook was designed to export all the frames to jpeg files for inference.\nIn order to reduce processing time, we created a custom data loader that can infer each frame captured by cv2.VideoCapture without going through the file.\nAlthough the implementation was not aimed at improving the score, the score increased slightly when the same pre-processing was performed as before.\nPerhaps the conventional method via jpeg files degraded the image quality and had a negative impact on the score.",
          "votes": 3
        },
        {
          "id": 1996625,
          "postDate": "2022-10-20T10:31:22.277Z",
          "content": "<p>Thanks for your answer and I found there was a big different between adding TTA before the softmax layer and after the softmax layer. The latter was much better than the former and I guessed the softmax layer may make a negative effect, but I didn't find a better method to replace it. So, thanks for your great idea and explanation.</p>",
          "rawMarkdown": "Thanks for your answer and I found there was a big different between adding TTA before the softmax layer and after the softmax layer. The latter was much better than the former and I guessed the softmax layer may make a negative effect, but I didn't find a better method to replace it. So, thanks for your great idea and explanation.",
          "votes": 1
        }
      ]
    },
    {
      "id": 2001906,
      "postDate": "2022-10-24T11:46:14.003Z",
      "content": "<p>Thx! Would you mind sharing the code?</p>",
      "rawMarkdown": "Thx! Would you mind sharing the code?",
      "votes": 1
    },
    {
      "id": 2012077,
      "postDate": "2022-11-01T04:34:57.903Z",
      "content": "<p><a href=\"https://www.kaggle.com/kmizunoster\" target=\"_blank\">@kmizunoster</a> Great solution with description. Would help in many applications too. Thanks ! 👍</p>",
      "rawMarkdown": "@kmizunoster Great solution with description. Would help in many applications too. Thanks ! 👍"
    },
    {
      "id": 1994756,
      "postDate": "2022-10-19T08:09:17.440Z",
      "content": "<p>Great share, I will learn and explore your method for some use cases.</p>",
      "rawMarkdown": "Great share, I will learn and explore your method for some use cases."
    },
    {
      "id": 1993933,
      "postDate": "2022-10-18T17:06:26.603Z",
      "content": "<p>Many thanks for the detailed process description and congrats for the results <a href=\"https://www.kaggle.com/kmizunoster\" target=\"_blank\">@kmizunoster</a> </p>",
      "rawMarkdown": "Many thanks for the detailed process description and congrats for the results @kmizunoster "
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-22T08:53:13Z",
  "competition_id": 37244,
  "competition_slug": "dfl-bundesliga-data-shootout",
  "team_id": 9142769,
  "solution_url": "/competitions/dfl-bundesliga-data-shootout/writeups/fixnet-a-simple-approach-based-on-baseline-noteboo"
}