{
  "id": 603222,
  "title": "Sequence Augmentation What i've learend from CMI 2025",
  "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/603222",
  "author_name": "",
  "post_date": "2025-09-01T10:04:52.673159700Z",
  "votes": 10,
  "comment_count": 1,
  "views": 0,
  "content": "<p>CMI participants performed series of repeated gesture while wearing the Helios device</p>\n<ol>\n<li>They began a transition from “rest” position and moved their hand to the appropriate location <strong>(Transition)</strong></li>\n<li>They followed this with a short pause wherein they did nothing <strong>(Pause)</strong></li>\n<li>Finally they performed a gesture from either the BFRB-like or non-BFRB-like category of movements <strong>(Gesture)</strong></li>\n</ol>\n<p>ex) </p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F824da99e5ca3d6fa7208f49e52657270%2FSEQ1.JPG?generation=1756720736329591&amp;alt=media\" alt=\"\"></p>\n<blockquote>\n  <p>A sequence consists of three movement phases: Transition, Pause, and Gesture.</p>\n  <p>When applying data augmentation, instead of augmenting the entire sequence, we use a <strong>Phased Lock</strong> approach, applying augmentation <strong>separately to each phase</strong>.</p>\n  <p>This allows us to preserve the characteristics of each phase while still generating diverse augmented data.</p>\n</blockquote>\n<table>\n<thead>\n<tr>\n<th>Aspect</th>\n<th>Phased Lock Augmentation</th>\n<th>Whole Sequence Augmentation</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Feature Preservation</td>\n<td>✅ Maintains phase-specific characteristics</td>\n<td>❌ Phase characteristics may get mixed</td>\n</tr>\n<tr>\n<td>Data Diversity</td>\n<td>✅ Allows diverse variations for each phase</td>\n<td>✅ Can augment overall sequence</td>\n</tr>\n<tr>\n<td>Model Training</td>\n<td>✅ Easier to learn phase-specific features</td>\n<td>❌ Harder to capture fine-grained phase features</td>\n</tr>\n<tr>\n<td>Distortion Minimization</td>\n<td>✅ Minimizes meaningless distortions</td>\n<td>❌ Important signals may be distorted</td>\n</tr>\n<tr>\n<td></td>\n<td></td>\n<td></td>\n</tr>\n</tbody>\n</table>\n<hr>\n<table>\n<thead>\n<tr>\n<th>Technique</th>\n<th>Features / Advantages</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td><strong>Phased Lock Time Shift</strong></td>\n<td>- Simple positional shift within the sequence- Preserves overall pattern- Can be applied to specific phases</td>\n</tr>\n<tr>\n<td><strong>Phased Lock Time Warp</strong></td>\n<td>- Simulates changes in sequence speed- Produces natural time variations- Phase-wise control of warping intensity</td>\n</tr>\n<tr>\n<td><strong>Phased Lock Amplitude Warp</strong></td>\n<td>- Diversifies data by varying amplitude- Maintains sequence patterns- Can apply spline smoothing for smooth variation</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<p><strong>Phased Lock Time Shift</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F338b529b5895be2bb26072b48afdfeb7%2FSEQ2.JPG?generation=1756720748351980&amp;alt=media\" alt=\"\"></p>\n<p><strong>Phased Lock Time Warp</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F587b0cce457b0bb7ee4a87192914b81c%2FSEQ3.JPG?generation=1756720760721088&amp;alt=media\" alt=\"\"></p>\n<p><strong>Phased Lock Amplitude Warp</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fde327bdabf3dac88aa250213f1316a5d%2FSEQ4.JPG?generation=1756720778381536&amp;alt=media\" alt=\"\"></p>\n<hr>\n<p>Besides phase-locked augmentation, there are many other types of sequence augmentations</p>\n<table>\n<thead>\n<tr>\n<th>Technique</th>\n<th>Features / Advantages</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td><strong>Mix Up</strong></td>\n<td>- Combines multiple sequences to generate new data points- Preserves overall temporal patterns- Enhances model generalization by interpolating between phases</td>\n</tr>\n<tr>\n<td><strong>Jitter / Scale</strong></td>\n<td>- Introduces slight temporal variations to simulate speed changes- Produces natural and smooth time distortions- Phase-wise control enables targeted augmentation</td>\n</tr>\n<tr>\n<td><strong>Quantize</strong></td>\n<td>- Reduces amplitude resolution to diversify signal characteristics- Maintains essential sequence patterns- Can apply smoothing for realistic transitions</td>\n</tr>\n<tr>\n<td><strong>Pool</strong></td>\n<td>- Reduces temporal resolution by aggregating frames- Emphasizes dominant patterns and suppresses noise- Useful for multi-scale sequence learning</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<p><strong>Mix Up</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F087a4760446867371ffd477c588e746a%2FSEQ5.JPG?generation=1756720962077444&amp;alt=media\" alt=\"\"></p>\n<p><strong>Jitter_Scale</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fa8e360c489836facef725f19af3debb2%2FSEQ6.JPG?generation=1756721004010967&amp;alt=media\" alt=\"\"></p>\n<p><strong>Quantize</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F186e33720f752a150fd521afdaea8931%2FSEQ7.JPG?generation=1756721017590180&amp;alt=media\" alt=\"\"></p>\n<p><strong></strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fffcc1bc74ab15a5f418360636e93f7a6%2FSEQ8.JPG?generation=1756721026857600&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": "3279721",
      "postDate": "09/01/2025 10:04:52",
      "content": "<p>CMI participants performed series of repeated gesture while wearing the Helios device</p>\n<ol>\n<li>They began a transition from “rest” position and moved their hand to the appropriate location <strong>(Transition)</strong></li>\n<li>They followed this with a short pause wherein they did nothing <strong>(Pause)</strong></li>\n<li>Finally they performed a gesture from either the BFRB-like or non-BFRB-like category of movements <strong>(Gesture)</strong></li>\n</ol>\n<p>ex) </p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F824da99e5ca3d6fa7208f49e52657270%2FSEQ1.JPG?generation=1756720736329591&amp;alt=media\" alt=\"\"></p>\n<blockquote>\n  <p>A sequence consists of three movement phases: Transition, Pause, and Gesture.</p>\n  <p>When applying data augmentation, instead of augmenting the entire sequence, we use a <strong>Phased Lock</strong> approach, applying augmentation <strong>separately to each phase</strong>.</p>\n  <p>This allows us to preserve the characteristics of each phase while still generating diverse augmented data.</p>\n</blockquote>\n<table>\n<thead>\n<tr>\n<th>Aspect</th>\n<th>Phased Lock Augmentation</th>\n<th>Whole Sequence Augmentation</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Feature Preservation</td>\n<td>✅ Maintains phase-specific characteristics</td>\n<td>❌ Phase characteristics may get mixed</td>\n</tr>\n<tr>\n<td>Data Diversity</td>\n<td>✅ Allows diverse variations for each phase</td>\n<td>✅ Can augment overall sequence</td>\n</tr>\n<tr>\n<td>Model Training</td>\n<td>✅ Easier to learn phase-specific features</td>\n<td>❌ Harder to capture fine-grained phase features</td>\n</tr>\n<tr>\n<td>Distortion Minimization</td>\n<td>✅ Minimizes meaningless distortions</td>\n<td>❌ Important signals may be distorted</td>\n</tr>\n<tr>\n<td></td>\n<td></td>\n<td></td>\n</tr>\n</tbody>\n</table>\n<hr>\n<table>\n<thead>\n<tr>\n<th>Technique</th>\n<th>Features / Advantages</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td><strong>Phased Lock Time Shift</strong></td>\n<td>- Simple positional shift within the sequence- Preserves overall pattern- Can be applied to specific phases</td>\n</tr>\n<tr>\n<td><strong>Phased Lock Time Warp</strong></td>\n<td>- Simulates changes in sequence speed- Produces natural time variations- Phase-wise control of warping intensity</td>\n</tr>\n<tr>\n<td><strong>Phased Lock Amplitude Warp</strong></td>\n<td>- Diversifies data by varying amplitude- Maintains sequence patterns- Can apply spline smoothing for smooth variation</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<p><strong>Phased Lock Time Shift</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F338b529b5895be2bb26072b48afdfeb7%2FSEQ2.JPG?generation=1756720748351980&amp;alt=media\" alt=\"\"></p>\n<p><strong>Phased Lock Time Warp</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F587b0cce457b0bb7ee4a87192914b81c%2FSEQ3.JPG?generation=1756720760721088&amp;alt=media\" alt=\"\"></p>\n<p><strong>Phased Lock Amplitude Warp</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fde327bdabf3dac88aa250213f1316a5d%2FSEQ4.JPG?generation=1756720778381536&amp;alt=media\" alt=\"\"></p>\n<hr>\n<p>Besides phase-locked augmentation, there are many other types of sequence augmentations</p>\n<table>\n<thead>\n<tr>\n<th>Technique</th>\n<th>Features / Advantages</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td><strong>Mix Up</strong></td>\n<td>- Combines multiple sequences to generate new data points- Preserves overall temporal patterns- Enhances model generalization by interpolating between phases</td>\n</tr>\n<tr>\n<td><strong>Jitter / Scale</strong></td>\n<td>- Introduces slight temporal variations to simulate speed changes- Produces natural and smooth time distortions- Phase-wise control enables targeted augmentation</td>\n</tr>\n<tr>\n<td><strong>Quantize</strong></td>\n<td>- Reduces amplitude resolution to diversify signal characteristics- Maintains essential sequence patterns- Can apply smoothing for realistic transitions</td>\n</tr>\n<tr>\n<td><strong>Pool</strong></td>\n<td>- Reduces temporal resolution by aggregating frames- Emphasizes dominant patterns and suppresses noise- Useful for multi-scale sequence learning</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<p><strong>Mix Up</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F087a4760446867371ffd477c588e746a%2FSEQ5.JPG?generation=1756720962077444&amp;alt=media\" alt=\"\"></p>\n<p><strong>Jitter_Scale</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fa8e360c489836facef725f19af3debb2%2FSEQ6.JPG?generation=1756721004010967&amp;alt=media\" alt=\"\"></p>\n<p><strong>Quantize</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F186e33720f752a150fd521afdaea8931%2FSEQ7.JPG?generation=1756721017590180&amp;alt=media\" alt=\"\"></p>\n<p><strong></strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fffcc1bc74ab15a5f418360636e93f7a6%2FSEQ8.JPG?generation=1756721026857600&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "CMI participants performed series of repeated gesture while wearing the Helios device\n\n1. They began a transition from “rest” position and moved their hand to the appropriate location **(Transition)**\n2. They followed this with a short pause wherein they did nothing **(Pause)**\n3. Finally they performed a gesture from either the BFRB-like or non-BFRB-like category of movements **(Gesture)**\n\nex) \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F824da99e5ca3d6fa7208f49e52657270%2FSEQ1.JPG?generation=1756720736329591&alt=media)\n\n> A sequence consists of three movement phases: Transition, Pause, and Gesture.\n> \n> \n> When applying data augmentation, instead of augmenting the entire sequence, we use a **Phased Lock** approach, applying augmentation **separately to each phase**.\n> \n> This allows us to preserve the characteristics of each phase while still generating diverse augmented data.\n> \n\n| Aspect | Phased Lock Augmentation | Whole Sequence Augmentation |\n| --- | --- | --- |\n| Feature Preservation | ✅ Maintains phase-specific characteristics | ❌ Phase characteristics may get mixed |\n| Data Diversity | ✅ Allows diverse variations for each phase | ✅ Can augment overall sequence |\n| Model Training | ✅ Easier to learn phase-specific features | ❌ Harder to capture fine-grained phase features |\n| Distortion Minimization | ✅ Minimizes meaningless distortions | ❌ Important signals may be distorted |\n|  |  |  |\n\n___\n\n| Technique | Features / Advantages |\n| --- | --- |\n| **Phased Lock Time Shift** | - Simple positional shift within the sequence- Preserves overall pattern- Can be applied to specific phases |\n| **Phased Lock Time Warp** | - Simulates changes in sequence speed- Produces natural time variations- Phase-wise control of warping intensity |\n| **Phased Lock Amplitude Warp** | - Diversifies data by varying amplitude- Maintains sequence patterns- Can apply spline smoothing for smooth variation |\n\n___\n\n**Phased Lock Time Shift**\n \n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F338b529b5895be2bb26072b48afdfeb7%2FSEQ2.JPG?generation=1756720748351980&alt=media)\n\n**Phased Lock Time Warp**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F587b0cce457b0bb7ee4a87192914b81c%2FSEQ3.JPG?generation=1756720760721088&alt=media)\n\n**Phased Lock Amplitude Warp**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fde327bdabf3dac88aa250213f1316a5d%2FSEQ4.JPG?generation=1756720778381536&alt=media)\n\n___\nBesides phase-locked augmentation, there are many other types of sequence augmentations\n\n| Technique | Features / Advantages |\n| --- | --- |\n| **Mix Up** | - Combines multiple sequences to generate new data points- Preserves overall temporal patterns- Enhances model generalization by interpolating between phases |\n| **Jitter / Scale** | - Introduces slight temporal variations to simulate speed changes- Produces natural and smooth time distortions- Phase-wise control enables targeted augmentation |\n| **Quantize** | - Reduces amplitude resolution to diversify signal characteristics- Maintains essential sequence patterns- Can apply smoothing for realistic transitions |\n| **Pool** | - Reduces temporal resolution by aggregating frames- Emphasizes dominant patterns and suppresses noise- Useful for multi-scale sequence learning |\n___\n\n**Mix Up**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F087a4760446867371ffd477c588e746a%2FSEQ5.JPG?generation=1756720962077444&alt=media)\n\n**Jitter_Scale**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fa8e360c489836facef725f19af3debb2%2FSEQ6.JPG?generation=1756721004010967&alt=media)\n\n**Quantize**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F186e33720f752a150fd521afdaea8931%2FSEQ7.JPG?generation=1756721017590180&alt=media)\n\n**<Pool>**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fffcc1bc74ab15a5f418360636e93f7a6%2FSEQ8.JPG?generation=1756721026857600&alt=media)",
      "votes": null
    },
    {
      "id": "3279858",
      "postDate": "09/01/2025 15:47:37",
      "content": "<p>I normally make this post so I am glad to see someone else kick it off :) for me most of my learning comes from using Bert for this task. You can find my code here: <a href=\"https://www.kaggle.com/code/cody11null/public-bert-training-attempt\" target=\"_blank\">https://www.kaggle.com/code/cody11null/public-bert-training-attempt</a> tho I did have some good fun with feature engineering and differing models like using 1d CNN with SE block, tabm, and ensembling these. Tho we made a crucial mistake in this that has limited how much we can optimize. Excited to see the results and hope that many above us have overfit too far haha</p>",
      "rawMarkdown": "I normally make this post so I am glad to see someone else kick it off :) for me most of my learning comes from using Bert for this task. You can find my code here: https://www.kaggle.com/code/cody11null/public-bert-training-attempt tho I did have some good fun with feature engineering and differing models like using 1d CNN with SE block, tabm, and ensembling these. Tho we made a crucial mistake in this that has limited how much we can optimize. Excited to see the results and hope that many above us have overfit too far haha",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3279858,
      "author_name": "cody11null",
      "author_url": "",
      "post_date": "09/01/2025 15:47:37",
      "content": "<p>I normally make this post so I am glad to see someone else kick it off :) for me most of my learning comes from using Bert for this task. You can find my code here: <a href=\"https://www.kaggle.com/code/cody11null/public-bert-training-attempt\" target=\"_blank\">https://www.kaggle.com/code/cody11null/public-bert-training-attempt</a> tho I did have some good fun with feature engineering and differing models like using 1d CNN with SE block, tabm, and ensembling these. Tho we made a crucial mistake in this that has limited how much we can optimize. Excited to see the results and hope that many above us have overfit too far haha</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3279721": "CMI participants performed series of repeated gesture while wearing the Helios device\n\n1. They began a transition from “rest” position and moved their hand to the appropriate location **(Transition)**\n2. They followed this with a short pause wherein they did nothing **(Pause)**\n3. Finally they performed a gesture from either the BFRB-like or non-BFRB-like category of movements **(Gesture)**\n\nex) \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F824da99e5ca3d6fa7208f49e52657270%2FSEQ1.JPG?generation=1756720736329591&alt=media)\n\n> A sequence consists of three movement phases: Transition, Pause, and Gesture.\n> \n> \n> When applying data augmentation, instead of augmenting the entire sequence, we use a **Phased Lock** approach, applying augmentation **separately to each phase**.\n> \n> This allows us to preserve the characteristics of each phase while still generating diverse augmented data.\n> \n\n| Aspect | Phased Lock Augmentation | Whole Sequence Augmentation |\n| --- | --- | --- |\n| Feature Preservation | ✅ Maintains phase-specific characteristics | ❌ Phase characteristics may get mixed |\n| Data Diversity | ✅ Allows diverse variations for each phase | ✅ Can augment overall sequence |\n| Model Training | ✅ Easier to learn phase-specific features | ❌ Harder to capture fine-grained phase features |\n| Distortion Minimization | ✅ Minimizes meaningless distortions | ❌ Important signals may be distorted |\n|  |  |  |\n\n___\n\n| Technique | Features / Advantages |\n| --- | --- |\n| **Phased Lock Time Shift** | - Simple positional shift within the sequence- Preserves overall pattern- Can be applied to specific phases |\n| **Phased Lock Time Warp** | - Simulates changes in sequence speed- Produces natural time variations- Phase-wise control of warping intensity |\n| **Phased Lock Amplitude Warp** | - Diversifies data by varying amplitude- Maintains sequence patterns- Can apply spline smoothing for smooth variation |\n\n___\n\n**Phased Lock Time Shift**\n \n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F338b529b5895be2bb26072b48afdfeb7%2FSEQ2.JPG?generation=1756720748351980&alt=media)\n\n**Phased Lock Time Warp**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F587b0cce457b0bb7ee4a87192914b81c%2FSEQ3.JPG?generation=1756720760721088&alt=media)\n\n**Phased Lock Amplitude Warp**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fde327bdabf3dac88aa250213f1316a5d%2FSEQ4.JPG?generation=1756720778381536&alt=media)\n\n___\nBesides phase-locked augmentation, there are many other types of sequence augmentations\n\n| Technique | Features / Advantages |\n| --- | --- |\n| **Mix Up** | - Combines multiple sequences to generate new data points- Preserves overall temporal patterns- Enhances model generalization by interpolating between phases |\n| **Jitter / Scale** | - Introduces slight temporal variations to simulate speed changes- Produces natural and smooth time distortions- Phase-wise control enables targeted augmentation |\n| **Quantize** | - Reduces amplitude resolution to diversify signal characteristics- Maintains essential sequence patterns- Can apply smoothing for realistic transitions |\n| **Pool** | - Reduces temporal resolution by aggregating frames- Emphasizes dominant patterns and suppresses noise- Useful for multi-scale sequence learning |\n___\n\n**Mix Up**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F087a4760446867371ffd477c588e746a%2FSEQ5.JPG?generation=1756720962077444&alt=media)\n\n**Jitter_Scale**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fa8e360c489836facef725f19af3debb2%2FSEQ6.JPG?generation=1756721004010967&alt=media)\n\n**Quantize**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2F186e33720f752a150fd521afdaea8931%2FSEQ7.JPG?generation=1756721017590180&alt=media)\n\n**<Pool>**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fffcc1bc74ab15a5f418360636e93f7a6%2FSEQ8.JPG?generation=1756721026857600&alt=media)",
    "3279858": "I normally make this post so I am glad to see someone else kick it off :) for me most of my learning comes from using Bert for this task. You can find my code here: https://www.kaggle.com/code/cody11null/public-bert-training-attempt tho I did have some good fun with feature engineering and differing models like using 1d CNN with SE block, tabm, and ensembling these. Tho we made a crucial mistake in this that has limited how much we can optimize. Excited to see the results and hope that many above us have overfit too far haha"
  },
  "source": "meta"
}