![\"point4\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2583778%2F57cdbdd2ddd4d33d9fd94cda9017fa35%2Fsolution.png?generation=1702132623612174&alt=media\")
Hello everyone,
\nI would like to express my gratitude to the organizers of this competition, the members who participated, and all of my teammates( @stgkrtua, @kuto0633, @copasta, @masatomatsui).
In this discussion, I will talk about my individual solution. Please have a look at our team solution here. I believe reading both will deepen your understanding as there might be some overlaps.
\nI focused on creating a stronger single model, as I believe even in ensembling, without a good single model, it's hard to see significant improvement.
\nI developed my model based on the code shared by tubo's public code. The scores for the models used in our team ensemble were as follows:
\n1DUnet
\n1DUnet
\nAlthough Ididn't adopt it in the end, simple 1DConv stacking of the team's predictions was effective. We didn't use it in our final submission due to concerns about CV overfitting, but I observed slight improvements upon later review.
\nCV: 0.8058
I utilized the 1DUnet created by @kmat2019, which was rewritten in PyTorch by @super13579
\nhttps://www.kaggle.com/code/super13579/u-net-1d-cnn-with-pytorch
Various improvements were made to this model for this competition.
\nEfforts based on the scores include:
Stacking consisted only of 1 layer in the Encoder and a matching Conv1d 1 layer. Adding the duplicate flag feature in stacking also proved to be effective.
", "messages": [ { "id": 2554930, "postDate": "2023-12-09T14:39:15.077Z", "content": "Hello everyone,
\nI would like to express my gratitude to the organizers of this competition, the members who participated, and all of my teammates( @stgkrtua, @kuto0633, @copasta, @masatomatsui).
In this discussion, I will talk about my individual solution. Please have a look at our team solution here. I believe reading both will deepen your understanding as there might be some overlaps.
\nI focused on creating a stronger single model, as I believe even in ensembling, without a good single model, it's hard to see significant improvement.
\nI developed my model based on the code shared by tubo's public code. The scores for the models used in our team ensemble were as follows:
\n1DUnet
\n1DUnet
\nAlthough Ididn't adopt it in the end, simple 1DConv stacking of the team's predictions was effective. We didn't use it in our final submission due to concerns about CV overfitting, but I observed slight improvements upon later review.
\nCV: 0.8058
I utilized the 1DUnet created by @kmat2019, which was rewritten in PyTorch by @super13579
\nhttps://www.kaggle.com/code/super13579/u-net-1d-cnn-with-pytorch
Various improvements were made to this model for this competition.
\nEfforts based on the scores include:
Stacking consisted only of 1 layer in the Encoder and a matching Conv1d 1 layer. Adding the duplicate flag feature in stacking also proved to be effective.
", "rawMarkdown": "Hello everyone,\nI would like to express my gratitude to the organizers of this competition, the members who participated, and all of my teammates( @stgkrtua, @kuto0633, @copasta, @masatomatsui).\n\nIn this discussion, I will talk about my individual solution. Please have a look at our team solution [here](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459703). I believe reading both will deepen your understanding as there might be some overlaps.\n\nI focused on creating a stronger single model, as I believe even in ensembling, without a good single model, it's hard to see significant improvement.\n\nI developed my model based on the code shared by tubo's public code. The scores for the models used in our team ensemble were as follows:\n\n# Model summary\n- 1DUnet\n - Duration 11440\n - downsample 2\n - CV:0.778 (Non PostProccess)\n - Public Score:0.751 (With LowPaassFilter PostProccess)\n - Private Score:0.806 (With LowPaassFilter PostProccess)\n\n- 1DUnet\n - Duration 17280\n - downsample 4\n - CV:0.801 (Non PostProccess)\n - Public Score:0.755 (With LowPaassFilter PostProccess)\n - Private Score:0.814 (With LowPaassFilter PostProccess)\n\nAlthough Ididn't adopt it in the end, simple 1DConv stacking of the team's predictions was effective. We didn't use it in our final submission due to concerns about CV overfitting, but I observed slight improvements upon later review.\nCV: 0.8058\n\n# About the Single Model used:\nI utilized the 1DUnet created by @kmat2019, which was rewritten in PyTorch by @super13579\nhttps://www.kaggle.com/code/super13579/u-net-1d-cnn-with-pytorch\n\nVarious improvements were made to this model for this competition.\nEfforts based on the scores include:\n\n1. Add L1Loss for sleep state prediction difference (refer to team solution for details)\n1. Duplicate flag feature (refer to team solution for details)\n1. Warmup\n1. Two rounds of loss calculation with targets\n1. Change to 5 layers in the Encoder and 4 layers in the Decoder\nFor point 4, I modified the output of the decoder in the following way. The reason is that upon analyzing the predictions, we found that sleep labels were not standing out enough, while onset or wakeup labels were being assigned, so Iwanted the onset and wakeup predictions to take sleep information into account.\n\n\n\nStacking consisted only of 1 layer in the Encoder and a matching Conv1d 1 layer. Adding the duplicate flag feature in stacking also proved to be effective.", "votes": 23 } ], "comments": [], "raw_markdown_by_id": { "2554930": "Hello everyone,\nI would like to express my gratitude to the organizers of this competition, the members who participated, and all of my teammates( @stgkrtua, @kuto0633, @copasta, @masatomatsui).\n\nIn this discussion, I will talk about my individual solution. Please have a look at our team solution [here](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459703). I believe reading both will deepen your understanding as there might be some overlaps.\n\nI focused on creating a stronger single model, as I believe even in ensembling, without a good single model, it's hard to see significant improvement.\n\nI developed my model based on the code shared by tubo's public code. The scores for the models used in our team ensemble were as follows:\n\n# Model summary\n- 1DUnet\n - Duration 11440\n - downsample 2\n - CV:0.778 (Non PostProccess)\n - Public Score:0.751 (With LowPaassFilter PostProccess)\n - Private Score:0.806 (With LowPaassFilter PostProccess)\n\n- 1DUnet\n - Duration 17280\n - downsample 4\n - CV:0.801 (Non PostProccess)\n - Public Score:0.755 (With LowPaassFilter PostProccess)\n - Private Score:0.814 (With LowPaassFilter PostProccess)\n\nAlthough Ididn't adopt it in the end, simple 1DConv stacking of the team's predictions was effective. We didn't use it in our final submission due to concerns about CV overfitting, but I observed slight improvements upon later review.\nCV: 0.8058\n\n# About the Single Model used:\nI utilized the 1DUnet created by @kmat2019, which was rewritten in PyTorch by @super13579\nhttps://www.kaggle.com/code/super13579/u-net-1d-cnn-with-pytorch\n\nVarious improvements were made to this model for this competition.\nEfforts based on the scores include:\n\n1. Add L1Loss for sleep state prediction difference (refer to team solution for details)\n1. Duplicate flag feature (refer to team solution for details)\n1. Warmup\n1. Two rounds of loss calculation with targets\n1. Change to 5 layers in the Encoder and 4 layers in the Decoder\nFor point 4, I modified the output of the decoder in the following way. The reason is that upon analyzing the predictions, we found that sleep labels were not standing out enough, while onset or wakeup labels were being assigned, so Iwanted the onset and wakeup predictions to take sleep information into account.\n\n\n\nStacking consisted only of 1 layer in the Encoder and a matching Conv1d 1 layer. Adding the duplicate flag feature in stacking also proved to be effective." } }