{ "id": 603402, "title": "Null values question", "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/603402", "author_name": "hyd", "post_date": "2025-09-02T11:27:20.398000", "votes": 6, "comment_count": 31, "views": 0, "content": "
\n

Expect approximately 3,500 sequences in the hidden test set.
\n Half of the hidden-test sequences are recorded with IMU only; the thermopile (thm_) and time-of-flight (tof__v*) columns are still present but contain null values for those sequences.

\n
\n

I find this issue very important, but after searching through many discussion forums, I haven't found anyone asking about this. With only about ten hours left, I'd like to ask: are these null values masked by sequence_ids or are they entirely masked based on the subject?

", "messages": [ { "id": 3280256, "postDate": "2025-09-02T12:10:18.020Z", "content": "

Translation: I have 0.91 CV and I am preparing my single submission show but I am worried that the distribution shift on the test set may ruin my show.

\n

Jokes aside, GL & HF.

", "rawMarkdown": "Translation: I have 0.91 CV and I am preparing my single submission show but I am worried that the distribution shift on the test set may ruin my show.\n\nJokes aside, GL & HF.", "votes": 13, "replies": [ { "id": 3280264, "postDate": "2025-09-02T12:16:34.400Z", "content": "

Too lazy to prob the test. If the test set isn't what I expected, there's no need to submit at all.🤣

", "rawMarkdown": "Too lazy to prob the test. If the test set isn't what I expected, there's no need to submit at all.🤣", "votes": 1 }, { "id": 3280486, "postDate": "2025-09-02T17:30:02.387Z", "content": "

with group fold?

", "rawMarkdown": "with group fold? " } ] }, { "id": 3280217, "postDate": "2025-09-02T11:27:20.400Z", "content": "
\n

Expect approximately 3,500 sequences in the hidden test set.
\n Half of the hidden-test sequences are recorded with IMU only; the thermopile (thm_) and time-of-flight (tof__v*) columns are still present but contain null values for those sequences.

\n
\n

I find this issue very important, but after searching through many discussion forums, I haven't found anyone asking about this. With only about ten hours left, I'd like to ask: are these null values masked by sequence_ids or are they entirely masked based on the subject?

", "rawMarkdown": ">Expect approximately 3,500 sequences in the hidden test set.\nHalf of the hidden-test sequences are recorded with IMU only; the thermopile (thm_) and time-of-flight (tof__v*) columns are still present but contain null values for those sequences.\n\nI find this issue very important, but after searching through many discussion forums, I haven't found anyone asking about this. With only about ten hours left, I'd like to ask: are these null values masked by sequence_ids or are they entirely masked based on the subject?", "votes": 6 }, { "id": 3280373, "postDate": "2025-09-02T14:20:25.990Z", "content": "

Needless to say I am not surprised at all to see you there :)

", "rawMarkdown": "Needless to say I am not surprised at all to see you there :) ", "votes": 1 }, { "id": 3280350, "postDate": "2025-09-02T13:50:50.383Z", "content": "

Continuing your one sub gold medal streak?

", "rawMarkdown": "Continuing your one sub gold medal streak?", "votes": 1, "replies": [ { "id": 3280363, "postDate": "2025-09-02T14:01:37.877Z", "content": "

I'm considering whether to submit. My score isn't high, and if I misunderstood the test data, I might get a silver medal.😂

", "rawMarkdown": "I'm considering whether to submit. My score isn't high, and if I misunderstood the test data, I might get a silver medal.😂", "votes": -1 }, { "id": 3280445, "postDate": "2025-09-02T16:14:39.440Z", "content": "

All of you GOAT guy how you achive top 10 position in just 1 or 2 sub

", "rawMarkdown": "All of you GOAT guy how you achive top 10 position in just 1 or 2 sub", "votes": 1 } ] }, { "id": 3280306, "postDate": "2025-09-02T12:50:31.877Z", "content": "

I think they are masked

", "rawMarkdown": "I think they are masked", "votes": 1 }, { "id": 3280227, "postDate": "2025-09-02T11:35:32.033Z", "content": "

Pretty timely and natural question from participant with 0 submits so far.

", "rawMarkdown": "Pretty timely and natural question from participant with 0 submits so far.", "votes": 2, "replies": [ { "id": 3280237, "postDate": "2025-09-02T11:40:54.197Z", "content": "

I think this is a question that someone would ask right after the competition starts, but I couldn't find the answer after searching through the discussion forums.

", "rawMarkdown": "I think this is a question that someone would ask right after the competition starts, but I couldn't find the answer after searching through the discussion forums.", "replies": [ { "id": 3280252, "postDate": "2025-09-02T12:06:42.940Z", "content": "

What would that change if I may ask ?

", "rawMarkdown": "What would that change if I may ask ?" } ] } ] }, { "id": 3280753, "postDate": "2025-09-03T05:30:48.703Z", "content": "

@hydantess can you now explain why exactly you asked this not so candid question 10h before end of comp without having done any sub?

\n

This pushed a lot of teams in a \"leak hunt\" which is kind of detrimental 10h before end of comp :)

", "rawMarkdown": "@hydantess can you now explain why exactly you asked this not so candid question 10h before end of comp without having done any sub?\n\nThis pushed a lot of teams in a \"leak hunt\" which is kind of detrimental 10h before end of comp :)", "replies": [ { "id": 3280769, "postDate": "2025-09-03T06:04:47.997Z", "content": "

Right from the start of this competition, I realized that training some subject-level models would be necessary, since that’s exactly what I did when I participated in the U.S. Patent Phrase to Phrase Matching competition. After competing for so many years, I honestly didn’t consider this to be some hugely important trick. So I went ahead and trained a few such models based on my own understanding.

\n

It wasn’t until there were about 10 hours left in the competition that I started writing the inference code. That’s when I realized the problem description on the competition page was ambiguous—the actual null value masking method had to be probed and figured out on our own. I didn’t want to waste my last five submission attempts, and I believed this was something the competition should have made clear in the description.

\n

What’s more, I assumed that all the top teams had trained these kinds of models, which is why I made that post. But unexpectedly, some of the leading teams seemed… pretty triggered.

\n

Maybe I shouldn’t have posted anything at all. And as punishment, my subject-level model threw errors, three other submissions timed out, and out of my final five attempts, only the sequence-level model actually scored.

", "rawMarkdown": "Right from the start of this competition, I realized that training some subject-level models would be necessary, since that’s exactly what I did when I participated in the U.S. Patent Phrase to Phrase Matching competition. After competing for so many years, I honestly didn’t consider this to be some hugely important trick. So I went ahead and trained a few such models based on my own understanding.\n\nIt wasn’t until there were about 10 hours left in the competition that I started writing the inference code. That’s when I realized the problem description on the competition page was ambiguous—the actual null value masking method had to be probed and figured out on our own. I didn’t want to waste my last five submission attempts, and I believed this was something the competition should have made clear in the description.\n\nWhat’s more, I assumed that all the top teams had trained these kinds of models, which is why I made that post. But unexpectedly, some of the leading teams seemed… pretty triggered.\n\nMaybe I shouldn’t have posted anything at all. And as punishment, my subject-level model threw errors, three other submissions timed out, and out of my final five attempts, only the sequence-level model actually scored.", "votes": 1, "replies": [ { "id": 3280775, "postDate": "2025-09-03T06:24:52.660Z", "content": "

Thanks for your answer! It was not obvious to me nor the organizers that this was the go to method :)

", "rawMarkdown": "Thanks for your answer! It was not obvious to me nor the organizers that this was the go to method :)" } ] } ] }, { "id": 3280598, "postDate": "2025-09-02T23:14:48.267Z", "content": "

This was my first competition on Kaggle, and it was quite a challenge for me. I put a lot of effort into it, but I couldn’t achieve the results I was hoping for. I’m really eager to understand which parts I might have overlooked and what approaches others used to reach scores above 86.

", "rawMarkdown": "This was my first competition on Kaggle, and it was quite a challenge for me. I put a lot of effort into it, but I couldn’t achieve the results I was hoping for. I’m really eager to understand which parts I might have overlooked and what approaches others used to reach scores above 86." }, { "id": 3280368, "postDate": "2025-09-02T14:10:33.303Z", "content": "

Not random, I personally tend to believe that \"50% are null values\" means that for a fixed 50% of the subjects, all sequences have null values for features other than IMU, while the other 50% of the subjects have all sequences with complete features.

", "rawMarkdown": "Not random, I personally tend to believe that \"50% are null values\" means that for a fixed 50% of the subjects, all sequences have null values for features other than IMU, while the other 50% of the subjects have all sequences with complete features." }, { "id": 3280302, "postDate": "2025-09-02T12:47:55.850Z", "content": "

I would like to give you an answer but I would need to understand the question first! Can you clarify please? :)

", "rawMarkdown": "I would like to give you an answer but I would need to understand the question first! Can you clarify please? :)", "replies": [ { "id": 3280305, "postDate": "2025-09-02T12:50:20.093Z", "content": "

As far as I understood, i think he means if the Nans are randomly distributed among the subject's sequences or if the whole sequence of a subject is masked. I would say the latter.

", "rawMarkdown": "As far as I understood, i think he means if the `Nan`s are randomly distributed among the subject's sequences or if the whole sequence of a subject is masked. I would say the latter.", "replies": [ { "id": 3280310, "postDate": "2025-09-02T12:53:46.253Z", "content": "

If that's the question it seemed obvious to me that half of the sequences have their entire THM and TOF missing completely. But I don't think that is the question.

", "rawMarkdown": "If that's the question it seemed obvious to me that half of the sequences have their entire THM and TOF missing completely. But I don't think that is the question." } ] }, { "id": 3280309, "postDate": "2025-09-02T12:52:20.173Z", "content": "

Or another possibility would be if a subject has all of its sequences masked or just some of its sequences masked

", "rawMarkdown": "Or another possibility would be if a subject has *all* of its sequences masked or just *some* of its sequences masked", "votes": 1, "replies": [ { "id": 3280314, "postDate": "2025-09-02T12:55:56.513Z", "content": "

Yes but what does that would change ?

", "rawMarkdown": "Yes but what does that would change ?" }, { "id": 3280315, "postDate": "2025-09-02T12:56:48.173Z", "content": "

That's what i ask~~~

", "rawMarkdown": "That's what i ask~~~ ", "votes": 1 }, { "id": 3280319, "postDate": "2025-09-02T13:01:43.457Z", "content": "

I've realized that my training approach was based on the assumption that the entire subject would either be completely masked or all have TOF, which could lead to some issues. That's why I started this thread to discuss it.

", "rawMarkdown": "I've realized that my training approach was based on the assumption that the entire subject would either be completely masked or all have TOF, which could lead to some issues. That's why I started this thread to discuss it." }, { "id": 3280330, "postDate": "2025-09-02T13:24:25.640Z", "content": "

@optimo Everything😀 exploits are possible from the simple fact that each subject does same gesture-orientation combination twice where the longer sequence has a static beginning pattern. This is sad if randomness is sequence based

", "rawMarkdown": "@optimo Everything😀 exploits are possible from the simple fact that each subject does same gesture-orientation combination twice where the longer sequence has a static beginning pattern. This is sad if randomness is sequence based" }, { "id": 3280576, "postDate": "2025-09-02T21:22:26.287Z", "rawMarkdown": "", "isDeleted": true } ] } ] }, { "id": 3280268, "postDate": "2025-09-02T12:21:40.083Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 3280275, "postDate": "2025-09-02T12:27:36.710Z", "content": "
\n

lets say, na values are not random and totally different than train data,

\n
\n

Thank you for telling me this.
\n So what exactly do you mean? Please tell me directly—I still have time to train a model and make a submission.

", "rawMarkdown": ">lets say, na values are not random and totally different than train data,\n\nThank you for telling me this.\n So what exactly do you mean? Please tell me directly—I still have time to train a model and make a submission.", "replies": [ { "id": 3280288, "postDate": "2025-09-02T12:35:55.833Z", "content": "

read it again

", "rawMarkdown": "read it again ", "votes": -1 }, { "id": 3280294, "postDate": "2025-09-02T12:40:04.793Z", "content": "

Of course it's different from the training set. Didn't the description mention that half of the sequence IDs in the test set are null? Isn't it a bit too sensitive to jump straight to private sharing and multi-account accusations?

", "rawMarkdown": "Of course it's different from the training set. Didn't the description mention that half of the sequence IDs in the test set are null? Isn't it a bit too sensitive to jump straight to private sharing and multi-account accusations?" }, { "id": 3280308, "postDate": "2025-09-02T12:51:48.417Z", "content": "

\"my assumption was based on with 0 submission so far, if private dataset is totally different than public one\" ( totally means really totally not sth deterministic or a simple rule based), and the solution requires lb probing / testing. that info must come from somewhere or maybe know sth we don't know. anyway, lets wait till the end. can't wait to see your solution.

", "rawMarkdown": "\"my assumption was based on with 0 submission so far, if private dataset is totally different than public one\" ( totally means really totally not sth deterministic or a simple rule based), and the solution requires lb probing / testing. that info must come from somewhere or maybe know sth we don't know. anyway, lets wait till the end. can't wait to see your solution. " }, { "id": 3280593, "postDate": "2025-09-02T22:36:18.600Z", "rawMarkdown": "", "isDeleted": true } ] } ] } ], "comments": [ { "id": 3280256, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2025-09-02T12:10:18.020000", "content": "

Translation: I have 0.91 CV and I am preparing my single submission show but I am worried that the distribution shift on the test set may ruin my show.

\n

Jokes aside, GL & HF.

", "votes": 13, "replies": [ { "id": 3280264, "author_name": "hyd", "author_url": "", "post_date": "2025-09-02T12:16:34.400000", "content": "

Too lazy to prob the test. If the test set isn't what I expected, there's no need to submit at all.🤣

", "votes": 1, "replies": [] }, { "id": 3280486, "author_name": "Cyrus", "author_url": "", "post_date": "2025-09-02T17:30:02.387000", "content": "

with group fold?

", "votes": 0, "replies": [] } ] }, { "id": 3280373, "author_name": "Theo Viel", "author_url": "", "post_date": "2025-09-02T14:20:25.990000", "content": "

Needless to say I am not surprised at all to see you there :)

", "votes": 1, "replies": [] }, { "id": 3280350, "author_name": "c-number", "author_url": "", "post_date": "2025-09-02T13:50:50.383000", "content": "

Continuing your one sub gold medal streak?

", "votes": 1, "replies": [ { "id": 3280363, "author_name": "hyd", "author_url": "", "post_date": "2025-09-02T14:01:37.877000", "content": "

I'm considering whether to submit. My score isn't high, and if I misunderstood the test data, I might get a silver medal.😂

", "votes": -1, "replies": [] }, { "id": 3280445, "author_name": "Sangram Patil", "author_url": "", "post_date": "2025-09-02T16:14:39.440000", "content": "

All of you GOAT guy how you achive top 10 position in just 1 or 2 sub

", "votes": 1, "replies": [] } ] }, { "id": 3280306, "author_name": "paperxd", "author_url": "", "post_date": "2025-09-02T12:50:31.877000", "content": "

I think they are masked

", "votes": 1, "replies": [] }, { "id": 3280227, "author_name": "Ogurtsov", "author_url": "", "post_date": "2025-09-02T11:35:32.033000", "content": "

Pretty timely and natural question from participant with 0 submits so far.

", "votes": 2, "replies": [ { "id": 3280237, "author_name": "hyd", "author_url": "", "post_date": "2025-09-02T11:40:54.197000", "content": "

I think this is a question that someone would ask right after the competition starts, but I couldn't find the answer after searching through the discussion forums.

", "votes": 0, "replies": [ { "id": 3280252, "author_name": "Optimo", "author_url": "", "post_date": "2025-09-02T12:06:42.940000", "content": "

What would that change if I may ask ?

", "votes": 0, "replies": [] } ] } ] }, { "id": 3280753, "author_name": "Optimo", "author_url": "", "post_date": "2025-09-03T05:30:48.703000", "content": "

@hydantess can you now explain why exactly you asked this not so candid question 10h before end of comp without having done any sub?

\n

This pushed a lot of teams in a \"leak hunt\" which is kind of detrimental 10h before end of comp :)

", "votes": 0, "replies": [ { "id": 3280769, "author_name": "hyd", "author_url": "", "post_date": "2025-09-03T06:04:47.997000", "content": "

Right from the start of this competition, I realized that training some subject-level models would be necessary, since that’s exactly what I did when I participated in the U.S. Patent Phrase to Phrase Matching competition. After competing for so many years, I honestly didn’t consider this to be some hugely important trick. So I went ahead and trained a few such models based on my own understanding.

\n

It wasn’t until there were about 10 hours left in the competition that I started writing the inference code. That’s when I realized the problem description on the competition page was ambiguous—the actual null value masking method had to be probed and figured out on our own. I didn’t want to waste my last five submission attempts, and I believed this was something the competition should have made clear in the description.

\n

What’s more, I assumed that all the top teams had trained these kinds of models, which is why I made that post. But unexpectedly, some of the leading teams seemed… pretty triggered.

\n

Maybe I shouldn’t have posted anything at all. And as punishment, my subject-level model threw errors, three other submissions timed out, and out of my final five attempts, only the sequence-level model actually scored.

", "votes": 1, "replies": [ { "id": 3280775, "author_name": "Optimo", "author_url": "", "post_date": "2025-09-03T06:24:52.660000", "content": "

Thanks for your answer! It was not obvious to me nor the organizers that this was the go to method :)

", "votes": 0, "replies": [] } ] } ] }, { "id": 3280598, "author_name": "Nima Kaveh", "author_url": "", "post_date": "2025-09-02T23:14:48.267000", "content": "

This was my first competition on Kaggle, and it was quite a challenge for me. I put a lot of effort into it, but I couldn’t achieve the results I was hoping for. I’m really eager to understand which parts I might have overlooked and what approaches others used to reach scores above 86.

", "votes": 0, "replies": [] }, { "id": 3280368, "author_name": "Timmy Juicehouse", "author_url": "", "post_date": "2025-09-02T14:10:33.303000", "content": "

Not random, I personally tend to believe that \"50% are null values\" means that for a fixed 50% of the subjects, all sequences have null values for features other than IMU, while the other 50% of the subjects have all sequences with complete features.

", "votes": 0, "replies": [] }, { "id": 3280302, "author_name": "Optimo", "author_url": "", "post_date": "2025-09-02T12:47:55.850000", "content": "

I would like to give you an answer but I would need to understand the question first! Can you clarify please? :)

", "votes": 0, "replies": [ { "id": 3280305, "author_name": "moth", "author_url": "", "post_date": "2025-09-02T12:50:20.093000", "content": "

As far as I understood, i think he means if the Nans are randomly distributed among the subject's sequences or if the whole sequence of a subject is masked. I would say the latter.

", "votes": 0, "replies": [ { "id": 3280310, "author_name": "Optimo", "author_url": "", "post_date": "2025-09-02T12:53:46.253000", "content": "

If that's the question it seemed obvious to me that half of the sequences have their entire THM and TOF missing completely. But I don't think that is the question.

", "votes": 0, "replies": [] } ] }, { "id": 3280309, "author_name": "moth", "author_url": "", "post_date": "2025-09-02T12:52:20.173000", "content": "

Or another possibility would be if a subject has all of its sequences masked or just some of its sequences masked

", "votes": 1, "replies": [ { "id": 3280314, "author_name": "Optimo", "author_url": "", "post_date": "2025-09-02T12:55:56.513000", "content": "

Yes but what does that would change ?

", "votes": 0, "replies": [] }, { "id": 3280315, "author_name": "hyd", "author_url": "", "post_date": "2025-09-02T12:56:48.173000", "content": "

That's what i ask~~~

", "votes": 1, "replies": [] }, { "id": 3280319, "author_name": "hyd", "author_url": "", "post_date": "2025-09-02T13:01:43.457000", "content": "

I've realized that my training approach was based on the assumption that the entire subject would either be completely masked or all have TOF, which could lead to some issues. That's why I started this thread to discuss it.

", "votes": 0, "replies": [] }, { "id": 3280330, "author_name": "Cihat Emre Çeliker", "author_url": "", "post_date": "2025-09-02T13:24:25.640000", "content": "

@optimo Everything😀 exploits are possible from the simple fact that each subject does same gesture-orientation combination twice where the longer sequence has a static beginning pattern. This is sad if randomness is sequence based

", "votes": 0, "replies": [] }, { "id": 3280576, "author_name": "", "author_url": "", "post_date": "2025-09-02T21:22:26.287000", "content": "", "votes": 0, "replies": [] } ] } ] }, { "id": 3280268, "author_name": "", "author_url": "", "post_date": "2025-09-02T12:21:40.083000", "content": "", "votes": 0, "replies": [ { "id": 3280275, "author_name": "hyd", "author_url": "", "post_date": "2025-09-02T12:27:36.710000", "content": "
\n

lets say, na values are not random and totally different than train data,

\n
\n

Thank you for telling me this.
\n So what exactly do you mean? Please tell me directly—I still have time to train a model and make a submission.

", "votes": 0, "replies": [ { "id": 3280288, "author_name": "Davut Polat", "author_url": "", "post_date": "2025-09-02T12:35:55.833000", "content": "

read it again

", "votes": -1, "replies": [] }, { "id": 3280294, "author_name": "hyd", "author_url": "", "post_date": "2025-09-02T12:40:04.793000", "content": "

Of course it's different from the training set. Didn't the description mention that half of the sequence IDs in the test set are null? Isn't it a bit too sensitive to jump straight to private sharing and multi-account accusations?

", "votes": 0, "replies": [] }, { "id": 3280308, "author_name": "Davut Polat", "author_url": "", "post_date": "2025-09-02T12:51:48.417000", "content": "

\"my assumption was based on with 0 submission so far, if private dataset is totally different than public one\" ( totally means really totally not sth deterministic or a simple rule based), and the solution requires lb probing / testing. that info must come from somewhere or maybe know sth we don't know. anyway, lets wait till the end. can't wait to see your solution.

", "votes": 0, "replies": [] }, { "id": 3280593, "author_name": "", "author_url": "", "post_date": "2025-09-02T22:36:18.600000", "content": "", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3280256": "Translation: I have 0.91 CV and I am preparing my single submission show but I am worried that the distribution shift on the test set may ruin my show.\n\nJokes aside, GL & HF.", "3280217": ">Expect approximately 3,500 sequences in the hidden test set.\nHalf of the hidden-test sequences are recorded with IMU only; the thermopile (thm_) and time-of-flight (tof__v*) columns are still present but contain null values for those sequences.\n\nI find this issue very important, but after searching through many discussion forums, I haven't found anyone asking about this. With only about ten hours left, I'd like to ask: are these null values masked by sequence_ids or are they entirely masked based on the subject?", "3280373": "Needless to say I am not surprised at all to see you there :) ", "3280350": "Continuing your one sub gold medal streak?", "3280306": "I think they are masked", "3280227": "Pretty timely and natural question from participant with 0 submits so far.", "3280753": "@hydantess can you now explain why exactly you asked this not so candid question 10h before end of comp without having done any sub?\n\nThis pushed a lot of teams in a \"leak hunt\" which is kind of detrimental 10h before end of comp :)", "3280598": "This was my first competition on Kaggle, and it was quite a challenge for me. I put a lot of effort into it, but I couldn’t achieve the results I was hoping for. I’m really eager to understand which parts I might have overlooked and what approaches others used to reach scores above 86.", "3280368": "Not random, I personally tend to believe that \"50% are null values\" means that for a fixed 50% of the subjects, all sequences have null values for features other than IMU, while the other 50% of the subjects have all sequences with complete features.", "3280302": "I would like to give you an answer but I would need to understand the question first! Can you clarify please? :)", "3280268": "" } }