Thanks for all the discussion and kernel made available by many Kagglers. \nSpecial thanks to @Thomas Rohwer, @Markus F, @Nanashi, @Ram Ramrakhya.
\n\nThis is my second competition. Thanks to all the public kernels, I learned a lot. Here is a brief description on my path in this competition:
\n\nIn the beginning, I am not satisfied with the short description of the data. It would be more helpful to give details on how the data is collected. Instead, everyone seems to be pointed to the quaternion to Euler angle conversion. I do not fully grasp why Euler angle is needed, nor do I think orientation is of any use. In this stage, I believe that the orientation data is a \"red herring\" (quoted from \"The Orientation Sensor\" or \"Science vs. Alchemy\"). Using orientation data would only lead to over-fitting.
After read about quaternion, I found that its derivative can be converted to angular velocity (this). Aha, maybe that is the use of orientation, I will have a 3dB cleaner angular velocity data. Then I started some feature engineering. One day later, I am looking at a data set with ~1400 features for each series, most of them being spectrum of DC removed data, and some other values with their low frequency portion filtered out. The rational behind DC removal is similar to that for orientation data: DC values are irrelevant to the surface type. For example, driving north, making turns, acceleration would not give me any hint on what surface you are on. Except for one key data: velocity, which I think strongly linked to the bumpiness of the surface and the robot's resonant mode. But, the data is missing and I could not afford to mislead my model using estimations from acceleration data without knowing initial conditions on each run. Then I setup a simple NN model and got 0.67 (private 0.79).
What is missing? I tried to clean the anomaly in the features but no better score. Then I read about this: The missing link. It is very interesting. Despite directly using run number to predict surface type, I can link the series to provide bigger picture of this competition. It gives us some insight on the data acquisition process. When converting the quaternion, I can get a estimate of the data sampling rate at around 300Hz. It means that each series contains data collected in 0.4s, which seems to be too short/noisy for our model to recognise. In real world, when we walk around or ride bicycle on different surface, we do not expect to jump to another surface in every 0.4s. Knowing some series belong to the same run would suggest that they are more likely on same surface. It also suggest that the robot is likely to encounter a new surface when you see sudden data behaviour changes in same run. These prior knowledge would greatly increase the surface recognition performance.
With these additional info, my prediction jumps from 0.67 to 0.87 (0.99). So should we use the orientation data and group id? The answer is yes and no. If the data is collected in contiguous run, linking series would greatly help (and indeed should be used in real world application). If the data is collected in a sparse manner, linking series would not work. In this competition, I think the data set is provided with group id and orientation with their reasons. If we do not link the series, the dependency between adjacent series is lost. Such info is quite useful in systems with memory (i.e. Kalman filter). Therefore it should be used in applications which have access to these info.\nOverall, the takeaway is: when driving a robot, recording temporal correlated data and exploit on the correlation can greatly increase surface prediction accuracy.
Thanks for reading.
", "comments": [ { "id": 516016, "author_name": "Ilham Firdausi Putra", "author_url": "", "post_date": "2019-04-13T14:21:27.037000", "content": "Thank you for sharing!
\n\nWhat do you mean by \"cleaner angular velocity data\". What's wrong with the angular velocity data provided in the data set?
", "votes": 1, "replies": [ { "id": 516164, "author_name": "Mark", "author_url": "", "post_date": "2019-04-13T17:51:01.730000", "content": "Thanks for the comment.\nThe data provided are quite noisy in each series due to the IMU limitation.\nIf you have multiple data from different measurements then your SNR would improve (see Signal_averaging).
", "votes": 0, "replies": [] } ] }, { "id": 515982, "author_name": "Alexander Abstreiter", "author_url": "", "post_date": "2019-04-13T13:31:21.103000", "content": "Thanks for sharing your insights and congratulations to the second place!
", "votes": 0, "replies": [] } ], "comment_count": 3, "messages": [ { "id": 515756, "postDate": "2019-04-13T04:34:32.487Z", "content": "Thanks for all the discussion and kernel made available by many Kagglers. \nSpecial thanks to @Thomas Rohwer, @Markus F, @Nanashi, @Ram Ramrakhya.
\n\nThis is my second competition. Thanks to all the public kernels, I learned a lot. Here is a brief description on my path in this competition:
\n\nIn the beginning, I am not satisfied with the short description of the data. It would be more helpful to give details on how the data is collected. Instead, everyone seems to be pointed to the quaternion to Euler angle conversion. I do not fully grasp why Euler angle is needed, nor do I think orientation is of any use. In this stage, I believe that the orientation data is a \"red herring\" (quoted from \"The Orientation Sensor\" or \"Science vs. Alchemy\"). Using orientation data would only lead to over-fitting.
After read about quaternion, I found that its derivative can be converted to angular velocity (this). Aha, maybe that is the use of orientation, I will have a 3dB cleaner angular velocity data. Then I started some feature engineering. One day later, I am looking at a data set with ~1400 features for each series, most of them being spectrum of DC removed data, and some other values with their low frequency portion filtered out. The rational behind DC removal is similar to that for orientation data: DC values are irrelevant to the surface type. For example, driving north, making turns, acceleration would not give me any hint on what surface you are on. Except for one key data: velocity, which I think strongly linked to the bumpiness of the surface and the robot's resonant mode. But, the data is missing and I could not afford to mislead my model using estimations from acceleration data without knowing initial conditions on each run. Then I setup a simple NN model and got 0.67 (private 0.79).
What is missing? I tried to clean the anomaly in the features but no better score. Then I read about this: The missing link. It is very interesting. Despite directly using run number to predict surface type, I can link the series to provide bigger picture of this competition. It gives us some insight on the data acquisition process. When converting the quaternion, I can get a estimate of the data sampling rate at around 300Hz. It means that each series contains data collected in 0.4s, which seems to be too short/noisy for our model to recognise. In real world, when we walk around or ride bicycle on different surface, we do not expect to jump to another surface in every 0.4s. Knowing some series belong to the same run would suggest that they are more likely on same surface. It also suggest that the robot is likely to encounter a new surface when you see sudden data behaviour changes in same run. These prior knowledge would greatly increase the surface recognition performance.
With these additional info, my prediction jumps from 0.67 to 0.87 (0.99). So should we use the orientation data and group id? The answer is yes and no. If the data is collected in contiguous run, linking series would greatly help (and indeed should be used in real world application). If the data is collected in a sparse manner, linking series would not work. In this competition, I think the data set is provided with group id and orientation with their reasons. If we do not link the series, the dependency between adjacent series is lost. Such info is quite useful in systems with memory (i.e. Kalman filter). Therefore it should be used in applications which have access to these info.\nOverall, the takeaway is: when driving a robot, recording temporal correlated data and exploit on the correlation can greatly increase surface prediction accuracy.
Thanks for reading.
", "rawMarkdown": "Thanks for all the discussion and kernel made available by many Kagglers. \nSpecial thanks to [@Thomas Rohwer](https://www.kaggle.com/trohwer64), [@Markus F](https://www.kaggle.com/friedchips), [@Nanashi](https://www.kaggle.com/jesucristo), [@Ram Ramrakhya](https://www.kaggle.com/axel81).\n\nThis is my second competition. Thanks to all the public kernels, I learned a lot. Here is a brief description on my path in this competition:\n\n1. In the beginning, I am not satisfied with the short description of the data. It would be more helpful to give details on how the data is collected. Instead, everyone seems to be pointed to the quaternion to Euler angle conversion. I do not fully grasp why Euler angle is needed, nor do I think orientation is of any use. In this stage, I believe that the orientation data is a \"red herring\" (quoted from [\"The Orientation Sensor\" or \"Science vs. Alchemy\"](https://www.kaggle.com/c/career-con-2019/discussion/87239#latest-512162)). Using orientation data would only lead to over-fitting.\n\n2. After read about quaternion, I found that its derivative can be converted to angular velocity ([this](https://math.stackexchange.com/questions/160908/how-to-get-angular-velocity-from-difference-orientation-quaternion-and-time)). Aha, maybe that is the use of orientation, I will have a 3dB cleaner angular velocity data. Then I started some feature engineering. One day later, I am looking at a data set with ~1400 features for each series, most of them being spectrum of DC removed data, and some other values with their low frequency portion filtered out. The rational behind DC removal is similar to that for orientation data: DC values are irrelevant to the surface type. For example, driving north, making turns, acceleration would not give me any hint on what surface you are on. Except for one key data: velocity, which I think strongly linked to the bumpiness of the surface and the robot's resonant mode. But, the data is missing and I could not afford to mislead my model using estimations from acceleration data without knowing initial conditions on each run. Then I setup a simple NN model and got 0.67 (private 0.79).\n\n3. What is missing? I tried to clean the anomaly in the features but no better score. Then I read about this: [The missing link](https://www.kaggle.com/friedchips/the-missing-link). It is very interesting. Despite directly using run number to predict surface type, I can link the series to provide bigger picture of this competition. It gives us some insight on the data acquisition process. When converting the quaternion, I can get a estimate of the data sampling rate at around 300Hz. It means that each series contains data collected in 0.4s, which seems to be too short/noisy for our model to recognise. In real world, when we walk around or ride bicycle on different surface, we do not expect to jump to another surface in every 0.4s. Knowing some series belong to the same run would suggest that they are more likely on same surface. It also suggest that the robot is likely to encounter a new surface when you see sudden data behaviour changes in same run. These prior knowledge would greatly increase the surface recognition performance.\n\n4. With these additional info, my prediction jumps from 0.67 to 0.87 (0.99). So should we use the orientation data and group id? The answer is yes and no. If the data is collected in contiguous run, linking series would greatly help (and indeed should be used in real world application). If the data is collected in a sparse manner, linking series would not work. In this competition, I think the data set is provided with group id and orientation with their reasons. If we do not link the series, the dependency between adjacent series is lost. Such info is quite useful in systems with memory (i.e. Kalman filter). Therefore it should be used in applications which have access to these info.\nOverall, the takeaway is: when driving a robot, recording temporal correlated data and exploit on the correlation can greatly increase surface prediction accuracy.\n\nThanks for reading.\n\n\n", "votes": 9 }, { "id": 516016, "postDate": "2019-04-13T14:21:27.037Z", "content": "Thank you for sharing!
\n\nWhat do you mean by \"cleaner angular velocity data\". What's wrong with the angular velocity data provided in the data set?
", "rawMarkdown": "Thank you for sharing! \n\nWhat do you mean by \"cleaner angular velocity data\". What's wrong with the angular velocity data provided in the data set? ", "votes": 1, "replies": [ { "id": 516164, "postDate": "2019-04-13T17:51:01.730Z", "content": "Thanks for the comment.\nThe data provided are quite noisy in each series due to the IMU limitation.\nIf you have multiple data from different measurements then your SNR would improve (see Signal_averaging).
", "rawMarkdown": "Thanks for the comment.\nThe data provided are quite noisy in each series due to the IMU limitation.\nIf you have multiple data from different measurements then your SNR would improve (see [Signal_averaging](https://en.wikipedia.org/wiki/Signal_averaging)).\n" } ] }, { "id": 515982, "postDate": "2019-04-13T13:31:21.103Z", "content": "Thanks for sharing your insights and congratulations to the second place!
", "rawMarkdown": "Thanks for sharing your insights and congratulations to the second place!" } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T15:43:38Z", "competition_id": 13242, "competition_slug": "career-con-2019", "team_id": 2987347, "solution_url": "/competitions/career-con-2019/writeups/markyff-some-thoughts-on-this-competition" }