{ "id": 5204, "title": "Trouble understanding the Data Preparation", "url": "/competitions/accelerometer-biometric-competition/discussion/5204", "author_name": "", "post_date": "2013-07-26T04:56:35.697Z", "votes": 1, "comment_count": 6, "views": 2755, "content": "

I apologize in advance if this is a lame question but I have trouble understanding the first pointer in the Data Preparation section. 

\n

After removing from the data set samples from devices with fewer than 6000 samples collected and sequences of samples during periods of rest exceeding 10 seconds, 60 million samples from 387 users were made available for the competition.

\n

Especially the 6000 samples collected and the periods of rest exceeding 10 seconds. Appreciate any clarification in this regard !

\n

 

", "messages": [ { "id": "27678", "postDate": "07/26/2013 04:56:35", "content": "

I apologize in advance if this is a lame question but I have trouble understanding the first pointer in the Data Preparation section. 

\n

After removing from the data set samples from devices with fewer than 6000 samples collected and sequences of samples during periods of rest exceeding 10 seconds, 60 million samples from 387 users were made available for the competition.

\n

Especially the 6000 samples collected and the periods of rest exceeding 10 seconds. Appreciate any clarification in this regard !

\n

 

", "rawMarkdown": "", "votes": null }, { "id": "27683", "postDate": "07/26/2013 10:30:02", "content": "

Part of your question would be answered by the "Sampling frequency" sub-section in the "More on Accelerometer Data" page.

\n

Let's say we only have 3 devices; A, B & C. The Android app on each of the devices queries the Accelerometer every 200ms and makes note of the G-readings on X, Y & Z axis(along with a timestamp). So, if the app was running for 10mins on each of the device then each device would have (10min*60secs*1000ms/200ms) = 3,000 samples collected on it. Furthermore, 300 consecutive samples are bunched into a sequence and given a unique sequence id.

\n

Now, let's say there was some trouble with device B and only 400 samples were collected instead of the anticipated 3,000 samples. As the number of samples is less than the threshold of 6000 samples required to include a device - we will exclude device B & and its data from further consideration.

\n

We are now left with devices A & C.

\n

Now, let's say we noticed a 5 min period (later half of the above mentioned 10-min period) of resting on device C. (I think a period of resting could have been defined as no change in G-readings on X, Y, & Z across a set of consecutive timestamps). All sequences (of samples) which fell in the resting period were excluded.

\n

So, in the end we would have 3,000 samples & 10 sequences from device A & 1,500 samples & 5 sequences from device C.

\n

Hope that clears things up.

", "rawMarkdown": "", "votes": null }, { "id": "27685", "postDate": "07/26/2013 12:57:20", "content": "

Thank you Sashikanth for your clarification.

\n

In other words:

\n

- Only devices that submitted more than 6000 samples were included in the data set.

\n

- Sequences of sample data that had no significant change in the G readings for a period of 10 seconds (~50 samples) were excluded from the data set.

\n

 

\n

 

", "rawMarkdown": "", "votes": null }, { "id": "27692", "postDate": "07/26/2013 16:00:38", "content": "

Thank you Sashikanth and geoffkl for the crystal clear explanation.

", "rawMarkdown": "", "votes": null }, { "id": "27745", "postDate": "07/29/2013 13:41:47", "content": "

Hi guys:

\n

 

\n

Sorry, maybe I am not getting right... So, as I understand, if a "sample" is a single "reading", then, if ~200ms sampling rate yields 5 samples per second, then in 10 min we suppose to get ~ 5 * 60sec * 10 min = 3000 samples, not 600.000 right?

", "rawMarkdown": "", "votes": null }, { "id": "27746", "postDate": "07/29/2013 13:45:10", "content": "

[quote=Pavel Senin;27745]

\n

Hi guys:

\n

 

\n

Sorry, maybe I am not getting right... So, as I understand, if a "sample" is a single "reading", then, if ~200ms sampling rate yields 5 samples per second, then in 10 min we suppose to get ~ 5 * 60sec * 10 min = 3000 samples, not 600.000 right?

\n

[/quote]

\n

You are correct. I will update my reply, above.

", "rawMarkdown": "", "votes": null }, { "id": "27758", "postDate": "07/29/2013 14:35:56", "content": "

Thank you for clarifying things.

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 27683, "author_name": "sashikanthdareddy", "author_url": "", "post_date": "07/26/2013 10:30:02", "content": "

Part of your question would be answered by the "Sampling frequency" sub-section in the "More on Accelerometer Data" page.

\n

Let's say we only have 3 devices; A, B & C. The Android app on each of the devices queries the Accelerometer every 200ms and makes note of the G-readings on X, Y & Z axis(along with a timestamp). So, if the app was running for 10mins on each of the device then each device would have (10min*60secs*1000ms/200ms) = 3,000 samples collected on it. Furthermore, 300 consecutive samples are bunched into a sequence and given a unique sequence id.

\n

Now, let's say there was some trouble with device B and only 400 samples were collected instead of the anticipated 3,000 samples. As the number of samples is less than the threshold of 6000 samples required to include a device - we will exclude device B & and its data from further consideration.

\n

We are now left with devices A & C.

\n

Now, let's say we noticed a 5 min period (later half of the above mentioned 10-min period) of resting on device C. (I think a period of resting could have been defined as no change in G-readings on X, Y, & Z across a set of consecutive timestamps). All sequences (of samples) which fell in the resting period were excluded.

\n

So, in the end we would have 3,000 samples & 10 sequences from device A & 1,500 samples & 5 sequences from device C.

\n

Hope that clears things up.

", "votes": null, "replies": [] }, { "id": 27685, "author_name": "geoff3", "author_url": "", "post_date": "07/26/2013 12:57:20", "content": "

Thank you Sashikanth for your clarification.

\n

In other words:

\n

- Only devices that submitted more than 6000 samples were included in the data set.

\n

- Sequences of sample data that had no significant change in the G readings for a period of 10 seconds (~50 samples) were excluded from the data set.

\n

 

\n

 

", "votes": null, "replies": [] }, { "id": 27692, "author_name": "jugeshsundram", "author_url": "", "post_date": "07/26/2013 16:00:38", "content": "

Thank you Sashikanth and geoffkl for the crystal clear explanation.

", "votes": null, "replies": [] }, { "id": 27745, "author_name": "seninp", "author_url": "", "post_date": "07/29/2013 13:41:47", "content": "

Hi guys:

\n

 

\n

Sorry, maybe I am not getting right... So, as I understand, if a "sample" is a single "reading", then, if ~200ms sampling rate yields 5 samples per second, then in 10 min we suppose to get ~ 5 * 60sec * 10 min = 3000 samples, not 600.000 right?

", "votes": null, "replies": [] }, { "id": 27746, "author_name": "sashikanthdareddy", "author_url": "", "post_date": "07/29/2013 13:45:10", "content": "

[quote=Pavel Senin;27745]

\n

Hi guys:

\n

 

\n

Sorry, maybe I am not getting right... So, as I understand, if a "sample" is a single "reading", then, if ~200ms sampling rate yields 5 samples per second, then in 10 min we suppose to get ~ 5 * 60sec * 10 min = 3000 samples, not 600.000 right?

\n

[/quote]

\n

You are correct. I will update my reply, above.

", "votes": null, "replies": [] }, { "id": 27758, "author_name": "seninp", "author_url": "", "post_date": "07/29/2013 14:35:56", "content": "

Thank you for clarifying things.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "27678": "", "27683": "", "27685": "", "27692": "", "27745": "", "27746": "", "27758": "" }, "source": "meta" }