{
  "id": 24254,
  "title": "How to perform Data Analysis and reveal the gold in data?",
  "url": "/competitions/bosch-production-line-performance/discussion/24254",
  "author_name": "",
  "post_date": "2016-10-10T17:52:48.010Z",
  "votes": null,
  "comment_count": 4,
  "views": 1372,
  "content": "<p>Hi,</p>\n\n<p>Can you please provide what are the best steps procedure / typed of tests to perform for Data Analysis? I know there are heated discussions going on in other posts on leaks /morality / usefulness etc. I believe its an art to do data analysis (at times you get lucky with leaks) but most of the times its thorough analysis which exposes the leaks or may be at times valuable insights to data. </p>\n\n<p>I have seen many master Kagglers find leaks and at times find the magic features among 100s and thousands of features. I dont know how they do it? Can you please tell me how you do it, I do very basic analysis and most often am unable to spot these details. </p>\n\n<p>Can you throw some light on your approaches please?</p>\n\n<p>Thanks and Regards\nSayantan</p>",
  "messages": [
    {
      "id": "138740",
      "postDate": "10/10/2016 17:52:48",
      "content": "<p>Hi,</p>\n\n<p>Can you please provide what are the best steps procedure / typed of tests to perform for Data Analysis? I know there are heated discussions going on in other posts on leaks /morality / usefulness etc. I believe its an art to do data analysis (at times you get lucky with leaks) but most of the times its thorough analysis which exposes the leaks or may be at times valuable insights to data. </p>\n\n<p>I have seen many master Kagglers find leaks and at times find the magic features among 100s and thousands of features. I dont know how they do it? Can you please tell me how you do it, I do very basic analysis and most often am unable to spot these details. </p>\n\n<p>Can you throw some light on your approaches please?</p>\n\n<p>Thanks and Regards\nSayantan</p>",
      "rawMarkdown": "Hi,\r\n\r\nCan you please provide what are the best steps procedure / typed of tests to perform for Data Analysis? I know there are heated discussions going on in other posts on leaks /morality / usefulness etc. I believe its an art to do data analysis (at times you get lucky with leaks) but most of the times its thorough analysis which exposes the leaks or may be at times valuable insights to data. \r\n\r\nI have seen many master Kagglers find leaks and at times find the magic features among 100s and thousands of features. I dont know how they do it? Can you please tell me how you do it, I do very basic analysis and most often am unable to spot these details. \r\n\r\nCan you throw some light on your approaches please?\r\n\r\nThanks and Regards\r\nSayantan",
      "votes": null
    },
    {
      "id": "138742",
      "postDate": "10/10/2016 18:14:33",
      "content": "<p>Check this very good notebook from dune_dweller from the leak on Telstra competition.</p>\n\n<p><a href=\"https://github.com/gereleth/kaggle-telstra/blob/master/Discovering the magic feature.ipynb\">https://github.com/gereleth/kaggle-telstra/blob/master/Discovering%20the%20magic%20feature.ipynb</a></p>",
      "rawMarkdown": "Check this very good notebook from dune_dweller from the leak on Telstra competition.\r\n\r\nhttps://github.com/gereleth/kaggle-telstra/blob/master/Discovering%20the%20magic%20feature.ipynb",
      "votes": null
    },
    {
      "id": "139120",
      "postDate": "10/12/2016 16:26:08",
      "content": "<ol>\n<li><p>Fill your pan so it is &#190; full of data. Submerge it in the code so it is just under the surface of the data.table.</p></li>\n<li><p>Shake the data.table vigorously several times. Shake it back and forth and side to side. Make sure you do not shake so intensely that golds get washed out of the pan.</p></li>\n<li><p>Stop shaking the data and begin using a gentle for loop motion. The data should begin spinning in a circle inside the data.table. Doing this will make most of the noise wash out of the data.table or dissolve. </p></li>\n<li><p>????</p></li>\n<li><p>PROFIT!</p></li>\n</ol>",
      "rawMarkdown": "1. Fill your pan so it is ¾ full of data. Submerge it in the code so it is just under the surface of the data.table.\r\n\r\n2. Shake the data.table vigorously several times. Shake it back and forth and side to side. Make sure you do not shake so intensely that golds get washed out of the pan.\r\n\r\n3. Stop shaking the data and begin using a gentle for loop motion. The data should begin spinning in a circle inside the data.table. Doing this will make most of the noise wash out of the data.table or dissolve. \r\n\r\n4. ????\r\n\r\n4. PROFIT!",
      "votes": null
    },
    {
      "id": "139209",
      "postDate": "10/12/2016 22:14:05",
      "content": "<p>Don't underestimate one of the best pattern recognition systems ever invented. I spent quite some looking at &quot;barcode&quot; pictures I created from the data.</p>",
      "rawMarkdown": "Don't underestimate one of the best pattern recognition systems ever invented. I spent quite some looking at \"barcode\" pictures I created from the data.",
      "votes": null
    },
    {
      "id": "139336",
      "postDate": "10/13/2016 14:14:52",
      "content": "<p>Hi SR,</p>\n\n<p>The ways to find magic features are complicated. We sometimes see raw data.<br>\nMany kagglers have already found magical features. We can learn about them from Forum.</p>",
      "rawMarkdown": "Hi SR,\r\n\r\nThe ways to find magic features are complicated. We sometimes see raw data.<br>\r\nMany kagglers have already found magical features. We can learn about them from Forum.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 138742,
      "author_name": "pvlima",
      "author_url": "",
      "post_date": "10/10/2016 18:14:33",
      "content": "<p>Check this very good notebook from dune_dweller from the leak on Telstra competition.</p>\n\n<p><a href=\"https://github.com/gereleth/kaggle-telstra/blob/master/Discovering the magic feature.ipynb\">https://github.com/gereleth/kaggle-telstra/blob/master/Discovering%20the%20magic%20feature.ipynb</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 139120,
      "author_name": "mikeskim",
      "author_url": "",
      "post_date": "10/12/2016 16:26:08",
      "content": "<ol>\n<li><p>Fill your pan so it is &#190; full of data. Submerge it in the code so it is just under the surface of the data.table.</p></li>\n<li><p>Shake the data.table vigorously several times. Shake it back and forth and side to side. Make sure you do not shake so intensely that golds get washed out of the pan.</p></li>\n<li><p>Stop shaking the data and begin using a gentle for loop motion. The data should begin spinning in a circle inside the data.table. Doing this will make most of the noise wash out of the data.table or dissolve. </p></li>\n<li><p>????</p></li>\n<li><p>PROFIT!</p></li>\n</ol>",
      "votes": null,
      "replies": []
    },
    {
      "id": 139209,
      "author_name": "kailicht",
      "author_url": "",
      "post_date": "10/12/2016 22:14:05",
      "content": "<p>Don't underestimate one of the best pattern recognition systems ever invented. I spent quite some looking at &quot;barcode&quot; pictures I created from the data.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 139336,
      "author_name": "toshik",
      "author_url": "",
      "post_date": "10/13/2016 14:14:52",
      "content": "<p>Hi SR,</p>\n\n<p>The ways to find magic features are complicated. We sometimes see raw data.<br>\nMany kagglers have already found magical features. We can learn about them from Forum.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "138740": "Hi,\r\n\r\nCan you please provide what are the best steps procedure / typed of tests to perform for Data Analysis? I know there are heated discussions going on in other posts on leaks /morality / usefulness etc. I believe its an art to do data analysis (at times you get lucky with leaks) but most of the times its thorough analysis which exposes the leaks or may be at times valuable insights to data. \r\n\r\nI have seen many master Kagglers find leaks and at times find the magic features among 100s and thousands of features. I dont know how they do it? Can you please tell me how you do it, I do very basic analysis and most often am unable to spot these details. \r\n\r\nCan you throw some light on your approaches please?\r\n\r\nThanks and Regards\r\nSayantan",
    "138742": "Check this very good notebook from dune_dweller from the leak on Telstra competition.\r\n\r\nhttps://github.com/gereleth/kaggle-telstra/blob/master/Discovering%20the%20magic%20feature.ipynb",
    "139120": "1. Fill your pan so it is ¾ full of data. Submerge it in the code so it is just under the surface of the data.table.\r\n\r\n2. Shake the data.table vigorously several times. Shake it back and forth and side to side. Make sure you do not shake so intensely that golds get washed out of the pan.\r\n\r\n3. Stop shaking the data and begin using a gentle for loop motion. The data should begin spinning in a circle inside the data.table. Doing this will make most of the noise wash out of the data.table or dissolve. \r\n\r\n4. ????\r\n\r\n4. PROFIT!",
    "139209": "Don't underestimate one of the best pattern recognition systems ever invented. I spent quite some looking at \"barcode\" pictures I created from the data.",
    "139336": "Hi SR,\r\n\r\nThe ways to find magic features are complicated. We sometimes see raw data.<br>\r\nMany kagglers have already found magical features. We can learn about them from Forum."
  },
  "source": "meta"
}