{
  "id": 23301,
  "title": "Discover data with Java Spark",
  "url": "/competitions/bosch-production-line-performance/discussion/23301",
  "author_name": "",
  "post_date": "2016-09-02T19:36:04.173Z",
  "votes": 6,
  "comment_count": 1,
  "views": 554,
  "content": "<p>Hi Kagglers,</p>\n\n<p>Here is a Java Spark program in order to discover Bosch datasets.\n<a href=\"https://github.com/aaudevart/bosch-spark\">https://github.com/aaudevart/bosch-spark</a></p>\n\n<p>The goal of this program is to find the list of all the components, which are manufactured in the same machines (identified by the line and the station).</p>\n\n<p>Steps :</p>\n\n<ul>\n<li>Load Numerical Data</li>\n<li>Find for each component (in the numerical file) the machines list where the component is manufactured</li>\n<li>Load Categorical Data</li>\n<li>Find for each component (in the categorical file) the machines list where the component is manufactured</li>\n<li>Merge the 2 machine lists in order to find the complete machines list where the component is manufactured. And finally, know all the components which share the same machines.</li>\n</ul>\n\n<p>The program runs in Spark local mode .\nThat take about 25min to execute this program on my own computer ( 16Go RAM) .</p>\n\n<p>I hope this program will help some of you !</p>",
  "messages": [
    {
      "id": "133565",
      "postDate": "09/02/2016 19:36:04",
      "content": "<p>Hi Kagglers,</p>\n\n<p>Here is a Java Spark program in order to discover Bosch datasets.\n<a href=\"https://github.com/aaudevart/bosch-spark\">https://github.com/aaudevart/bosch-spark</a></p>\n\n<p>The goal of this program is to find the list of all the components, which are manufactured in the same machines (identified by the line and the station).</p>\n\n<p>Steps :</p>\n\n<ul>\n<li>Load Numerical Data</li>\n<li>Find for each component (in the numerical file) the machines list where the component is manufactured</li>\n<li>Load Categorical Data</li>\n<li>Find for each component (in the categorical file) the machines list where the component is manufactured</li>\n<li>Merge the 2 machine lists in order to find the complete machines list where the component is manufactured. And finally, know all the components which share the same machines.</li>\n</ul>\n\n<p>The program runs in Spark local mode .\nThat take about 25min to execute this program on my own computer ( 16Go RAM) .</p>\n\n<p>I hope this program will help some of you !</p>",
      "rawMarkdown": "Hi Kagglers,\r\n\r\nHere is a Java Spark program in order to discover Bosch datasets.\r\nhttps://github.com/aaudevart/bosch-spark\r\n\r\nThe goal of this program is to find the list of all the components, which are manufactured in the same machines (identified by the line and the station).\r\n\r\nSteps :\r\n\r\n- Load Numerical Data\r\n- Find for each component (in the numerical file) the machines list where the component is manufactured\r\n- Load Categorical Data\r\n- Find for each component (in the categorical file) the machines list where the component is manufactured\r\n- Merge the 2 machine lists in order to find the complete machines list where the component is manufactured. And finally, know all the components which share the same machines.\r\n\r\nThe program runs in Spark local mode .\r\nThat take about 25min to execute this program on my own computer ( 16Go RAM) .\r\n\r\nI hope this program will help some of you !",
      "votes": null
    },
    {
      "id": "153943",
      "postDate": "01/04/2017 06:27:01",
      "content": "<p>Thanks!</p>",
      "rawMarkdown": "Thanks!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 153943,
      "author_name": "dmitriyrudenskiy",
      "author_url": "",
      "post_date": "01/04/2017 06:27:01",
      "content": "<p>Thanks!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "133565": "Hi Kagglers,\r\n\r\nHere is a Java Spark program in order to discover Bosch datasets.\r\nhttps://github.com/aaudevart/bosch-spark\r\n\r\nThe goal of this program is to find the list of all the components, which are manufactured in the same machines (identified by the line and the station).\r\n\r\nSteps :\r\n\r\n- Load Numerical Data\r\n- Find for each component (in the numerical file) the machines list where the component is manufactured\r\n- Load Categorical Data\r\n- Find for each component (in the categorical file) the machines list where the component is manufactured\r\n- Merge the 2 machine lists in order to find the complete machines list where the component is manufactured. And finally, know all the components which share the same machines.\r\n\r\nThe program runs in Spark local mode .\r\nThat take about 25min to execute this program on my own computer ( 16Go RAM) .\r\n\r\nI hope this program will help some of you !",
    "153943": "Thanks!"
  },
  "source": "meta"
}