{
  "id": 24496,
  "title": "Segments of production parts with biclustering?",
  "url": "/competitions/bosch-production-line-performance/discussion/24496",
  "author_name": "Jüri Kuusik",
  "post_date": "2016-10-17T08:20:59.390000",
  "votes": 3,
  "comment_count": 0,
  "views": 413,
  "content": "<p>Hi.\nAs You know there is no explicit information regarding to segments of products then biclustering would help to do dimension reduction ?! The point is to use as complete data as possible for each feature in each bicluster.\nI used R package iBBig for that using the whole training data set of numeric features.</p>\n\n<p>R code used:</p>\n\n<p>library(iBBiG)\nlibrary(data.table)\nlibrary(stringr)</p>\n\n<p>dt_train_num &lt;- fread(input = '/home/ruser/Kaggle/Bosch/Data/train_numeric.csv'\n                                       ,showProgress = TRUE\n                                       ,data.table = TRUE\n)</p>\n\n<p>vec_names_of_all_features &lt;- names(dt_train_num)[str_detect(names(dt_train_num),&quot;L\\d+_S\\d+_F\\d+&quot;)]</p>\n\n<h1>Creating 0/1 data frame indicating 1 - non NA value; 0  - NA value</h1>\n\n<p>dt_train_num &lt;- dt_train_num[,vec_names_of_all_features,with=F]</p>\n\n<p>dt_train_num[!is.na(dt_train_num)] &lt;- 1\ndt_train_num[is.na(dt_train_num)] &lt;- 0</p>\n\n<p>dt_train_num &lt;- as.matrix(dt_train_num)\nstr(dt_train_num)</p>\n\n<h1>Using the whole data set requires more RAM and patient  ;)</h1>\n\n<h1>One could use sampling</h1>\n\n<p>iBBiG_train_num &lt;- iBBiG(dt_train_num, nModules=6)\nsetwd(&quot;~/Kaggle/Bosch/Models/Biclusters&quot;)\nsave(iBBiG_train_num,file='dt_train_num_iBBiG.RData')</p>\n\n<p>pdf('Biclusters_train_num.pdf',width=20,height=10)\nplot(iBBiG_train_num)\ndev.off()</p>\n\n<p>str(iBBiG_train_num)</p>\n\n<h1>Number of biclusters</h1>\n\n<p>Number(iBBiG_train_num)</p>\n\n<h1>Which rows belong to which biclusters ?</h1>\n\n<p>RowxNumber(iBBiG_train_num)[1:2,]</p>\n\n<h1>Which features belong to which clusters ?</h1>\n\n<p>NumberxCol(iBBiG_train_num)[,1:10]</p>\n\n<h1>The same has been done for test numeric data set...</h1>",
  "messages": [
    {
      "id": 139909,
      "postDate": "2016-10-17T08:20:59.390Z",
      "content": "<p>Hi.\nAs You know there is no explicit information regarding to segments of products then biclustering would help to do dimension reduction ?! The point is to use as complete data as possible for each feature in each bicluster.\nI used R package iBBig for that using the whole training data set of numeric features.</p>\n\n<p>R code used:</p>\n\n<p>library(iBBiG)\nlibrary(data.table)\nlibrary(stringr)</p>\n\n<p>dt_train_num &lt;- fread(input = '/home/ruser/Kaggle/Bosch/Data/train_numeric.csv'\n                                       ,showProgress = TRUE\n                                       ,data.table = TRUE\n)</p>\n\n<p>vec_names_of_all_features &lt;- names(dt_train_num)[str_detect(names(dt_train_num),&quot;L\\d+_S\\d+_F\\d+&quot;)]</p>\n\n<h1>Creating 0/1 data frame indicating 1 - non NA value; 0  - NA value</h1>\n\n<p>dt_train_num &lt;- dt_train_num[,vec_names_of_all_features,with=F]</p>\n\n<p>dt_train_num[!is.na(dt_train_num)] &lt;- 1\ndt_train_num[is.na(dt_train_num)] &lt;- 0</p>\n\n<p>dt_train_num &lt;- as.matrix(dt_train_num)\nstr(dt_train_num)</p>\n\n<h1>Using the whole data set requires more RAM and patient  ;)</h1>\n\n<h1>One could use sampling</h1>\n\n<p>iBBiG_train_num &lt;- iBBiG(dt_train_num, nModules=6)\nsetwd(&quot;~/Kaggle/Bosch/Models/Biclusters&quot;)\nsave(iBBiG_train_num,file='dt_train_num_iBBiG.RData')</p>\n\n<p>pdf('Biclusters_train_num.pdf',width=20,height=10)\nplot(iBBiG_train_num)\ndev.off()</p>\n\n<p>str(iBBiG_train_num)</p>\n\n<h1>Number of biclusters</h1>\n\n<p>Number(iBBiG_train_num)</p>\n\n<h1>Which rows belong to which biclusters ?</h1>\n\n<p>RowxNumber(iBBiG_train_num)[1:2,]</p>\n\n<h1>Which features belong to which clusters ?</h1>\n\n<p>NumberxCol(iBBiG_train_num)[,1:10]</p>\n\n<h1>The same has been done for test numeric data set...</h1>",
      "rawMarkdown": "Hi.\r\nAs You know there is no explicit information regarding to segments of products then biclustering would help to do dimension reduction ?! The point is to use as complete data as possible for each feature in each bicluster.\r\nI used R package iBBig for that using the whole training data set of numeric features.\r\n\r\nR code used:\r\n\r\nlibrary(iBBiG)\r\nlibrary(data.table)\r\nlibrary(stringr)\r\n\r\ndt_train_num <- fread(input = '/home/ruser/Kaggle/Bosch/Data/train_numeric.csv'\r\n                                       ,showProgress = TRUE\r\n                                       ,data.table = TRUE\r\n)\r\n\r\nvec_names_of_all_features <- names(dt_train_num)[str_detect(names(dt_train_num),\"L\\\\d+_S\\\\d+_F\\\\d+\")]\r\n\r\n# Creating 0/1 data frame indicating 1 - non NA value; 0  - NA value\r\ndt_train_num <- dt_train_num[,vec_names_of_all_features,with=F]\r\n\r\ndt_train_num[!is.na(dt_train_num)] <- 1\r\ndt_train_num[is.na(dt_train_num)] <- 0\r\n\r\ndt_train_num <- as.matrix(dt_train_num)\r\nstr(dt_train_num)\r\n\r\n\r\n# Using the whole data set requires more RAM and patient  ;)\r\n# One could use sampling\r\niBBiG_train_num <- iBBiG(dt_train_num, nModules=6)\r\nsetwd(\"~/Kaggle/Bosch/Models/Biclusters\")\r\nsave(iBBiG_train_num,file='dt_train_num_iBBiG.RData')\r\n\r\npdf('Biclusters_train_num.pdf',width=20,height=10)\r\nplot(iBBiG_train_num)\r\ndev.off()\r\n\r\nstr(iBBiG_train_num)\r\n# Number of biclusters\r\nNumber(iBBiG_train_num)\r\n# Which rows belong to which biclusters ?\r\nRowxNumber(iBBiG_train_num)[1:2,]\r\n# Which features belong to which clusters ?\r\nNumberxCol(iBBiG_train_num)[,1:10]\r\n\r\n# The same has been done for test numeric data set...\r\n",
      "votes": 3
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "139909": "Hi.\r\nAs You know there is no explicit information regarding to segments of products then biclustering would help to do dimension reduction ?! The point is to use as complete data as possible for each feature in each bicluster.\r\nI used R package iBBig for that using the whole training data set of numeric features.\r\n\r\nR code used:\r\n\r\nlibrary(iBBiG)\r\nlibrary(data.table)\r\nlibrary(stringr)\r\n\r\ndt_train_num <- fread(input = '/home/ruser/Kaggle/Bosch/Data/train_numeric.csv'\r\n                                       ,showProgress = TRUE\r\n                                       ,data.table = TRUE\r\n)\r\n\r\nvec_names_of_all_features <- names(dt_train_num)[str_detect(names(dt_train_num),\"L\\\\d+_S\\\\d+_F\\\\d+\")]\r\n\r\n# Creating 0/1 data frame indicating 1 - non NA value; 0  - NA value\r\ndt_train_num <- dt_train_num[,vec_names_of_all_features,with=F]\r\n\r\ndt_train_num[!is.na(dt_train_num)] <- 1\r\ndt_train_num[is.na(dt_train_num)] <- 0\r\n\r\ndt_train_num <- as.matrix(dt_train_num)\r\nstr(dt_train_num)\r\n\r\n\r\n# Using the whole data set requires more RAM and patient  ;)\r\n# One could use sampling\r\niBBiG_train_num <- iBBiG(dt_train_num, nModules=6)\r\nsetwd(\"~/Kaggle/Bosch/Models/Biclusters\")\r\nsave(iBBiG_train_num,file='dt_train_num_iBBiG.RData')\r\n\r\npdf('Biclusters_train_num.pdf',width=20,height=10)\r\nplot(iBBiG_train_num)\r\ndev.off()\r\n\r\nstr(iBBiG_train_num)\r\n# Number of biclusters\r\nNumber(iBBiG_train_num)\r\n# Which rows belong to which biclusters ?\r\nRowxNumber(iBBiG_train_num)[1:2,]\r\n# Which features belong to which clusters ?\r\nNumberxCol(iBBiG_train_num)[,1:10]\r\n\r\n# The same has been done for test numeric data set...\r\n"
  }
}