{
  "id": 42993,
  "title": "Missing Values for onpromotion (RevoScaleR)",
  "url": "/competitions/favorita-grocery-sales-forecasting/discussion/42993",
  "author_name": "SrikanthPotukuchi",
  "post_date": "2017-11-08T10:30:19.418000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Hi all,</p>\n\n<p>I'm using Microsoft R Server(RevoScaleR package) for this competition. I used rxImport function to convert csv file to xdf format but onpromotion values are NA. I did check tail and even after date 2014-04-01, it's still NA.</p>\n\n<p>Edit: I solved it! I had to use rowsToSniff option in RxTextData.</p>\n\n<p>My code:\n&gt; rxImport(inData = inputFile,outFile = outputFile2, numRows =30000000,\n+          reportProgress = 1,rowsPerRead = 5000000,type = \"text\",\n+          overwrite=TRUE)</p>\n\n<p>Rows Processed: 5000000\nRows Processed: 10000000\nRows Processed: 15000000\nRows Processed: 20000000\nRows Processed: 25000000\nRows Processed: 30000000 \n&gt; rxGetInfo(outputFile2,getVarInfo = TRUE)\nFile name: D:\\Kaggle\\CF Grocery Sales Forecasting\\train2.xdf \nNumber of observations: 3e+07 \nNumber of variables: 6 \nNumber of blocks: 6 \nCompression type: zlib \nVariable information: \nVar 1: id, Type: integer, Low/High: (0, 29999999)\nVar 2: date, Type: Date, Low/High: (2013-01-01, 2014-09-01)\nVar 3: store_nbr, Type: integer, Low/High: (1, 54)\nVar 4: item_nbr, Type: integer, Low/High: (96995, 1909755)\nVar 5: unit_sales, Type: numeric, Storage: float32, Low/High: (-1451.0000, 12021.0000)\nVar 6: onpromotion, Type: numeric, Low/High: (NA, NA)</p>\n\n<p>&gt; tail(dataSource2,n=10)\n               id       date store_nbr item_nbr unit_sales onpromotion\n29999991 29999990 2014-09-01         7  1473476     30.860        </p>",
  "messages": [
    {
      "id": 241221,
      "postDate": "2017-11-08T10:30:19.420Z",
      "content": "<p>Hi all,</p>\n\n<p>I'm using Microsoft R Server(RevoScaleR package) for this competition. I used rxImport function to convert csv file to xdf format but onpromotion values are NA. I did check tail and even after date 2014-04-01, it's still NA.</p>\n\n<p>Edit: I solved it! I had to use rowsToSniff option in RxTextData.</p>\n\n<p>My code:\n&gt; rxImport(inData = inputFile,outFile = outputFile2, numRows =30000000,\n+          reportProgress = 1,rowsPerRead = 5000000,type = \"text\",\n+          overwrite=TRUE)</p>\n\n<p>Rows Processed: 5000000\nRows Processed: 10000000\nRows Processed: 15000000\nRows Processed: 20000000\nRows Processed: 25000000\nRows Processed: 30000000 \n&gt; rxGetInfo(outputFile2,getVarInfo = TRUE)\nFile name: D:\\Kaggle\\CF Grocery Sales Forecasting\\train2.xdf \nNumber of observations: 3e+07 \nNumber of variables: 6 \nNumber of blocks: 6 \nCompression type: zlib \nVariable information: \nVar 1: id, Type: integer, Low/High: (0, 29999999)\nVar 2: date, Type: Date, Low/High: (2013-01-01, 2014-09-01)\nVar 3: store_nbr, Type: integer, Low/High: (1, 54)\nVar 4: item_nbr, Type: integer, Low/High: (96995, 1909755)\nVar 5: unit_sales, Type: numeric, Storage: float32, Low/High: (-1451.0000, 12021.0000)\nVar 6: onpromotion, Type: numeric, Low/High: (NA, NA)</p>\n\n<p>&gt; tail(dataSource2,n=10)\n               id       date store_nbr item_nbr unit_sales onpromotion\n29999991 29999990 2014-09-01         7  1473476     30.860        </p>",
      "rawMarkdown": "Hi all,\n\nI'm using Microsoft R Server(RevoScaleR package) for this competition. I used rxImport function to convert csv file to xdf format but onpromotion values are NA. I did check tail and even after date 2014-04-01, it's still NA.\n\nEdit: I solved it! I had to use rowsToSniff option in RxTextData.\n\nMy code:\n&gt; rxImport(inData = inputFile,outFile = outputFile2, numRows =30000000,\n+          reportProgress = 1,rowsPerRead = 5000000,type = \"text\",\n+          overwrite=TRUE)\n\nRows Processed: 5000000\nRows Processed: 10000000\nRows Processed: 15000000\nRows Processed: 20000000\nRows Processed: 25000000\nRows Processed: 30000000 \n&gt; rxGetInfo(outputFile2,getVarInfo = TRUE)\nFile name: D:\\Kaggle\\CF Grocery Sales Forecasting\\train2.xdf \nNumber of observations: 3e+07 \nNumber of variables: 6 \nNumber of blocks: 6 \nCompression type: zlib \nVariable information: \nVar 1: id, Type: integer, Low/High: (0, 29999999)\nVar 2: date, Type: Date, Low/High: (2013-01-01, 2014-09-01)\nVar 3: store_nbr, Type: integer, Low/High: (1, 54)\nVar 4: item_nbr, Type: integer, Low/High: (96995, 1909755)\nVar 5: unit_sales, Type: numeric, Storage: float32, Low/High: (-1451.0000, 12021.0000)\nVar 6: onpromotion, Type: numeric, Low/High: (NA, NA)\n\n&gt; tail(dataSource2,n=10)\n               id       date store_nbr item_nbr unit_sales onpromotion\n29999991 29999990 2014-09-01         7  1473476     30.860        "
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "241221": "Hi all,\n\nI'm using Microsoft R Server(RevoScaleR package) for this competition. I used rxImport function to convert csv file to xdf format but onpromotion values are NA. I did check tail and even after date 2014-04-01, it's still NA.\n\nEdit: I solved it! I had to use rowsToSniff option in RxTextData.\n\nMy code:\n&gt; rxImport(inData = inputFile,outFile = outputFile2, numRows =30000000,\n+          reportProgress = 1,rowsPerRead = 5000000,type = \"text\",\n+          overwrite=TRUE)\n\nRows Processed: 5000000\nRows Processed: 10000000\nRows Processed: 15000000\nRows Processed: 20000000\nRows Processed: 25000000\nRows Processed: 30000000 \n&gt; rxGetInfo(outputFile2,getVarInfo = TRUE)\nFile name: D:\\Kaggle\\CF Grocery Sales Forecasting\\train2.xdf \nNumber of observations: 3e+07 \nNumber of variables: 6 \nNumber of blocks: 6 \nCompression type: zlib \nVariable information: \nVar 1: id, Type: integer, Low/High: (0, 29999999)\nVar 2: date, Type: Date, Low/High: (2013-01-01, 2014-09-01)\nVar 3: store_nbr, Type: integer, Low/High: (1, 54)\nVar 4: item_nbr, Type: integer, Low/High: (96995, 1909755)\nVar 5: unit_sales, Type: numeric, Storage: float32, Low/High: (-1451.0000, 12021.0000)\nVar 6: onpromotion, Type: numeric, Low/High: (NA, NA)\n\n&gt; tail(dataSource2,n=10)\n               id       date store_nbr item_nbr unit_sales onpromotion\n29999991 29999990 2014-09-01         7  1473476     30.860        "
  }
}