{"cells":[
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# This R environment comes with all of CRAN preinstalled, as well as many other helpful packages\n# The environment is defined by the kaggle/rstats docker image: https://github.com/kaggle/docker-rstats\n# For example, here's several helpful packages to load in \n\nlibrary(ggplot2) # Data visualization\nlibrary(readr) # CSV file I/O, e.g. the read_csv function\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nsystem(\"ls ../input\")\n\n# Any results you write to the current directory are saved as output."
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# Load packages\nlibrary('data.table')\n\n#library('devtools')\n#install_github('rBatt/rbLib')\n#library('rbLib')\n\nprint(sessionInfo())\n"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "# Read in and Print Data\nUsing data.table b/c it's fast and I <3 it."
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# Read and print data set\ntest <- fread(\"../input/test.csv\")\nprint(test)"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "# Basic Visualization\nScatter plots of pairs of columns, and histograms.\n\nSubsetting to 10,000 random rows so it's a bit faster to plot"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# lapply(test, summary)\npanel.hist <- function(x, ...){ # ~copied from ?pairs\n    usr <- par(\"usr\"); on.exit(par(usr))\n    par(usr = c(usr[1:2], 0, 1.5) )\n    h <- hist(x, plot = FALSE)\n    breaks <- h$breaks; nB <- length(breaks)\n    y <- h$counts; y <- y/max(y)\n    r2 <- function(..., col)rect(breaks[-nB], 0, breaks[-1], y, col = \"cyan\", ...)\n    #rect(breaks[-nB], 0, breaks[-1], y, col = \"cyan\", ...)\n    r2()\n}\npairs(test[sample(1:nrow(test), 1E4),list(x, y, accuracy, time)], panel=panel.smooth, diag.panel=panel.hist, col=adjustcolor('black', alpha=0.01))\n"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "# Features of time stamps\nDefinitely a couple gotchas revealed in the following.\n\n1. not sorted by time\n2. most of the time stamps are duplicates\n3. the spacing between (unique) time stamps is mostly regular, with one exception\n4. there appears to be a large amount of temporal pattern in activity (cycles, trend)\n    \n  "
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "test[,summary(diff(sort(time)))]\ncat(\"\\n\\n\")\n\ntest[,data.table(percent_unique=length(unique(time))/nrow(test), n_unique=length(unique(time)))]\ncat(\"\\n\\n\")\n\ntest[,j={unique(diff(sort(unique(time))))}]\ncat(\"\\n\\n\")\n\n\ntest[,j={table(diff(sort(unique(time))))}]\ncat(\"\\n\\n\")\n\ntime_tbl <- test[,table(time)]\ntbl_x <- as.integer(names(time_tbl))"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "par(mar=c(5,4,4,4))\nplot(tbl_x, time_tbl, type=\"l\", xlab=\"time\", ylab=\"# observations\", col=adjustcolor('black', 0.25))\npar(new=TRUE)\n\ntest[,plot(density(time, from=min(tbl_x), to=max(tbl_x)), col=\"red\", xaxt=\"n\", yaxt=\"n\", xlab=\"\",ylab=\"\", main=\"\")]\naxis(side=4, col.ticks=\"red\", col=\"red\")\nmtext(\"density\", side=4, line=3)"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "test_n_time <- test[,.N, by=\"time\"]\ntest_n_time[,summary(diff(time))]\ntest_n_time[,plot(.SD, type=\"l\")]"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": ""
 }
],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"}}, "nbformat": 4, "nbformat_minor": 0}