{"cells":[
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "suppressPackageStartupMessages(library('data.table'))\nsuppressPackageStartupMessages(library('bit64'))\nsuppressMessages(train_full <- fread(\"../input/train.csv\"))\n\n# Make groups\n# doign train_full <- just to suppress printing of the data.table (grr)\ntrain_full <- train_full[,c(\"rg_x\",\"rg_y\"):=list(cut(x,14), cut(y,100))]\ntrain_full <- train_full[,c(\"xy_group\"):=list(paste(rg_x, rg_y))]\ntrain_full <- train_full[,c(\"rg_x\",\"rg_y\"):=NULL]\n\nhead(train_full)\n\n"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "I just split the data into ~1400 groups, that each contains <1000 `place_id's`"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": " # just a conventient function I use; in rbLib package: https://github.com/rBatt/rbLib\nzCol <- function(nCols, Z){\n\tcols <- colorRampPalette(c(\"#000099\", \"#00FEFF\", \"#45FE4F\", \"#FCFF00\", \"#FF9400\", \"#FF3100\"))(nCols)\n\tcolVec_ind <- cut(Z, breaks=nCols)\n\tcolVec <- cols[colVec_ind]\n}\n\n# Makes a cool plot similar to what's on the data description page\n# multiple panels just to see a few places\n# note that at this scaling, you can see the usefulness of x \nxy_loops <- train_full[,sample(unique(xy_group), 6)]\npar(mfrow=c(3,2), mar=c(1,1,0.5,0.5), mgp=c(1,0.1,0), tcl=-0.1, ps=9, cex=1)\nfor(r in 1:6){\n\ttrain_full[xy_group==xy_loops[[r]],plot(x,y, col=adjustcolor(zCol(256, as.integer(place_id)), 0.25), cex=0.5, pch=20)]\n}"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "# Why I use this\n1. The visualization is manageable, but still zoomed-in (doesn't happen if you select random ID's)\n2. Can replace color coding with any other variable (continuous or `as.integer(variable)`) to find patterns\n3. There are less than 1000 `place_id`'s per `xy_group`, which might be easier on classification models\n\nBecause the groups are defined w/o needing to know any information about the `place_id` you can group the test data like this before analysis. \n\nAnalyzing 100k+ classes it tough, but 1k might be manageable (even if you have to do it 1.4k times).\n\nNote: it might be important to `rbind()` the test data set to the training data set before defining groups.\n\nI'm interested in hearing any thoughts on the approach of splitting up the data into subsets before applying others models."
 }
],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"}}, "nbformat": 4, "nbformat_minor": 0}