{"cells":[
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "A few people have done very nice fourier transformation analysis on the time data, and have convincingly show that the time is in minutes.  Here, my purpose is just to show that, by basic reasoning and simpler analysis, we can still pretty convincingly see the same point.\n\nI present several pieces of evidence here.\n\nA little bit of basic reasoning tells us that the time is most likely minutes.  \n\nLet's estimate how long it took for Facebook to collect this many check-in events:\nA typical large US city has a population of ~1M.  The largest population is found in NY, which is 8.5M (complete list found here: https://en.wikipedia.org/wiki/List_of_United_States_cities_by_population).  Let's way, we are looking at a large city with 3M people (slightly larger than Chicago), and Facebook has collected this 30M check-ins.  Let us further suppose, that through out the time, on an average day, about 1% of the population used check-in once.  This will result in 0.03M check-ins per day.  \n\nTo get 30M check-ins, it takes 1000 days.  our assumptions may not be accurate, so let's give it a range, say it takes ~500 to 2000 days to collect this many check-ins.  Now, look at the 'time' column in the data.  This ranges from 1 to 800,000. So each unit is roughly 1/800 day (assuming 1000 days), or 1/1600 to 1/400 days.  We notice that 1 min is exactly 1/1440 day, sitting nicely within the range of our estimation.  \n\nIf the unit is sec, we will have to change our assumptions drastically to make it fit, so drastic that it becomes unrealistic.  So, most likely, the time unit is min, unless Facebook is using an arbitrary unit here."
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "Now, to confirm, let us do a little bit of simple tests."
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "import pandas as pd\nfrom matplotlib import pyplot as plt\nfrom matplotlib import cm as cm\nimport numpy as np\n%matplotlib inline\n\ntrain=pd.read_csv('../input/train.csv')\ntrain.describe()"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# first take a look at the whole picture of time data:\n# train['time'].plot(kind='hist', bins=500)\ntrain['timeBin']=(train['time'].divide((786239.0/5000))).apply(int)\ntimeHist=pd.value_counts(train['timeBin'])\ntimeHist.sort_index(inplace=True, ascending=True)\ntimeHist.plot()"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "Notice nice little peaks on top of the picture show a periodic component of ~10000, very close to the number of minutes in a week: 10080\n\n\nNow, let us take a look at what a compound week data look like. This will require us to assume either the time is in minutes or seconds, and the resulting graphs will serve as a test.\n\nAnalysis of the time values.\n\nHypothesis 1: time in sec\n\nHypothesis 2: time in min\n\nSimple method: For each hypothesis, compute for each time data point to get its corresponding hour in a week. For each of the 168 hours in a week, get a total count of check-ins during that hour. Plot the compound hourly activity in the week range. The correct hypothsis should:\n\n    exhibit some degree of 24 hour cycle\n    should show some degree of weekend-weekday variation.\n    because weeks are periodic, the start activity and the end activity in the correct week plot should be somewhat close to each other\n\n"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# Hypothesis 1, time in sec:\nhourly=(train['time'].divide(3600)).apply(int)\npd.Series([len(hourly[(hourly%168)==i]) for i in range(168)]).plot()"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "This has no 24 hour cycle for sure.\n\nThere might be weekend-weekday variation\n\nThe data at starting and ending of the week are far apart.\n\nDoes not look very likely.\n"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# Hypothesis 2, time in min:\nhourly=(train['time'].divide(60)).apply(int)\npd.Series([len(hourly[(hourly%168)==i]) for i in range(168)]).plot()"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "There might be 24 hour cycle component, but not clear.\n\nThere might be weekend-weekday variation.\n\nThe data at starting and ending of the week are very close to each other.\n\nThis is more likely to be, although not very convincing.\n\nBelow I randomly picked some hourly numbers to serve as controls, to get a sense of what random graphs look like. Most of them look like Hypothesis 1 (seconds). This means the graphs produced by Hypothesis 1 is not distinguishable from random graphs.\n\nThus, this serves as an additional support for Hypothesis 2 (minutes).\n"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# random control 1:\nhourly=(train['time'].divide(1485)).apply(int)\npd.Series([len(hourly[(hourly%168)==i]) for i in range(168)]).plot()"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# random control 2:\nhourly=(train['time'].divide(18)).apply(int)\npd.Series([len(hourly[(hourly%168)==i]) for i in range(168)]).plot()"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# random control 3:\nhourly=(train['time'].divide(83)).apply(int)\npd.Series([len(hourly[(hourly%168)==i]) for i in range(168)]).plot()"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# random control 4:\nhourly=(train['time'].divide(634)).apply(int)\npd.Series([len(hourly[(hourly%168)==i]) for i in range(168)]).plot()"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# random control 5:\nhourly=(train['time'].divide(20059)).apply(int)\npd.Series([len(hourly[(hourly%168)==i]) for i in range(168)]).plot()"
 }
],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"}}, "nbformat": 4, "nbformat_minor": 0}