{"cells":[
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport matplotlib.pyplot as plt\n\ndf_train = pd.read_csv('../input/train.csv')\ndf_test = pd.read_csv('../input/test.csv')\n\n# Any results you write to the current directory are saved as output."
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "df_train.head()\n"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "JSab and Michael Campos provided very detailed analysis on the time data.\n\nHere I would like to share my method, and I think it confirms their conclusion that the unit of the time is minute in this dataset.\n\nThe method I used to figure out the time definition is through Fourier transform."
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# Compute the histogram of the event time\ntime = df_train['time']\nhist = hist = np.histogram(time,5000)\n\n# To know the unit definition of Time\n# we can look into the frequency structure of the histogram\nhist_fft = np.absolute(np.fft.fft(hist[0]))\nplt.plot(hist_fft)\nplt.xlim([0,2500])\nplt.ylim([0,1e6])\nplt.title('FFT of event time histogram')\nplt.xlabel('1/T')\nplt.grid(True)\nplt.show()"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "mu, sigma = 100, 15\nx = mu + sigma*np.random.randn(10000)\n\n#plt.plot(hist)\n\n"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "The peaks in the FFT curve indicate strong periodic structure at that frequency.\nLet's zoom-in the see the numbers."
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "plt.plot(hist_fft)\nplt.xlim([0,250])\nplt.ylim([0,1e6])\nplt.title('FFT of event time histogram')\nplt.xlabel('1/T')\nplt.grid(True)\nplt.show()"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "The first peak (fundamental frequency) is at 78\nwhich means the time histogram has a period of:"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "print(time.max()/78)"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "The overall event histogram averages over all place_id, so I look at one of the most popular place_id to find more information."
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "time = df_train[df_train['place_id']==8772469670]['time']\nhist = np.histogram(time,5000)\nhist_fft = np.absolute(np.fft.fft(hist[0]))\n\nplt.plot(hist_fft)\nplt.xlim([0,2500])\nplt.title('FFT of event time histogram')\nplt.xlabel('1/T')\nplt.grid(True)\nplt.show()"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "Small peak at 64 and large peak at 451, they conrespond to periods:"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "T1 = time.max()/64\nT2 = time.max()/451\nprint('period T1:', T1)\nprint('period T2:', T2)"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "T1 is the same period as the one found in overall histogram.\n\nBut the interesting part is that the ratio of T1 and T2 happen to be about 7, so they are very likely week and day.\n\nAnd 1440 is the minute time for a day."
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# Another place_id for confirmation\ntime = df_train[df_train['place_id']==4823777529]['time']\nhist = np.histogram(time,5000)\nhist_fft = np.absolute(np.fft.fft(hist[0]))\n\nplt.plot(hist_fft)\nplt.xlim([0,2500])\nplt.title('FFT of event time histogram')\nplt.xlabel('1/T')\nplt.grid(True)\nplt.show()"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# peaks at 77 and 539, same periods: 10080 and 1440\nT1 = time.max()/77\nT2 = time.max()/539\nprint('period T1:', T1)\nprint('period T2:', T2)"
 }
],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"}}, "nbformat": 4, "nbformat_minor": 0}