{"cells":[
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "### Validaing seasonality using Autocorrelation Function (ACF)"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "This script provides an easy way to valdiate whether \"time\" feature in the dataset represents minute.\nThe logic of ACF is simple: to see the correlation of a time series between its values at time t and time t-x.\nFor example, we expect to see a quarterly series is highly correlated to 4 periods ago of itself.\nACF is normally used to test stationary in a time series or determine coefficients in ARIMA time series model,\nbut it is also extremely useful to determine/validate the seasonal period.\n\nThank you for the people who share their insights about \"time\" is minute,\nso I can validate it qucikly using ACF without further processing and testing."
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "import pandas as pd\nimport numpy as np\nimport seaborn as sns\nfrom statsmodels.tsa.stattools import acf"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "df_train = pd.read_csv(\"../input/train.csv\", dtype={\"place_id\": object})\n\n# assume time is minute\ndf_train['hour'] = df_train.time // 60\ndf_train['day'] = df_train.time // (60*24)\n\n# use the place id with the most checkins as an example\ncheckins_by_place = df_train.place_id.value_counts()\ndf_sample_place = df_train[df_train.place_id == checkins_by_place.index[0]]"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# this function plots the ACF when you input frequency series with time index\ndef check_acf(counts_by_time):\n\n    # fill in the gap with 0 to create a series with fixed intervals\n    time_index = np.arange(\n        counts_by_time.index.min(),\n        counts_by_time.index.max() + 1)\n    count_by_time_filled = counts_by_time.reindex(time_index)\n    count_by_time_filled.fillna(0, inplace=True)\n\n    # ACF\n    acf_raw = acf(count_by_time_filled)\n    \n    # plot\n    sns.barplot(x=np.arange(0, acf_raw.size), y=acf_raw)"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "### Check seasonality of day if we assume feature \"time\" is minute"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# plot the ACF of \"day\"\ncheck_acf(df_sample_place.day.value_counts())"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "In the graph above, y is correlation, and peaks can be observed when x equals 7 and its multiples.\nThat means the \"day\" series at time t is higly correlated with its values in 7 periods ago.\nThis shows that \"day\" has a weekly seasonal period of 7,\nso our assumption where \"time\" is minute is probably correct.\n\nThe correlation is 1 when x=0 since the values at time t is perfectly correlated with itself at t-0.\nThe correlation decays when x increases because trend exists in this time series,\nso the older values you used to compare with the base values, the less correlated they are.\nSometimes the effect of trend may affect the seasonality heavily so that you cannot visualize the seasonal peaks well,\nin those cases, you may want to do differencing of the series to remove trending effect before plotting ACF,\nthen a clear seasonal pattern can be observed.\n\n\n\n"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "### Check seasonality of hour if we assume feature \"time\" is minute"
 },
 {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {
   "collapsed": false
  },
  "outputs": [],
  "source": "# plot the ACF of hour\ncheck_acf(df_sample_place.hour.value_counts())"
 },
 {
  "cell_type": "markdown",
  "metadata": {},
  "source": "Similarly, y is correlation, and peaks can be observed when x = 24.\nThat means the \"hour\" series at time t is higly correlated with its values in 24 periods ago.\nThis shows that \"hour\" has a hourly seasonal period of 24,\nso our assumption where \"time\" is minute is probably correct."
 }
],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"}}, "nbformat": 4, "nbformat_minor": 0}