In this kernel you going to learn about how to guide a data analysis based on CRISP-DM methodology. We provided question of each of phases and we applied to elo competition. We use R language to develop all analysis.
\n\nYou will learn in this kernel:
\nAll of us know that one of the most important parts of a project are the hipothesis that business expert have\naround some interest problem, topic or area. In the same way, the questions are extremely important, because those give us different ways to do an analysis. We 'll see a list of typical questions that we define when start an data analysis.
\n\nLet me introduce CRISP-DM methodoloy, one of the most famous ways to develop analytical projects. We going to use this methodology to remark some of the possible questions that we can propose to start a data analysis.
\n\nThe following are the ciclycal phases of CRISP-DM:
\n\n* Train.csv/Test.csv:
We have six variables: card_id, first_active_month, feature_1, feature_2, feature_3, target. \n* Historical_transactions.csv/new_merchant_transactions.csv:
We have fourteen variables: card_id, month_lag, purchase_date, authorized_flag, category_3, installments, category_1, merchant_category_id, subsector_id, merchant_id, purchase_amount, city_id, state_id, category_2. \n* merchant.csv:
We have twenty two variables; merchant_id, merchant_group_id, merchant_category_id, subsector_id, numerical_1, numerical_2, category_1, most_recent_sales_range, most_recent_purchases_range, avg_sales_lag3, avg_purchases_lag3, active_months_lag3, avg_sales_lag6, avg_purchases_lag6, active_months_lag6, avg_sales_lag12, avg_purchases_lag12, active_months_lag12, cateogy_4, city_id, state_id, category_2.\n\n\n\n\n\n"},{"metadata":{"_uuid":"a863af90620406a6d1ee1c172d7341133a5718a3"},"cell_type":"markdown","source":" * What kind of variables do we have? "},{"metadata":{"trusted":true,"_uuid":"113bc055fe361e283d163f8a3ce8a364c799ccf9"},"cell_type":"code","source":"# Before start with the variable analysis we need to load libraries\nlibrary(dplyr,suppressMessages);library(tidyr,suppressMessages);library(tidyverse,suppressMessages)\nlibrary(ggplot2,suppressMessages);library(caret,suppressMessages)\nsuppressMessages(library(readr))\nsuppressMessages(library(psych))\nsuppressMessages(library(e1071))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"417918024141978f502a9ec8a4f2a592e8ef7edf"},"cell_type":"code","source":"# We load files to analysis\nlist.files(path = \"../input\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"19a45fe69d56b63934c2fc69dc96ab372cef8352"},"cell_type":"markdown","source":"We are going to load and analyze .csv files "},{"metadata":{"trusted":true,"_uuid":"5baabf2738bff264c461adfb83c757c194d30082"},"cell_type":"code","source":"#Uploading .csv files\nhist_trxns <- read_csv(\"../input/historical_transactions.csv\")\nmerch <- read_csv(\"../input/merchants.csv\")\nnew_merch_trxns <- read_csv(\"../input/new_merchant_transactions.csv\")\ntest <- read_csv(\"../input/test.csv\")\ntrain <- read_csv(\"../input/train.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"dc58291c4fdec5267e5a1189e6aa82f671349981"},"cell_type":"markdown","source":"As we can see we have seven files to develop the algorithms. Now we can see the kind of variables. \n\n * How many registers do we have? "},{"metadata":{"trusted":true,"_uuid":"c278ecfb9e357fb3f9165c6e935fe30ed9c3678c"},"cell_type":"code","source":"# Looking the dimension of each structure\ncat(\"Dimensions of hist_trxns file is: \",\"\\n\");dim(hist_trxns)\ncat(\"Dimensions of merch file is: \",\"\\n\");dim(merch)\ncat(\"Dimensions of new_merch_trxns file is: \",\"\\n\");dim(new_merch_trxns)\ncat(\"Dimensions of test file is: \",\"\\n\");dim(test)\ncat(\"Dimensions of train file is: \",\"\\n\");dim(train)\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1f057afba0debef8f270e6752ef48c548f5ded11"},"cell_type":"markdown","source":"Aditionally we want to know how csv look like. Let me use head() and tail() function to learn more about files."},{"metadata":{"trusted":true,"_uuid":"989091e607a62bdf3ebff318d6296816c0ed261a"},"cell_type":"code","source":"# Head and tail for each file\ncat(\"First five registers of historical transactional dataset\");head(hist_trxns, n = 5)\ncat(\"First five registers of merchant dataset\");head(merch,n = 5)\ncat(\"First five registers of new merchant transactions dataset\");head(new_merch_trxns, n = 5)\ncat(\"First five registers of test dataset\");head(test, n = 5)\ncat(\"First five registers of train dataset\");head(train, n = 5)\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8baebf4546b0703c50ba246e79cefadb25a4b48d"},"cell_type":"markdown","source":"