Hi All,\nThis is to share some features I created.\nSince I am running on a laptop with 8GM ram, data analysis and aggregations on the hist sets was out of the question. Therefore I switched to SQL. I did most of data manipulation and feature engineering with SQL.\nOne of the features I built was \nSum transactions per month for every card_id and then calculate for example\nLN(May_Transactions/Jan_Transactions) where N is the logarithm.
\n\nWhy?
\n\nI observed that there was a different trend between outliers and nonoutliers for sum of monthly transactions and wanted to capture this trend over the year.
\n\nI also did the same thing for each month-year, meaning that in the above there were 12 buckets whereas in the latter case 16 buckets.
\n\nCheers
\n\nP.S \nDoes anybody use Postgresql for competitions?
", "messages": [ { "id": "480922", "postDate": "02/28/2019 21:40:42", "content": "Hi All,\nThis is to share some features I created.\nSince I am running on a laptop with 8GM ram, data analysis and aggregations on the hist sets was out of the question. Therefore I switched to SQL. I did most of data manipulation and feature engineering with SQL.\nOne of the features I built was \nSum transactions per month for every card_id and then calculate for example\nLN(May_Transactions/Jan_Transactions) where N is the logarithm.
\n\nWhy?
\n\nI observed that there was a different trend between outliers and nonoutliers for sum of monthly transactions and wanted to capture this trend over the year.
\n\nI also did the same thing for each month-year, meaning that in the above there were 12 buckets whereas in the latter case 16 buckets.
\n\nCheers
\n\nP.S \nDoes anybody use Postgresql for competitions?
", "rawMarkdown": "Hi All,\nThis is to share some features I created.\nSince I am running on a laptop with 8GM ram, data analysis and aggregations on the hist sets was out of the question. Therefore I switched to SQL. I did most of data manipulation and feature engineering with SQL.\nOne of the features I built was \nSum transactions per month for every card_id and then calculate for example\nLN(May_Transactions/Jan_Transactions) where N is the logarithm.\n\nWhy?\n\nI observed that there was a different trend between outliers and nonoutliers for sum of monthly transactions and wanted to capture this trend over the year.\n\nI also did the same thing for each month-year, meaning that in the above there were 12 buckets whereas in the latter case 16 buckets.\n\nCheers \n\nP.S \nDoes anybody use Postgresql for competitions?", "votes": null }, { "id": "481390", "postDate": "03/01/2019 10:45:08", "content": "...data analysis and aggregations on the hist sets was out of the question with R
", "rawMarkdown": "...data analysis and aggregations on the hist sets was out of the question with R", "votes": null }, { "id": "481401", "postDate": "03/01/2019 11:02:36", "content": "I disagree Anton.\nI use only R in this competition, and successfully calculated hundreds of features and aggregations based on hist, merchantsb etc.
\n\nI have 16 GB ram.
\n\nWith 8 GB ram this indeed could be challenging
", "rawMarkdown": "I disagree Anton.\nI use only R in this competition, and successfully calculated hundreds of features and aggregations based on hist, merchantsb etc. \n\nI have 16 GB ram.\n\nWith 8 GB ram this indeed could be challenging", "votes": null }, { "id": "482315", "postDate": "03/02/2019 17:51:18", "content": "Yes , I was topping 8GB easy in R , I'm building a machine with 32 now
", "rawMarkdown": "Yes , I was topping 8GB easy in R , I'm building a machine with 32 now", "votes": null } ], "comments": [ { "id": 481390, "author_name": "antonspartan", "author_url": "", "post_date": "03/01/2019 10:45:08", "content": "...data analysis and aggregations on the hist sets was out of the question with R
", "votes": null, "replies": [ { "id": 481401, "author_name": "tpthegreat", "author_url": "", "post_date": "03/01/2019 11:02:36", "content": "I disagree Anton.\nI use only R in this competition, and successfully calculated hundreds of features and aggregations based on hist, merchantsb etc.
\n\nI have 16 GB ram.
\n\nWith 8 GB ram this indeed could be challenging
", "votes": null, "replies": [] } ] }, { "id": 482315, "author_name": "antonspartan", "author_url": "", "post_date": "03/02/2019 17:51:18", "content": "Yes , I was topping 8GB easy in R , I'm building a machine with 32 now
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "480922": "Hi All,\nThis is to share some features I created.\nSince I am running on a laptop with 8GM ram, data analysis and aggregations on the hist sets was out of the question. Therefore I switched to SQL. I did most of data manipulation and feature engineering with SQL.\nOne of the features I built was \nSum transactions per month for every card_id and then calculate for example\nLN(May_Transactions/Jan_Transactions) where N is the logarithm.\n\nWhy?\n\nI observed that there was a different trend between outliers and nonoutliers for sum of monthly transactions and wanted to capture this trend over the year.\n\nI also did the same thing for each month-year, meaning that in the above there were 12 buckets whereas in the latter case 16 buckets.\n\nCheers \n\nP.S \nDoes anybody use Postgresql for competitions?", "481390": "...data analysis and aggregations on the hist sets was out of the question with R", "481401": "I disagree Anton.\nI use only R in this competition, and successfully calculated hundreds of features and aggregations based on hist, merchantsb etc. \n\nI have 16 GB ram.\n\nWith 8 GB ram this indeed could be challenging", "482315": "Yes , I was topping 8GB easy in R , I'm building a machine with 32 now" }, "source": "meta" }