![\"\"](\"http://youdao.pythonplus.cn/answer20181213.PNG\")
Welcome to Elo Merchant recommendation competition!
\n\nWe are really enthusiastic to host this competition and we believe kagglers are going to join it and help us make a difference in our customers’ lives.
\n\nPlease feel free to ask your questions in this thread, we are happy to help!
\n\nWe wish you all the best of luck! Have fun!
\n\nTeam Elo
", "messages": [ { "id": 429468, "postDate": "2018-11-28T22:38:43.380Z", "content": "Welcome to Elo Merchant recommendation competition!
\n\nWe are really enthusiastic to host this competition and we believe kagglers are going to join it and help us make a difference in our customers’ lives.
\n\nPlease feel free to ask your questions in this thread, we are happy to help!
\n\nWe wish you all the best of luck! Have fun!
\n\nTeam Elo
", "rawMarkdown": "Welcome to Elo Merchant recommendation competition!\n \nWe are really enthusiastic to host this competition and we believe kagglers are going to join it and help us make a difference in our customers’ lives.\n \nPlease feel free to ask your questions in this thread, we are happy to help!\n\nWe wish you all the best of luck! Have fun!\n\nTeam Elo\n", "votes": 54 }, { "id": 444375, "postDate": "2018-12-23T22:33:27.380Z", "content": "My take on these questions (which don't seem to be answered by the Admins). Note this is all my opinion after going through the data for a while.
\n\nThe loyalty score has little, if anything, to do with the data we have been given. Elo wants to see if the transaction and merchant data they have can be used to predict this loyalty score. And I think we can use the given data and external data in any way we choose.
For the transaction data files, ignore the descriptions on the data page. No one can figure out what they mean. I combined the historical and new files and used the complete file to build aggregates and other features. You need to be careful about the mysterious reference date when doing this - some cards have different reference dates than others.
Reference date. I believe this is the date that the loyalty score was calculated. There is no information (that I have seen) that states when the reference date is. I used month_lag to determine the \"reference month\" for each card (reference_month = purchase_date_month - month_lag). Then I defined the reference date as the last day of the reference month, but there are other ways to do it also.
Merchants. The merchant data is a little messy as there are disconnects between the merchant file and the merchant columns in the transaction data. This is a very real-world problem and you'll have to decide which file to go with. I deleted the common columns out of the transaction data and then replaced them using the merchants file (then both agree and every merchant has one subsector, city, state, etc.). That is one way, but may not be the best way, to proceed.
Hope that helps a bit.
", "rawMarkdown": "My take on these questions (which don't seem to be answered by the Admins). Note this is all my opinion after going through the data for a while.\n\n1. The loyalty score has little, if anything, to do with the data we have been given. Elo wants to see if the transaction and merchant data they have can be used to predict this loyalty score. And I think we can use the given data and external data in any way we choose.\n\n2. For the transaction data files, ignore the descriptions on the data page. No one can figure out what they mean. I combined the historical and new files and used the complete file to build aggregates and other features. You need to be careful about the mysterious reference date when doing this - some cards have different reference dates than others.\n\n3. Reference date. I believe this is the date that the loyalty score was calculated. There is no information (that I have seen) that states when the reference date is. I used month\\_lag to determine the \"reference month\" for each card (reference\\_month = purchase\\_date_month - month\\_lag). Then I defined the reference date as the last day of the reference month, but there are other ways to do it also.\n\n4. Merchants. The merchant data is a little messy as there are disconnects between the merchant file and the merchant columns in the transaction data. This is a very real-world problem and you'll have to decide which file to go with. I deleted the common columns out of the transaction data and then replaced them using the merchants file (then both agree and every merchant has one subsector, city, state, etc.). That is one way, but may not be the best way, to proceed.\n\nHope that helps a bit.\n\n", "votes": 39, "replies": [ { "id": 446026, "postDate": "2018-12-27T11:16:03.507Z", "content": "Thank you for your comment Michael! In my opinion, it's really helpful.
", "rawMarkdown": "Thank you for your comment Michael! In my opinion, it's really helpful.", "votes": 4 }, { "id": 446077, "postDate": "2018-12-27T13:13:05.377Z", "content": "Thanks, it's very reasonable!
", "rawMarkdown": "Thanks, it's very reasonable!", "votes": 2 }, { "id": 473604, "postDate": "2019-02-18T08:51:01.193Z", "content": "It's very useful, thanks
", "rawMarkdown": "It's very useful, thanks" } ] }, { "id": 450570, "postDate": "2019-01-05T09:36:56.700Z", "content": "Hi Valeria,\nThere is a authorized_flag column inhistory.csv which I believe indicates whether the transaction is authorized or not. If the transaction is not authorized then it's not a valid transaction . But many of these not authorized transactions have installments ranging from 1 to 10 which is confusing. Please clarify.
", "rawMarkdown": "Hi Valeria,\nThere is a authorized_flag column inhistory.csv which I believe indicates whether the transaction is authorized or not. If the transaction is not authorized then it's not a valid transaction . But many of these not authorized transactions have installments ranging from 1 to 10 which is confusing. Please clarify.", "votes": 23, "replies": [ { "id": 466589, "postDate": "2019-02-05T17:00:23.420Z", "content": "thanks for asking
", "rawMarkdown": "thanks for asking", "votes": 1 }, { "id": 1733771, "postDate": "2022-03-24T15:51:27.170Z", "content": "thanking for your ask
", "rawMarkdown": "thanking for your ask" } ] }, { "id": 446376, "postDate": "2018-12-28T01:15:27.657Z", "content": "Hi @elovaleria,\nIn the historical_transactions file, column purcharse_amount is defined as normalized purcharse amount. But I found that the maximum value of the purcharse_amount column is 6010604 and the number of values larger than 1.5 is only account for 0.84% of the whole dataset, so is the outlier caused by missing value or any other reasons?
Hi Valeria,
\n\nIn the data dictionary (corresponding train.csv), target is describe as \"Loyalty numerical score calculated 2 months after historical and evaluation period\". What does this evaluation period mean ? Is it the new merchant transaction period ?
", "rawMarkdown": "Hi Valeria,\n\nIn the data dictionary (corresponding train.csv), target is describe as \"Loyalty numerical score calculated 2 months after historical and evaluation period\". What does this evaluation period mean ? Is it the new merchant transaction period ? ", "votes": 15, "replies": [ { "id": 443062, "postDate": "2018-12-21T00:49:04.570Z", "content": "Good question! I also have this question.
", "rawMarkdown": "Good question! I also have this question." }, { "id": 443167, "postDate": "2018-12-21T06:13:51.417Z", "content": "I have the same problem, too. Besides, I'm so confused because of the feature_1, 2, 3: they're both separated variables, and lack of details.
", "rawMarkdown": "I have the same problem, too. Besides, I'm so confused because of the feature_1, 2, 3: they're both separated variables, and lack of details. " }, { "id": 444544, "postDate": "2018-12-24T08:30:18.163Z", "content": "I am confused about the same problem.
", "rawMarkdown": "I am confused about the same problem." }, { "id": 446041, "postDate": "2018-12-27T11:48:03.423Z", "content": "Hello all,
\n\nYes, \"Evaluation period\" in target description is the period in which new merchant transactions happen. The loyalty score is calculated after historical and evaluation transaction period are given in order to avoid data leakage.
", "rawMarkdown": "Hello all,\n\nYes, \"Evaluation period\" in target description is the period in which new merchant transactions happen. The loyalty score is calculated after historical and evaluation transaction period are given in order to avoid data leakage.", "votes": 7 }, { "id": 450059, "postDate": "2019-01-04T08:17:57.710Z", "content": "Hi Valeria,
\n\nJust to clarify is \"Evaluation period\" the overall New transaction period which is 2017-03 - 2018-04 or is this specific to each card_id as the reference date is different for different cards ? This would also answer whether loyalty scores for all cards are calculated as same time or different time points. Thanks !
", "rawMarkdown": "Hi Valeria,\n\nJust to clarify is \"Evaluation period\" the overall New transaction period which is 2017-03 - 2018-04 or is this specific to each card_id as the reference date is different for different cards ? This would also answer whether loyalty scores for all cards are calculated as same time or different time points. Thanks ! ", "votes": 2 }, { "id": 454563, "postDate": "2019-01-11T20:39:56.450Z", "content": "Hi @Valeria Elo, the evaluation period is to avoid data leakage, so did you guys find any sign of leakage?
", "rawMarkdown": "Hi @Valeria Elo, the evaluation period is to avoid data leakage, so did you guys find any sign of leakage? " }, { "id": 1826297, "postDate": "2022-06-20T08:38:57.127Z", "content": "Good question!
", "rawMarkdown": "Good question! " } ] }, { "id": 429544, "postDate": "2018-11-29T02:10:23.380Z", "content": "Hi,
\n\nI am unclear on the relationship between the transactions and \"loyalty score\". The description sets up the problem as a collaborative filtering/ recommendation system where we need to predict which new stores a user would try, but the relation between that and loyalty is unclear to me from data description.
\n\nAlso is loyalty strictly calculated from visits to \"new\" stores in next two months, or a combination of store visits in last 3 months and new store visits? What about repeated visit to stores (in next 2 months) the customer has been to in past 3 months.
\n\nThank you.
", "rawMarkdown": "Hi,\n\nI am unclear on the relationship between the transactions and \"loyalty score\". The description sets up the problem as a collaborative filtering/ recommendation system where we need to predict which new stores a user would try, but the relation between that and loyalty is unclear to me from data description.\n\nAlso is loyalty strictly calculated from visits to \"new\" stores in next two months, or a combination of store visits in last 3 months and new store visits? What about repeated visit to stores (in next 2 months) the customer has been to in past 3 months.\n\nThank you.", "votes": 15, "replies": [ { "id": 430388, "postDate": "2018-11-30T10:00:42.767Z", "content": "Hello Darshak,
\n\nThose are pretty good questions!
\n\nThis competition is not a traditional collaborative filtering/recommendation system challenge since we don’t have a direct definition of what is great merchant or experience. Despite that, we want to link historical information of merchants visits and purchases to a loyalty score, which is basically a business metric that considered both future spending and retention as main components. We expect you to link this information in order to find what triggers higher loyalty scores and you can use any kind of modeling, including recommendation system, but not exclusively. You can be as creative as you want!
\n\nI hope it’s clearer now, but please feel free to ask more questions if needed.
\n\nBest,
", "rawMarkdown": "Hello Darshak,\n \nThose are pretty good questions!\n \nThis competition is not a traditional collaborative filtering/recommendation system challenge since we don’t have a direct definition of what is great merchant or experience. Despite that, we want to link historical information of merchants visits and purchases to a loyalty score, which is basically a business metric that considered both future spending and retention as main components. We expect you to link this information in order to find what triggers higher loyalty scores and you can use any kind of modeling, including recommendation system, but not exclusively. You can be as creative as you want!\n \nI hope it’s clearer now, but please feel free to ask more questions if needed.\n \nBest,", "votes": 29 }, { "id": 430826, "postDate": "2018-12-01T02:29:01.130Z", "content": "Thank you Valeria.
", "rawMarkdown": "Thank you Valeria." }, { "id": 447863, "postDate": "2018-12-30T18:17:51.700Z", "content": "Hello Valeria, I'm a bit confused by your answer.\nI'm assuming that you currently have a formula for the \"loyalty score\", so if you want to link loyalty score with historical data you should provide a time series for the loyalty score (which is not presented) so we can link changes in history (presented in the historical_transaction.csv + new_merchant_transactions.csv) with changes in score (not available in this dataset). If the loyalty score doesn't change over time, there is no point on any of this.
\n\nIf you only provided a snapshot of the loyalty score (for February), then I'm not sure why you provided transactions older than the time lag that the loyalty score uses... unless there is something else going on.... which, in any case, you already know because you have the formula for the loyalty score.... therefore you already have the triggers...
\n\nIf you want to obtain a new formula for the loyalty score, I'm not sure why you don't provide the current formula and describe what are the pitfalls, what do you want to fix, etc. Because, if we develop and train model using the current loyalty score, these models are going to have the same problem that the current score because from the modeling respective the current score is considered the \"truth\"...
\n\nSo, I'm confused... if you have the formula we would need some information about why you want to change it. If you don't have a formula, how did you get the current loyalty scores?
", "rawMarkdown": "Hello Valeria, I'm a bit confused by your answer.\nI'm assuming that you currently have a formula for the \"loyalty score\", so if you want to link loyalty score with historical data you should provide a time series for the loyalty score (which is not presented) so we can link changes in history (presented in the historical_transaction.csv + new_merchant_transactions.csv) with changes in score (not available in this dataset). If the loyalty score doesn't change over time, there is no point on any of this. \n\nIf you only provided a snapshot of the loyalty score (for February), then I'm not sure why you provided transactions older than the time lag that the loyalty score uses... unless there is something else going on.... which, in any case, you already know because you have the formula for the loyalty score.... therefore you already have the triggers... \n\nIf you want to obtain a new formula for the loyalty score, I'm not sure why you don't provide the current formula and describe what are the pitfalls, what do you want to fix, etc. Because, if we develop and train model using the current loyalty score, these models are going to have the same problem that the current score because from the modeling respective the current score is considered the \"truth\"... \n\nSo, I'm confused... if you have the formula we would need some information about why you want to change it. If you don't have a formula, how did you get the current loyalty scores?", "votes": 5 } ] }, { "id": 440403, "postDate": "2018-12-17T14:04:58.387Z", "content": "Hi, I have two doubts：\n1、\nThere is a paragraph in the overview,：\n\"Elo has built machine learning models to understand the most important aspects and preferences in their customers’ lifecycle, from food to shopping. But so far none of them is specifically tailored for an individual or profile. This is where you come in“；\nI think the meaning of this sentence is that Kagglers need \"specifically tailored for an individual or profile\" for this project, is that right?\n2、\nWhat is the purpose of obtaining loyalty score, in order to evaluate whether the ELO's recommendation system is effective for people? Use the past history and the data for the next two months to fit the loyalty score. So I think this is not a predictive project. Can give us a clear explanation, what is loyalty, or what is it used for? Thanks
", "rawMarkdown": "Hi, I have two doubts：\n1、\nThere is a paragraph in the overview,：\n\"Elo has built machine learning models to understand the most important aspects and preferences in their customers’ lifecycle, from food to shopping. But so far none of them is specifically tailored for an individual or profile. This is where you come in“；\nI think the meaning of this sentence is that Kagglers need \"specifically tailored for an individual or profile\" for this project, is that right?\n2、\nWhat is the purpose of obtaining loyalty score, in order to evaluate whether the ELO's recommendation system is effective for people? Use the past history and the data for the next two months to fit the loyalty score. So I think this is not a predictive project. Can give us a clear explanation, what is loyalty, or what is it used for? Thanks", "votes": 9, "replies": [ { "id": 448016, "postDate": "2018-12-31T04:02:45.163Z", "content": "I agree with you. I don't understand why you should predict the loyalty score. Loyalty score is often calculated using some formulae . Are we attempting to find the formulae ELO is using ? . If recommendations based on loyalty score doesn't work for ELO they should look review how they are calculating the loyalty score.
", "rawMarkdown": "I agree with you. I don't understand why you should predict the loyalty score. Loyalty score is often calculated using some formulae . Are we attempting to find the formulae ELO is using ? . If recommendations based on loyalty score doesn't work for ELO they should look review how they are calculating the loyalty score.", "votes": 2 }, { "id": 457510, "postDate": "2019-01-17T15:23:05.730Z", "content": "I have same question.\nIs there any one who knows what \"loyalty\" means?
", "rawMarkdown": "I have same question.\nIs there any one who knows what \"loyalty\" means?" }, { "id": 463222, "postDate": "2019-01-29T16:29:44.187Z", "content": "a dog knows
", "rawMarkdown": "a dog knows" } ] }, { "id": 437626, "postDate": "2018-12-12T08:10:26.273Z", "content": "Hi Valeria,
\n\nI found some card_id's purchase_date is earlier than first_active_month,for example a card_id\nC_ID_a51e74c7c1's first_active_monthis 2017-10,but earliest purchase_date is '2017-08-04 12:31:03' in historical_transactions.\nis it reasonable?
", "rawMarkdown": "Hi Valeria,\n\nI found some card_id's purchase_date is earlier than first_active_month,for example a card_id\nC_ID_a51e74c7c1's first_active_monthis 2017-10,but earliest purchase_date is '2017-08-04 12:31:03' in historical_transactions.\nis it reasonable?", "votes": 6, "replies": [ { "id": 438915, "postDate": "2018-12-14T12:07:45.343Z", "content": "Hello senkin13! Although it might seem counterintuitive, there are cases in which it could happen.
", "rawMarkdown": "Hello senkin13! Although it might seem counterintuitive, there are cases in which it could happen." }, { "id": 440495, "postDate": "2018-12-17T16:09:14.493Z", "content": "Valeria, under what circumstances would something like this occur? Does the first active month feature mean something other than the first month of purchases made using the card, does it mean the first month of reaching a certain threshold in activity or something else?
", "rawMarkdown": "Valeria, under what circumstances would something like this occur? Does the *first active month* feature mean something other than the first month of purchases made using the card, does it mean the first month of reaching a certain threshold in activity or something else?", "votes": 5 } ] }, { "id": 436706, "postDate": "2018-12-10T19:48:36.813Z", "content": "Same questions as @Joshi (sort of). I am assuming the reference date is the date when the loyalty score was calculated. So, 28 February 2018, the last date in the historical transactions, would be the reference date on which the train.target is based, correct? Second question, what reference date are we predicting for? The last date in the new transactions file is 30 April 2018, so I would assume that is the reference date for our predictions? If so, the month_lag is not correct in the new transaction file. Or, are we to predict for 28 February but we use the purchase transactions after that date?
", "rawMarkdown": "Same questions as @Joshi (sort of). I am assuming the reference date is the date when the loyalty score was calculated. So, 28 February 2018, the last date in the historical transactions, would be the reference date on which the train.target is based, correct? Second question, what reference date are we predicting for? The last date in the new transactions file is 30 April 2018, so I would assume that is the reference date for our predictions? If so, the month_lag is not correct in the new transaction file. Or, are we to predict for 28 February but we use the purchase transactions after that date?", "votes": 6, "replies": [ { "id": 446160, "postDate": "2018-12-27T16:20:14.270Z", "content": "@Michael,\nAccording to data dictionary, It's mentioned that loyalty score is calculated 2 months after historical and evaluation period (new transaction ).
", "rawMarkdown": "@Michael,\nAccording to data dictionary, It's mentioned that loyalty score is calculated 2 months after historical and evaluation period (new transaction ). ", "votes": 1 } ] }, { "id": 440657, "postDate": "2018-12-17T21:13:03.607Z", "content": "HI!\nI have a question like everyone here. Maybe you've discussed it.\nWhat does 999 value of installments mean? Is Number of installments unlimited?\nThanks)
", "rawMarkdown": "HI!\nI have a question like everyone here. Maybe you've discussed it.\nWhat does 999 value of installments mean? Is Number of installments unlimited?\nThanks)", "votes": 3, "replies": [ { "id": 444215, "postDate": "2018-12-23T14:57:39.800Z", "content": "It can be outliers.
", "rawMarkdown": "It can be outliers.", "votes": 1 }, { "id": 446043, "postDate": "2018-12-27T11:49:49.113Z", "content": "Hello all,
\n\n999 in installments indicates missing values.
", "rawMarkdown": "Hello all,\n\n999 in installments indicates missing values.", "votes": 3 }, { "id": 446181, "postDate": "2018-12-27T16:53:14.007Z", "content": "Hi Valeria, What does -1 in installments imply ?\nThanks!
", "rawMarkdown": "Hi Valeria, What does -1 in installments imply ?\nThanks!", "votes": 1 }, { "id": 446189, "postDate": "2018-12-27T17:10:16.387Z", "content": "Hi! \"-1\" are also missing values.
", "rawMarkdown": "Hi! \"-1\" are also missing values.", "votes": 5 }, { "id": 454207, "postDate": "2019-01-11T08:41:56.183Z", "content": "So is there any different between 999 and -1 as missing values? Or it is kind of same treatment like \"Okay start right now, we never use -1 as missing values, using 999 instead\" (or vice versa)?
I thought -1 could be a refund :-)\nBtw, I don't see also the difference between 0 and 1 installment, understanding 1 installment as the complete pay of the transaction. Or maybe we should understand that there are 2 monthly payments?
", "rawMarkdown": "I thought -1 could be a refund :-)\nBtw, I don't see also the difference between 0 and 1 installment, understanding 1 installment as the complete pay of the transaction. Or maybe we should understand that there are 2 monthly payments?", "votes": 1 } ] }, { "id": 438907, "postDate": "2018-12-14T11:40:56.463Z", "content": "Valéria, please, could you confirm that newMerchantTransactions.csv file doesn't account for all transactions a given cardId did in the period but only the transactions that were done by that cardId in a merchantId that never happened before?
", "rawMarkdown": "Valéria, please, could you confirm that newMerchantTransactions.csv file doesn't account for all transactions a given cardId did in the period but only the transactions that were done by that cardId in a merchantId that never happened before?", "votes": 3, "replies": [ { "id": 438914, "postDate": "2018-12-14T11:59:17.737Z", "content": "Hello Leandro,
\n\nYou are correct. 'newmerchanttransactions.csv' accounts only for transactions in merchants never visited before for each card_id, therefore it doesn't have all transactions.
", "rawMarkdown": "Hello Leandro,\n\nYou are correct. 'newmerchanttransactions.csv' accounts only for transactions in merchants never visited before for each card_id, therefore it doesn't have all transactions." }, { "id": 438925, "postDate": "2018-12-14T12:31:47.800Z", "content": "Thanks! Second thoughts then... what kind of information could be hidden in the actual transactions set that each cardId did during the \"new\" period of time? I mean, are you guys doing any kind of filtering or there could be cardIds that didn't buy anything even from their previous merchantIds?
", "rawMarkdown": "Thanks! Second thoughts then... what kind of information could be hidden in the actual transactions set that each cardId did during the \"new\" period of time? I mean, are you guys doing any kind of filtering or there could be cardIds that didn't buy anything even from their previous merchantIds?", "votes": 1 } ] }, { "id": 430126, "postDate": "2018-11-29T21:44:36.047Z", "content": "How does \"loyalty score\" not depend on the merchant id? What kind of recommendation this is?
", "rawMarkdown": "How does \"loyalty score\" not depend on the merchant id? What kind of recommendation this is?", "votes": 3, "replies": [ { "id": 430380, "postDate": "2018-11-30T09:41:17.677Z", "content": "I think here the loyalty is measured toward Elo the payment company and not the merchant.
", "rawMarkdown": "I think here the loyalty is measured toward Elo the payment company and not the merchant.", "votes": 7 }, { "id": 445402, "postDate": "2018-12-26T11:41:18.487Z", "content": "but when it's not depend on the merchant_id, what the meaning of merchant.csv file?
", "rawMarkdown": "but when it's not depend on the merchant_id, what the meaning of merchant.csv file?", "votes": 1 } ] }, { "id": 435332, "postDate": "2018-12-07T21:46:33.360Z", "content": "Hi Valeria, \nI had a quick question about the dates in historical transactions and the new transactions. Is there any significance in the reference date 2018-02?
", "rawMarkdown": "Hi Valeria, \nI had a quick question about the dates in historical transactions and the new transactions. Is there any significance in the reference date 2018-02?", "votes": 4 }, { "id": 430638, "postDate": "2018-11-30T19:10:26.910Z", "content": "Hello Valeria,\ncan you please clarify the target? are we supposed to predict the loyalty score for the dates 1/5/2018 to 30/6/2018?
", "rawMarkdown": "Hello Valeria,\ncan you please clarify the target? are we supposed to predict the loyalty score for the dates 1/5/2018 to 30/6/2018? ", "votes": 1, "replies": [ { "id": 430701, "postDate": "2018-11-30T21:27:16.313Z", "content": "That is correct - you are predicting loyalty score.
", "rawMarkdown": "That is correct - you are predicting loyalty score.", "votes": 2 }, { "id": 430713, "postDate": "2018-11-30T21:46:49.430Z", "content": "thank you for your clarification
", "rawMarkdown": "thank you for your clarification" }, { "id": 436632, "postDate": "2018-12-10T17:02:48.557Z", "content": "@Dimosthenis Karaflos just double check, so we are supposed to predict the target (loyalty score) for dates 1/5/2018 to 30/6/2018? where did you get the dates from?
\n\n@Valeria Elo In the data description, \"historical_transactions.csv - up to 3 months' worth of historical transactions for each card_id\", but when I checked the data, it seems like some card_ids has way more than 3 month purchase_dates, did I misunderstand something?
", "rawMarkdown": "@Dimosthenis Karaflos just double check, so we are supposed to predict the target (loyalty score) for dates 1/5/2018 to 30/6/2018? where did you get the dates from?\n\n@Valeria Elo In the data description, \"historical_transactions.csv - up to 3 months' worth of historical transactions for each card_id\", but when I checked the data, it seems like some card_ids has way more than 3 month purchase_dates, did I misunderstand something? ", "votes": 1 }, { "id": 436646, "postDate": "2018-12-10T17:38:12.627Z", "content": "File \"historicaltransactions.csv\" has at least 3 months of historical data for each card.
", "rawMarkdown": "File \"historicaltransactions.csv\" has at least 3 months of historical data for each card.", "votes": 1 } ] }, { "id": 443117, "postDate": "2018-12-21T03:34:06.833Z", "content": "Hi,\nI have a question? what is the meaning of month_lag in history and new transaction?
", "rawMarkdown": "Hi,\nI have a question? what is the meaning of month_lag in history and new transaction?", "votes": 2, "replies": [ { "id": 445352, "postDate": "2018-12-26T09:00:42.297Z", "content": "I have the same question
", "rawMarkdown": "I have the same question" }, { "id": 446158, "postDate": "2018-12-27T16:11:53.063Z", "content": "Month_lag is the difference in months between purchase date month and reference date month, I'm assuming reference month is feb 2018.
", "rawMarkdown": "Month_lag is the difference in months between purchase date month and reference date month, I'm assuming reference month is feb 2018.", "votes": 1 }, { "id": 447722, "postDate": "2018-12-30T11:58:06.203Z", "content": "Based on other comments,I just figured out what reference date is. It is the date when loyalty score is calculated.\nSo , if a transaction as a purchase date as 1 June 2018 and reference date as 1 Feb 2018, We would have month_lag as their difference which would be -8.
\n\nPlease free to add/edit comments to it if my understanding is incorrect.
", "rawMarkdown": "Based on other comments,I just figured out what reference date is. It is the date when loyalty score is calculated.\nSo , if a transaction as a purchase date as 1 June 2018 and reference date as 1 Feb 2018, We would have month_lag as their difference which would be -8.\n\nPlease free to add/edit comments to it if my understanding is incorrect.", "votes": 1 }, { "id": 453801, "postDate": "2019-01-10T19:58:29.587Z", "content": "@HarshitMehta No, if you look at data, reference date/month can be different for different customers(card_id).
", "rawMarkdown": "@HarshitMehta No, if you look at data, reference date/month can be different for different customers(card_id).", "votes": 1 } ] }, { "id": 437983, "postDate": "2018-12-12T22:16:54.393Z", "content": "a question please: \nCan you explain what exactly \"new_merchant_transactions.csv contains the transactions at new merchants (merchant_ids that this particular card_id has not yet visited) over a period of two months\" means? (from the data description). How is it possible that a card_id hasn't visited the merchant and has a purchase_amount?
\n\nAlso, it's mentioned that the file contains transactions \"over a period of two months\", but if I get the min and max of the purchase_date, I get 2017-03-01 03:24:51 and 2018-04-30 23:59:59 respectively, which is more than a year!
\n\nThanks
", "rawMarkdown": "a question please: \nCan you explain what exactly \"new\\_merchant\\_transactions.csv contains the transactions at new merchants (merchant_ids that this particular card\\_id has not yet visited) over a period of two months\" means? (from the data description). How is it possible that a card_id hasn't visited the merchant and has a **purchase\\_amount**?\n\nAlso, it's mentioned that the file contains transactions \"over a period of two months\", but if I get the min and max of the purchase_date, I get 2017-03-01 03:24:51 and 2018-04-30 23:59:59 respectively, which is more than a year!\n\nThanks", "votes": 2, "replies": [ { "id": 438098, "postDate": "2018-12-13T05:09:52.527Z", "content": "I think the time range is more tant two months. Someone has asked the same question in this thread.(See below)\n
Thank you, Zhuang Jia.
\n\nHow about the other question? do you know what is the difference between historical transactions and new merchant transactions?\nnew merchant transactions: \"merchant ids that this particular card id has not yet visited\". How can a transaction occur if the customer hasn't visited the merchant? Are they online transactions maybe?
", "rawMarkdown": "Thank you, Zhuang Jia.\n\nHow about the other question? do you know what is the difference between historical transactions and new merchant transactions?\nnew merchant transactions: \"merchant ids that this particular card id has not yet visited\". How can a transaction occur if the customer hasn't visited the merchant? Are they online transactions maybe?", "votes": 1 }, { "id": 438171, "postDate": "2018-12-13T07:55:48.077Z", "content": "I also have many questions about this two files. I guess they collect the transactions for a period of time(2~14 months for each card_id) and put this information to historical transactions. And then they focus on the new merchant each card_id visits, so they only collect information about merchant that card_id not yet visited before and put this information to new_merchant_transactions.csv.
My English is not very good. I hope I have make my opinion clear. I do some check about this two files. You can have a look if you like. Basic info about historical/new transactions
", "rawMarkdown": "I also have many questions about this two files. I guess they collect the transactions for a period of time(2~14 months for each `card_id`) and put this information to `historical transactions`. And then they focus on the new merchant each `card_id` visits, so they only collect information about merchant that `card_id` **not yet visited before** and put this information to `new_merchant_transactions.csv`.\n\nMy English is not very good. I hope I have make my opinion clear. I do some check about this two files. You can have a look if you like. [Basic info about historical/new transactions](https://www.kaggle.com/jiazhuang/basic-info-about-historical-new-transactions)", "votes": 5 } ] }, { "id": 437478, "postDate": "2018-12-12T01:52:18.580Z", "content": "Hi Valeria,\nAccording to the Data Dictionary, merchant id should be an unique merchant identifier. However, there are 334,633 unique merchant IDs but there are 334,696 rows in the merchants.csv. Some merchant_id repeated 4 times.
\nCould you please review the merchants.csv and re-publish it ASAP?\nThanks
Hi Valeria ....\nWe all teams love Elo.
", "rawMarkdown": "Hi Valeria ....\nWe all teams love Elo.", "votes": 1 }, { "id": 435046, "postDate": "2018-12-07T11:46:14.100Z", "content": "Dear, I hope this can be built in Python as well right? pls advise
", "rawMarkdown": "Dear, I hope this can be built in Python as well right? pls advise", "votes": -6, "replies": [ { "id": 454120, "postDate": "2019-01-11T06:37:13.787Z", "content": "Language is never the constraint.
", "rawMarkdown": "Language is never the constraint. ", "votes": 1 } ] }, { "id": 1246286, "postDate": "2021-03-20T16:04:03.403Z", "content": "Hello Valeria Mam,
\nthis competition is featured. Thanks for providing this meaningful competition, I don't know where to see the winner's code , will the winner's code be publicly sharing?
", "rawMarkdown": "this competition is featured. Thanks for providing this meaningful competition, I don't know where to see the winner's code , will the winner's code be publicly sharing? " }, { "id": 477684, "postDate": "2019-02-25T05:04:31.350Z", "content": "Hi Will you give one or two more days on the submission deadline since kaggle was down for one day?
", "rawMarkdown": "Hi Will you give one or two more days on the submission deadline since kaggle was down for one day? ", "replies": [ { "id": 477692, "postDate": "2019-02-25T05:16:35.110Z", "content": "Hi - we will not be adjusting competition timelines for the Kaggle site maintenance
", "rawMarkdown": "Hi - we will not be adjusting competition timelines for the Kaggle site maintenance" } ] }, { "id": 475541, "postDate": "2019-02-20T21:44:38.303Z", "content": "Hi @elovaleria Some city have more than one state. City 331 in state 16 and 20, how it possible?
", "rawMarkdown": "Hi @elovaleria Some city have more than one state. City 331 in state 16 and 20, how it possible?" }, { "id": 466292, "postDate": "2019-02-05T04:05:34.147Z", "content": "Could someone please elaborate on the values for columns feature 1, feature 2, and feature 3? Are they suppose to be categorical or numeric. I am not understanding what it means by Anonymized card categorical feature.
", "rawMarkdown": "Could someone please elaborate on the values for columns feature 1, feature 2, and feature 3? Are they suppose to be categorical or numeric. I am not understanding what it means by Anonymized card categorical feature.", "replies": [ { "id": 466295, "postDate": "2019-02-05T04:15:46.883Z", "content": "Try all the possible ways you can do with these features, whether it's treating them as numerical, or categorical, or using some interaction between them or with some other features, as long your score is improving. For me treating them as categorical and finding interaction w.r.t some other features, helped to improve the score.
", "rawMarkdown": "Try all the possible ways you can do with these features, whether it's treating them as numerical, or categorical, or using some interaction between them or with some other features, as long your score is improving. For me treating them as categorical and finding interaction w.r.t some other features, helped to improve the score.", "votes": 3 }, { "id": 466301, "postDate": "2019-02-05T04:38:56.923Z", "content": "Thank you for the help and insights!
", "rawMarkdown": "Thank you for the help and insights!", "votes": 1 } ] }, { "id": 465641, "postDate": "2019-02-03T16:58:52.933Z", "content": "Hello everyone,\nI have seen some discussions regarding this matter and still I am not able to realize WHEN the score is calculated.
\n\nThe way I see it until some period X all customers had some transactions made (that may or may not be presented in the data) . Afterwards, ELO has used some ML model to advertise to customers ,then new transactions occurred (which definitely should be in the data) for some period Y . \nat the end of periods X+Y ELO has presented a \"loyalty score \"\nCould someone please explain to me how to obtain X +Y, X,Y from the data or at least contribute with some relevant idea.\nOtherwise, I don't understand how are we suppose to obtain any insights from the transaction data.
\n\nThank you
", "rawMarkdown": "Hello everyone,\nI have seen some discussions regarding this matter and still I am not able to realize WHEN the score is calculated.\n \nThe way I see it until some period X all customers had some transactions made (that may or may not be presented in the data) . Afterwards, ELO has used some ML model to advertise to customers ,then new transactions occurred (which definitely should be in the data) for some period Y . \nat the end of periods X+Y ELO has presented a \"loyalty score \"\nCould someone please explain to me how to obtain X +Y, X,Y from the data or at least contribute with some relevant idea.\nOtherwise, I don't understand how are we suppose to obtain any insights from the transaction data.\n\nThank you" }, { "id": 465533, "postDate": "2019-02-03T11:42:56.833Z", "content": "Hi Valéria, this challenge seems to be perfect as a topic for my thesis in a MBA business analytics course at the FGV. Can I use the data for this purpose besides competing?
", "rawMarkdown": "Hi Valéria, this challenge seems to be perfect as a topic for my thesis in a MBA business analytics course at the FGV. Can I use the data for this purpose besides competing? " }, { "id": 460896, "postDate": "2019-01-24T16:26:32.947Z", "content": "Hi Valeria,
\n\nWhere is the significate that a merchant appear two times or four times in merchant table?
\n\nThanks
", "rawMarkdown": "Hi Valeria,\n\nWhere is the significate that a merchant appear two times or four times in merchant table? \n\nThanks" }, { "id": 456970, "postDate": "2019-01-16T20:01:42.847Z", "content": "Hi Valeria,
\n\nCan you explain about the active_month_lagx column in merchant data. Like how it is calculated.
", "rawMarkdown": "Hi Valeria,\n\nCan you explain about the active_month_lagx column in merchant data. Like how it is calculated." }, { "id": 456147, "postDate": "2019-01-15T08:27:46.893Z", "content": "Hi Valeria,\nis the historical and evaluation period the same for the train and test dataset?\nSpecifically, is the date range from 2018-05 to 2018-06?
", "rawMarkdown": "Hi Valeria,\nis the historical and evaluation period the same for the train and test dataset?\nSpecifically, is the date range from 2018-05 to 2018-06?\n" }, { "id": 454145, "postDate": "2019-01-11T07:08:08.127Z", "content": "Hi all, the train data consists of feature_1, feature_2 & feature_3. As these columns are absent from any other data given, does that mean we need to decide upon the 3 features that will impact the loyalty score(assuming 'loyalty score=target') - in that case how do we ensure that target values are true for any 3 features selected ?
", "rawMarkdown": "Hi all, the train data consists of feature_1, feature_2 & feature_3. As these columns are absent from any other data given, does that mean we need to decide upon the 3 features that will impact the loyalty score(assuming 'loyalty score=target') - in that case how do we ensure that target values are true for any 3 features selected ?" }, { "id": 452349, "postDate": "2019-01-08T15:56:08.920Z", "content": "I looked through the discussions carefully but I'm still confused with 'reference date'. Is 'loyalty score' calculated at reference date, or is it calculated 2 months after the last date in new merchant transactions for each card id? If latter is true, what is the use of 'reference date'?
", "rawMarkdown": "I looked through the discussions carefully but I'm still confused with 'reference date'. Is 'loyalty score' calculated at reference date, or is it calculated 2 months after the last date in new merchant transactions for each card id? If latter is true, what is the use of 'reference date'?", "replies": [ { "id": 452377, "postDate": "2019-01-08T16:27:08.150Z", "content": "It is calculated two months after historical and new transactions. Check data dictionary : https://www.kaggle.com/c/elo-merchant-category-recommendation/data
", "rawMarkdown": "It is calculated two months after historical and new transactions. Check data dictionary : https://www.kaggle.com/c/elo-merchant-category-recommendation/data", "votes": 1 }, { "id": 452413, "postDate": "2019-01-08T17:32:17.667Z", "content": "@HarshitMehta do you know why there would be a data leakage if the evaluation period (new trans) were not given? According to the host:
\n\nevaluation transaction period are given in order to avoid data leakage.\nGood question @samshipengs. I don’t know about that. May be someone else can explain that.
", "rawMarkdown": "Good question @samshipengs. I don’t know about that. May be someone else can explain that." }, { "id": 452589, "postDate": "2019-01-08T23:27:22.310Z", "content": "@HarshitMehta, thanks for the reply. But I have two more questions if loyalty is not calculated at reference date.
What is the need of splitting transaction data into historical and new ones? So what is evaluation period actually doing?
Also, I've observed from https://www.kaggle.com/fabiendaniel/elo-world that month_diff, which represents oldness of reference date, is an important feature when calculating loyalty.\nWhy would reference date which is no more than just a 'last date in historical transactions for each card' have so much impact on loyalty?
@samshipengs Your quote is actually [The loyalty score is calculated after historical and evaluation transaction period are given] [in order to avoid data leakage.]
Hi @kimYooSun, Frankly i don't have the answers for that.
\n1) Evaluation period as mentioned by host and restated by @samshipengs, is avoiding the data leakage, but how, i don't know. \n2) Certainly Month diff is an important feature, i am relating this with the oldness of purchase date/transaction instead of reference date, suppose there are customers \"A\" and \"B\", Customer A has historical transaction 6 months back(month diff) and B has 2 months back from certain merchant, since customer A didn't show up in the last 6 months, it is more likely that customer A will have lesser affinity/loyalty towards merchant than B, although this is not the correct reasoning for sure as there can be a lot of other drivers.
@HarshitMehta, thank you again for the reply! Yeah I agree that if month_diff is calculated as the oldness of purchase_date, it would have impact on loyalty.
I wish hosts explain more about what loyalty, reference date, and evaluation period are. At least we should know what we are doing...
", "rawMarkdown": "@HarshitMehta, thank you again for the reply! Yeah I agree that if `month_diff` is calculated as the oldness of `purchase_date`, it would have impact on `loyalty`.\n\nI wish hosts explain more about what loyalty, reference date, and evaluation period are. At least we should know what we are doing..." }, { "id": 453144, "postDate": "2019-01-09T18:28:40.023Z", "content": "See my post below. Valeria states my guesses are helpful, but doesn't say they are correct.
", "rawMarkdown": "See my post below. Valeria states my guesses are helpful, but doesn't say they are correct.", "votes": 1 } ] }, { "id": 449970, "postDate": "2019-01-04T03:42:18.647Z", "content": "Hi @elovaleria, is -1 in subsector_id, city_id, state_id, merchant_category_id missing values ?
-1 implies missing data in all the ids.
", "rawMarkdown": "-1 implies missing data in all the ids." }, { "id": 450105, "postDate": "2019-01-04T10:10:19.430Z", "content": "Thank you very much, and hope it can be verified by the competition host.
", "rawMarkdown": "Thank you very much, and hope it can be verified by the competition host." } ] }, { "id": 448128, "postDate": "2018-12-31T09:54:34.897Z", "content": "Hi @elovaleria, just curious can we assume all the merchants are having MCC (merchant category id here in dataset) are serving for Hotel,Restaurant category? just to avoid unnecessary data..
\n\nThanks
", "rawMarkdown": "Hi @elovaleria, just curious can we assume all the merchants are having MCC (merchant category id here in dataset) are serving for Hotel,Restaurant category? just to avoid unnecessary data..\n\nThanks" }, { "id": 447115, "postDate": "2018-12-29T06:54:55.907Z", "content": "Hello, how do you use the merchant data?
", "rawMarkdown": "Hello, how do you use the merchant data?" }, { "id": 447030, "postDate": "2018-12-29T02:44:24.830Z", "content": "Hi, Valeria Elo, what's the difference between -1 and 999 in installment column? And what's the difference between 0 and 1 in installment column?
", "rawMarkdown": "Hi, Valeria Elo, what's the difference between -1 and 999 in installment column? And what's the difference between 0 and 1 in installment column?", "replies": [ { "id": 447169, "postDate": "2018-12-29T09:24:04.303Z", "content": "-1 & 999 are both missing value
Hi , Wenchao , I wanna be your teammate if I can touch 3.67x
", "rawMarkdown": "Hi , Wenchao , I wanna be your teammate if I can touch 3.67x", "votes": 1 }, { "id": 452619, "postDate": "2019-01-09T00:40:39.390Z", "content": "Hi Wenchao, are you still looking for teammate? If so, could we have a talk? :)
", "rawMarkdown": "Hi Wenchao, are you still looking for teammate? If so, could we have a talk? :)" }, { "id": 454688, "postDate": "2019-01-12T02:24:47.410Z", "content": "my weixin is xwc940512
", "rawMarkdown": "my weixin is xwc940512" } ] }, { "id": 445704, "postDate": "2018-12-27T00:46:06.997Z", "content": "Hi Valeria,
\n\nI have two questions about the score.
\n\nFrom the competition description i can understand that there is not any model to produce the loyalty scores. Hence, are the target scores derived from a determenistic formula or from a vanilla model?
\n\non the historical transactions and new merchants datasets, has the normalization been applied by the total column values by each seperated dataset or the normalization has been applied to the total dataset?
", "rawMarkdown": "Hi Valeria,\n\nI have two questions about the score.\n\nFrom the competition description i can understand that there is not any model to produce the loyalty scores. Hence, are the target scores derived from a determenistic formula or from a vanilla model?\n\non the historical transactions and new merchants datasets, has the normalization been applied by the total column values by each seperated dataset or the normalization has been applied to the total dataset?" }, { "id": 445498, "postDate": "2018-12-26T15:24:47.180Z", "content": "Hi Valeria,
\n\nWhy does not all users have \"month_lag == 0\" in historical_transactions data??
\n\nThanks
", "rawMarkdown": "Hi Valeria,\n\nWhy does not all users have \"month_lag == 0\" in historical_transactions data??\n\nThanks" }, { "id": 444239, "postDate": "2018-12-23T16:11:21.733Z", "content": "Hi Valeria,
\n\nCould you please explain more about \"reference date\" which appearing in the description of the field \"month_lag\" as saying that \"month lag to reference date\"?
\n\nAlso, i believe that it should be the \"reference date\" in the merchants data set, i am a little confused, could you help me?
\n\nMy English is not good, sorry for that.
\n\nThanks : )
", "rawMarkdown": "Hi Valeria,\n\nCould you please explain more about \"reference date\" which appearing in the description of the field \"month_lag\" as saying that \"month lag to reference date\"?\n\nAlso, i believe that it should be the \"reference date\" in the merchants data set, i am a little confused, could you help me?\n\nMy English is not good, sorry for that.\n\nThanks : )" }, { "id": 443764, "postDate": "2018-12-22T11:31:38.583Z", "content": "Hi Valeria,
\n\nCan you explain that the data in the table 'merchants' is based on '2018-04-30' or someday else?
\n\nBest,\nZhang.
", "rawMarkdown": "Hi Valeria,\n\nCan you explain that the data in the table 'merchants' is based on '2018-04-30' or someday else?\n\nBest,\nZhang." }, { "id": 443723, "postDate": "2018-12-22T08:36:54.203Z", "content": "Hi Valeria,
\n\nIn merchants.csv every merchant_id has 1 combination of [\"merchant_id\",\"merchant_category_id\",\"subsector_id\",\"city_id\",\"state_id\"]
\n\nBut in *historical/new_merchants_transactions*, rows with this merchant_id have different combiantions of [\"merchant_id\",\"merchant_category_id\",\"subsector_id\",\"city_id\",\"state_id\"]
\n\nDoes it make sense? How a merchant_id can have different identifiers in other datasets than merchant.csv?\nNote: Attached photo with example (table 1 from merchant.csv, table 2 from historical/new_merchants_transactions)
", "rawMarkdown": "Hi Valeria,\n\nIn *merchants.csv* every merchant_id has 1 combination of [\"merchant_id\",\"merchant_category_id\",\"subsector_id\",\"city_id\",\"state_id\"]\n\nBut in *historical/new_merchants_transactions*, rows with this merchant_id have different combiantions of [\"merchant_id\",\"merchant_category_id\",\"subsector_id\",\"city_id\",\"state_id\"]\n\nDoes it make sense? How a merchant_id can have different identifiers in other datasets than merchant.csv?\nNote: Attached photo with example (table 1 from merchant.csv, table 2 from historical/new_merchants_transactions)", "replies": [ { "id": 445398, "postDate": "2018-12-26T11:31:36.313Z", "content": "I am confused about this problem too.
", "rawMarkdown": "I am **confused** about this problem too." }, { "id": 452299, "postDate": "2019-01-08T14:17:18.883Z", "content": "I am too confused. Any updates?
", "rawMarkdown": "I am too confused. Any updates?" } ] }, { "id": 440032, "postDate": "2018-12-16T22:46:18.610Z", "content": "Hi, I have just started analyzing transaction information provided on the competition. If possible, can you please elaborate on how is normalized purchase amount calculated. Specifically:\n- Is it mean normalized?\n- Across which dimension is the normalization done (is it across card_id only or card_id X merchant_id or such). Also, is the normalization done using the same data or different reference data?
\n\nBest,\nPawas
", "rawMarkdown": "Hi, I have just started analyzing transaction information provided on the competition. If possible, can you please elaborate on how is normalized purchase amount calculated. Specifically:\n- Is it mean normalized?\n- Across which dimension is the normalization done (is it across card_id only or card_id X merchant_id or such). Also, is the normalization done using the same data or different reference data?\n\nBest,\nPawas", "replies": [ { "id": 445401, "postDate": "2018-12-26T11:39:57Z", "content": "There are much outliers in the purchase_amount column, which makes me very confused.
", "rawMarkdown": "There are much **outliers** in the purchase_amount column, which makes me very confused." } ] }, { "id": 437655, "postDate": "2018-12-12T09:29:50.160Z", "content": "why many people not use nn for this task ? who can tell me answer
", "rawMarkdown": "why many people not use nn for this task ? who can tell me answer", "replies": [ { "id": 438100, "postDate": "2018-12-13T05:12:33.987Z", "rawMarkdown": "", "votes": 2, "isDeleted": true }, { "id": 445407, "postDate": "2018-12-26T11:52:44.547Z", "content": "Sorry... what does NN mean? is it the abbreviation for Nearest Neighborhood?
", "rawMarkdown": "Sorry... what does NN mean? is it the abbreviation for Nearest Neighborhood? ", "votes": 3 }, { "id": 445411, "postDate": "2018-12-26T11:59:49.037Z", "content": "神经网络
", "rawMarkdown": "神经网络", "votes": -1 }, { "id": 447740, "postDate": "2018-12-30T12:36:00.960Z", "content": "Neural Network
", "rawMarkdown": "Neural Network" } ] }, { "id": 2466192, "postDate": "2023-10-03T16:20:38.420Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2119147, "postDate": "2023-01-28T14:13:51.087Z", "rawMarkdown": "", "isDeleted": true }, { "id": 482108, "postDate": "2019-03-02T10:42:17.083Z", "rawMarkdown": "", "isDeleted": true }, { "id": 479484, "postDate": "2019-02-27T07:25:44.913Z", "rawMarkdown": "", "isDeleted": true }, { "id": 477357, "postDate": "2019-02-24T11:45:33.730Z", "rawMarkdown": "", "isDeleted": true }, { "id": 446866, "postDate": "2018-12-28T19:57:44.737Z", "rawMarkdown": "", "isDeleted": true }, { "id": 436844, "postDate": "2018-12-11T02:41:02.620Z", "rawMarkdown": "", "isDeleted": true }, { "id": 434975, "postDate": "2018-12-07T08:45:12.517Z", "rawMarkdown": "", "isDeleted": true }, { "id": 484636, "postDate": "2019-03-06T09:07:40.977Z", "content": "Thank you, competition was nice!
", "rawMarkdown": "Thank you, competition was nice!", "votes": 1 }, { "id": 464770, "postDate": "2019-02-01T13:28:00.373Z", "content": "Thank you! :)
", "rawMarkdown": "Thank you! :)" } ], "comments": [ { "id": 444375, "author_name": "Michael Maguire", "author_url": "", "post_date": "2018-12-23T22:33:27.380000", "content": "My take on these questions (which don't seem to be answered by the Admins). Note this is all my opinion after going through the data for a while.
\n\nThe loyalty score has little, if anything, to do with the data we have been given. Elo wants to see if the transaction and merchant data they have can be used to predict this loyalty score. And I think we can use the given data and external data in any way we choose.
For the transaction data files, ignore the descriptions on the data page. No one can figure out what they mean. I combined the historical and new files and used the complete file to build aggregates and other features. You need to be careful about the mysterious reference date when doing this - some cards have different reference dates than others.
Reference date. I believe this is the date that the loyalty score was calculated. There is no information (that I have seen) that states when the reference date is. I used month_lag to determine the \"reference month\" for each card (reference_month = purchase_date_month - month_lag). Then I defined the reference date as the last day of the reference month, but there are other ways to do it also.
Merchants. The merchant data is a little messy as there are disconnects between the merchant file and the merchant columns in the transaction data. This is a very real-world problem and you'll have to decide which file to go with. I deleted the common columns out of the transaction data and then replaced them using the merchants file (then both agree and every merchant has one subsector, city, state, etc.). That is one way, but may not be the best way, to proceed.
Hope that helps a bit.
", "votes": 39, "replies": [ { "id": 446026, "author_name": "Valeria Elo", "author_url": "", "post_date": "2018-12-27T11:16:03.507000", "content": "Thank you for your comment Michael! In my opinion, it's really helpful.
", "votes": 4, "replies": [] }, { "id": 446077, "author_name": "Zhuang Jia", "author_url": "", "post_date": "2018-12-27T13:13:05.377000", "content": "Thanks, it's very reasonable!
", "votes": 2, "replies": [] }, { "id": 473604, "author_name": "", "author_url": "", "post_date": "2019-02-18T08:51:01.193000", "content": "It's very useful, thanks
", "votes": 0, "replies": [] } ] }, { "id": 450570, "author_name": "Rajesh Vikraman", "author_url": "", "post_date": "2019-01-05T09:36:56.700000", "content": "Hi Valeria,\nThere is a authorized_flag column inhistory.csv which I believe indicates whether the transaction is authorized or not. If the transaction is not authorized then it's not a valid transaction . But many of these not authorized transactions have installments ranging from 1 to 10 which is confusing. Please clarify.
", "votes": 23, "replies": [ { "id": 466589, "author_name": "Peter", "author_url": "", "post_date": "2019-02-05T17:00:23.420000", "content": "thanks for asking
", "votes": 1, "replies": [] }, { "id": 1733771, "author_name": "xiaofei007-", "author_url": "", "post_date": "2022-03-24T15:51:27.170000", "content": "thanking for your ask
", "votes": 0, "replies": [] } ] }, { "id": 446376, "author_name": "Manchoi", "author_url": "", "post_date": "2018-12-28T01:15:27.657000", "content": "Hi @elovaleria,\nIn the historical_transactions file, column purcharse_amount is defined as normalized purcharse amount. But I found that the maximum value of the purcharse_amount column is 6010604 and the number of values larger than 1.5 is only account for 0.84% of the whole dataset, so is the outlier caused by missing value or any other reasons?
Hi Valeria,
\n\nIn the data dictionary (corresponding train.csv), target is describe as \"Loyalty numerical score calculated 2 months after historical and evaluation period\". What does this evaluation period mean ? Is it the new merchant transaction period ?
", "votes": 15, "replies": [ { "id": 443062, "author_name": "Zhuang Jia", "author_url": "", "post_date": "2018-12-21T00:49:04.570000", "content": "Good question! I also have this question.
", "votes": 0, "replies": [] }, { "id": 443167, "author_name": "Rui-Qin Li", "author_url": "", "post_date": "2018-12-21T06:13:51.417000", "content": "I have the same problem, too. Besides, I'm so confused because of the feature_1, 2, 3: they're both separated variables, and lack of details.
", "votes": 0, "replies": [] }, { "id": 444544, "author_name": "Manchoi", "author_url": "", "post_date": "2018-12-24T08:30:18.163000", "content": "I am confused about the same problem.
", "votes": 0, "replies": [] }, { "id": 446041, "author_name": "Valeria Elo", "author_url": "", "post_date": "2018-12-27T11:48:03.423000", "content": "Hello all,
\n\nYes, \"Evaluation period\" in target description is the period in which new merchant transactions happen. The loyalty score is calculated after historical and evaluation transaction period are given in order to avoid data leakage.
", "votes": 7, "replies": [] }, { "id": 450059, "author_name": "Gosset", "author_url": "", "post_date": "2019-01-04T08:17:57.710000", "content": "Hi Valeria,
\n\nJust to clarify is \"Evaluation period\" the overall New transaction period which is 2017-03 - 2018-04 or is this specific to each card_id as the reference date is different for different cards ? This would also answer whether loyalty scores for all cards are calculated as same time or different time points. Thanks !
", "votes": 2, "replies": [] }, { "id": 454563, "author_name": "samshipengs", "author_url": "", "post_date": "2019-01-11T20:39:56.450000", "content": "Hi @Valeria Elo, the evaluation period is to avoid data leakage, so did you guys find any sign of leakage?
", "votes": 0, "replies": [] }, { "id": 1826297, "author_name": "Anna Rus", "author_url": "", "post_date": "2022-06-20T08:38:57.127000", "content": "Good question!
", "votes": 0, "replies": [] } ] }, { "id": 429544, "author_name": "Darshak", "author_url": "", "post_date": "2018-11-29T02:10:23.380000", "content": "Hi,
\n\nI am unclear on the relationship between the transactions and \"loyalty score\". The description sets up the problem as a collaborative filtering/ recommendation system where we need to predict which new stores a user would try, but the relation between that and loyalty is unclear to me from data description.
\n\nAlso is loyalty strictly calculated from visits to \"new\" stores in next two months, or a combination of store visits in last 3 months and new store visits? What about repeated visit to stores (in next 2 months) the customer has been to in past 3 months.
\n\nThank you.
", "votes": 15, "replies": [ { "id": 430388, "author_name": "Valeria Elo", "author_url": "", "post_date": "2018-11-30T10:00:42.767000", "content": "Hello Darshak,
\n\nThose are pretty good questions!
\n\nThis competition is not a traditional collaborative filtering/recommendation system challenge since we don’t have a direct definition of what is great merchant or experience. Despite that, we want to link historical information of merchants visits and purchases to a loyalty score, which is basically a business metric that considered both future spending and retention as main components. We expect you to link this information in order to find what triggers higher loyalty scores and you can use any kind of modeling, including recommendation system, but not exclusively. You can be as creative as you want!
\n\nI hope it’s clearer now, but please feel free to ask more questions if needed.
\n\nBest,
", "votes": 29, "replies": [] }, { "id": 430826, "author_name": "Darshak", "author_url": "", "post_date": "2018-12-01T02:29:01.130000", "content": "Thank you Valeria.
", "votes": 0, "replies": [] }, { "id": 447863, "author_name": "Diego Ubuntu", "author_url": "", "post_date": "2018-12-30T18:17:51.700000", "content": "Hello Valeria, I'm a bit confused by your answer.\nI'm assuming that you currently have a formula for the \"loyalty score\", so if you want to link loyalty score with historical data you should provide a time series for the loyalty score (which is not presented) so we can link changes in history (presented in the historical_transaction.csv + new_merchant_transactions.csv) with changes in score (not available in this dataset). If the loyalty score doesn't change over time, there is no point on any of this.
\n\nIf you only provided a snapshot of the loyalty score (for February), then I'm not sure why you provided transactions older than the time lag that the loyalty score uses... unless there is something else going on.... which, in any case, you already know because you have the formula for the loyalty score.... therefore you already have the triggers...
\n\nIf you want to obtain a new formula for the loyalty score, I'm not sure why you don't provide the current formula and describe what are the pitfalls, what do you want to fix, etc. Because, if we develop and train model using the current loyalty score, these models are going to have the same problem that the current score because from the modeling respective the current score is considered the \"truth\"...
\n\nSo, I'm confused... if you have the formula we would need some information about why you want to change it. If you don't have a formula, how did you get the current loyalty scores?
", "votes": 5, "replies": [] } ] }, { "id": 440403, "author_name": "u2", "author_url": "", "post_date": "2018-12-17T14:04:58.387000", "content": "Hi, I have two doubts：\n1、\nThere is a paragraph in the overview,：\n\"Elo has built machine learning models to understand the most important aspects and preferences in their customers’ lifecycle, from food to shopping. But so far none of them is specifically tailored for an individual or profile. This is where you come in“；\nI think the meaning of this sentence is that Kagglers need \"specifically tailored for an individual or profile\" for this project, is that right?\n2、\nWhat is the purpose of obtaining loyalty score, in order to evaluate whether the ELO's recommendation system is effective for people? Use the past history and the data for the next two months to fit the loyalty score. So I think this is not a predictive project. Can give us a clear explanation, what is loyalty, or what is it used for? Thanks
", "votes": 9, "replies": [ { "id": 448016, "author_name": "Rajesh Vikraman", "author_url": "", "post_date": "2018-12-31T04:02:45.163000", "content": "I agree with you. I don't understand why you should predict the loyalty score. Loyalty score is often calculated using some formulae . Are we attempting to find the formulae ELO is using ? . If recommendations based on loyalty score doesn't work for ELO they should look review how they are calculating the loyalty score.
", "votes": 2, "replies": [] }, { "id": 457510, "author_name": "Olivia", "author_url": "", "post_date": "2019-01-17T15:23:05.730000", "content": "I have same question.\nIs there any one who knows what \"loyalty\" means?
", "votes": 0, "replies": [] }, { "id": 463222, "author_name": "Matias Thayer", "author_url": "", "post_date": "2019-01-29T16:29:44.187000", "content": "a dog knows
", "votes": 0, "replies": [] } ] }, { "id": 437626, "author_name": "senkin13", "author_url": "", "post_date": "2018-12-12T08:10:26.273000", "content": "Hi Valeria,
\n\nI found some card_id's purchase_date is earlier than first_active_month,for example a card_id\nC_ID_a51e74c7c1's first_active_monthis 2017-10,but earliest purchase_date is '2017-08-04 12:31:03' in historical_transactions.\nis it reasonable?
", "votes": 6, "replies": [ { "id": 438915, "author_name": "Valeria Elo", "author_url": "", "post_date": "2018-12-14T12:07:45.343000", "content": "Hello senkin13! Although it might seem counterintuitive, there are cases in which it could happen.
", "votes": 0, "replies": [] }, { "id": 440495, "author_name": "Timothy C Willard", "author_url": "", "post_date": "2018-12-17T16:09:14.493000", "content": "Valeria, under what circumstances would something like this occur? Does the first active month feature mean something other than the first month of purchases made using the card, does it mean the first month of reaching a certain threshold in activity or something else?
", "votes": 5, "replies": [] } ] }, { "id": 436706, "author_name": "Michael Maguire", "author_url": "", "post_date": "2018-12-10T19:48:36.813000", "content": "Same questions as @Joshi (sort of). I am assuming the reference date is the date when the loyalty score was calculated. So, 28 February 2018, the last date in the historical transactions, would be the reference date on which the train.target is based, correct? Second question, what reference date are we predicting for? The last date in the new transactions file is 30 April 2018, so I would assume that is the reference date for our predictions? If so, the month_lag is not correct in the new transaction file. Or, are we to predict for 28 February but we use the purchase transactions after that date?
", "votes": 6, "replies": [ { "id": 446160, "author_name": "HarshitMehta", "author_url": "", "post_date": "2018-12-27T16:20:14.270000", "content": "@Michael,\nAccording to data dictionary, It's mentioned that loyalty score is calculated 2 months after historical and evaluation period (new transaction ).
", "votes": 1, "replies": [] } ] }, { "id": 440657, "author_name": "LaughingGirl", "author_url": "", "post_date": "2018-12-17T21:13:03.607000", "content": "HI!\nI have a question like everyone here. Maybe you've discussed it.\nWhat does 999 value of installments mean? Is Number of installments unlimited?\nThanks)
", "votes": 3, "replies": [ { "id": 444215, "author_name": "HarshitMehta", "author_url": "", "post_date": "2018-12-23T14:57:39.800000", "content": "It can be outliers.
", "votes": 1, "replies": [] }, { "id": 446043, "author_name": "Valeria Elo", "author_url": "", "post_date": "2018-12-27T11:49:49.113000", "content": "Hello all,
\n\n999 in installments indicates missing values.
", "votes": 3, "replies": [] }, { "id": 446181, "author_name": "HarshitMehta", "author_url": "", "post_date": "2018-12-27T16:53:14.007000", "content": "Hi Valeria, What does -1 in installments imply ?\nThanks!
", "votes": 1, "replies": [] }, { "id": 446189, "author_name": "Valeria Elo", "author_url": "", "post_date": "2018-12-27T17:10:16.387000", "content": "Hi! \"-1\" are also missing values.
", "votes": 5, "replies": [] }, { "id": 454207, "author_name": "Badak dan Ikan", "author_url": "", "post_date": "2019-01-11T08:41:56.183000", "content": "So is there any different between 999 and -1 as missing values? Or it is kind of same treatment like \"Okay start right now, we never use -1 as missing values, using 999 instead\" (or vice versa)?
I thought -1 could be a refund :-)\nBtw, I don't see also the difference between 0 and 1 installment, understanding 1 installment as the complete pay of the transaction. Or maybe we should understand that there are 2 monthly payments?
", "votes": 1, "replies": [] } ] }, { "id": 438907, "author_name": "Leandro Bianchini", "author_url": "", "post_date": "2018-12-14T11:40:56.463000", "content": "Valéria, please, could you confirm that newMerchantTransactions.csv file doesn't account for all transactions a given cardId did in the period but only the transactions that were done by that cardId in a merchantId that never happened before?
", "votes": 3, "replies": [ { "id": 438914, "author_name": "Valeria Elo", "author_url": "", "post_date": "2018-12-14T11:59:17.737000", "content": "Hello Leandro,
\n\nYou are correct. 'newmerchanttransactions.csv' accounts only for transactions in merchants never visited before for each card_id, therefore it doesn't have all transactions.
", "votes": 0, "replies": [] }, { "id": 438925, "author_name": "Leandro Bianchini", "author_url": "", "post_date": "2018-12-14T12:31:47.800000", "content": "Thanks! Second thoughts then... what kind of information could be hidden in the actual transactions set that each cardId did during the \"new\" period of time? I mean, are you guys doing any kind of filtering or there could be cardIds that didn't buy anything even from their previous merchantIds?
", "votes": 1, "replies": [] } ] }, { "id": 430126, "author_name": "Lucas Cusinato", "author_url": "", "post_date": "2018-11-29T21:44:36.047000", "content": "How does \"loyalty score\" not depend on the merchant id? What kind of recommendation this is?
", "votes": 3, "replies": [ { "id": 430380, "author_name": "Darshak", "author_url": "", "post_date": "2018-11-30T09:41:17.677000", "content": "I think here the loyalty is measured toward Elo the payment company and not the merchant.
", "votes": 7, "replies": [] }, { "id": 445402, "author_name": "Manchoi", "author_url": "", "post_date": "2018-12-26T11:41:18.487000", "content": "but when it's not depend on the merchant_id, what the meaning of merchant.csv file?
", "votes": 1, "replies": [] } ] }, { "id": 435332, "author_name": "Joshi", "author_url": "", "post_date": "2018-12-07T21:46:33.360000", "content": "Hi Valeria, \nI had a quick question about the dates in historical transactions and the new transactions. Is there any significance in the reference date 2018-02?
", "votes": 4, "replies": [] }, { "id": 430638, "author_name": "Dimosthenis Karaflos", "author_url": "", "post_date": "2018-11-30T19:10:26.910000", "content": "Hello Valeria,\ncan you please clarify the target? are we supposed to predict the loyalty score for the dates 1/5/2018 to 30/6/2018?
", "votes": 1, "replies": [ { "id": 430701, "author_name": "Addison Howard", "author_url": "", "post_date": "2018-11-30T21:27:16.313000", "content": "That is correct - you are predicting loyalty score.
", "votes": 2, "replies": [] }, { "id": 430713, "author_name": "Dimosthenis Karaflos", "author_url": "", "post_date": "2018-11-30T21:46:49.430000", "content": "thank you for your clarification
", "votes": 0, "replies": [] }, { "id": 436632, "author_name": "samshipengs", "author_url": "", "post_date": "2018-12-10T17:02:48.557000", "content": "@Dimosthenis Karaflos just double check, so we are supposed to predict the target (loyalty score) for dates 1/5/2018 to 30/6/2018? where did you get the dates from?
\n\n@Valeria Elo In the data description, \"historical_transactions.csv - up to 3 months' worth of historical transactions for each card_id\", but when I checked the data, it seems like some card_ids has way more than 3 month purchase_dates, did I misunderstand something?
", "votes": 1, "replies": [] }, { "id": 436646, "author_name": "Valeria Elo", "author_url": "", "post_date": "2018-12-10T17:38:12.627000", "content": "File \"historicaltransactions.csv\" has at least 3 months of historical data for each card.
", "votes": 1, "replies": [] } ] }, { "id": 443117, "author_name": "tien bui", "author_url": "", "post_date": "2018-12-21T03:34:06.833000", "content": "Hi,\nI have a question? what is the meaning of month_lag in history and new transaction?
", "votes": 2, "replies": [ { "id": 445352, "author_name": "Shuhan Lu", "author_url": "", "post_date": "2018-12-26T09:00:42.297000", "content": "I have the same question
", "votes": 0, "replies": [] }, { "id": 446158, "author_name": "HarshitMehta", "author_url": "", "post_date": "2018-12-27T16:11:53.063000", "content": "Month_lag is the difference in months between purchase date month and reference date month, I'm assuming reference month is feb 2018.
", "votes": 1, "replies": [] }, { "id": 447722, "author_name": "Aditya Kansal", "author_url": "", "post_date": "2018-12-30T11:58:06.203000", "content": "Based on other comments,I just figured out what reference date is. It is the date when loyalty score is calculated.\nSo , if a transaction as a purchase date as 1 June 2018 and reference date as 1 Feb 2018, We would have month_lag as their difference which would be -8.
\n\nPlease free to add/edit comments to it if my understanding is incorrect.
", "votes": 1, "replies": [] }, { "id": 453801, "author_name": "Wei Zhao", "author_url": "", "post_date": "2019-01-10T19:58:29.587000", "content": "@HarshitMehta No, if you look at data, reference date/month can be different for different customers(card_id).
", "votes": 1, "replies": [] } ] }, { "id": 437983, "author_name": "Matias Thayer", "author_url": "", "post_date": "2018-12-12T22:16:54.393000", "content": "a question please: \nCan you explain what exactly \"new_merchant_transactions.csv contains the transactions at new merchants (merchant_ids that this particular card_id has not yet visited) over a period of two months\" means? (from the data description). How is it possible that a card_id hasn't visited the merchant and has a purchase_amount?
\n\nAlso, it's mentioned that the file contains transactions \"over a period of two months\", but if I get the min and max of the purchase_date, I get 2017-03-01 03:24:51 and 2018-04-30 23:59:59 respectively, which is more than a year!
\n\nThanks
", "votes": 2, "replies": [ { "id": 438098, "author_name": "Zhuang Jia", "author_url": "", "post_date": "2018-12-13T05:09:52.527000", "content": "I think the time range is more tant two months. Someone has asked the same question in this thread.(See below)\n
Thank you, Zhuang Jia.
\n\nHow about the other question? do you know what is the difference between historical transactions and new merchant transactions?\nnew merchant transactions: \"merchant ids that this particular card id has not yet visited\". How can a transaction occur if the customer hasn't visited the merchant? Are they online transactions maybe?
", "votes": 1, "replies": [] }, { "id": 438171, "author_name": "Zhuang Jia", "author_url": "", "post_date": "2018-12-13T07:55:48.077000", "content": "I also have many questions about this two files. I guess they collect the transactions for a period of time(2~14 months for each card_id) and put this information to historical transactions. And then they focus on the new merchant each card_id visits, so they only collect information about merchant that card_id not yet visited before and put this information to new_merchant_transactions.csv.
My English is not very good. I hope I have make my opinion clear. I do some check about this two files. You can have a look if you like. Basic info about historical/new transactions
", "votes": 5, "replies": [] } ] }, { "id": 437478, "author_name": "Sarada Lee", "author_url": "", "post_date": "2018-12-12T01:52:18.580000", "content": "Hi Valeria,\nAccording to the Data Dictionary, merchant id should be an unique merchant identifier. However, there are 334,633 unique merchant IDs but there are 334,696 rows in the merchants.csv. Some merchant_id repeated 4 times.
\nCould you please review the merchants.csv and re-publish it ASAP?\nThanks
Hi Valeria ....\nWe all teams love Elo.
", "votes": 1, "replies": [] }, { "id": 435046, "author_name": "Giri", "author_url": "", "post_date": "2018-12-07T11:46:14.100000", "content": "Dear, I hope this can be built in Python as well right? pls advise
", "votes": -6, "replies": [ { "id": 454120, "author_name": "eajamat", "author_url": "", "post_date": "2019-01-11T06:37:13.787000", "content": "Language is never the constraint.
", "votes": 1, "replies": [] } ] }, { "id": 1246286, "author_name": "Amit Misra", "author_url": "", "post_date": "2021-03-20T16:04:03.403000", "content": "Hello Valeria Mam,
\nthis competition is featured. Thanks for providing this meaningful competition, I don't know where to see the winner's code , will the winner's code be publicly sharing?
", "votes": 0, "replies": [] }, { "id": 477684, "author_name": "hanzh0420", "author_url": "", "post_date": "2019-02-25T05:04:31.350000", "content": "Hi Will you give one or two more days on the submission deadline since kaggle was down for one day?
", "votes": 0, "replies": [ { "id": 477692, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-02-25T05:16:35.110000", "content": "Hi - we will not be adjusting competition timelines for the Kaggle site maintenance
", "votes": 0, "replies": [] } ] }, { "id": 475541, "author_name": "Leigh", "author_url": "", "post_date": "2019-02-20T21:44:38.303000", "content": "Hi @elovaleria Some city have more than one state. City 331 in state 16 and 20, how it possible?
", "votes": 0, "replies": [] }, { "id": 466292, "author_name": "Amber Arriaga", "author_url": "", "post_date": "2019-02-05T04:05:34.147000", "content": "Could someone please elaborate on the values for columns feature 1, feature 2, and feature 3? Are they suppose to be categorical or numeric. I am not understanding what it means by Anonymized card categorical feature.
", "votes": 0, "replies": [ { "id": 466295, "author_name": "HarshitMehta", "author_url": "", "post_date": "2019-02-05T04:15:46.883000", "content": "Try all the possible ways you can do with these features, whether it's treating them as numerical, or categorical, or using some interaction between them or with some other features, as long your score is improving. For me treating them as categorical and finding interaction w.r.t some other features, helped to improve the score.
", "votes": 3, "replies": [] }, { "id": 466301, "author_name": "Amber Arriaga", "author_url": "", "post_date": "2019-02-05T04:38:56.923000", "content": "Thank you for the help and insights!
", "votes": 1, "replies": [] } ] }, { "id": 465641, "author_name": "YossiBondarev", "author_url": "", "post_date": "2019-02-03T16:58:52.933000", "content": "Hello everyone,\nI have seen some discussions regarding this matter and still I am not able to realize WHEN the score is calculated.
\n\nThe way I see it until some period X all customers had some transactions made (that may or may not be presented in the data) . Afterwards, ELO has used some ML model to advertise to customers ,then new transactions occurred (which definitely should be in the data) for some period Y . \nat the end of periods X+Y ELO has presented a \"loyalty score \"\nCould someone please explain to me how to obtain X +Y, X,Y from the data or at least contribute with some relevant idea.\nOtherwise, I don't understand how are we suppose to obtain any insights from the transaction data.
\n\nThank you
", "votes": 0, "replies": [] }, { "id": 465533, "author_name": "Ersin Erkan", "author_url": "", "post_date": "2019-02-03T11:42:56.833000", "content": "Hi Valéria, this challenge seems to be perfect as a topic for my thesis in a MBA business analytics course at the FGV. Can I use the data for this purpose besides competing?
", "votes": 0, "replies": [] }, { "id": 460896, "author_name": "Francisco Javier", "author_url": "", "post_date": "2019-01-24T16:26:32.947000", "content": "Hi Valeria,
\n\nWhere is the significate that a merchant appear two times or four times in merchant table?
\n\nThanks
", "votes": 0, "replies": [] }, { "id": 456970, "author_name": "vijayabaskar santhanam", "author_url": "", "post_date": "2019-01-16T20:01:42.847000", "content": "Hi Valeria,
\n\nCan you explain about the active_month_lagx column in merchant data. Like how it is calculated.
", "votes": 0, "replies": [] }, { "id": 456147, "author_name": "Dimosthenis Karaflos", "author_url": "", "post_date": "2019-01-15T08:27:46.893000", "content": "Hi Valeria,\nis the historical and evaluation period the same for the train and test dataset?\nSpecifically, is the date range from 2018-05 to 2018-06?
", "votes": 0, "replies": [] }, { "id": 454145, "author_name": "eajamat", "author_url": "", "post_date": "2019-01-11T07:08:08.127000", "content": "Hi all, the train data consists of feature_1, feature_2 & feature_3. As these columns are absent from any other data given, does that mean we need to decide upon the 3 features that will impact the loyalty score(assuming 'loyalty score=target') - in that case how do we ensure that target values are true for any 3 features selected ?
", "votes": 0, "replies": [] }, { "id": 452349, "author_name": "", "author_url": "", "post_date": "2019-01-08T15:56:08.920000", "content": "", "votes": 0, "replies": [ { "id": 452377, "author_name": "", "author_url": "", "post_date": "2019-01-08T16:27:08.150000", "content": "", "votes": 1, "replies": [] }, { "id": 452413, "author_name": "", "author_url": "", "post_date": "2019-01-08T17:32:17.667000", "content": "", "votes": 1, "replies": [] }, { "id": 452580, "author_name": "", "author_url": "", "post_date": "2019-01-08T23:10:25.483000", "content": "", "votes": 0, "replies": [] }, { "id": 452589, "author_name": "", "author_url": "", "post_date": "2019-01-08T23:27:22.310000", "content": "", "votes": 1, "replies": [] }, { "id": 452602, "author_name": "", "author_url": "", "post_date": "2019-01-09T00:01:10.753000", "content": "", "votes": 2, "replies": [] }, { "id": 452607, "author_name": "", "author_url": "", "post_date": "2019-01-09T00:07:24.273000", "content": "", "votes": 2, "replies": [] }, { "id": 452616, "author_name": "", "author_url": "", "post_date": "2019-01-09T00:31:28.130000", "content": "", "votes": 0, "replies": [] }, { "id": 453144, "author_name": "", "author_url": "", "post_date": "2019-01-09T18:28:40.023000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 449970, "author_name": "", "author_url": "", "post_date": "2019-01-04T03:42:18.647000", "content": "", "votes": 0, "replies": [ { "id": 449983, "author_name": "", "author_url": "", "post_date": "2019-01-04T04:14:46.530000", "content": "", "votes": 0, "replies": [] }, { "id": 450105, "author_name": "", "author_url": "", "post_date": "2019-01-04T10:10:19.430000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 448128, "author_name": "", "author_url": "", "post_date": "2018-12-31T09:54:34.897000", "content": "", "votes": 0, "replies": [] }, { "id": 447115, "author_name": "", "author_url": "", "post_date": "2018-12-29T06:54:55.907000", "content": "", "votes": 0, "replies": [] }, { "id": 447030, "author_name": "", "author_url": "", "post_date": "2018-12-29T02:44:24.830000", "content": "", "votes": 0, "replies": [ { "id": 447169, "author_name": "", "author_url": "", "post_date": "2018-12-29T09:24:04.303000", "content": "", "votes": 1, "replies": [] }, { "id": 449553, "author_name": "", "author_url": "", "post_date": "2019-01-03T10:47:52.280000", "content": "", "votes": 1, "replies": [] }, { "id": 452619, "author_name": "", "author_url": "", "post_date": "2019-01-09T00:40:39.390000", "content": "", "votes": 0, "replies": [] }, { "id": 454688, "author_name": "", "author_url": "", "post_date": "2019-01-12T02:24:47.410000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 445704, "author_name": "", "author_url": "", "post_date": "2018-12-27T00:46:06.997000", "content": "", "votes": 0, "replies": [] }, { "id": 445498, "author_name": "", "author_url": "", "post_date": "2018-12-26T15:24:47.180000", "content": "", "votes": 0, "replies": [] }, { "id": 444239, "author_name": "", "author_url": "", "post_date": "2018-12-23T16:11:21.733000", "content": "", "votes": 0, "replies": [] }, { "id": 443764, "author_name": "", "author_url": "", "post_date": "2018-12-22T11:31:38.583000", "content": "", "votes": 0, "replies": [] }, { "id": 443723, "author_name": "", "author_url": "", "post_date": "2018-12-22T08:36:54.203000", "content": "", "votes": 0, "replies": [ { "id": 445398, "author_name": "", "author_url": "", "post_date": "2018-12-26T11:31:36.313000", "content": "", "votes": 0, "replies": [] }, { "id": 452299, "author_name": "", "author_url": "", "post_date": "2019-01-08T14:17:18.883000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 440032, "author_name": "", "author_url": "", "post_date": "2018-12-16T22:46:18.610000", "content": "", "votes": 0, "replies": [ { "id": 445401, "author_name": "", "author_url": "", "post_date": "2018-12-26T11:39:57", "content": "", "votes": 0, "replies": [] } ] }, { "id": 437655, "author_name": "", "author_url": "", "post_date": "2018-12-12T09:29:50.160000", "content": "", "votes": 0, "replies": [ { "id": 438100, "author_name": "", "author_url": "", "post_date": "2018-12-13T05:12:33.987000", "content": "", "votes": 2, "replies": [] }, { "id": 445407, "author_name": "", "author_url": "", "post_date": "2018-12-26T11:52:44.547000", "content": "", "votes": 3, "replies": [] }, { "id": 445411, "author_name": "", "author_url": "", "post_date": "2018-12-26T11:59:49.037000", "content": "", "votes": -1, "replies": [] }, { "id": 447740, "author_name": "", "author_url": "", "post_date": "2018-12-30T12:36:00.960000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2466192, "author_name": "", "author_url": "", "post_date": "2023-10-03T16:20:38.420000", "content": "", "votes": 0, "replies": [] }, { "id": 2119147, "author_name": "", "author_url": "", "post_date": "2023-01-28T14:13:51.087000", "content": "", "votes": 0, "replies": [] }, { "id": 482108, "author_name": "", "author_url": "", "post_date": "2019-03-02T10:42:17.083000", "content": "", "votes": 0, "replies": [] }, { "id": 479484, "author_name": "", "author_url": "", "post_date": "2019-02-27T07:25:44.913000", "content": "", "votes": 0, "replies": [] }, { "id": 477357, "author_name": "", "author_url": "", "post_date": "2019-02-24T11:45:33.730000", "content": "", "votes": 0, "replies": [] }, { "id": 446866, "author_name": "", "author_url": "", "post_date": "2018-12-28T19:57:44.737000", "content": "", "votes": 0, "replies": [] }, { "id": 436844, "author_name": "", "author_url": "", "post_date": "2018-12-11T02:41:02.620000", "content": "", "votes": 0, "replies": [] }, { "id": 434975, "author_name": "", "author_url": "", "post_date": "2018-12-07T08:45:12.517000", "content": "", "votes": 0, "replies": [] }, { "id": 484636, "author_name": "", "author_url": "", "post_date": "2019-03-06T09:07:40.977000", "content": "", "votes": 1, "replies": [] }, { "id": 464770, "author_name": "", "author_url": "", "post_date": "2019-02-01T13:28:00.373000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "429468": "Welcome to Elo Merchant recommendation competition!\n \nWe are really enthusiastic to host this competition and we believe kagglers are going to join it and help us make a difference in our customers’ lives.\n \nPlease feel free to ask your questions in this thread, we are happy to help!\n\nWe wish you all the best of luck! Have fun!\n\nTeam Elo\n", "444375": "My take on these questions (which don't seem to be answered by the Admins). Note this is all my opinion after going through the data for a while.\n\n1. The loyalty score has little, if anything, to do with the data we have been given. Elo wants to see if the transaction and merchant data they have can be used to predict this loyalty score. And I think we can use the given data and external data in any way we choose.\n\n2. For the transaction data files, ignore the descriptions on the data page. No one can figure out what they mean. I combined the historical and new files and used the complete file to build aggregates and other features. You need to be careful about the mysterious reference date when doing this - some cards have different reference dates than others.\n\n3. Reference date. I believe this is the date that the loyalty score was calculated. There is no information (that I have seen) that states when the reference date is. I used month\\_lag to determine the \"reference month\" for each card (reference\\_month = purchase\\_date_month - month\\_lag). Then I defined the reference date as the last day of the reference month, but there are other ways to do it also.\n\n4. Merchants. The merchant data is a little messy as there are disconnects between the merchant file and the merchant columns in the transaction data. This is a very real-world problem and you'll have to decide which file to go with. I deleted the common columns out of the transaction data and then replaced them using the merchants file (then both agree and every merchant has one subsector, city, state, etc.). That is one way, but may not be the best way, to proceed.\n\nHope that helps a bit.\n\n", "450570": "Hi Valeria,\nThere is a authorized_flag column inhistory.csv which I believe indicates whether the transaction is authorized or not. If the transaction is not authorized then it's not a valid transaction . But many of these not authorized transactions have installments ranging from 1 to 10 which is confusing. Please clarify.", "446376": "Hi @elovaleria,\nIn the historical_transactions file, column `purcharse_amount` is defined as `normalized purcharse amount`. But I found that the maximum value of the `purcharse_amount` column is `6010604` and the number of values larger than `1.5` is only account for `0.84%` of the whole dataset, so is the outlier caused by `missing value` or any other reasons?", "442869": "Hi Valeria,\n\nIn the data dictionary (corresponding train.csv), target is describe as \"Loyalty numerical score calculated 2 months after historical and evaluation period\". What does this evaluation period mean ? Is it the new merchant transaction period ? ", "429544": "Hi,\n\nI am unclear on the relationship between the transactions and \"loyalty score\". The description sets up the problem as a collaborative filtering/ recommendation system where we need to predict which new stores a user would try, but the relation between that and loyalty is unclear to me from data description.\n\nAlso is loyalty strictly calculated from visits to \"new\" stores in next two months, or a combination of store visits in last 3 months and new store visits? What about repeated visit to stores (in next 2 months) the customer has been to in past 3 months.\n\nThank you.", "440403": "Hi, I have two doubts：\n1、\nThere is a paragraph in the overview,：\n\"Elo has built machine learning models to understand the most important aspects and preferences in their customers’ lifecycle, from food to shopping. But so far none of them is specifically tailored for an individual or profile. This is where you come in“；\nI think the meaning of this sentence is that Kagglers need \"specifically tailored for an individual or profile\" for this project, is that right?\n2、\nWhat is the purpose of obtaining loyalty score, in order to evaluate whether the ELO's recommendation system is effective for people? Use the past history and the data for the next two months to fit the loyalty score. So I think this is not a predictive project. Can give us a clear explanation, what is loyalty, or what is it used for? Thanks", "437626": "Hi Valeria,\n\nI found some card_id's purchase_date is earlier than first_active_month,for example a card_id\nC_ID_a51e74c7c1's first_active_monthis 2017-10,but earliest purchase_date is '2017-08-04 12:31:03' in historical_transactions.\nis it reasonable?", "436706": "Same questions as @Joshi (sort of). I am assuming the reference date is the date when the loyalty score was calculated. So, 28 February 2018, the last date in the historical transactions, would be the reference date on which the train.target is based, correct? Second question, what reference date are we predicting for? The last date in the new transactions file is 30 April 2018, so I would assume that is the reference date for our predictions? If so, the month_lag is not correct in the new transaction file. Or, are we to predict for 28 February but we use the purchase transactions after that date?", "440657": "HI!\nI have a question like everyone here. Maybe you've discussed it.\nWhat does 999 value of installments mean? Is Number of installments unlimited?\nThanks)", "438907": "Valéria, please, could you confirm that newMerchantTransactions.csv file doesn't account for all transactions a given cardId did in the period but only the transactions that were done by that cardId in a merchantId that never happened before?", "430126": "How does \"loyalty score\" not depend on the merchant id? What kind of recommendation this is?", "435332": "Hi Valeria, \nI had a quick question about the dates in historical transactions and the new transactions. Is there any significance in the reference date 2018-02?", "430638": "Hello Valeria,\ncan you please clarify the target? are we supposed to predict the loyalty score for the dates 1/5/2018 to 30/6/2018? ", "443117": "Hi,\nI have a question? what is the meaning of month_lag in history and new transaction?", "437983": "a question please: \nCan you explain what exactly \"new\\_merchant\\_transactions.csv contains the transactions at new merchants (merchant_ids that this particular card\\_id has not yet visited) over a period of two months\" means? (from the data description). How is it possible that a card_id hasn't visited the merchant and has a **purchase\\_amount**?\n\nAlso, it's mentioned that the file contains transactions \"over a period of two months\", but if I get the min and max of the purchase_date, I get 2017-03-01 03:24:51 and 2018-04-30 23:59:59 respectively, which is more than a year!\n\nThanks", "437478": "Hi Valeria,\nAccording to the Data Dictionary, merchant id should be an unique merchant identifier. However, there are 334,633 unique merchant IDs but there are 334,696 rows in the merchants.csv. Some merchant_id repeated 4 times. \nCould you please review the merchants.csv and re-publish it ASAP?\nThanks", "461836": "Hi Valeria ....\nWe all teams love Elo.", "435046": "Dear, I hope this can be built in Python as well right? pls advise", "1246286": "Hello Valeria Mam,\n\n1. Can you give some idea of categories.", "487066": "this competition is featured. Thanks for providing this meaningful competition, I don't know where to see the winner's code , will the winner's code be publicly sharing? ", "477684": "Hi Will you give one or two more days on the submission deadline since kaggle was down for one day? ", "475541": "Hi @elovaleria Some city have more than one state. City 331 in state 16 and 20, how it possible?", "466292": "Could someone please elaborate on the values for columns feature 1, feature 2, and feature 3? Are they suppose to be categorical or numeric. I am not understanding what it means by Anonymized card categorical feature.", "465641": "Hello everyone,\nI have seen some discussions regarding this matter and still I am not able to realize WHEN the score is calculated.\n \nThe way I see it until some period X all customers had some transactions made (that may or may not be presented in the data) . Afterwards, ELO has used some ML model to advertise to customers ,then new transactions occurred (which definitely should be in the data) for some period Y . \nat the end of periods X+Y ELO has presented a \"loyalty score \"\nCould someone please explain to me how to obtain X +Y, X,Y from the data or at least contribute with some relevant idea.\nOtherwise, I don't understand how are we suppose to obtain any insights from the transaction data.\n\nThank you", "465533": "Hi Valéria, this challenge seems to be perfect as a topic for my thesis in a MBA business analytics course at the FGV. Can I use the data for this purpose besides competing? ", "460896": "Hi Valeria,\n\nWhere is the significate that a merchant appear two times or four times in merchant table? \n\nThanks", "456970": "Hi Valeria,\n\nCan you explain about the active_month_lagx column in merchant data. Like how it is calculated.", "456147": "Hi Valeria,\nis the historical and evaluation period the same for the train and test dataset?\nSpecifically, is the date range from 2018-05 to 2018-06?\n", "454145": "Hi all, the train data consists of feature_1, feature_2 & feature_3. As these columns are absent from any other data given, does that mean we need to decide upon the 3 features that will impact the loyalty score(assuming 'loyalty score=target') - in that case how do we ensure that target values are true for any 3 features selected ?", "452349": "I looked through the discussions carefully but I'm still confused with 'reference date'. Is 'loyalty score' calculated at reference date, or is it calculated 2 months after the last date in new merchant transactions for each card id? If latter is true, what is the use of 'reference date'?", "449970": "Hi @elovaleria, is `-1` in `subsector_id`, `city_id`, `state_id`, `merchant_category_id` missing values ?", "448128": "Hi @elovaleria, just curious can we assume all the merchants are having MCC (merchant category id here in dataset) are serving for Hotel,Restaurant category? just to avoid unnecessary data..\n\nThanks", "447115": "Hello, how do you use the merchant data?", "447030": "Hi, Valeria Elo, what's the difference between -1 and 999 in installment column? And what's the difference between 0 and 1 in installment column?", "445704": "Hi Valeria,\n\nI have two questions about the score.\n\nFrom the competition description i can understand that there is not any model to produce the loyalty scores. Hence, are the target scores derived from a determenistic formula or from a vanilla model?\n\non the historical transactions and new merchants datasets, has the normalization been applied by the total column values by each seperated dataset or the normalization has been applied to the total dataset?", "445498": "Hi Valeria,\n\nWhy does not all users have \"month_lag == 0\" in historical_transactions data??\n\nThanks", "444239": "Hi Valeria,\n\nCould you please explain more about \"reference date\" which appearing in the description of the field \"month_lag\" as saying that \"month lag to reference date\"?\n\nAlso, i believe that it should be the \"reference date\" in the merchants data set, i am a little confused, could you help me?\n\nMy English is not good, sorry for that.\n\nThanks : )", "443764": "Hi Valeria,\n\nCan you explain that the data in the table 'merchants' is based on '2018-04-30' or someday else?\n\nBest,\nZhang.", "443723": "Hi Valeria,\n\nIn *merchants.csv* every merchant_id has 1 combination of [\"merchant_id\",\"merchant_category_id\",\"subsector_id\",\"city_id\",\"state_id\"]\n\nBut in *historical/new_merchants_transactions*, rows with this merchant_id have different combiantions of [\"merchant_id\",\"merchant_category_id\",\"subsector_id\",\"city_id\",\"state_id\"]\n\nDoes it make sense? How a merchant_id can have different identifiers in other datasets than merchant.csv?\nNote: Attached photo with example (table 1 from merchant.csv, table 2 from historical/new_merchants_transactions)", "440032": "Hi, I have just started analyzing transaction information provided on the competition. If possible, can you please elaborate on how is normalized purchase amount calculated. Specifically:\n- Is it mean normalized?\n- Across which dimension is the normalization done (is it across card_id only or card_id X merchant_id or such). Also, is the normalization done using the same data or different reference data?\n\nBest,\nPawas", "437655": "why many people not use nn for this task ? who can tell me answer", "2466192": "", "2119147": "", "482108": "", "479484": "", "477357": "", "446866": "", "436844": "", "434975": "", "484636": "Thank you, competition was nice!", "464770": "Thank you! :)" } }