Might be useful for someone: I decided to play around with the holiday info, tried joining it with the main table - and it didn't work. Not inner, not outer, matching blew up as well... I was slowly starting to question my (1.01. level required here) sql-type skills, until it dawned on me: maybe there are duplicates? Lo and behold, indeed there are:
\n\n> xholiday <- fread('../input/holidays_events.csv', select = c('date', 'type', 'locale', 'transferred'))\n> dim(xholiday)\n[1] 350 4\n> dim(unique(xholiday))\n[1] 337 4\nSo there you have it.
", "messages": [ { "id": "235619", "postDate": "10/25/2017 18:52:21", "content": "Might be useful for someone: I decided to play around with the holiday info, tried joining it with the main table - and it didn't work. Not inner, not outer, matching blew up as well... I was slowly starting to question my (1.01. level required here) sql-type skills, until it dawned on me: maybe there are duplicates? Lo and behold, indeed there are:
\n\n> xholiday <- fread('../input/holidays_events.csv', select = c('date', 'type', 'locale', 'transferred'))\n> dim(xholiday)\n[1] 350 4\n> dim(unique(xholiday))\n[1] 337 4\nSo there you have it.
", "rawMarkdown": "Might be useful for someone: I decided to play around with the holiday info, tried joining it with the main table - and it didn't work. Not inner, not outer, matching blew up as well... I was slowly starting to question my (1.01. level required here) sql-type skills, until it dawned on me: maybe there are duplicates? Lo and behold, indeed there are:\n\n > xholiday <- fread('../input/holidays_events.csv', select = c('date', 'type', 'locale', 'transferred'))\n > dim(xholiday)\n [1] 350 4\n > dim(unique(xholiday))\n [1] 337 4\n\nSo there you have it.", "votes": null }, { "id": "235872", "postDate": "10/26/2017 07:47:52", "content": "Holidays in current data sets cannot be joined directly. However, the 'locale_name' column in holiday list contains reference of city name, state name and country name.
\n\nSo possible joining strategy could be,
\n\nFeel free to correct me if anything seems to be irrelevant in above.
", "rawMarkdown": "Holidays in current data sets cannot be joined directly. However, the 'locale_name' column in holiday list contains reference of city name, state name and country name.\n\nSo possible joining strategy could be,\n\n 1. If 'locale_name' is some city, append respective 'locale' and 'transferred' columns from holidays to all the stores with given located in that city on that date in train data joined with stores\n 2. If 'locale_name' is some state, append respective 'locale' and 'transferred' columns from holidays to all the stores with given located in that state on that date in train data joined with stores\n 3. Otherwise, append respective 'locale' and 'transferred' columns from holidays to all the stores on that date.\n\nFeel free to correct me if anything seems to be irrelevant in above.", "votes": null }, { "id": "239700", "postDate": "11/04/2017 08:00:24", "content": "One way of finding the holidays is to find missing dates in transactions.csv. There are indeed some missing dates in the transactions data, e.g. 3/22/2016 and 3/23/2016 for store number 17. The actual national holiday is on 3/25/2016. The holidays' data could be used for \"test\" file. But, how to find the mysterious breaks like 3/22/16 and 3/23/16 which are not there in the stores other than 17. Or do we just ignore the transactions data? Can someone please clarify?
", "rawMarkdown": "One way of finding the holidays is to find missing dates in transactions.csv. There are indeed some missing dates in the transactions data, e.g. 3/22/2016 and 3/23/2016 for store number 17. The actual national holiday is on 3/25/2016. The holidays' data could be used for \"test\" file. But, how to find the mysterious breaks like 3/22/16 and 3/23/16 which are not there in the stores other than 17. Or do we just ignore the transactions data? Can someone please clarify?", "votes": null }, { "id": "240392", "postDate": "11/06/2017 13:23:44", "content": "Possibly the mysterious breaks are due to the fact that store 17 was closed that day. E.g. for renovation.
\n\nI had a lot of trouble with the holidays dataset as well. Especially I'm wondering if an event can be interpreted as \"holiday\" and why there are work days mentioned.
", "rawMarkdown": "Possibly the mysterious breaks are due to the fact that store 17 was closed that day. E.g. for renovation.\n\nI had a lot of trouble with the holidays dataset as well. Especially I'm wondering if an event can be interpreted as \"holiday\" and why there are work days mentioned.", "votes": null }, { "id": "240684", "postDate": "11/07/2017 06:52:54", "content": "Thanks, CEbel! To be fair, the test data too has missing items from id 128 867 504 to id 128 900 000.
", "rawMarkdown": "Thanks, CEbel! To be fair, the test data too has missing items from id 128 867 504 to id 128 900 000.", "votes": null } ], "comments": [ { "id": 235872, "author_name": "varunvyas", "author_url": "", "post_date": "10/26/2017 07:47:52", "content": "Holidays in current data sets cannot be joined directly. However, the 'locale_name' column in holiday list contains reference of city name, state name and country name.
\n\nSo possible joining strategy could be,
\n\nFeel free to correct me if anything seems to be irrelevant in above.
", "votes": null, "replies": [] }, { "id": 239700, "author_name": "robby75", "author_url": "", "post_date": "11/04/2017 08:00:24", "content": "One way of finding the holidays is to find missing dates in transactions.csv. There are indeed some missing dates in the transactions data, e.g. 3/22/2016 and 3/23/2016 for store number 17. The actual national holiday is on 3/25/2016. The holidays' data could be used for \"test\" file. But, how to find the mysterious breaks like 3/22/16 and 3/23/16 which are not there in the stores other than 17. Or do we just ignore the transactions data? Can someone please clarify?
", "votes": null, "replies": [] }, { "id": 240392, "author_name": "christophebel", "author_url": "", "post_date": "11/06/2017 13:23:44", "content": "Possibly the mysterious breaks are due to the fact that store 17 was closed that day. E.g. for renovation.
\n\nI had a lot of trouble with the holidays dataset as well. Especially I'm wondering if an event can be interpreted as \"holiday\" and why there are work days mentioned.
", "votes": null, "replies": [] }, { "id": 240684, "author_name": "robby75", "author_url": "", "post_date": "11/07/2017 06:52:54", "content": "Thanks, CEbel! To be fair, the test data too has missing items from id 128 867 504 to id 128 900 000.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "235619": "Might be useful for someone: I decided to play around with the holiday info, tried joining it with the main table - and it didn't work. Not inner, not outer, matching blew up as well... I was slowly starting to question my (1.01. level required here) sql-type skills, until it dawned on me: maybe there are duplicates? Lo and behold, indeed there are:\n\n > xholiday <- fread('../input/holidays_events.csv', select = c('date', 'type', 'locale', 'transferred'))\n > dim(xholiday)\n [1] 350 4\n > dim(unique(xholiday))\n [1] 337 4\n\nSo there you have it.", "235872": "Holidays in current data sets cannot be joined directly. However, the 'locale_name' column in holiday list contains reference of city name, state name and country name.\n\nSo possible joining strategy could be,\n\n 1. If 'locale_name' is some city, append respective 'locale' and 'transferred' columns from holidays to all the stores with given located in that city on that date in train data joined with stores\n 2. If 'locale_name' is some state, append respective 'locale' and 'transferred' columns from holidays to all the stores with given located in that state on that date in train data joined with stores\n 3. Otherwise, append respective 'locale' and 'transferred' columns from holidays to all the stores on that date.\n\nFeel free to correct me if anything seems to be irrelevant in above.", "239700": "One way of finding the holidays is to find missing dates in transactions.csv. There are indeed some missing dates in the transactions data, e.g. 3/22/2016 and 3/23/2016 for store number 17. The actual national holiday is on 3/25/2016. The holidays' data could be used for \"test\" file. But, how to find the mysterious breaks like 3/22/16 and 3/23/16 which are not there in the stores other than 17. Or do we just ignore the transactions data? Can someone please clarify?", "240392": "Possibly the mysterious breaks are due to the fact that store 17 was closed that day. E.g. for renovation.\n\nI had a lot of trouble with the holidays dataset as well. Especially I'm wondering if an event can be interpreted as \"holiday\" and why there are work days mentioned.", "240684": "Thanks, CEbel! To be fair, the test data too has missing items from id 128 867 504 to id 128 900 000." }, "source": "meta" }