{
  "id": 9498,
  "title": "Data Usage beyond the contest",
  "url": "/competitions/expedia-personalized-sort/discussion/9498",
  "author_name": "",
  "post_date": "2014-06-19T14:35:03.010Z",
  "votes": null,
  "comment_count": 42,
  "views": 10540,
  "content": "<p>dear adam,</p>\n<p>is it possible to use the competition data we already downloaded in scientific publications? is it publicly accessible?</p>\n\n<p>best,</p>\n<p>claudio</p>",
  "messages": [
    {
      "id": "49301",
      "postDate": "06/19/2014 14:35:03",
      "content": "<p>dear adam,</p>\n<p>is it possible to use the competition data we already downloaded in scientific publications? is it publicly accessible?</p>\n\n<p>best,</p>\n<p>claudio</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51044",
      "postDate": "07/22/2014 09:35:29",
      "content": "<p>Hi Claudio,</p>\n<p>Yes, you can use it in your research but it is subject to the same restrictions as specified in the rules of the contest (e.g. Expedia competitors cannot use/analyze it).</p>\n<p>Adam</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "60227",
      "postDate": "12/15/2014 10:50:49",
      "content": "<p>Dear Adam,</p>\n<p>The data is at the search id level. Is it possible to know the anonymized user id connected to a search?</p>\n<p>Thank you</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "60228",
      "postDate": "12/15/2014 10:59:58",
      "content": "<p>Hi,</p>\n\n<p>Anonymized user ids are not part of this dataset. Currently, there are not plans to release this information.</p>\n\n<p>Regards,</p>\n<p>Adam</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "62665",
      "postDate": "01/12/2015 20:37:19",
      "content": "<p>Hi Adam,</p>\n<p>If I may, I have three questions about how the data was generated for this contest.</p>\n<p>1. The data description says &#8220;Most of the data are for searches that resulted in a purchase, but a small proportion are for searches not leading to a purchase.&#8221; My question is how was the pool of searches that ended in a transaction chosen? Were they randomly chosen from all searches ending in a transaction?</p>\n<p>2. I know that the data contains some search impressions where hotels were randomly ranked. Does this mean that each hotel in a certain region had an equal probability of being placed in a given position as any other hotel in that region?</p>\n<p>3. Finally, do you have any idea why most search impressions (over 90 percent of them) have exactly one click? Since searches were chosen so that they end in a transaction, I expected the minimum number of clicks in a search impression to be one, but why is the maximum also one?</p>\n<p>Thank you!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "62667",
      "postDate": "01/12/2015 21:17:02",
      "content": "<p>Hi,</p>\n<p>Answering your questions:</p>\n<p>1. No, converting impressions were oversampled from random impressions.</p>\n<p>2. In principle yes, but some impressions that are labeled as random were not truly random - this was a result of a limitation of our raw logging data.</p>\n<p>3. First, this is a result of a limitation of our raw data, the second reason is simply that users on average do not click many hotels from an impression.</p>\n\n<p>Adam</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "62669",
      "postDate": "01/12/2015 21:22:28",
      "content": "<p>Hi Adam,</p>\n<p>Thank you very much. About 2, is there any way to detect which impressions were truly random and which were not? Or do you know of the percentage of error in mislabeling them?</p>\n<p>Thank you,</p>\n<p>Raluca</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "62670",
      "postDate": "01/12/2015 21:26:17",
      "content": "<p>There is not simple way, but you can see what hotels are over sampled in random impressions, based on which you can detect what impressions are likely not random.</p>\n<p>I don't remember exact number but mislabeled impressions should not be very common.</p>\n<p>Adam</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "62672",
      "postDate": "01/12/2015 23:15:51",
      "content": "<p>Great, thank you very much! </p>\n<p>-Raluca</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "62797",
      "postDate": "01/16/2015 17:24:22",
      "content": "<p>Hi Adam,</p>\n<p>Is it possible to find out how some search impressions saw the random ranking while others saw the default one? Was this also done randomly based on time of arrival at website or were there other criteria involved?</p>\n<p>Thank you!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "62798",
      "postDate": "01/16/2015 19:08:30",
      "content": "<p>Users we assigned randomly to different types of impressions.</p>\n\n<p>Adam</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63724",
      "postDate": "02/06/2015 20:55:33",
      "content": "<p>Hi Adam,</p>\n<p>Just to clarify a point I asked you about previously. The data description says &#8220;Most of the data are for searches that resulted in a purchase, but a small proportion are for searches not leading to a purchase.&#8221; In the data, there is a large difference in the number of transactions made under the default ranking and the random ranking. Should I be worried that this is difference is due to the way the data was selected or is this representative of the difference in performance of the two rankings?</p>\n<p>Thank you!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63779",
      "postDate": "02/08/2015 19:51:55",
      "content": "<p>The proportions are not representative on the way the different types of rankings perform. Moreover, the proportions between different types of rankings were arbitrary set by the organizers of the competition.</p>\n\n<p>Adam</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63780",
      "postDate": "02/08/2015 20:11:32",
      "content": "<p>From your reply, I understand that the data set provided contains only a fraction of the random-ranking impressions (and presumably, a fraction of the default ranking one, too). Can I assume that the sampling was done so that the properties of the two rankings were preserved? For instance, I notice that, in the random ranking case, only a small fractions of the clicks turn into transactions. Can I assume that this observation stands for the whole set of impressions or could this be an artifact of the sampling for the provided data set?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63781",
      "postDate": "02/08/2015 20:25:42",
      "content": "<p>This could be an artifact of the samplings.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63783",
      "postDate": "02/08/2015 21:35:04",
      "content": "<p>could I find out how the sampling of search impressions leading to a purchase was done? so for a given ranking, was the sampling done randomly or was it done based on an algorithm, and if so, can I find out more about this algorithm?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63784",
      "postDate": "02/08/2015 21:40:55",
      "content": "<p>Unfortunately, I cannot share the exact procedure because it could potentially will reveal sensitive information (e.g. conversion rates).</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63785",
      "postDate": "02/08/2015 21:43:00",
      "content": "<p>Thank you.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64050",
      "postDate": "02/11/2015 17:56:26",
      "content": "<p>Thank you again for your previous reply. I am not interested in determining absolute values (like conversion rates of the two rankings), but is there any way I could get a sense of the relative numbers? For my research I would really appreciate the opportunity to learn more about this or talk to someone on this issue.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64051",
      "postDate": "02/11/2015 18:21:03",
      "content": "<p>In general: the majority of impressions are non-random, and the majority of impressions and non-converting. I cannot reveal more details.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64175",
      "postDate": "02/13/2015 18:47:05",
      "content": "<p>Thank you very much for your previous reply.</p>\n<p>About destination ID: what is the definition of the destination ID? Is it a city, county, neighborhood?</p>\n<p>About hotel ID: is this the name of the hotel or is this the name of the chain? I am asking because I can see that the same hotel ID can appear in multiple destination IDs. Is this because there is an overlap in destination ID, so that the same hotel can appear if one searches for NYC or for Manhattan, or is there a different reason?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64186",
      "postDate": "02/13/2015 20:02:44",
      "content": "<p>destination ID can be anything but most frequently is of type city or multi-city; destinations are overlapping.</p>\n<p>HotleID is a hash of a name of hotel name and it can appear in several destinations.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64191",
      "postDate": "02/13/2015 21:18:26",
      "content": "<p>Thank you! So a hotel called &quot;Hotel A-airport&quot; and another called &quot;Hotel A-city center&quot; would be coded as different hotels?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64439",
      "postDate": "02/17/2015 23:25:30",
      "content": "<p>that's correct.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64819",
      "postDate": "02/24/2015 20:26:53",
      "content": "<p>is there a&nbsp; way to tell whether the hotel that was shown in a given position was a sponsored ad or was placed there by the algorithm (either in the random or the non-random data), or where sponsored hotels not part of the data set?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64821",
      "postDate": "02/24/2015 20:31:05",
      "content": "<p>Sponsored ads were part of the dataset, but there is no way to identify them.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64822",
      "postDate": "02/24/2015 20:39:35",
      "content": "<p>Thank you! So even for the random ranking there were sponsored ads included?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65007",
      "postDate": "02/27/2015 04:33:35",
      "content": "<p>Hi Adam,</p>\n<p>Just a quick question: how was the variable srch_query_affinity_score constructed? Any details?</p>\n<p>Thank you,</p>\n<p>Raluca</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65384",
      "postDate": "03/04/2015 05:31:25",
      "content": "<p>For the non-random ranking, a better hotel is presumably shown in a better position, so if it gets a click or a purchase, it is not clear which factor contributed more (i.e. there's a position bias). Other than using the random ranking where this problem doesn't show up, is there a way to disentangle the two effects (position and quality of hotel)?</p>\n<p>Thank you</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65409",
      "postDate": "03/04/2015 14:16:47",
      "content": "<p>We hope that you'll be able solve this problem :-)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65412",
      "postDate": "03/04/2015 14:33:01",
      "content": "<p>I am doing my best! All that is needed is something correlated with the position of the hotel but that does not influence the decision of the consumer to click/purchase.</p>\n<p>Thank you!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75060",
      "postDate": "04/27/2015 16:32:53",
      "content": "<p>Hi Adam,</p>\n<p>One more question: Is there a way to recover the order in which a click was made? I know that the click variable is boolean, but was the order that the data was given informative about the click order or is there anything else that I can use?</p>\n<p>Thank you,</p>\n<p>Raluca</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75096",
      "postDate": "04/27/2015 19:20:53",
      "content": "<p>Hi,</p>\n<p>It's not possible to recover the click order from the released data.</p>\n\n<p>Adam</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "77646",
      "postDate": "05/08/2015 19:03:12",
      "content": "<p>Is there a way to tell whether the consumer sorted results by price or stars instead of seeing Expedia's ranking or the random ranking?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "77796",
      "postDate": "05/10/2015 20:41:12",
      "content": "<p>The released data only contains either impressions sorted by our algorithm or random impressions.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "78306",
      "postDate": "05/13/2015 20:51:00",
      "content": "<p>Thank you! And I know that sponsored search ads are included in the data, but there is no way to identify them. Are they also included in the random ranking data?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "79517",
      "postDate": "05/20/2015 20:42:18",
      "content": "<p>I was also wondering is searches where made only from desktops or mobile devices too?</p>\n<p>Thank you!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "83072",
      "postDate": "07/01/2015 12:00:15",
      "content": "<p>Hi there!</p>\n<p>I have noticed that user Raluca Ursu asked about the Hotel ID's and the Location ID's. My hounors project deals with visulaizing the data. It would be highly beneficiall if I knew the location ID's and even possibly the hotel names. Is there any way to obtain this information?</p>\n<p>Regards</p>\n<p>Pelonomi</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "83077",
      "postDate": "07/01/2015 13:26:10",
      "content": "<p>Hi, Unfortunately we cannot reveal this information.</p>\n<p>Adam</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "137591",
      "postDate": "10/03/2016 03:53:42",
      "content": "<p>Hi Adam,</p>\n<p>I wanted to implement your suggestion below - you noted that among the random impressions, the hotel order was randomized within region (or at least, you said &quot;In principle yes&quot; to Raluca's comment asserting this). &nbsp;But I wasn't sure which &quot;region&quot; you were referring to - what variable in the data defines the regions within which the hotel orders were randomized? - or is there some other way to reconstruct these regions? &nbsp;The only geographic identifier I saw was countries - I assume you did not actually randomize the order for every country, correct?</p>\n<p>One thought I had was to construct subsets of hotels which ever appear in the same search as other hotels as an ad hoc way of defining regions. &nbsp;Does that make sense and do you have other suggestions?</p>\n<p>Thanks in advance for your help with this!</p>\n<p>Jason</p>\n\n<p>[quote=Adam;62670]</p>\n<p>There is not simple way, but you can see what hotels are over sampled in random impressions, based on which you can detect what impressions are likely not random.</p>\n<p>I don't remember exact number but mislabeled impressions should not be very common.</p>\n<p>Adam</p>\n<p>[/quote]</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "137630",
      "postDate": "10/03/2016 14:00:29",
      "content": "<p>Hotels were randomized in the context of srch_id. By &quot;region&quot; I was referring to &quot;srch_destination_id&quot; which is search region. </p>",
      "rawMarkdown": "Hotels were randomized in the context of srch_id. By \"region\" I was referring to \"srch_destination_id\" which is search region.",
      "votes": null
    },
    {
      "id": "150789",
      "postDate": "12/16/2016 18:30:39",
      "content": "<p>Hi @Adam - it looks like there's between 5 and 35 results for each search (how many times a unique searchID is associated with a row). Does the dataset only show the search results on the first page? Why does it range like this and what's the right way to look at this number? </p>\n\n<p>Thanks!</p>",
      "rawMarkdown": "Hi @Adam - it looks like there's between 5 and 35 results for each search (how many times a unique searchID is associated with a row). Does the dataset only show the search results on the first page? Why does it range like this and what's the right way to look at this number? \r\n\r\nThanks!",
      "votes": null
    },
    {
      "id": "151193",
      "postDate": "12/19/2016 10:09:49",
      "content": "<p>Yes, it's only the first impression page. The # of hotels can very because of the mixture large/small markets and limited hotels availability.</p>",
      "rawMarkdown": "Yes, it's only the first impression page. The # of hotels can very because of the mixture large/small markets and limited hotels availability.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 51044,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "07/22/2014 09:35:29",
      "content": "<p>Hi Claudio,</p>\n<p>Yes, you can use it in your research but it is subject to the same restrictions as specified in the rules of the contest (e.g. Expedia competitors cannot use/analyze it).</p>\n<p>Adam</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 60227,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "12/15/2014 10:50:49",
      "content": "<p>Dear Adam,</p>\n<p>The data is at the search id level. Is it possible to know the anonymized user id connected to a search?</p>\n<p>Thank you</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 60228,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "12/15/2014 10:59:58",
      "content": "<p>Hi,</p>\n\n<p>Anonymized user ids are not part of this dataset. Currently, there are not plans to release this information.</p>\n\n<p>Regards,</p>\n<p>Adam</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 62665,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "01/12/2015 20:37:19",
      "content": "<p>Hi Adam,</p>\n<p>If I may, I have three questions about how the data was generated for this contest.</p>\n<p>1. The data description says &#8220;Most of the data are for searches that resulted in a purchase, but a small proportion are for searches not leading to a purchase.&#8221; My question is how was the pool of searches that ended in a transaction chosen? Were they randomly chosen from all searches ending in a transaction?</p>\n<p>2. I know that the data contains some search impressions where hotels were randomly ranked. Does this mean that each hotel in a certain region had an equal probability of being placed in a given position as any other hotel in that region?</p>\n<p>3. Finally, do you have any idea why most search impressions (over 90 percent of them) have exactly one click? Since searches were chosen so that they end in a transaction, I expected the minimum number of clicks in a search impression to be one, but why is the maximum also one?</p>\n<p>Thank you!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 62667,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "01/12/2015 21:17:02",
      "content": "<p>Hi,</p>\n<p>Answering your questions:</p>\n<p>1. No, converting impressions were oversampled from random impressions.</p>\n<p>2. In principle yes, but some impressions that are labeled as random were not truly random - this was a result of a limitation of our raw logging data.</p>\n<p>3. First, this is a result of a limitation of our raw data, the second reason is simply that users on average do not click many hotels from an impression.</p>\n\n<p>Adam</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 62669,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "01/12/2015 21:22:28",
      "content": "<p>Hi Adam,</p>\n<p>Thank you very much. About 2, is there any way to detect which impressions were truly random and which were not? Or do you know of the percentage of error in mislabeling them?</p>\n<p>Thank you,</p>\n<p>Raluca</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 62670,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "01/12/2015 21:26:17",
      "content": "<p>There is not simple way, but you can see what hotels are over sampled in random impressions, based on which you can detect what impressions are likely not random.</p>\n<p>I don't remember exact number but mislabeled impressions should not be very common.</p>\n<p>Adam</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 62672,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "01/12/2015 23:15:51",
      "content": "<p>Great, thank you very much! </p>\n<p>-Raluca</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 62797,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "01/16/2015 17:24:22",
      "content": "<p>Hi Adam,</p>\n<p>Is it possible to find out how some search impressions saw the random ranking while others saw the default one? Was this also done randomly based on time of arrival at website or were there other criteria involved?</p>\n<p>Thank you!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 62798,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "01/16/2015 19:08:30",
      "content": "<p>Users we assigned randomly to different types of impressions.</p>\n\n<p>Adam</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63724,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "02/06/2015 20:55:33",
      "content": "<p>Hi Adam,</p>\n<p>Just to clarify a point I asked you about previously. The data description says &#8220;Most of the data are for searches that resulted in a purchase, but a small proportion are for searches not leading to a purchase.&#8221; In the data, there is a large difference in the number of transactions made under the default ranking and the random ranking. Should I be worried that this is difference is due to the way the data was selected or is this representative of the difference in performance of the two rankings?</p>\n<p>Thank you!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63779,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "02/08/2015 19:51:55",
      "content": "<p>The proportions are not representative on the way the different types of rankings perform. Moreover, the proportions between different types of rankings were arbitrary set by the organizers of the competition.</p>\n\n<p>Adam</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63780,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "02/08/2015 20:11:32",
      "content": "<p>From your reply, I understand that the data set provided contains only a fraction of the random-ranking impressions (and presumably, a fraction of the default ranking one, too). Can I assume that the sampling was done so that the properties of the two rankings were preserved? For instance, I notice that, in the random ranking case, only a small fractions of the clicks turn into transactions. Can I assume that this observation stands for the whole set of impressions or could this be an artifact of the sampling for the provided data set?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63781,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "02/08/2015 20:25:42",
      "content": "<p>This could be an artifact of the samplings.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63783,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "02/08/2015 21:35:04",
      "content": "<p>could I find out how the sampling of search impressions leading to a purchase was done? so for a given ranking, was the sampling done randomly or was it done based on an algorithm, and if so, can I find out more about this algorithm?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63784,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "02/08/2015 21:40:55",
      "content": "<p>Unfortunately, I cannot share the exact procedure because it could potentially will reveal sensitive information (e.g. conversion rates).</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63785,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "02/08/2015 21:43:00",
      "content": "<p>Thank you.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64050,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "02/11/2015 17:56:26",
      "content": "<p>Thank you again for your previous reply. I am not interested in determining absolute values (like conversion rates of the two rankings), but is there any way I could get a sense of the relative numbers? For my research I would really appreciate the opportunity to learn more about this or talk to someone on this issue.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64051,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "02/11/2015 18:21:03",
      "content": "<p>In general: the majority of impressions are non-random, and the majority of impressions and non-converting. I cannot reveal more details.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64175,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "02/13/2015 18:47:05",
      "content": "<p>Thank you very much for your previous reply.</p>\n<p>About destination ID: what is the definition of the destination ID? Is it a city, county, neighborhood?</p>\n<p>About hotel ID: is this the name of the hotel or is this the name of the chain? I am asking because I can see that the same hotel ID can appear in multiple destination IDs. Is this because there is an overlap in destination ID, so that the same hotel can appear if one searches for NYC or for Manhattan, or is there a different reason?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64186,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "02/13/2015 20:02:44",
      "content": "<p>destination ID can be anything but most frequently is of type city or multi-city; destinations are overlapping.</p>\n<p>HotleID is a hash of a name of hotel name and it can appear in several destinations.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64191,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "02/13/2015 21:18:26",
      "content": "<p>Thank you! So a hotel called &quot;Hotel A-airport&quot; and another called &quot;Hotel A-city center&quot; would be coded as different hotels?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64439,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "02/17/2015 23:25:30",
      "content": "<p>that's correct.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64819,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "02/24/2015 20:26:53",
      "content": "<p>is there a&nbsp; way to tell whether the hotel that was shown in a given position was a sponsored ad or was placed there by the algorithm (either in the random or the non-random data), or where sponsored hotels not part of the data set?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64821,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "02/24/2015 20:31:05",
      "content": "<p>Sponsored ads were part of the dataset, but there is no way to identify them.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64822,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "02/24/2015 20:39:35",
      "content": "<p>Thank you! So even for the random ranking there were sponsored ads included?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65007,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "02/27/2015 04:33:35",
      "content": "<p>Hi Adam,</p>\n<p>Just a quick question: how was the variable srch_query_affinity_score constructed? Any details?</p>\n<p>Thank you,</p>\n<p>Raluca</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65384,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "03/04/2015 05:31:25",
      "content": "<p>For the non-random ranking, a better hotel is presumably shown in a better position, so if it gets a click or a purchase, it is not clear which factor contributed more (i.e. there's a position bias). Other than using the random ranking where this problem doesn't show up, is there a way to disentangle the two effects (position and quality of hotel)?</p>\n<p>Thank you</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65409,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "03/04/2015 14:16:47",
      "content": "<p>We hope that you'll be able solve this problem :-)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65412,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "03/04/2015 14:33:01",
      "content": "<p>I am doing my best! All that is needed is something correlated with the position of the hotel but that does not influence the decision of the consumer to click/purchase.</p>\n<p>Thank you!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75060,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "04/27/2015 16:32:53",
      "content": "<p>Hi Adam,</p>\n<p>One more question: Is there a way to recover the order in which a click was made? I know that the click variable is boolean, but was the order that the data was given informative about the click order or is there anything else that I can use?</p>\n<p>Thank you,</p>\n<p>Raluca</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75096,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "04/27/2015 19:20:53",
      "content": "<p>Hi,</p>\n<p>It's not possible to recover the click order from the released data.</p>\n\n<p>Adam</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 77646,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "05/08/2015 19:03:12",
      "content": "<p>Is there a way to tell whether the consumer sorted results by price or stars instead of seeing Expedia's ranking or the random ranking?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 77796,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "05/10/2015 20:41:12",
      "content": "<p>The released data only contains either impressions sorted by our algorithm or random impressions.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 78306,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "05/13/2015 20:51:00",
      "content": "<p>Thank you! And I know that sponsored search ads are included in the data, but there is no way to identify them. Are they also included in the random ranking data?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 79517,
      "author_name": "ralucaursu",
      "author_url": "",
      "post_date": "05/20/2015 20:42:18",
      "content": "<p>I was also wondering is searches where made only from desktops or mobile devices too?</p>\n<p>Thank you!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 83072,
      "author_name": "pelonomi",
      "author_url": "",
      "post_date": "07/01/2015 12:00:15",
      "content": "<p>Hi there!</p>\n<p>I have noticed that user Raluca Ursu asked about the Hotel ID's and the Location ID's. My hounors project deals with visulaizing the data. It would be highly beneficiall if I knew the location ID's and even possibly the hotel names. Is there any way to obtain this information?</p>\n<p>Regards</p>\n<p>Pelonomi</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 83077,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "07/01/2015 13:26:10",
      "content": "<p>Hi, Unfortunately we cannot reveal this information.</p>\n<p>Adam</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 137591,
      "author_name": "jabaluck",
      "author_url": "",
      "post_date": "10/03/2016 03:53:42",
      "content": "<p>Hi Adam,</p>\n<p>I wanted to implement your suggestion below - you noted that among the random impressions, the hotel order was randomized within region (or at least, you said &quot;In principle yes&quot; to Raluca's comment asserting this). &nbsp;But I wasn't sure which &quot;region&quot; you were referring to - what variable in the data defines the regions within which the hotel orders were randomized? - or is there some other way to reconstruct these regions? &nbsp;The only geographic identifier I saw was countries - I assume you did not actually randomize the order for every country, correct?</p>\n<p>One thought I had was to construct subsets of hotels which ever appear in the same search as other hotels as an ad hoc way of defining regions. &nbsp;Does that make sense and do you have other suggestions?</p>\n<p>Thanks in advance for your help with this!</p>\n<p>Jason</p>\n\n<p>[quote=Adam;62670]</p>\n<p>There is not simple way, but you can see what hotels are over sampled in random impressions, based on which you can detect what impressions are likely not random.</p>\n<p>I don't remember exact number but mislabeled impressions should not be very common.</p>\n<p>Adam</p>\n<p>[/quote]</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 137630,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "10/03/2016 14:00:29",
      "content": "<p>Hotels were randomized in the context of srch_id. By &quot;region&quot; I was referring to &quot;srch_destination_id&quot; which is search region. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 150789,
      "author_name": "josephvb",
      "author_url": "",
      "post_date": "12/16/2016 18:30:39",
      "content": "<p>Hi @Adam - it looks like there's between 5 and 35 results for each search (how many times a unique searchID is associated with a row). Does the dataset only show the search results on the first page? Why does it range like this and what's the right way to look at this number? </p>\n\n<p>Thanks!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 151193,
      "author_name": "adamwoz",
      "author_url": "",
      "post_date": "12/19/2016 10:09:49",
      "content": "<p>Yes, it's only the first impression page. The # of hotels can very because of the mixture large/small markets and limited hotels availability.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "49301": "",
    "51044": "",
    "60227": "",
    "60228": "",
    "62665": "",
    "62667": "",
    "62669": "",
    "62670": "",
    "62672": "",
    "62797": "",
    "62798": "",
    "63724": "",
    "63779": "",
    "63780": "",
    "63781": "",
    "63783": "",
    "63784": "",
    "63785": "",
    "64050": "",
    "64051": "",
    "64175": "",
    "64186": "",
    "64191": "",
    "64439": "",
    "64819": "",
    "64821": "",
    "64822": "",
    "65007": "",
    "65384": "",
    "65409": "",
    "65412": "",
    "75060": "",
    "75096": "",
    "77646": "",
    "77796": "",
    "78306": "",
    "79517": "",
    "83072": "",
    "83077": "",
    "137591": "",
    "137630": "Hotels were randomized in the context of srch_id. By \"region\" I was referring to \"srch_destination_id\" which is search region.",
    "150789": "Hi @Adam - it looks like there's between 5 and 35 results for each search (how many times a unique searchID is associated with a row). Does the dataset only show the search results on the first page? Why does it range like this and what's the right way to look at this number? \r\n\r\nThanks!",
    "151193": "Yes, it's only the first impression page. The # of hotels can very because of the mixture large/small markets and limited hotels availability."
  },
  "source": "meta"
}