{ "id": 14043, "title": "Admins, Please Address Auction-Merchandise Concerns", "url": "/competitions/facebook-recruiting-iv-human-or-bot/discussion/14043", "author_name": "", "post_date": "2015-05-05T12:42:01.683Z", "votes": 6, "comment_count": 8, "views": 1897, "content": "

Admins - I imagine you were busy with the submission issues yesterday, so thank you for your work on that.

\n

Could you please address the concerns about the Merchandise feature? The question has been raised here and here. Wendy having responded to the first thread, but I do not believe that addresses the concerns, which I've copied again below.

\n

I don't think anyone would expect clean data (though online systems for newer companies should be better than legacy companies who never expected to be used for extensive analysis). That being said, a sanity check turns up some results that would seem to indicate an error in the way the data was built.

\n

When looking at merchandise in the bids data you see two things:

\n
    \n
  1. All bidders except for one only placed bids in one merchandise category. You would expect bidders' bids to be concentrated in one category, but not exclusive. Particularly with the explanation provided by Wendy earlier - if a bidder intended to always bid in "furniture", but the furniture auctions were sometimes listed as "home goods", then you would expect a person to unintentionally have bids in multiple categories.
    2. \n
  2. The counts of categories by auction is also quite revealing. Fewer than half the auctions have just 1 or 2 merchandise categories, and over 1500 have 7 or more (out of the 10) merchandise categories.
    3. \n
\n

These points certainly seem to point to something besides categories that are not curated well.

\n

Thank you.

", "messages": [ { "id": "77011", "postDate": "05/05/2015 12:42:01", "content": "

Admins - I imagine you were busy with the submission issues yesterday, so thank you for your work on that.

\n

Could you please address the concerns about the Merchandise feature? The question has been raised here and here. Wendy having responded to the first thread, but I do not believe that addresses the concerns, which I've copied again below.

\n

I don't think anyone would expect clean data (though online systems for newer companies should be better than legacy companies who never expected to be used for extensive analysis). That being said, a sanity check turns up some results that would seem to indicate an error in the way the data was built.

\n

When looking at merchandise in the bids data you see two things:

\n
    \n
  1. All bidders except for one only placed bids in one merchandise category. You would expect bidders' bids to be concentrated in one category, but not exclusive. Particularly with the explanation provided by Wendy earlier - if a bidder intended to always bid in "furniture", but the furniture auctions were sometimes listed as "home goods", then you would expect a person to unintentionally have bids in multiple categories.
    2. \n
  2. The counts of categories by auction is also quite revealing. Fewer than half the auctions have just 1 or 2 merchandise categories, and over 1500 have 7 or more (out of the 10) merchandise categories.
    3. \n
\n

These points certainly seem to point to something besides categories that are not curated well.

\n

Thank you.

", "rawMarkdown": "", "votes": null }, { "id": "77021", "postDate": "05/05/2015 13:02:04", "content": "

Not to mention the fact that there is a single auction, 'jqx39', which spans a whopping 537,347 bids and 9 out of 10 merchandise categories.

", "rawMarkdown": "", "votes": null }, { "id": "77022", "postDate": "05/05/2015 13:04:29", "content": "

so what?

", "rawMarkdown": "", "votes": null }, { "id": "77024", "postDate": "05/05/2015 13:07:44", "content": "

[quote=Abhishek;77022]

\n

so what?

\n

[/quote]

\n

That's not exactly constructive. Can you explain these observations?

", "rawMarkdown": "", "votes": null }, { "id": "77026", "postDate": "05/05/2015 13:20:25", "content": "

@Abishek - Given that there was already one problem acknowledged with the data provided for this competition (see the rescore thread) and that basic checks on the data don't seem to make sense, I think it's reasonable to confirm that the data we're working on is correct.

", "rawMarkdown": "", "votes": null }, { "id": "77157", "postDate": "05/05/2015 20:17:48", "content": "

I agree with Abhishek on this one. The aim of this contest seems to be to test us on how we can handle messy noisy data.

\n

A quote from the competition admin Sofus: "The difficulty of dealing with real and noisy data is what we live with every day at Facebook, and this problem is reflected in the competition."

\n

It might help to pretend that you are working on your own data and find that there are some inconsistencies. Since your evidence seems to point that it was not the data generating process that caused the problems then perhaps the data was aggregated from multiple databases, and things were lost.

\n

The data page says the data is not from facebook, so maybe they purchased it, and now find there are flaws. If there is no one who knows what happened how do you move forward with what you have in front of you?

", "rawMarkdown": "", "votes": null }, { "id": "77187", "postDate": "05/06/2015 00:14:28", "content": "

Kind of interesting to think about the idea of Facebook using a Kaggle competition to do due diligence in finding out if a potential buyout of an auction site (taking on eBay realm but seeing how reliable the companies bot detection back end or whatever is) and the only thing they are having to pay for is potentially hire a new data scientist.

\n\n

In this case it makes total sense of why they've completely obfuscated everything as it is basically insider information

", "rawMarkdown": "", "votes": null }, { "id": "77213", "postDate": "05/06/2015 02:27:22", "content": "

If you don't want the job, you don't need to document your solution and deliver a model. Note in the rules this is only a Winner's Obligation, and you may decline to be nominated as a winner. I wouldn't anyway, if you wanted a job at facebook, just apply - they even link to the position in the prize section.

\n

Personally, I wouldn't want to work in the US or the UK, so it's all wasted on me.

\n\n

Secondly, we're not really improving bot detection. If anything we're fitting whatever bot detection was already used to find the labels.

", "rawMarkdown": "", "votes": null }, { "id": "77230", "postDate": "05/06/2015 04:17:57", "content": "

@Chotch

\n

Thank you for pointing this out. I'm sorry the data description was misleading. This column "Merchandise" is the category of the auction site campaign, which means the bidder might come to this site by way of searching for "home goods" but ended up bidding for "sporting goods" - and that leads to this field being "home goods". This categorical field could be a search term, or online advertisement. This field was aggregated from different sources, but overall it gives you an idea of where the bidder originated from, but not the product itself. And this is the reason that the same auction would have multiple categories. 

\n

I hope this is helpful for your understanding the data.

\n

In the mean time, I will update the data descriptions to avoid future confusions. 

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 77021, "author_name": "cfhlearns", "author_url": "", "post_date": "05/05/2015 13:02:04", "content": "

Not to mention the fact that there is a single auction, 'jqx39', which spans a whopping 537,347 bids and 9 out of 10 merchandise categories.

", "votes": null, "replies": [] }, { "id": 77022, "author_name": "abhishek", "author_url": "", "post_date": "05/05/2015 13:04:29", "content": "

so what?

", "votes": null, "replies": [] }, { "id": 77024, "author_name": "cfhlearns", "author_url": "", "post_date": "05/05/2015 13:07:44", "content": "

[quote=Abhishek;77022]

\n

so what?

\n

[/quote]

\n

That's not exactly constructive. Can you explain these observations?

", "votes": null, "replies": [] }, { "id": 77026, "author_name": "chotch", "author_url": "", "post_date": "05/05/2015 13:20:25", "content": "

@Abishek - Given that there was already one problem acknowledged with the data provided for this competition (see the rescore thread) and that basic checks on the data don't seem to make sense, I think it's reasonable to confirm that the data we're working on is correct.

", "votes": null, "replies": [] }, { "id": 77157, "author_name": "devinanzelmo", "author_url": "", "post_date": "05/05/2015 20:17:48", "content": "

I agree with Abhishek on this one. The aim of this contest seems to be to test us on how we can handle messy noisy data.

\n

A quote from the competition admin Sofus: "The difficulty of dealing with real and noisy data is what we live with every day at Facebook, and this problem is reflected in the competition."

\n

It might help to pretend that you are working on your own data and find that there are some inconsistencies. Since your evidence seems to point that it was not the data generating process that caused the problems then perhaps the data was aggregated from multiple databases, and things were lost.

\n

The data page says the data is not from facebook, so maybe they purchased it, and now find there are flaws. If there is no one who knows what happened how do you move forward with what you have in front of you?

", "votes": null, "replies": [] }, { "id": 77187, "author_name": "hassiktir", "author_url": "", "post_date": "05/06/2015 00:14:28", "content": "

Kind of interesting to think about the idea of Facebook using a Kaggle competition to do due diligence in finding out if a potential buyout of an auction site (taking on eBay realm but seeing how reliable the companies bot detection back end or whatever is) and the only thing they are having to pay for is potentially hire a new data scientist.

\n\n

In this case it makes total sense of why they've completely obfuscated everything as it is basically insider information

", "votes": null, "replies": [] }, { "id": 77213, "author_name": "g1685786", "author_url": "", "post_date": "05/06/2015 02:27:22", "content": "

If you don't want the job, you don't need to document your solution and deliver a model. Note in the rules this is only a Winner's Obligation, and you may decline to be nominated as a winner. I wouldn't anyway, if you wanted a job at facebook, just apply - they even link to the position in the prize section.

\n

Personally, I wouldn't want to work in the US or the UK, so it's all wasted on me.

\n\n

Secondly, we're not really improving bot detection. If anything we're fitting whatever bot detection was already used to find the labels.

", "votes": null, "replies": [] }, { "id": 77230, "author_name": "wendykan", "author_url": "", "post_date": "05/06/2015 04:17:57", "content": "

@Chotch

\n

Thank you for pointing this out. I'm sorry the data description was misleading. This column "Merchandise" is the category of the auction site campaign, which means the bidder might come to this site by way of searching for "home goods" but ended up bidding for "sporting goods" - and that leads to this field being "home goods". This categorical field could be a search term, or online advertisement. This field was aggregated from different sources, but overall it gives you an idea of where the bidder originated from, but not the product itself. And this is the reason that the same auction would have multiple categories. 

\n

I hope this is helpful for your understanding the data.

\n

In the mean time, I will update the data descriptions to avoid future confusions. 

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "77011": "", "77021": "", "77022": "", "77024": "", "77026": "", "77157": "", "77187": "", "77213": "", "77230": "" }, "source": "meta" }