{ "id": 13823, "title": "unique identifier of auction should be one item right?", "url": "/competitions/facebook-recruiting-iv-human-or-bot/discussion/13823", "author_name": "", "post_date": "2015-04-28T04:59:03.300Z", "votes": 8, "comment_count": 11, "views": 3083, "content": "

why do some auction id's have multiple different merchandise categories, or does that mean it may fall under multiple categories?  

", "messages": [ { "id": "75183", "postDate": "04/28/2015 04:59:03", "content": "

why do some auction id's have multiple different merchandise categories, or does that mean it may fall under multiple categories?  

", "rawMarkdown": "", "votes": null }, { "id": "75292", "postDate": "04/28/2015 17:39:03", "content": "

The merchandise categories (as well as content) aren't always very well curated on an online auction site. You might want to only use one category and ignore the others, but it's your call. This is part of the challenge in any data science projects. 

", "rawMarkdown": "", "votes": null }, { "id": "76063", "postDate": "05/02/2015 12:30:12", "content": "

I don't think anyone would expect clean data (though online systems for newer companies should be better than legacy companies who never expected to be used for extensive analysis). That being said, a sanity check turns up some results that would seem to indicate an error in the way the data was built.

\n

I'll copy the R code below, but when looking at merchandise in the bids data you see two things:

\n
    \n
  1. All bidders except for one only placed bids in one merchandise category. You would expect bidders' bids to be concentrated in one category, but not exclusive. Particularly with the explanation provided above - if a bidder intended to always bid in "furniture", but the furniture auctions were sometimes listed as "home goods", then you would expect a person to unintentionally have bids in multiple categories.
    2. \n
  2. The counts of categories by auction is also quite revealing. Fewer than half the auctions have just 1 or 2 merchandise categories, and over 1500 have 7 or more (out of the 10) merchandise categories.
    3. \n
\n

These points certainly seem to point to something besides categories that are not curated well.

\n

R code:

\n

library(data.table)
library(bit64)

\n

# Read in bids #
bids <- fread(input="source/bids.csv", header=TRUE)

\n

# Get unique list of bidders and merchandise categories #
merch.cnt.bidder <- bids[, length(unique(merchandise)), by=bidder_id]
setnames(merch.cnt.bidder, c("bidder_id", "merchandise.cnt"))
table(merch.cnt.bidder$merchandise.cnt)

\n

# Get unique list of auctions and merchandise categories #
merch.cnt.auction <- bids[, length(unique(merchandise)), by=auction]
setnames(merch.cnt.auction, c("auction", "merchandise.cnt"))
table(merch.cnt.auction$merchandise.cnt)

", "rawMarkdown": "", "votes": null }, { "id": "76089", "postDate": "05/02/2015 14:24:41", "content": "

Note that the rules forbid sharing the code here.

\n

I agree on point 1, but for 2 it's less clear. We don't really know what does "auction" mean in this dataset. My guess is that it's some kind of aggregation of many auctions. Perhaps during the obfuscation of auction IDs many of them collapsed? Their IDs are rather short compared to  others.

", "rawMarkdown": "", "votes": null }, { "id": "76286", "postDate": "05/03/2015 11:32:35", "content": "

Or maybe during one "auction" many different items can be sold? In any case, that would not correspond to the usual understanding of auction.

", "rawMarkdown": "", "votes": null }, { "id": "76339", "postDate": "05/03/2015 15:42:32", "content": "

The description of an "auction" in the Data section for this competition seems to state pretty clearly that it only refers to one auction:

\n

"Unique identifier of an auction"

\n

Since the auctions are 5 digits long and alphanumeric, there are over 60 million possible combinations, which would be plenty for this data.

\n

I'm still having a tough time coming up with an explanation (other than an error in the data) that would address both points I made above.

", "rawMarkdown": "", "votes": null }, { "id": "77214", "postDate": "05/06/2015 02:42:27", "content": "

Pretty sure this'll just be idiots listing items under multiple categories, and the auction platform will just list the bid under which ever category the bidder found it under.

", "rawMarkdown": "", "votes": null }, { "id": "77232", "postDate": "05/06/2015 04:22:17", "content": "

Thanks for bringing this up. Please see this post I had in the other thread. 

", "rawMarkdown": "", "votes": null }, { "id": "77908", "postDate": "05/11/2015 17:04:39", "content": "

there may be multiple referring channels for the auction.

\n

different channels may have somewhat different classification systems, like home furnishings for one and furniture for another.

", "rawMarkdown": "", "votes": null }, { "id": "77911", "postDate": "05/11/2015 17:39:19", "content": "

I'm still unclear about what an auction is.   There are multiple instances of a single person making multiple bids on a single auction while being the only bidder on that auction.   

\n\n

What kind of auction would a person (not a bot) want to continue to bid when they are the only bidder?

", "rawMarkdown": "", "votes": null }, { "id": "77913", "postDate": "05/11/2015 17:51:45", "content": "

@Bryan Smith - see discussion in Multiple bids by same bidder

", "rawMarkdown": "", "votes": null }, { "id": "77956", "postDate": "05/11/2015 21:49:53", "content": "

people game trading systems.

\n

they have complex strategies for manipulating the outcome in their favor.

\n

by making an auction seem crowded by using a cast of alts it may scare away competition. It may also help the botmeister hide from the guys running the auction.

\n

by hanging back and then making a full court press in the closing seconds they may try to keep the price from escalating too high.

\n

they guy who was bagged in London recently was making "fake" bids that were quickly cancelled to get others to enter the fray so he could dump to them.

\n

to get a good humorous feel for it watch the pit scene in Trading Places.

\n

Pit Scene

\n

Old school market panic.

\n

professional traders, even guys who make auction bots, have deep understanding of how market psychology works and many tricks to manipulate it.

\n

look also at the controversy over High Frequency Trading, where traders get an edge by having the fastest network connections possible.

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 75292, "author_name": "wendykan", "author_url": "", "post_date": "04/28/2015 17:39:03", "content": "

The merchandise categories (as well as content) aren't always very well curated on an online auction site. You might want to only use one category and ignore the others, but it's your call. This is part of the challenge in any data science projects. 

", "votes": null, "replies": [] }, { "id": 76063, "author_name": "chotch", "author_url": "", "post_date": "05/02/2015 12:30:12", "content": "

I don't think anyone would expect clean data (though online systems for newer companies should be better than legacy companies who never expected to be used for extensive analysis). That being said, a sanity check turns up some results that would seem to indicate an error in the way the data was built.

\n

I'll copy the R code below, but when looking at merchandise in the bids data you see two things:

\n
    \n
  1. All bidders except for one only placed bids in one merchandise category. You would expect bidders' bids to be concentrated in one category, but not exclusive. Particularly with the explanation provided above - if a bidder intended to always bid in "furniture", but the furniture auctions were sometimes listed as "home goods", then you would expect a person to unintentionally have bids in multiple categories.
    2. \n
  2. The counts of categories by auction is also quite revealing. Fewer than half the auctions have just 1 or 2 merchandise categories, and over 1500 have 7 or more (out of the 10) merchandise categories.
    3. \n
\n

These points certainly seem to point to something besides categories that are not curated well.

\n

R code:

\n

library(data.table)
library(bit64)

\n

# Read in bids #
bids <- fread(input="source/bids.csv", header=TRUE)

\n

# Get unique list of bidders and merchandise categories #
merch.cnt.bidder <- bids[, length(unique(merchandise)), by=bidder_id]
setnames(merch.cnt.bidder, c("bidder_id", "merchandise.cnt"))
table(merch.cnt.bidder$merchandise.cnt)

\n

# Get unique list of auctions and merchandise categories #
merch.cnt.auction <- bids[, length(unique(merchandise)), by=auction]
setnames(merch.cnt.auction, c("auction", "merchandise.cnt"))
table(merch.cnt.auction$merchandise.cnt)

", "votes": null, "replies": [] }, { "id": 76089, "author_name": "redstr", "author_url": "", "post_date": "05/02/2015 14:24:41", "content": "

Note that the rules forbid sharing the code here.

\n

I agree on point 1, but for 2 it's less clear. We don't really know what does "auction" mean in this dataset. My guess is that it's some kind of aggregation of many auctions. Perhaps during the obfuscation of auction IDs many of them collapsed? Their IDs are rather short compared to  others.

", "votes": null, "replies": [] }, { "id": 76286, "author_name": "lorenzo", "author_url": "", "post_date": "05/03/2015 11:32:35", "content": "

Or maybe during one "auction" many different items can be sold? In any case, that would not correspond to the usual understanding of auction.

", "votes": null, "replies": [] }, { "id": 76339, "author_name": "chotch", "author_url": "", "post_date": "05/03/2015 15:42:32", "content": "

The description of an "auction" in the Data section for this competition seems to state pretty clearly that it only refers to one auction:

\n

"Unique identifier of an auction"

\n

Since the auctions are 5 digits long and alphanumeric, there are over 60 million possible combinations, which would be plenty for this data.

\n

I'm still having a tough time coming up with an explanation (other than an error in the data) that would address both points I made above.

", "votes": null, "replies": [] }, { "id": 77214, "author_name": "g1685786", "author_url": "", "post_date": "05/06/2015 02:42:27", "content": "

Pretty sure this'll just be idiots listing items under multiple categories, and the auction platform will just list the bid under which ever category the bidder found it under.

", "votes": null, "replies": [] }, { "id": 77232, "author_name": "wendykan", "author_url": "", "post_date": "05/06/2015 04:22:17", "content": "

Thanks for bringing this up. Please see this post I had in the other thread. 

", "votes": null, "replies": [] }, { "id": 77908, "author_name": "charlesdavis", "author_url": "", "post_date": "05/11/2015 17:04:39", "content": "

there may be multiple referring channels for the auction.

\n

different channels may have somewhat different classification systems, like home furnishings for one and furniture for another.

", "votes": null, "replies": [] }, { "id": 77911, "author_name": "bryantravissmith", "author_url": "", "post_date": "05/11/2015 17:39:19", "content": "

I'm still unclear about what an auction is.   There are multiple instances of a single person making multiple bids on a single auction while being the only bidder on that auction.   

\n\n

What kind of auction would a person (not a bot) want to continue to bid when they are the only bidder?

", "votes": null, "replies": [] }, { "id": 77913, "author_name": "chotch", "author_url": "", "post_date": "05/11/2015 17:51:45", "content": "

@Bryan Smith - see discussion in Multiple bids by same bidder

", "votes": null, "replies": [] }, { "id": 77956, "author_name": "charlesdavis", "author_url": "", "post_date": "05/11/2015 21:49:53", "content": "

people game trading systems.

\n

they have complex strategies for manipulating the outcome in their favor.

\n

by making an auction seem crowded by using a cast of alts it may scare away competition. It may also help the botmeister hide from the guys running the auction.

\n

by hanging back and then making a full court press in the closing seconds they may try to keep the price from escalating too high.

\n

they guy who was bagged in London recently was making "fake" bids that were quickly cancelled to get others to enter the fray so he could dump to them.

\n

to get a good humorous feel for it watch the pit scene in Trading Places.

\n

Pit Scene

\n

Old school market panic.

\n

professional traders, even guys who make auction bots, have deep understanding of how market psychology works and many tricks to manipulate it.

\n

look also at the controversy over High Frequency Trading, where traders get an edge by having the fastest network connections possible.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "75183": "", "75292": "", "76063": "", "76089": "", "76286": "", "76339": "", "77214": "", "77232": "", "77908": "", "77911": "", "77913": "", "77956": "" }, "source": "meta" }