{ "id": 10782, "title": "Q&A", "url": "/competitions/avazu-ctr-prediction/discussion/10782", "author_name": "", "post_date": "2014-10-29T07:32:22.723Z", "votes": 6, "comment_count": 30, "views": 18122, "content": "

1. Q: Why CTR is so high? Normally the industry average is more like 0.1%, yet the CTR of the training data set is between 10 to 25%. How is that even possible?

\n

A: The click records and non-click records are subsampled based on different sampling strategies. We have subsampled much fewer non-click records, which makes the CTR really high.

\n

2. Q: By the way, are there any explanations why CTR is maximal on Wednesday?

\n

A: We made the data containing nearly 200k records per hour, that is, we first sampled some clicked records, and then added non-clicked records to make them adding up to nearly 200k, so perhaps there is no strong correlation between sampled CTR and true CTR with respect to time features

\n

3. Q: Was the test-data also subsampled based on different sample strategies?

\n

A: I use uniform sampling this time，so the data is as IID distributed as possible now

\n

4. Q: Could you tell me whether the first field data are wrong.

\n

A: yes, the fisrt field is record id

", "messages": [ { "id": "56963", "postDate": "10/29/2014 07:32:22", "content": "

1. Q: Why CTR is so high? Normally the industry average is more like 0.1%, yet the CTR of the training data set is between 10 to 25%. How is that even possible?

\n

A: The click records and non-click records are subsampled based on different sampling strategies. We have subsampled much fewer non-click records, which makes the CTR really high.

\n

2. Q: By the way, are there any explanations why CTR is maximal on Wednesday?

\n

A: We made the data containing nearly 200k records per hour, that is, we first sampled some clicked records, and then added non-clicked records to make them adding up to nearly 200k, so perhaps there is no strong correlation between sampled CTR and true CTR with respect to time features

\n

3. Q: Was the test-data also subsampled based on different sample strategies?

\n

A: I use uniform sampling this time，so the data is as IID distributed as possible now

\n

4. Q: Could you tell me whether the first field data are wrong.

\n

A: yes, the fisrt field is record id

", "rawMarkdown": "", "votes": null }, { "id": "56974", "postDate": "10/29/2014 10:19:32", "content": "

Besides, we can not provide detailed information related to TRUE hourly CTR, as it's a business secret.

\n

If you wonder there are some patterns in time series data, you can modeling them.

", "rawMarkdown": "", "votes": null }, { "id": "57198", "postDate": "11/02/2014 16:45:08", "content": "

Can you answer two questions:

\n

1.What is the difference between features represented as 0xXXXXXXXX(for example device_geo_country) and features represented as int values(for example C24)

\n

2.  -1 means that value is absent? For example in C23 feature.

", "rawMarkdown": "", "votes": null }, { "id": "57200", "postDate": "11/02/2014 17:41:19", "content": "

[quote=Shalnovv;57198]

\n

Can you answer two questions:

\n

1.What is the difference between features represented as 0xXXXXXXXX(for example device_geo_country) and features represented as int values(for example C24)

\n

2.  -1 means that value is absent? For example in C23 feature.

\n

[/quote]

\n

they are all categorical values, some are strings, and we hashed them, others kept original values

\n

I have little knowledge of special values of certain column of the data, sorry for that.

", "rawMarkdown": "", "votes": null }, { "id": "57254", "postDate": "11/03/2014 16:44:56", "content": "

1. Was the test-data also subsampled based on different sample strategies? If the train and the test data are not equivalent in terms of CTR, do you think inferring about the test-data using the train-data will produce good results? 

", "rawMarkdown": "", "votes": null }, { "id": "57265", "postDate": "11/04/2014 03:22:54", "content": "

[quote=achilles;57254]

\n

1. Was the test-data also subsampled based on different sample strategies? If the train and the test data are not equivalent in terms of CTR, do you think inferring about the test-data using the train-data will produce good results? 

\n

[/quote]

\n

I use uniform sampling this time，so the data is as IID distributed as possible now

", "rawMarkdown": "", "votes": null }, { "id": "57316", "postDate": "11/04/2014 22:03:16", "content": "

I don't agree Steve Wang's last comment.

\n

The ultimate goal of any machine learning algorithm is the ability to generalize. We usually, if not always, assume the test data is coming from the same distribution of training data. Deviations are inevitable (who knows about future before it comes?), but if the test data is intentionally sampled from a different distribution, then all machine learning techniques becomes, frankly speaking, useless. 

\n

In practice, this is often represented by a large deviation of results obtained from leader board and offline cross validation, which is misleading and confusing, and often causes a waste of competitors' time.

", "rawMarkdown": "", "votes": null }, { "id": "57318", "postDate": "11/05/2014 00:34:57", "content": "

So is the test data unfiltered?

", "rawMarkdown": "", "votes": null }, { "id": "57335", "postDate": "11/05/2014 03:42:25", "content": "

[quote=byronyi;57316]

\n

I don't agree Steve Wang's last comment.

\n

The ultimate goal of any machine learning algorithm is the ability to generalize. We usually, if not always, assume the test data is coming from the same distribution of training data. Deviations are inevitable (who knows about future before it comes?), but if the test data is intentionally sampled from a different distribution, then all machine learning techniques becomes, frankly speaking, useless. 

\n

In practice, this is often represented by a large deviation of results obtained from leader board and offline cross validation, which is misleading and confusing, and often causes a waste of competitors' time.

\n

[/quote]

\n

Your comment is quite reasonable，so I made a private benchmark several days ago，the train error and test error matched very well，using a standard algorithm，with no tricks

", "rawMarkdown": "", "votes": null }, { "id": "57417", "postDate": "11/06/2014 16:55:16", "content": "

Based on the competition admin's answers, this seems like an odd setup:

\n
    \n
  1. The training data were constructed by sampling click and non-click using different, unspecified procedures. (Reply #1)
    2. \n
  2. The training and test data were constructed using different sampling procedures. (Replies #6 and #9)
    3. \n
\n

#1 means that models are conditional on the two different sampling procedures, so it's unclear how the contest relates to Avazu's business needs or to any realistic application.  byroni has stated the problem with #2.

", "rawMarkdown": "", "votes": null }, { "id": "57420", "postDate": "11/06/2014 17:09:21", "content": "

True. I do not know if getting the train-data by "judgement sampling" and using it to model the test-data (which did not came from the same sapling technique) is a very good idea or not. I am sure someone would get the best result among all all the competitors but I am afraid all the competitors are ,unknowingly, wasting a lot of their time.

\n

Nonetheless, my comments originate from my personal opinion and Avazu and Kaggle have the ultimate right to put whatever problem they want in front of the competitors and offer prizes.

\n

Best wishes. 

", "rawMarkdown": "", "votes": null }, { "id": "57421", "postDate": "11/06/2014 17:11:05", "content": "

My own guessing is, given test set is exactly 1/10 of training data, it's a 1:10 split after the whole sampling process. Just think it as your 11-fold cross validation.

\n\n

A little reminder: CTR prediction has its class imbalance problem and thank you to your sampling process, it will not be bothering us in this contest. However, things come at a price so if you want to apply the model into reality, this problem has to be addressed. 

", "rawMarkdown": "", "votes": null }, { "id": "57469", "postDate": "11/07/2014 15:18:25", "content": "

Does "This leaderboard is calculated on approximately 20% of the test data." mean, that every time I submit a file, 20% of rows are randomly selected and score is calculated, so, submitting the same file several times may result in different scores on the leaderboard?

", "rawMarkdown": "", "votes": null }, { "id": "57505", "postDate": "11/08/2014 09:27:52", "content": "

I have one question that how to apply sampled CTR back into real CTR, as business scenario mostly needs an estimated CTR other than classification result, so it's regression other than classification problem, any good function to solve that ? Thanks

", "rawMarkdown": "", "votes": null }, { "id": "57510", "postDate": "11/08/2014 12:14:59", "content": "

[quote=Arcady27;57469]

\n

Does "This leaderboard is calculated on approximately 20% of the test data." mean, that every time I submit a file, 20% of rows are randomly selected and score is calculated, so, submitting the same file several times may result in different scores on the leaderboard?

\n

[/quote]

\n

No. Its the same 20%.

", "rawMarkdown": "", "votes": null }, { "id": "58671", "postDate": "11/22/2014 07:08:38", "content": "

I have a quick question. How many lines are there in the current training data? My network is acting really weird lately and it took me 3 times to download the data, so I just want to make sure that the files I downloaded are not corrupted.

\n

Here is what I've got:

\n

wc -l train
40428968 train

\n

Thanks.

", "rawMarkdown": "", "votes": null }, { "id": "58672", "postDate": "11/22/2014 07:19:56", "content": "

I got the same number of lines if that helps:

\n

wc -l train
40428968 train

", "rawMarkdown": "", "votes": null }, { "id": "58676", "postDate": "11/22/2014 08:59:40", "content": "

for that extra secure feeling :)

\n

md5sum train
f5d49ff28f41dc993b9ecb2372abb033 train

\n

md5sum test
04d0e9fc15a32dfc73d21e0d9551eb5d test

", "rawMarkdown": "", "votes": null }, { "id": "59073", "postDate": "11/27/2014 08:18:30", "content": "

Data in order to predict whether the advertisement is to click.However, we have no advertising information. Could you tell me whether the first field data are wrong.

", "rawMarkdown": "", "votes": null }, { "id": "59079", "postDate": "11/27/2014 08:40:58", "content": "

any one can help me?

", "rawMarkdown": "", "votes": null }, { "id": "59083", "postDate": "11/27/2014 09:06:41", "content": "

[quote=chang yangfan;59073]

\n

Data in order to predict whether the advertisement is to click.However, we have no advertising information. Could you tell me whether the first field data are wrong.

\n

[/quote]

\n

yes, the fisrt field is record id

", "rawMarkdown": "", "votes": null }, { "id": "59087", "postDate": "11/27/2014 09:32:32", "content": "

would u update the data?

", "rawMarkdown": "", "votes": null }, { "id": "59258", "postDate": "11/30/2014 05:56:31", "content": "

@chang yangfan - the first column has id, to identify an AD. This will be different in train and test. 

\n

let me know if it helps.

", "rawMarkdown": "", "votes": null }, { "id": "59370", "postDate": "12/02/2014 01:20:53", "content": "

Anyone want to post what they have for sql?

Respectfully asked :) 

", "rawMarkdown": "", "votes": null }, { "id": "59479", "postDate": "12/03/2014 16:47:09", "content": "

I need help. i am doing this for a school project. I am struggling with scientific notation on this dataset. nothing i do is working. please help.

", "rawMarkdown": "", "votes": null }, { "id": "59676", "postDate": "12/06/2014 13:51:55", "content": "

Since click and non-click data is sub-sampled already, so the data is not "complete", and it will be incorrect to engineer any new features which assume relation between records from same IP or same device right?

\n

For instance, I cannot make a pass through data to add a feature like - "# of visits by this IP in this day sofar", even though the data is chronologically ordered. Because it's entirely possible that if an IP has single record, it's merely due to sub-sampling.

\n

So device IP / device ID columns are effectively useless, and so is the fact that data is chronologically ordered. All the rows are effectively independent and as good as random.

\n

If I am right in my statements above, then it's rather unfortunate and rather impractical. I work for a digital marketing product and know that click prediction relies good deal on user session activity.

\n

If my assumptions are wrong, would someone please clarify / correct me on how the chronological ordering / IP / device ID columns are relevant for modeling ?

", "rawMarkdown": "", "votes": null }, { "id": "61453", "postDate": "12/18/2014 08:24:08", "content": "

[quote=Steve Wang;56963]

\n

1. Q: Why CTR is so high? Normally the industry average is more like 0.1%, yet the CTR of the training data set is between 10 to 25%. How is that even possible?

\n

A: The click records and non-click records are subsampled based on different sampling strategies. We have subsampled much fewer non-click records, which makes the CTR really high.

\n

2. Q: By the way, are there any explanations why CTR is maximal on Wednesday?

\n

A: We made the data containing nearly 200k records per hour, that is, we first sampled some clicked records, and then added non-clicked records to make them adding up to nearly 200k, so perhaps there is no strong correlation between sampled CTR and true CTR with respect to time features

\n

3. Q: Was the test-data also subsampled based on different sample strategies?

\n

A: I use uniform sampling this time，so the data is as IID distributed as possible now

\n

4. Q: Could you tell me whether the first field data are wrong.

\n

A: yes, the fisrt field is record id

\n

[/quote]

\n

Is the answer to 3 valid after the last release of data?

", "rawMarkdown": "", "votes": null }, { "id": "62837", "postDate": "01/17/2015 17:28:58", "content": "

in submission:

\n

1) Should ads be in the same order as the test file?
2) Can I submit CTR in exponential form (with "e-XX')? Thanks

", "rawMarkdown": "", "votes": null }, { "id": "63402", "postDate": "02/01/2015 05:38:50", "content": "

Confusion regarding private leaderboard score:

\n

"This leaderboard is calculated on approximately 20% of the test data.
The final results will be based on the other 80%, so the final standings may be different."

\n

When I submit solution for test.csv test file, I'm submitting solution for 20% of the data. How is that submission used to calculate results on other 80% unseen data, if we are not submitting our code/algorithm to run against that 80%.

", "rawMarkdown": "", "votes": null }, { "id": "63403", "postDate": "02/01/2015 06:26:48", "content": "

You are submitting predictions for 100% of the data, but the score is shown to you for only 20% on the public leader board. You get to see the other 80% when then competition is over.

", "rawMarkdown": "", "votes": null }, { "id": "64624", "postDate": "02/20/2015 04:11:58", "content": "

This IP is a real IP address?

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 56974, "author_name": "stevewang0", "author_url": "", "post_date": "10/29/2014 10:19:32", "content": "

Besides, we can not provide detailed information related to TRUE hourly CTR, as it's a business secret.

\n

If you wonder there are some patterns in time series data, you can modeling them.

", "votes": null, "replies": [] }, { "id": 57198, "author_name": "shalnovv", "author_url": "", "post_date": "11/02/2014 16:45:08", "content": "

Can you answer two questions:

\n

1.What is the difference between features represented as 0xXXXXXXXX(for example device_geo_country) and features represented as int values(for example C24)

\n

2.  -1 means that value is absent? For example in C23 feature.

", "votes": null, "replies": [] }, { "id": 57200, "author_name": "stevewang0", "author_url": "", "post_date": "11/02/2014 17:41:19", "content": "

[quote=Shalnovv;57198]

\n

Can you answer two questions:

\n

1.What is the difference between features represented as 0xXXXXXXXX(for example device_geo_country) and features represented as int values(for example C24)

\n

2.  -1 means that value is absent? For example in C23 feature.

\n

[/quote]

\n

they are all categorical values, some are strings, and we hashed them, others kept original values

\n

I have little knowledge of special values of certain column of the data, sorry for that.

", "votes": null, "replies": [] }, { "id": 57254, "author_name": "arjunpjhalda", "author_url": "", "post_date": "11/03/2014 16:44:56", "content": "

1. Was the test-data also subsampled based on different sample strategies? If the train and the test data are not equivalent in terms of CTR, do you think inferring about the test-data using the train-data will produce good results? 

", "votes": null, "replies": [] }, { "id": 57265, "author_name": "stevewang0", "author_url": "", "post_date": "11/04/2014 03:22:54", "content": "

[quote=achilles;57254]

\n

1. Was the test-data also subsampled based on different sample strategies? If the train and the test data are not equivalent in terms of CTR, do you think inferring about the test-data using the train-data will produce good results? 

\n

[/quote]

\n

I use uniform sampling this time，so the data is as IID distributed as possible now

", "votes": null, "replies": [] }, { "id": 57316, "author_name": "", "author_url": "", "post_date": "11/04/2014 22:03:16", "content": "

I don't agree Steve Wang's last comment.

\n

The ultimate goal of any machine learning algorithm is the ability to generalize. We usually, if not always, assume the test data is coming from the same distribution of training data. Deviations are inevitable (who knows about future before it comes?), but if the test data is intentionally sampled from a different distribution, then all machine learning techniques becomes, frankly speaking, useless. 

\n

In practice, this is often represented by a large deviation of results obtained from leader board and offline cross validation, which is misleading and confusing, and often causes a waste of competitors' time.

", "votes": null, "replies": [] }, { "id": 57318, "author_name": "nicholasconnor", "author_url": "", "post_date": "11/05/2014 00:34:57", "content": "

So is the test data unfiltered?

", "votes": null, "replies": [] }, { "id": 57335, "author_name": "stevewang0", "author_url": "", "post_date": "11/05/2014 03:42:25", "content": "

[quote=byronyi;57316]

\n

I don't agree Steve Wang's last comment.

\n

The ultimate goal of any machine learning algorithm is the ability to generalize. We usually, if not always, assume the test data is coming from the same distribution of training data. Deviations are inevitable (who knows about future before it comes?), but if the test data is intentionally sampled from a different distribution, then all machine learning techniques becomes, frankly speaking, useless. 

\n

In practice, this is often represented by a large deviation of results obtained from leader board and offline cross validation, which is misleading and confusing, and often causes a waste of competitors' time.

\n

[/quote]

\n

Your comment is quite reasonable，so I made a private benchmark several days ago，the train error and test error matched very well，using a standard algorithm，with no tricks

", "votes": null, "replies": [] }, { "id": 57417, "author_name": "eagle159065", "author_url": "", "post_date": "11/06/2014 16:55:16", "content": "

Based on the competition admin's answers, this seems like an odd setup:

\n
    \n
  1. The training data were constructed by sampling click and non-click using different, unspecified procedures. (Reply #1)
    2. \n
  2. The training and test data were constructed using different sampling procedures. (Replies #6 and #9)
    3. \n
\n

#1 means that models are conditional on the two different sampling procedures, so it's unclear how the contest relates to Avazu's business needs or to any realistic application.  byroni has stated the problem with #2.

", "votes": null, "replies": [] }, { "id": 57420, "author_name": "arjunpjhalda", "author_url": "", "post_date": "11/06/2014 17:09:21", "content": "

True. I do not know if getting the train-data by "judgement sampling" and using it to model the test-data (which did not came from the same sapling technique) is a very good idea or not. I am sure someone would get the best result among all all the competitors but I am afraid all the competitors are ,unknowingly, wasting a lot of their time.

\n

Nonetheless, my comments originate from my personal opinion and Avazu and Kaggle have the ultimate right to put whatever problem they want in front of the competitors and offer prizes.

\n

Best wishes. 

", "votes": null, "replies": [] }, { "id": 57421, "author_name": "", "author_url": "", "post_date": "11/06/2014 17:11:05", "content": "

My own guessing is, given test set is exactly 1/10 of training data, it's a 1:10 split after the whole sampling process. Just think it as your 11-fold cross validation.

\n\n

A little reminder: CTR prediction has its class imbalance problem and thank you to your sampling process, it will not be bothering us in this contest. However, things come at a price so if you want to apply the model into reality, this problem has to be addressed. 

", "votes": null, "replies": [] }, { "id": 57469, "author_name": "arcady27", "author_url": "", "post_date": "11/07/2014 15:18:25", "content": "

Does "This leaderboard is calculated on approximately 20% of the test data." mean, that every time I submit a file, 20% of rows are randomly selected and score is calculated, so, submitting the same file several times may result in different scores on the leaderboard?

", "votes": null, "replies": [] }, { "id": 57505, "author_name": "linpingta", "author_url": "", "post_date": "11/08/2014 09:27:52", "content": "

I have one question that how to apply sampled CTR back into real CTR, as business scenario mostly needs an estimated CTR other than classification result, so it's regression other than classification problem, any good function to solve that ? Thanks

", "votes": null, "replies": [] }, { "id": 57510, "author_name": "rohanrao", "author_url": "", "post_date": "11/08/2014 12:14:59", "content": "

[quote=Arcady27;57469]

\n

Does "This leaderboard is calculated on approximately 20% of the test data." mean, that every time I submit a file, 20% of rows are randomly selected and score is calculated, so, submitting the same file several times may result in different scores on the leaderboard?

\n

[/quote]

\n

No. Its the same 20%.

", "votes": null, "replies": [] }, { "id": 58671, "author_name": "ggglhf", "author_url": "", "post_date": "11/22/2014 07:08:38", "content": "

I have a quick question. How many lines are there in the current training data? My network is acting really weird lately and it took me 3 times to download the data, so I just want to make sure that the files I downloaded are not corrupted.

\n

Here is what I've got:

\n

wc -l train
40428968 train

\n

Thanks.

", "votes": null, "replies": [] }, { "id": 58672, "author_name": "cdjbee", "author_url": "", "post_date": "11/22/2014 07:19:56", "content": "

I got the same number of lines if that helps:

\n

wc -l train
40428968 train

", "votes": null, "replies": [] }, { "id": 58676, "author_name": "vvarkey", "author_url": "", "post_date": "11/22/2014 08:59:40", "content": "

for that extra secure feeling :)

\n

md5sum train
f5d49ff28f41dc993b9ecb2372abb033 train

\n

md5sum test
04d0e9fc15a32dfc73d21e0d9551eb5d test

", "votes": null, "replies": [] }, { "id": 59073, "author_name": "changyangfan", "author_url": "", "post_date": "11/27/2014 08:18:30", "content": "

Data in order to predict whether the advertisement is to click.However, we have no advertising information. Could you tell me whether the first field data are wrong.

", "votes": null, "replies": [] }, { "id": 59079, "author_name": "changyangfan", "author_url": "", "post_date": "11/27/2014 08:40:58", "content": "

any one can help me?

", "votes": null, "replies": [] }, { "id": 59083, "author_name": "stevewang0", "author_url": "", "post_date": "11/27/2014 09:06:41", "content": "

[quote=chang yangfan;59073]

\n

Data in order to predict whether the advertisement is to click.However, we have no advertising information. Could you tell me whether the first field data are wrong.

\n

[/quote]

\n

yes, the fisrt field is record id

", "votes": null, "replies": [] }, { "id": 59087, "author_name": "changyangfan", "author_url": "", "post_date": "11/27/2014 09:32:32", "content": "

would u update the data?

", "votes": null, "replies": [] }, { "id": 59258, "author_name": "", "author_url": "", "post_date": "11/30/2014 05:56:31", "content": "

@chang yangfan - the first column has id, to identify an AD. This will be different in train and test. 

\n

let me know if it helps.

", "votes": null, "replies": [] }, { "id": 59370, "author_name": "lasdivine", "author_url": "", "post_date": "12/02/2014 01:20:53", "content": "

Anyone want to post what they have for sql?

Respectfully asked :) 

", "votes": null, "replies": [] }, { "id": 59479, "author_name": "amandachristinawashington", "author_url": "", "post_date": "12/03/2014 16:47:09", "content": "

I need help. i am doing this for a school project. I am struggling with scientific notation on this dataset. nothing i do is working. please help.

", "votes": null, "replies": [] }, { "id": 59676, "author_name": "kapilmalik", "author_url": "", "post_date": "12/06/2014 13:51:55", "content": "

Since click and non-click data is sub-sampled already, so the data is not "complete", and it will be incorrect to engineer any new features which assume relation between records from same IP or same device right?

\n

For instance, I cannot make a pass through data to add a feature like - "# of visits by this IP in this day sofar", even though the data is chronologically ordered. Because it's entirely possible that if an IP has single record, it's merely due to sub-sampling.

\n

So device IP / device ID columns are effectively useless, and so is the fact that data is chronologically ordered. All the rows are effectively independent and as good as random.

\n

If I am right in my statements above, then it's rather unfortunate and rather impractical. I work for a digital marketing product and know that click prediction relies good deal on user session activity.

\n

If my assumptions are wrong, would someone please clarify / correct me on how the chronological ordering / IP / device ID columns are relevant for modeling ?

", "votes": null, "replies": [] }, { "id": 61453, "author_name": "blindape", "author_url": "", "post_date": "12/18/2014 08:24:08", "content": "

[quote=Steve Wang;56963]

\n

1. Q: Why CTR is so high? Normally the industry average is more like 0.1%, yet the CTR of the training data set is between 10 to 25%. How is that even possible?

\n

A: The click records and non-click records are subsampled based on different sampling strategies. We have subsampled much fewer non-click records, which makes the CTR really high.

\n

2. Q: By the way, are there any explanations why CTR is maximal on Wednesday?

\n

A: We made the data containing nearly 200k records per hour, that is, we first sampled some clicked records, and then added non-clicked records to make them adding up to nearly 200k, so perhaps there is no strong correlation between sampled CTR and true CTR with respect to time features

\n

3. Q: Was the test-data also subsampled based on different sample strategies?

\n

A: I use uniform sampling this time，so the data is as IID distributed as possible now

\n

4. Q: Could you tell me whether the first field data are wrong.

\n

A: yes, the fisrt field is record id

\n

[/quote]

\n

Is the answer to 3 valid after the last release of data?

", "votes": null, "replies": [] }, { "id": 62837, "author_name": "justwajdi", "author_url": "", "post_date": "01/17/2015 17:28:58", "content": "

in submission:

\n

1) Should ads be in the same order as the test file?
2) Can I submit CTR in exponential form (with "e-XX')? Thanks

", "votes": null, "replies": [] }, { "id": 63402, "author_name": "kromil", "author_url": "", "post_date": "02/01/2015 05:38:50", "content": "

Confusion regarding private leaderboard score:

\n

"This leaderboard is calculated on approximately 20% of the test data.
The final results will be based on the other 80%, so the final standings may be different."

\n

When I submit solution for test.csv test file, I'm submitting solution for 20% of the data. How is that submission used to calculate results on other 80% unseen data, if we are not submitting our code/algorithm to run against that 80%.

", "votes": null, "replies": [] }, { "id": 63403, "author_name": "jfkingiii", "author_url": "", "post_date": "02/01/2015 06:26:48", "content": "

You are submitting predictions for 100% of the data, but the score is shown to you for only 20% on the public leader board. You get to see the other 80% when then competition is over.

", "votes": null, "replies": [] }, { "id": 64624, "author_name": "changyangfan", "author_url": "", "post_date": "02/20/2015 04:11:58", "content": "

This IP is a real IP address?

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "56963": "", "56974": "", "57198": "", "57200": "", "57254": "", "57265": "", "57316": "", "57318": "", "57335": "", "57417": "", "57420": "", "57421": "", "57469": "", "57505": "", "57510": "", "58671": "", "58672": "", "58676": "", "59073": "", "59079": "", "59083": "", "59087": "", "59258": "", "59370": "", "59479": "", "59676": "", "61453": "", "62837": "", "63402": "", "63403": "", "64624": "" }, "source": "meta" }