{ "id": 11090, "title": "Ratio click vs non-click", "url": "/competitions/avazu-ctr-prediction/discussion/11090", "author_name": "", "post_date": "2014-11-30T06:02:41.177Z", "votes": null, "comment_count": 4, "views": 1829, "content": "

Do you think the ratio # click / # non-click is controlled in the test set to mimic the reality?

\n

What may be this rate 20%, 30%?

\n

Thks,

\n

HR

", "messages": [ { "id": "59259", "postDate": "11/30/2014 06:02:41", "content": "

Do you think the ratio # click / # non-click is controlled in the test set to mimic the reality?

\n

What may be this rate 20%, 30%?

\n

Thks,

\n

HR

", "rawMarkdown": "", "votes": null }, { "id": "59284", "postDate": "11/30/2014 19:13:55", "content": "

The CTRs in the training data are as follow:

\n

21: 17.4%
22: 15.7%
23: 18.2%
24: 17.5%
25: 18.2%
26: 18.3%
27: 18.1%
28: 15.2%
29: 15.7%
30: 16.9%

\n\n

For whatever reason my classifier is very sensitive to the CTR of the day I test it on, I can see the logloss vary the same way as the CTR.

\n\n

My LB score falls between my validation score on 29 and 30, thus I assume the CTR on the test set is between 15.7% and 16.9% (at least for the 20% of data used in the LB).

\n\n

Now obviously these are not real-world CTRs, I'm not sure what you meant by "mimic the reality"?

", "rawMarkdown": "", "votes": null }, { "id": "59288", "postDate": "11/30/2014 19:29:22", "content": "

As you said, I mean the real world CTR (what Avazu sees from their data).

\n

But the range from which CTR varies should be approximately known in the field of online advertising  particularly for this kind of advertising, right?

", "rawMarkdown": "", "votes": null }, { "id": "59328", "postDate": "12/01/2014 13:27:55", "content": "

The real-world CTRs are probably lower by a factor of 10 (in other words, ~1.5%).

\n

As a though experiment, ask yourself - Do you click on 15% of the online ads you see?

", "rawMarkdown": "", "votes": null }, { "id": "59343", "postDate": "12/01/2014 17:01:22", "content": "

To be more specific, in my experience, these are some real-world CTRs taking into consideration network and other factors:

\n\n

SEARCH network  (advertising within Google, Yahoo or Bing)

\n

- Brand Terms  (the keyword is advertiser's brand):  Avg CTR for Top3 positions: 10%

\n

- Generic Terms (i.e.: "online shoes"):                       Avg CTR for Top3 positions:   2%

\n

 

\n

DISPLAY network (advertising within any site except Search Engines)

\n

- Text Ads:       0,1%

\n

- Banner Ads:   0,3%   (obviously depends on banner size)

\n

 

\n

There are also other classification for ads:

\n

- Contextually oriented ads (ad shows only on theme related sites):  Higher CTR

\n

- Interest oriented ads  (ad shows wherever, only to people with interests related to Ad): Low CTR

\n

 

\n

Take into account that ads 'below the fold' (below what a person with a normal resolution would see without scrolling) could have extremely low CTRs.

\n

Then, there are other considerations like remarketing ads, video ads, etc with slightly different CTRs.

\n

Avazu's data is probably only for the Display Network, so CTRs in the training data are very far from real-world metrics.

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 59284, "author_name": "alexistaugeron", "author_url": "", "post_date": "11/30/2014 19:13:55", "content": "

The CTRs in the training data are as follow:

\n

21: 17.4%
22: 15.7%
23: 18.2%
24: 17.5%
25: 18.2%
26: 18.3%
27: 18.1%
28: 15.2%
29: 15.7%
30: 16.9%

\n\n

For whatever reason my classifier is very sensitive to the CTR of the day I test it on, I can see the logloss vary the same way as the CTR.

\n\n

My LB score falls between my validation score on 29 and 30, thus I assume the CTR on the test set is between 15.7% and 16.9% (at least for the 20% of data used in the LB).

\n\n

Now obviously these are not real-world CTRs, I'm not sure what you meant by "mimic the reality"?

", "votes": null, "replies": [] }, { "id": 59288, "author_name": "herimanitra", "author_url": "", "post_date": "11/30/2014 19:29:22", "content": "

As you said, I mean the real world CTR (what Avazu sees from their data).

\n

But the range from which CTR varies should be approximately known in the field of online advertising  particularly for this kind of advertising, right?

", "votes": null, "replies": [] }, { "id": 59328, "author_name": "inversion", "author_url": "", "post_date": "12/01/2014 13:27:55", "content": "

The real-world CTRs are probably lower by a factor of 10 (in other words, ~1.5%).

\n

As a though experiment, ask yourself - Do you click on 15% of the online ads you see?

", "votes": null, "replies": [] }, { "id": 59343, "author_name": "fernandoconstantino", "author_url": "", "post_date": "12/01/2014 17:01:22", "content": "

To be more specific, in my experience, these are some real-world CTRs taking into consideration network and other factors:

\n\n

SEARCH network  (advertising within Google, Yahoo or Bing)

\n

- Brand Terms  (the keyword is advertiser's brand):  Avg CTR for Top3 positions: 10%

\n

- Generic Terms (i.e.: "online shoes"):                       Avg CTR for Top3 positions:   2%

\n

 

\n

DISPLAY network (advertising within any site except Search Engines)

\n

- Text Ads:       0,1%

\n

- Banner Ads:   0,3%   (obviously depends on banner size)

\n

 

\n

There are also other classification for ads:

\n

- Contextually oriented ads (ad shows only on theme related sites):  Higher CTR

\n

- Interest oriented ads  (ad shows wherever, only to people with interests related to Ad): Low CTR

\n

 

\n

Take into account that ads 'below the fold' (below what a person with a normal resolution would see without scrolling) could have extremely low CTRs.

\n

Then, there are other considerations like remarketing ads, video ads, etc with slightly different CTRs.

\n

Avazu's data is probably only for the Display Network, so CTRs in the training data are very far from real-world metrics.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "59259": "", "59284": "", "59288": "", "59328": "", "59343": "" }, "source": "meta" }