{
  "id": 11016,
  "title": "0.39 - more to the dataset than meets the eye?",
  "url": "/competitions/avazu-ctr-prediction/discussion/11016",
  "author_name": "",
  "post_date": "2014-11-22T13:52:51.133Z",
  "votes": 6,
  "comment_count": 6,
  "views": 2845,
  "content": "<p>I've ran lots of validation programs but haven't made an entry yet. What I've found interesting is that 0.39 seems to be very common. I have totally changed the data to be unrecognisable but still get 0.39. I expected better or worst not the same considering the changes I had made. I then removed each variable and ran Tinrtgu's v3 - you get 0.39 all the way down to only 4 features left. You can get .42 using just 2 features from the original dataset. I think a breakthrough may come through data prep not code this time. Anyone else notice oddities?</p>",
  "messages": [
    {
      "id": "58680",
      "postDate": "11/22/2014 13:52:51",
      "content": "<p>I've ran lots of validation programs but haven't made an entry yet. What I've found interesting is that 0.39 seems to be very common. I have totally changed the data to be unrecognisable but still get 0.39. I expected better or worst not the same considering the changes I had made. I then removed each variable and ran Tinrtgu's v3 - you get 0.39 all the way down to only 4 features left. You can get .42 using just 2 features from the original dataset. I think a breakthrough may come through data prep not code this time. Anyone else notice oddities?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "58681",
      "postDate": "11/22/2014 14:00:43",
      "content": "<p>I tried training on only 10% of the data and... yep, 0.39.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "58683",
      "postDate": "11/22/2014 14:22:41",
      "content": "<p>What is the score for optimal constant predictions?</p>\n<p>Edit: hmm, using two features 0.42. Maybe there is small number of clusters? And then basically the only thing what the algorithm would be doing is calculating different means for each cluster.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "58684",
      "postDate": "11/22/2014 14:27:23",
      "content": "<p>[quote=Herra Huu;58683]</p>\n<p>What is the score for optimal constant predictions?</p>\n<p>[/quote]</p>\n<p>0.44 I believe</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "58685",
      "postDate": "11/22/2014 14:27:31",
      "content": "",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "58687",
      "postDate": "11/22/2014 15:58:40",
      "content": "<p>Yep, I'm a few steps behind you all but I had noticed that the official benchmark should be around 0.45 (constant prediction) and simple averaging across subsets can get you down to 0.42.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "58691",
      "postDate": "11/22/2014 16:46:36",
      "content": "<p>[quote=tinrtgu;58684]</p>\n<p>[quote=Herra Huu;58683]</p>\n<p>What is the score for optimal constant predictions?</p>\n<p>[/quote]</p>\n<p>0.44 I believe</p>\n<p>[/quote]</p>\n<p>Yup. My submission right now just uses the average ctr by hour, not quite constant but pretty close. Score is 0.443</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 58681,
      "author_name": "ggglhf",
      "author_url": "",
      "post_date": "11/22/2014 14:00:43",
      "content": "<p>I tried training on only 10% of the data and... yep, 0.39.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 58683,
      "author_name": "herrahuu",
      "author_url": "",
      "post_date": "11/22/2014 14:22:41",
      "content": "<p>What is the score for optimal constant predictions?</p>\n<p>Edit: hmm, using two features 0.42. Maybe there is small number of clusters? And then basically the only thing what the algorithm would be doing is calculating different means for each cluster.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 58684,
      "author_name": "ggglhf",
      "author_url": "",
      "post_date": "11/22/2014 14:27:23",
      "content": "<p>[quote=Herra Huu;58683]</p>\n<p>What is the score for optimal constant predictions?</p>\n<p>[/quote]</p>\n<p>0.44 I believe</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 58685,
      "author_name": "ggglhf",
      "author_url": "",
      "post_date": "11/22/2014 14:27:31",
      "content": "",
      "votes": null,
      "replies": []
    },
    {
      "id": 58687,
      "author_name": "cartographic",
      "author_url": "",
      "post_date": "11/22/2014 15:58:40",
      "content": "<p>Yep, I'm a few steps behind you all but I had noticed that the official benchmark should be around 0.45 (constant prediction) and simple averaging across subsets can get you down to 0.42.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 58691,
      "author_name": "liamhuber",
      "author_url": "",
      "post_date": "11/22/2014 16:46:36",
      "content": "<p>[quote=tinrtgu;58684]</p>\n<p>[quote=Herra Huu;58683]</p>\n<p>What is the score for optimal constant predictions?</p>\n<p>[/quote]</p>\n<p>0.44 I believe</p>\n<p>[/quote]</p>\n<p>Yup. My submission right now just uses the average ctr by hour, not quite constant but pretty close. Score is 0.443</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "58680": "",
    "58681": "",
    "58683": "",
    "58684": "",
    "58685": "",
    "58687": "",
    "58691": ""
  },
  "source": "meta"
}