{
  "id": 11148,
  "title": "Playing with data: a novice work",
  "url": "/competitions/avazu-ctr-prediction/discussion/11148",
  "author_name": "ighyboo",
  "post_date": "2014-12-05T17:53:09.477000",
  "votes": 4,
  "comment_count": 0,
  "views": 900,
  "content": "<p>Hello everyone,</p>\n<p>this is my first competition and my first post so please forgive me if I say something obvious or silly :)</p>\n<p>I'm not an expert in modelling but I have a passion for data and I'm slowly learning the ropes in R.</p>\n<p>I did some exploratory data visualization of the dataset hoping to get some insights about good predictors but it was only at the very end that I read that click VS non-click were sampled in different manner.. So I'm not sure if my analysis is still valid.. Nonetheless... Here's a link to a&nbsp;<a href=\"https://public.tableausoftware.com/workbooks/CTR_Exploratory\">Tableau Public Workbook</a></p>\n<p>Here some notes:</p>\n<ul>\n<li><span style=\"line-height: 1.4\">I loaded the data in a DB and sampled ~400k records</span></li>\n<li><span style=\"line-height: 1.4\">For each tab I compared one of the variable with the CTR as percentage of the records with such value, as well as reporting the total number of records (once you open the sheet it should make more sense)</span></li>\n<li><span style=\"line-height: 1.4\">For each variable I only considered those value that had at least 100 records/observations</span></li>\n<li><span style=\"line-height: 1.4\">Size is calculated as C15*C16</span></li>\n<li><span style=\"line-height: 1.4\">Banner_overview is calculated as concatenated string of banner_pos+C15+C16</span></li>\n<li><span style=\"line-height: 1.4\">I did not include all varibles in my analysis (just those that had few category)</span></li>\n</ul>\n\n<p>Based on that analysis here some more speculations:</p>\n<ul>\n<li><span style=\"line-height: 1.4\">C18 seems the best candidate for a potential Time-zone (maybe a region like EMEA/APAC/etc..), in particular C18=0 and C18=3 seem to be displayed at different time of the day (looking at raw count), once split into days though it's harder to see that pattern</span></li>\n<li><span style=\"line-height: 1.4\">C20-21 could be Background-Foreground colours (web safe pehaps?) with C20=-1 when no BG</span></li>\n</ul>\n<p>I don't know if this is useful nor if it's statistically correct but any feedback would be greatly appreciated...</p>\n<p>I guess the next step is to incorporate this knowledge into a model, I would like to use random forest or SVM as from my understanding they are quite good &quot;black boxes&quot;..<br>Is that correct? Should I transform the features into categories then? or encode them in some ways?</p>\n<p>(this is where my knowledge gets very patch)</p>\n<p>Thanks a lot :)</p>\n<p>Igor</p>",
  "messages": [
    {
      "id": 59632,
      "postDate": "2014-12-05T17:53:09.477Z",
      "content": "<p>Hello everyone,</p>\n<p>this is my first competition and my first post so please forgive me if I say something obvious or silly :)</p>\n<p>I'm not an expert in modelling but I have a passion for data and I'm slowly learning the ropes in R.</p>\n<p>I did some exploratory data visualization of the dataset hoping to get some insights about good predictors but it was only at the very end that I read that click VS non-click were sampled in different manner.. So I'm not sure if my analysis is still valid.. Nonetheless... Here's a link to a&nbsp;<a href=\"https://public.tableausoftware.com/workbooks/CTR_Exploratory\">Tableau Public Workbook</a></p>\n<p>Here some notes:</p>\n<ul>\n<li><span style=\"line-height: 1.4\">I loaded the data in a DB and sampled ~400k records</span></li>\n<li><span style=\"line-height: 1.4\">For each tab I compared one of the variable with the CTR as percentage of the records with such value, as well as reporting the total number of records (once you open the sheet it should make more sense)</span></li>\n<li><span style=\"line-height: 1.4\">For each variable I only considered those value that had at least 100 records/observations</span></li>\n<li><span style=\"line-height: 1.4\">Size is calculated as C15*C16</span></li>\n<li><span style=\"line-height: 1.4\">Banner_overview is calculated as concatenated string of banner_pos+C15+C16</span></li>\n<li><span style=\"line-height: 1.4\">I did not include all varibles in my analysis (just those that had few category)</span></li>\n</ul>\n\n<p>Based on that analysis here some more speculations:</p>\n<ul>\n<li><span style=\"line-height: 1.4\">C18 seems the best candidate for a potential Time-zone (maybe a region like EMEA/APAC/etc..), in particular C18=0 and C18=3 seem to be displayed at different time of the day (looking at raw count), once split into days though it's harder to see that pattern</span></li>\n<li><span style=\"line-height: 1.4\">C20-21 could be Background-Foreground colours (web safe pehaps?) with C20=-1 when no BG</span></li>\n</ul>\n<p>I don't know if this is useful nor if it's statistically correct but any feedback would be greatly appreciated...</p>\n<p>I guess the next step is to incorporate this knowledge into a model, I would like to use random forest or SVM as from my understanding they are quite good &quot;black boxes&quot;..<br>Is that correct? Should I transform the features into categories then? or encode them in some ways?</p>\n<p>(this is where my knowledge gets very patch)</p>\n<p>Thanks a lot :)</p>\n<p>Igor</p>",
      "votes": 4
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "59632": ""
  }
}