{
  "id": 81500,
  "title": "Final Score Meaning",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/81500",
  "author_name": "",
  "post_date": "2019-02-22T01:26:03.151477800Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>The competition has been judged.  In the first stage before the leak, the top score was around 0.55 I think.</p>\n\n<p>Now after second stage results, the top score is 0.88</p>\n\n<p>As a Product Manager, should I accept this solution or discard it.</p>\n\n<p>I am a newbie to this field. Kindly humor me, as I am unable to decipher it.</p>",
  "messages": [
    {
      "id": "476324",
      "postDate": "02/22/2019 01:26:03",
      "content": "<p>The competition has been judged.  In the first stage before the leak, the top score was around 0.55 I think.</p>\n\n<p>Now after second stage results, the top score is 0.88</p>\n\n<p>As a Product Manager, should I accept this solution or discard it.</p>\n\n<p>I am a newbie to this field. Kindly humor me, as I am unable to decipher it.</p>",
      "rawMarkdown": "The competition has been judged.  In the first stage before the leak, the top score was around 0.55 I think.\n\nNow after second stage results, the top score is 0.88\n\nAs a Product Manager, should I accept this solution or discard it.\n\nI am a newbie to this field. Kindly humor me, as I am unable to decipher it.",
      "votes": null
    },
    {
      "id": "476339",
      "postDate": "02/22/2019 01:54:24",
      "content": "<p>In my opinion, this competition has poor business value mostly because the metric chosen is not adapted to this kind of problems. A classification task (or even multi-classification) would have been more useful. As a result, the best model scores 0.007 better (in terms of rmse!) than only-zeros prediction. I think it's not significant enough to consider it as a useful model in real life... So rather than discarding the solution (congratulation to the winners by the way), discarding the framework of the problem would have been a better idea ! Being able to assess the impact a model can have beforehand is more important than actually tuning it. </p>\n\n<p>By the way, I slightly did better than 0-pred by taking the average revenue of regular-users present in the test set. As for the definition of regular user, you can choose it as you want. I evaluated them as the users that came back purchasing every month in the train.</p>",
      "rawMarkdown": "In my opinion, this competition has poor business value mostly because the metric chosen is not adapted to this kind of problems. A classification task (or even multi-classification) would have been more useful. As a result, the best model scores 0.007 better (in terms of rmse!) than only-zeros prediction. I think it's not significant enough to consider it as a useful model in real life... So rather than discarding the solution (congratulation to the winners by the way), discarding the framework of the problem would have been a better idea ! Being able to assess the impact a model can have beforehand is more important than actually tuning it. \n\nBy the way, I slightly did better than 0-pred by taking the average revenue of regular-users present in the test set. As for the definition of regular user, you can choose it as you want. I evaluated them as the users that came back purchasing every month in the train.",
      "votes": null
    },
    {
      "id": "476343",
      "postDate": "02/22/2019 02:05:18",
      "content": "<p>Yes, congratulations to the Winner's. Apologies for any misinterpretations. </p>\n\n<p>Thank you for your answer. </p>\n\n<p>Score is not not significant enough for pursuing further.</p>",
      "rawMarkdown": "Yes, congratulations to the Winner's. Apologies for any misinterpretations. \n\n Thank you for your answer. \n\nScore is not not significant enough for pursuing further.",
      "votes": null
    },
    {
      "id": "476374",
      "postDate": "02/22/2019 03:29:48",
      "content": "<p>Just my opinion and own interpretation no worries</p>",
      "rawMarkdown": "Just my opinion and own interpretation no worries",
      "votes": null
    },
    {
      "id": "479447",
      "postDate": "02/27/2019 06:32:48",
      "content": "<p>YES,a classification target is more suitable for this competition.</p>",
      "rawMarkdown": "YES,a classification target is more suitable for this competition.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 476339,
      "author_name": "tita1708",
      "author_url": "",
      "post_date": "02/22/2019 01:54:24",
      "content": "<p>In my opinion, this competition has poor business value mostly because the metric chosen is not adapted to this kind of problems. A classification task (or even multi-classification) would have been more useful. As a result, the best model scores 0.007 better (in terms of rmse!) than only-zeros prediction. I think it's not significant enough to consider it as a useful model in real life... So rather than discarding the solution (congratulation to the winners by the way), discarding the framework of the problem would have been a better idea ! Being able to assess the impact a model can have beforehand is more important than actually tuning it. </p>\n\n<p>By the way, I slightly did better than 0-pred by taking the average revenue of regular-users present in the test set. As for the definition of regular user, you can choose it as you want. I evaluated them as the users that came back purchasing every month in the train.</p>",
      "votes": null,
      "replies": [
        {
          "id": 476343,
          "author_name": "viva32105",
          "author_url": "",
          "post_date": "02/22/2019 02:05:18",
          "content": "<p>Yes, congratulations to the Winner's. Apologies for any misinterpretations. </p>\n\n<p>Thank you for your answer. </p>\n\n<p>Score is not not significant enough for pursuing further.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 476374,
          "author_name": "tita1708",
          "author_url": "",
          "post_date": "02/22/2019 03:29:48",
          "content": "<p>Just my opinion and own interpretation no worries</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 479447,
          "author_name": "twopence",
          "author_url": "",
          "post_date": "02/27/2019 06:32:48",
          "content": "<p>YES,a classification target is more suitable for this competition.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "476324": "The competition has been judged.  In the first stage before the leak, the top score was around 0.55 I think.\n\nNow after second stage results, the top score is 0.88\n\nAs a Product Manager, should I accept this solution or discard it.\n\nI am a newbie to this field. Kindly humor me, as I am unable to decipher it.",
    "476339": "In my opinion, this competition has poor business value mostly because the metric chosen is not adapted to this kind of problems. A classification task (or even multi-classification) would have been more useful. As a result, the best model scores 0.007 better (in terms of rmse!) than only-zeros prediction. I think it's not significant enough to consider it as a useful model in real life... So rather than discarding the solution (congratulation to the winners by the way), discarding the framework of the problem would have been a better idea ! Being able to assess the impact a model can have beforehand is more important than actually tuning it. \n\nBy the way, I slightly did better than 0-pred by taking the average revenue of regular-users present in the test set. As for the definition of regular user, you can choose it as you want. I evaluated them as the users that came back purchasing every month in the train.",
    "476343": "Yes, congratulations to the Winner's. Apologies for any misinterpretations. \n\n Thank you for your answer. \n\nScore is not not significant enough for pursuing further.",
    "476374": "Just my opinion and own interpretation no worries",
    "479447": "YES,a classification target is more suitable for this competition."
  },
  "source": "meta"
}