{
  "id": 301300,
  "title": "Missing targets - Reconstruct and then model?",
  "url": "/competitions/g-research-crypto-forecasting/discussion/301300",
  "author_name": "",
  "post_date": "2022-01-17T00:52:44.442727600Z",
  "votes": 4,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hey guys! </p>\n<p>I was wondering what you do when modeling/predicting the data. For some crypto assets there are more than 100 or 200K missing target values. Do you drop them? Do you reconstruct the target via some of the notebooks out there and use the target_reconstruct column as the dependent variable? What is a good approach here??</p>\n<p>Thanks in advance!!<br>\nTarget stats for BTC and DOGE:</p>\n<p>Number of BTC target values that are null: 937<br>\nNumber of DOGE target values that are null: 273132</p>\n<p>Vincent </p>",
  "messages": [
    {
      "id": "1652758",
      "postDate": "01/17/2022 00:52:44",
      "content": "<p>Hey guys! </p>\n<p>I was wondering what you do when modeling/predicting the data. For some crypto assets there are more than 100 or 200K missing target values. Do you drop them? Do you reconstruct the target via some of the notebooks out there and use the target_reconstruct column as the dependent variable? What is a good approach here??</p>\n<p>Thanks in advance!!<br>\nTarget stats for BTC and DOGE:</p>\n<p>Number of BTC target values that are null: 937<br>\nNumber of DOGE target values that are null: 273132</p>\n<p>Vincent </p>",
      "rawMarkdown": "Hey guys! \n\nI was wondering what you do when modeling/predicting the data. For some crypto assets there are more than 100 or 200K missing target values. Do you drop them? Do you reconstruct the target via some of the notebooks out there and use the target_reconstruct column as the dependent variable? What is a good approach here??\n\nThanks in advance!!\nTarget stats for BTC and DOGE:\n\nNumber of BTC target values that are null: 937\nNumber of DOGE target values that are null: 273132\n\nVincent",
      "votes": null
    },
    {
      "id": "1654944",
      "postDate": "01/19/2022 01:21:40",
      "content": "<p>Anyone? I feel it's a pretty important question..</p>",
      "rawMarkdown": "Anyone? I feel it's a pretty important question..",
      "votes": null
    },
    {
      "id": "1654953",
      "postDate": "01/19/2022 01:40:37",
      "content": "<p>because if i do y = df['target'] the model will complain of NaN values feeded to .fit()</p>",
      "rawMarkdown": "because if i do y = df['target'] the model will complain of NaN values feeded to .fit()",
      "votes": null
    },
    {
      "id": "1656006",
      "postDate": "01/19/2022 03:24:18",
      "content": "<p>The time period in which each coin's data is collected differs.  I think Doge doesn't have any 2018 data which exaggerates the problem.  For me, I just use forward fill to replace na values.</p>",
      "rawMarkdown": "The time period in which each coin's data is collected differs.  I think Doge doesn't have any 2018 data which exaggerates the problem.  For me, I just use forward fill to replace na values.",
      "votes": null
    },
    {
      "id": "1656020",
      "postDate": "01/19/2022 03:41:22",
      "content": "<p>Thx <a href=\"https://www.kaggle.com/petersk20\" target=\"_blank\">@petersk20</a> , also the Target missing target values you forward fill? There's some notebooks that recreate the target for NaN target values, so could use those maybe instead of Forward Fill or dropping these rows</p>",
      "rawMarkdown": "Thx @petersk20 , also the Target missing target values you forward fill? There's some notebooks that recreate the target for NaN target values, so could use those maybe instead of Forward Fill or dropping these rows",
      "votes": null
    },
    {
      "id": "1656600",
      "postDate": "01/19/2022 13:23:21",
      "content": "<p>I would remove those from training if the target itself was missing, but you could also try to calculate it.  This will take time to do though in your submission notebook and might cause the notebook to become more complicated than it needs to be.  There aren't many NAs compared to the number of training rows so it won't make a difference either way.</p>",
      "rawMarkdown": "I would remove those from training if the target itself was missing, but you could also try to calculate it.  This will take time to do though in your submission notebook and might cause the notebook to become more complicated than it needs to be.  There aren't many NAs compared to the number of training rows so it won't make a difference either way.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1654944,
      "author_name": "vweimer",
      "author_url": "",
      "post_date": "01/19/2022 01:21:40",
      "content": "<p>Anyone? I feel it's a pretty important question..</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1654953,
      "author_name": "vweimer",
      "author_url": "",
      "post_date": "01/19/2022 01:40:37",
      "content": "<p>because if i do y = df['target'] the model will complain of NaN values feeded to .fit()</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1656006,
      "author_name": "petersk20",
      "author_url": "",
      "post_date": "01/19/2022 03:24:18",
      "content": "<p>The time period in which each coin's data is collected differs.  I think Doge doesn't have any 2018 data which exaggerates the problem.  For me, I just use forward fill to replace na values.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1656020,
      "author_name": "vweimer",
      "author_url": "",
      "post_date": "01/19/2022 03:41:22",
      "content": "<p>Thx <a href=\"https://www.kaggle.com/petersk20\" target=\"_blank\">@petersk20</a> , also the Target missing target values you forward fill? There's some notebooks that recreate the target for NaN target values, so could use those maybe instead of Forward Fill or dropping these rows</p>",
      "votes": null,
      "replies": [
        {
          "id": 1656600,
          "author_name": "petersk20",
          "author_url": "",
          "post_date": "01/19/2022 13:23:21",
          "content": "<p>I would remove those from training if the target itself was missing, but you could also try to calculate it.  This will take time to do though in your submission notebook and might cause the notebook to become more complicated than it needs to be.  There aren't many NAs compared to the number of training rows so it won't make a difference either way.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1652758": "Hey guys! \n\nI was wondering what you do when modeling/predicting the data. For some crypto assets there are more than 100 or 200K missing target values. Do you drop them? Do you reconstruct the target via some of the notebooks out there and use the target_reconstruct column as the dependent variable? What is a good approach here??\n\nThanks in advance!!\nTarget stats for BTC and DOGE:\n\nNumber of BTC target values that are null: 937\nNumber of DOGE target values that are null: 273132\n\nVincent",
    "1654944": "Anyone? I feel it's a pretty important question..",
    "1654953": "because if i do y = df['target'] the model will complain of NaN values feeded to .fit()",
    "1656006": "The time period in which each coin's data is collected differs.  I think Doge doesn't have any 2018 data which exaggerates the problem.  For me, I just use forward fill to replace na values.",
    "1656020": "Thx @petersk20 , also the Target missing target values you forward fill? There's some notebooks that recreate the target for NaN target values, so could use those maybe instead of Forward Fill or dropping these rows",
    "1656600": "I would remove those from training if the target itself was missing, but you could also try to calculate it.  This will take time to do though in your submission notebook and might cause the notebook to become more complicated than it needs to be.  There aren't many NAs compared to the number of training rows so it won't make a difference either way."
  },
  "source": "meta"
}