{
  "id": 285676,
  "title": "target values do not match definition of the tutorial",
  "url": "/competitions/g-research-crypto-forecasting/discussion/285676",
  "author_name": "",
  "post_date": "2021-11-05T17:57:42.392373400Z",
  "votes": 7,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hi there,</p>\n<p>I tried to recover the target values using the definition from <a href=\"https://www.kaggle.com/cstein06/tutorial-to-the-g-research-crypto-competition\" target=\"_blank\">https://www.kaggle.com/cstein06/tutorial-to-the-g-research-crypto-competition</a> with the closing price for P and a \"smoothing\"/rolling mean  of n=int(3750/60.) steps  (3750 minute windows). I find a close match but not exact so I was wondering how the target values are computed ?</p>\n<p>Thanks,<br>\nEle</p>",
  "messages": [
    {
      "id": "1572496",
      "postDate": "11/05/2021 17:57:42",
      "content": "<p>Hi there,</p>\n<p>I tried to recover the target values using the definition from <a href=\"https://www.kaggle.com/cstein06/tutorial-to-the-g-research-crypto-competition\" target=\"_blank\">https://www.kaggle.com/cstein06/tutorial-to-the-g-research-crypto-competition</a> with the closing price for P and a \"smoothing\"/rolling mean  of n=int(3750/60.) steps  (3750 minute windows). I find a close match but not exact so I was wondering how the target values are computed ?</p>\n<p>Thanks,<br>\nEle</p>",
      "rawMarkdown": "Hi there,\n\nI tried to recover the target values using the definition from https://www.kaggle.com/cstein06/tutorial-to-the-g-research-crypto-competition with the closing price for P and a \"smoothing\"/rolling mean  of n=int(3750/60.) steps  (3750 minute windows). I find a close match but not exact so I was wondering how the target values are computed ?\n\nThanks,\nEle",
      "votes": null
    },
    {
      "id": "1572592",
      "postDate": "11/05/2021 19:10:13",
      "content": "<p>Hi, could you detail how you managed to calculate the beta_a? I am also struggling to calculate as you can see in this topic <a href=\"https://www.kaggle.com/c/g-research-crypto-forecasting/discussion/285624\" target=\"_blank\">https://www.kaggle.com/c/g-research-crypto-forecasting/discussion/285624</a>, I thought I was close, but the results seems far to match. I appreciate if you could check it out.</p>\n<p>Also, why do you are dividing 3750 by 60?</p>\n<p>Cheers</p>",
      "rawMarkdown": "Hi, could you detail how you managed to calculate the beta_a? I am also struggling to calculate as you can see in this topic https://www.kaggle.com/c/g-research-crypto-forecasting/discussion/285624, I thought I was close, but the results seems far to match. I appreciate if you could check it out.\n\nAlso, why do you are dividing 3750 by 60?\n\nCheers",
      "votes": null
    },
    {
      "id": "1573176",
      "postDate": "11/06/2021 11:23:47",
      "content": "<p>Here is what I do:</p>\n<p>Ra=np.zeros((ydim,Nasset))<br>\nM=np.zeros(ydim)<br>\nw=np.ones(Nasset)</p>\n<p>for i in range(len(asset_details)):<br>\n    ia=asset_details.Asset_ID<br>\n    w[ia]=asset_details.Weight</p>\n<p>for it in range(ydim-16):<br>\n    for ia in range(Nasset):<br>\n        Ra[it,ia]=np.log(Xfeat[it+16,ia]/Xfeat[it+1,ia]) # log return over 15 mins</p>\n<pre><code># weighted average market return given our assets \nM[it]=np.sum(Ra[it,:]*w[:])/np.sum(w)\n</code></pre>\n<h1>n=int(3750/60.)</h1>\n<p>n=int(3750/60.)    </p>\n<p>beta=np.zeros(Nasset)<br>\nTarget=np.zeros((ydim,Nasset))</p>\n<p>for ia in range(Nasset):<br>\n    for it in range(ydim-n):<br>\n        beta[ia]=np.sum(M[it:it+n]<em>Ra[it:it+n,ia])/np.sum(M[it:it+n]</em><em>2)\n        #moving_average(M[it:it+n]</em>Ra[it:it+n,ia],n)/moving_average(M[it:it+n]*M[it:it+n],n)</p>\n<pre><code>    Target[it,ia]=Ra[it,ia]-beta[ia]*M[it]\n</code></pre>",
      "rawMarkdown": "Here is what I do:\n\nRa=np.zeros((ydim,Nasset))\nM=np.zeros(ydim)\nw=np.ones(Nasset)\n\nfor i in range(len(asset_details)):\n    ia=asset_details.Asset_ID\n    w[ia]=asset_details.Weight\n\n\nfor it in range(ydim-16):\n    for ia in range(Nasset):\n        Ra[it,ia]=np.log(Xfeat[it+16,ia]/Xfeat[it+1,ia]) # log return over 15 mins\n    \n    # weighted average market return given our assets \n    M[it]=np.sum(Ra[it,:]*w[:])/np.sum(w)\n    \n#n=int(3750/60.)    \nn=int(3750/60.)    \n\nbeta=np.zeros(Nasset)\nTarget=np.zeros((ydim,Nasset))\n\nfor ia in range(Nasset):\n    for it in range(ydim-n):\n        beta[ia]=np.sum(M[it:it+n]*Ra[it:it+n,ia])/np.sum(M[it:it+n]**2)\n        #moving_average(M[it:it+n]*Ra[it:it+n,ia],n)/moving_average(M[it:it+n]*M[it:it+n],n)\n    \n        Target[it,ia]=Ra[it,ia]-beta[ia]*M[it]",
      "votes": null
    },
    {
      "id": "1573177",
      "postDate": "11/06/2021 11:24:19",
      "content": "<p>I divide by 60 because the data are in sec.</p>",
      "rawMarkdown": "I divide by 60 because the data are in sec.",
      "votes": null
    },
    {
      "id": "1573252",
      "postDate": "11/06/2021 12:40:56",
      "content": "<p>Just published notebook with <a href=\"https://www.kaggle.com/alexfir/recreating-target\" target=\"_blank\">Target calculation</a>, this is not ideal, but probably can be helpful.</p>",
      "rawMarkdown": "Just published notebook with [Target calculation](https://www.kaggle.com/alexfir/recreating-target), this is not ideal, but probably can be helpful.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1572592,
      "author_name": "romulomadu",
      "author_url": "",
      "post_date": "11/05/2021 19:10:13",
      "content": "<p>Hi, could you detail how you managed to calculate the beta_a? I am also struggling to calculate as you can see in this topic <a href=\"https://www.kaggle.com/c/g-research-crypto-forecasting/discussion/285624\" target=\"_blank\">https://www.kaggle.com/c/g-research-crypto-forecasting/discussion/285624</a>, I thought I was close, but the results seems far to match. I appreciate if you could check it out.</p>\n<p>Also, why do you are dividing 3750 by 60?</p>\n<p>Cheers</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1573176,
      "author_name": "elonore",
      "author_url": "",
      "post_date": "11/06/2021 11:23:47",
      "content": "<p>Here is what I do:</p>\n<p>Ra=np.zeros((ydim,Nasset))<br>\nM=np.zeros(ydim)<br>\nw=np.ones(Nasset)</p>\n<p>for i in range(len(asset_details)):<br>\n    ia=asset_details.Asset_ID<br>\n    w[ia]=asset_details.Weight</p>\n<p>for it in range(ydim-16):<br>\n    for ia in range(Nasset):<br>\n        Ra[it,ia]=np.log(Xfeat[it+16,ia]/Xfeat[it+1,ia]) # log return over 15 mins</p>\n<pre><code># weighted average market return given our assets \nM[it]=np.sum(Ra[it,:]*w[:])/np.sum(w)\n</code></pre>\n<h1>n=int(3750/60.)</h1>\n<p>n=int(3750/60.)    </p>\n<p>beta=np.zeros(Nasset)<br>\nTarget=np.zeros((ydim,Nasset))</p>\n<p>for ia in range(Nasset):<br>\n    for it in range(ydim-n):<br>\n        beta[ia]=np.sum(M[it:it+n]<em>Ra[it:it+n,ia])/np.sum(M[it:it+n]</em><em>2)\n        #moving_average(M[it:it+n]</em>Ra[it:it+n,ia],n)/moving_average(M[it:it+n]*M[it:it+n],n)</p>\n<pre><code>    Target[it,ia]=Ra[it,ia]-beta[ia]*M[it]\n</code></pre>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1573177,
      "author_name": "elonore",
      "author_url": "",
      "post_date": "11/06/2021 11:24:19",
      "content": "<p>I divide by 60 because the data are in sec.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1573252,
      "author_name": "alexfir",
      "author_url": "",
      "post_date": "11/06/2021 12:40:56",
      "content": "<p>Just published notebook with <a href=\"https://www.kaggle.com/alexfir/recreating-target\" target=\"_blank\">Target calculation</a>, this is not ideal, but probably can be helpful.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1572496": "Hi there,\n\nI tried to recover the target values using the definition from https://www.kaggle.com/cstein06/tutorial-to-the-g-research-crypto-competition with the closing price for P and a \"smoothing\"/rolling mean  of n=int(3750/60.) steps  (3750 minute windows). I find a close match but not exact so I was wondering how the target values are computed ?\n\nThanks,\nEle",
    "1572592": "Hi, could you detail how you managed to calculate the beta_a? I am also struggling to calculate as you can see in this topic https://www.kaggle.com/c/g-research-crypto-forecasting/discussion/285624, I thought I was close, but the results seems far to match. I appreciate if you could check it out.\n\nAlso, why do you are dividing 3750 by 60?\n\nCheers",
    "1573176": "Here is what I do:\n\nRa=np.zeros((ydim,Nasset))\nM=np.zeros(ydim)\nw=np.ones(Nasset)\n\nfor i in range(len(asset_details)):\n    ia=asset_details.Asset_ID\n    w[ia]=asset_details.Weight\n\n\nfor it in range(ydim-16):\n    for ia in range(Nasset):\n        Ra[it,ia]=np.log(Xfeat[it+16,ia]/Xfeat[it+1,ia]) # log return over 15 mins\n    \n    # weighted average market return given our assets \n    M[it]=np.sum(Ra[it,:]*w[:])/np.sum(w)\n    \n#n=int(3750/60.)    \nn=int(3750/60.)    \n\nbeta=np.zeros(Nasset)\nTarget=np.zeros((ydim,Nasset))\n\nfor ia in range(Nasset):\n    for it in range(ydim-n):\n        beta[ia]=np.sum(M[it:it+n]*Ra[it:it+n,ia])/np.sum(M[it:it+n]**2)\n        #moving_average(M[it:it+n]*Ra[it:it+n,ia],n)/moving_average(M[it:it+n]*M[it:it+n],n)\n    \n        Target[it,ia]=Ra[it,ia]-beta[ia]*M[it]",
    "1573177": "I divide by 60 because the data are in sec.",
    "1573252": "Just published notebook with [Target calculation](https://www.kaggle.com/alexfir/recreating-target), this is not ideal, but probably can be helpful."
  },
  "source": "meta"
}