{
  "id": 614932,
  "title": "Has anyone tried applying a log transformation? ",
  "url": "/competitions/csiro-biomass/discussion/614932",
  "author_name": "",
  "post_date": "2025-11-07T13:37:29.617153500Z",
  "votes": null,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I’m wondering if it could make the R² score more stable. For example, transforming the labels into log space and then converting them back to gram (g) space when calculating the score — would that yield more stable results compared to not using the log transform? I’d really appreciate it if anyone could share their insights!</p>\n<p><code>y = np.log1p(row[self.target_cols].astype(np.float32).values)</code></p>\n<p><code>y_true_g = np.expm1(y_true_log.detach().cpu().numpy()).astype(np.float64)</code></p>\n<p><code>y_pred_g = np.expm1(y_pred_log.detach().cpu().numpy()).astype(np.float64)</code></p>",
  "messages": [
    {
      "id": "3312581",
      "postDate": "11/07/2025 13:37:29",
      "content": "<p>I’m wondering if it could make the R² score more stable. For example, transforming the labels into log space and then converting them back to gram (g) space when calculating the score — would that yield more stable results compared to not using the log transform? I’d really appreciate it if anyone could share their insights!</p>\n<p><code>y = np.log1p(row[self.target_cols].astype(np.float32).values)</code></p>\n<p><code>y_true_g = np.expm1(y_true_log.detach().cpu().numpy()).astype(np.float64)</code></p>\n<p><code>y_pred_g = np.expm1(y_pred_log.detach().cpu().numpy()).astype(np.float64)</code></p>",
      "rawMarkdown": "I’m wondering if it could make the R² score more stable. For example, transforming the labels into log space and then converting them back to gram (g) space when calculating the score — would that yield more stable results compared to not using the log transform? I’d really appreciate it if anyone could share their insights!\n\n`y = np.log1p(row[self.target_cols].astype(np.float32).values)`\n\n`y_true_g = np.expm1(y_true_log.detach().cpu().numpy()).astype(np.float64)`\n\n`y_pred_g = np.expm1(y_pred_log.detach().cpu().numpy()).astype(np.float64)`",
      "votes": null
    },
    {
      "id": "3312622",
      "postDate": "11/07/2025 15:12:37",
      "content": "<p>Just have to be careful about things exploding at first!</p>",
      "rawMarkdown": "Just have to be careful about things exploding at first!",
      "votes": null
    },
    {
      "id": "3312818",
      "postDate": "11/08/2025 03:35:49",
      "content": "<p>Was not useful for me. I think because somewhere in the paper for this competition it was mentioned that they calculate R2 on your log pred and log target.</p>",
      "rawMarkdown": "Was not useful for me. I think because somewhere in the paper for this competition it was mentioned that they calculate R2 on your log pred and log target.",
      "votes": null
    },
    {
      "id": "3312928",
      "postDate": "11/08/2025 09:16:33",
      "content": "<p>From my experiments, using a log transformation seems to make the CV score more stable.\nI’m not really sure if TTA makes a difference. From my current experiments, the LB score hasn’t changed much.</p>",
      "rawMarkdown": "From my experiments, using a log transformation seems to make the CV score more stable.\nI’m not really sure if TTA makes a difference. From my current experiments, the LB score hasn’t changed much.",
      "votes": null
    },
    {
      "id": "3312961",
      "postDate": "11/08/2025 10:43:08",
      "content": "<p>I gave it a try and find score become more stable but not improved</p>",
      "rawMarkdown": "I gave it a try and find score become more stable but not improved",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3312622,
      "author_name": "bklynch",
      "author_url": "",
      "post_date": "11/07/2025 15:12:37",
      "content": "<p>Just have to be careful about things exploding at first!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3312818,
      "author_name": "paritoshtripathi5",
      "author_url": "",
      "post_date": "11/08/2025 03:35:49",
      "content": "<p>Was not useful for me. I think because somewhere in the paper for this competition it was mentioned that they calculate R2 on your log pred and log target.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3312928,
      "author_name": "gunianxue",
      "author_url": "",
      "post_date": "11/08/2025 09:16:33",
      "content": "<p>From my experiments, using a log transformation seems to make the CV score more stable.\nI’m not really sure if TTA makes a difference. From my current experiments, the LB score hasn’t changed much.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3312961,
      "author_name": "forrestxlz",
      "author_url": "",
      "post_date": "11/08/2025 10:43:08",
      "content": "<p>I gave it a try and find score become more stable but not improved</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3312581": "I’m wondering if it could make the R² score more stable. For example, transforming the labels into log space and then converting them back to gram (g) space when calculating the score — would that yield more stable results compared to not using the log transform? I’d really appreciate it if anyone could share their insights!\n\n`y = np.log1p(row[self.target_cols].astype(np.float32).values)`\n\n`y_true_g = np.expm1(y_true_log.detach().cpu().numpy()).astype(np.float64)`\n\n`y_pred_g = np.expm1(y_pred_log.detach().cpu().numpy()).astype(np.float64)`",
    "3312622": "Just have to be careful about things exploding at first!",
    "3312818": "Was not useful for me. I think because somewhere in the paper for this competition it was mentioned that they calculate R2 on your log pred and log target.",
    "3312928": "From my experiments, using a log transformation seems to make the CV score more stable.\nI’m not really sure if TTA makes a difference. From my current experiments, the LB score hasn’t changed much.",
    "3312961": "I gave it a try and find score become more stable but not improved"
  },
  "source": "meta"
}