{
  "id": 671122,
  "title": "The use of tabular data has failed",
  "url": "/competitions/csiro-biomass/discussion/671122",
  "author_name": "",
  "post_date": "2026-01-31T08:09:41.730382800Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Has anyone tried this method? In the first stage, we predicted several sets of table data such as NDVI. In the second stage, we retrained the regression head, but the score R^2 remained stuck at around 0.55.🤔</p>",
  "messages": [
    {
      "id": "3399731",
      "postDate": "01/31/2026 08:09:41",
      "content": "<p>Has anyone tried this method? In the first stage, we predicted several sets of table data such as NDVI. In the second stage, we retrained the regression head, but the score R^2 remained stuck at around 0.55.🤔</p>",
      "rawMarkdown": "Has anyone tried this method? In the first stage, we predicted several sets of table data such as NDVI. In the second stage, we retrained the regression head, but the score R^2 remained stuck at around 0.55.🤔",
      "votes": null
    },
    {
      "id": "3399745",
      "postDate": "01/31/2026 08:35:23",
      "content": "<p>There are many solution writeups that shows improvement using meta data. Please refer them.</p>",
      "rawMarkdown": "There are many solution writeups that shows improvement using meta data. Please refer them.",
      "votes": null
    },
    {
      "id": "3399749",
      "postDate": "01/31/2026 08:46:55",
      "content": "<p>`\nimport torch\nimport torch.nn as nn\nimport timm\nimport os\nfrom timm.models.vision_transformer import resample_abs_pos_embed</p>\n<p>class AuxBiomassModel(nn.Module):\n    def <strong>init</strong>(self, model_name, pretrained=False, dropout=0.2, num_states=4, num_seasons=4, local_path=None):\n        super().<strong>init</strong>()\n        self.backbone = timm.create_model(model_name, pretrained=False, num_classes=0, global_pool='')\n        nf = self.backbone.num_features</p>\n<pre><code>    if hasattr(self.backbone, 'set_grad_checkpointing'):\n        self.backbone.set_grad_checkpointing(True)\n\n    self.load_pretrained(local_path)   \n    self.global_pool = nn.AdaptiveAvgPool1d(1)\n    self.film = FiLM(nf)\n\n    def make_aux_head(out_dim):\n        return nn.Sequential(\n            nn.Linear(nf * 2, nf), nn.GELU(), nn.Dropout(dropout),\n            nn.Linear(nf, out_dim)\n        )\n\n    self.aux_ndvi_head = make_aux_head(1)\n    self.aux_height_head = make_aux_head(1)\n    self.aux_state_head = make_aux_head(num_states)\n    self.aux_season_head = make_aux_head(num_seasons)\n\n    def make_biomass_head():\n        return nn.Sequential(\n            nn.Linear(nf * 2, nf), nn.GELU(), nn.Dropout(dropout),\n            nn.Linear(nf, 1), nn.Softplus()\n        )\n\n    self.head_green_raw  = make_biomass_head()\n    self.head_clover_raw = make_biomass_head()\n    self.head_gdm_raw    = make_biomass_head()\n    self.head_total_raw  = make_biomass_head()\n    self.freeze_backbone()\n\ndef load_pretrained(self, local_path: str = None):\n    try:\n        if local_path and os.path.isfile(local_path):\n            ckpt = torch.load(local_path, map_location='cpu')\n            ckpt = ckpt.get('model', ckpt)\n            ckpt = ckpt.get('state_dict', ckpt)\n            bb_dict = {k: v for k, v in ckpt.items() if not k.startswith(('head', 'classifier'))}\n        else:\n            bb_dict = timm.create_model(\n                self.backbone.default_cfg['architecture'],\n                pretrained=True, num_classes=0, global_pool='').state_dict()\n\n        if 'pos_embed' in bb_dict and hasattr(self.backbone, 'pos_embed'):\n            pe_ck, pe_m = bb_dict['pos_embed'], self.backbone.pos_embed\n            if pe_ck.shape != pe_m.shape:\n                new_grid_size = self.backbone.patch_embed.grid_size[0]\n                new_num_patches = new_grid_size * new_grid_size\n                bb_dict['pos_embed'] = resample_abs_pos_embed(pe_ck, new_size=new_num_patches, num_prefix_tokens=1)\n        self.backbone.load_state_dict(bb_dict, strict=False)\n    except Exception as e:\n        print(f\"[WARN]: {e}\")\n\ndef freeze_backbone(self):\n    for p in self.backbone.parameters():\n        p.requires_grad = False\n\ndef unfreeze_backbone(self):\n    for p in self.backbone.parameters():\n        p.requires_grad = True\n\ndef forward(self, left, right, mode='biomass'):\n    lf_tokens = self.backbone(left)\n    rf_tokens = self.backbone(right)\n\n    lf = self.global_pool(lf_tokens.transpose(1, 2)).flatten(1)\n    rf = self.global_pool(rf_tokens.transpose(1, 2)).flatten(1)\n\n    ctx = (lf + rf) / 2.0\n    gamma, beta = self.film(ctx)\n    lf_mod = lf * (1 + gamma) + beta\n    rf_mod = rf * (1 + gamma) + beta\n\n    x = torch.cat([lf_mod, rf_mod], dim=1)\n\n    if mode == 'auxiliary':\n        return {\n            'ndvi': self.aux_ndvi_head(x),\n            'height': self.aux_height_head(x),\n            'state': self.aux_state_head(x),\n            'season': self.aux_season_head(x)\n        }\n    else:\n        green  = self.head_green_raw(x)\n        clover = self.head_clover_raw(x)\n        gdm    = self.head_gdm_raw(x)\n        total  = self.head_total_raw(x)\n        dead   = total - gdm\n        return total, gdm, green, clover, dead\n\ndef get_biomass_parameters(self):\n    params = []\n    for name, param in self.named_parameters():\n        if any(k in name for k in ['head_', 'softplus', 'film', 'global_pool']):\n            params.append(param)\n    return params\n\ndef get_auxiliary_parameters(self):\n    return [param for name, param in self.named_parameters() if 'aux_' in name]\n</code></pre>\n<p>`\nthis is my net.</p>",
      "rawMarkdown": "`\nimport torch\nimport torch.nn as nn\nimport timm\nimport os\nfrom timm.models.vision_transformer import resample_abs_pos_embed\n\nclass AuxBiomassModel(nn.Module):\n    def __init__(self, model_name, pretrained=False, dropout=0.2, num_states=4, num_seasons=4, local_path=None):\n        super().__init__()\n        self.backbone = timm.create_model(model_name, pretrained=False, num_classes=0, global_pool='')\n        nf = self.backbone.num_features\n\n        if hasattr(self.backbone, 'set_grad_checkpointing'):\n            self.backbone.set_grad_checkpointing(True)\n\n        self.load_pretrained(local_path)   \n        self.global_pool = nn.AdaptiveAvgPool1d(1)\n        self.film = FiLM(nf)\n\n        def make_aux_head(out_dim):\n            return nn.Sequential(\n                nn.Linear(nf * 2, nf), nn.GELU(), nn.Dropout(dropout),\n                nn.Linear(nf, out_dim)\n            )\n        \n        self.aux_ndvi_head = make_aux_head(1)\n        self.aux_height_head = make_aux_head(1)\n        self.aux_state_head = make_aux_head(num_states)\n        self.aux_season_head = make_aux_head(num_seasons)\n        \n        def make_biomass_head():\n            return nn.Sequential(\n                nn.Linear(nf * 2, nf), nn.GELU(), nn.Dropout(dropout),\n                nn.Linear(nf, 1), nn.Softplus()\n            )\n        \n        self.head_green_raw  = make_biomass_head()\n        self.head_clover_raw = make_biomass_head()\n        self.head_gdm_raw    = make_biomass_head()\n        self.head_total_raw  = make_biomass_head()\n        self.freeze_backbone()\n\n    def load_pretrained(self, local_path: str = None):\n        try:\n            if local_path and os.path.isfile(local_path):\n                ckpt = torch.load(local_path, map_location='cpu')\n                ckpt = ckpt.get('model', ckpt)\n                ckpt = ckpt.get('state_dict', ckpt)\n                bb_dict = {k: v for k, v in ckpt.items() if not k.startswith(('head', 'classifier'))}\n            else:\n                bb_dict = timm.create_model(\n                    self.backbone.default_cfg['architecture'],\n                    pretrained=True, num_classes=0, global_pool='').state_dict()\n\n            if 'pos_embed' in bb_dict and hasattr(self.backbone, 'pos_embed'):\n                pe_ck, pe_m = bb_dict['pos_embed'], self.backbone.pos_embed\n                if pe_ck.shape != pe_m.shape:\n                    new_grid_size = self.backbone.patch_embed.grid_size[0]\n                    new_num_patches = new_grid_size * new_grid_size\n                    bb_dict['pos_embed'] = resample_abs_pos_embed(pe_ck, new_size=new_num_patches, num_prefix_tokens=1)\n            self.backbone.load_state_dict(bb_dict, strict=False)\n        except Exception as e:\n            print(f\"[WARN]: {e}\")\n    \n    def freeze_backbone(self):\n        for p in self.backbone.parameters():\n            p.requires_grad = False\n    \n    def unfreeze_backbone(self):\n        for p in self.backbone.parameters():\n            p.requires_grad = True\n    \n    def forward(self, left, right, mode='biomass'):\n        lf_tokens = self.backbone(left)\n        rf_tokens = self.backbone(right)\n        \n        lf = self.global_pool(lf_tokens.transpose(1, 2)).flatten(1)\n        rf = self.global_pool(rf_tokens.transpose(1, 2)).flatten(1)\n        \n        ctx = (lf + rf) / 2.0\n        gamma, beta = self.film(ctx)\n        lf_mod = lf * (1 + gamma) + beta\n        rf_mod = rf * (1 + gamma) + beta\n        \n        x = torch.cat([lf_mod, rf_mod], dim=1)\n        \n        if mode == 'auxiliary':\n            return {\n                'ndvi': self.aux_ndvi_head(x),\n                'height': self.aux_height_head(x),\n                'state': self.aux_state_head(x),\n                'season': self.aux_season_head(x)\n            }\n        else:\n            green  = self.head_green_raw(x)\n            clover = self.head_clover_raw(x)\n            gdm    = self.head_gdm_raw(x)\n            total  = self.head_total_raw(x)\n            dead   = total - gdm\n            return total, gdm, green, clover, dead\n\n    def get_biomass_parameters(self):\n        params = []\n        for name, param in self.named_parameters():\n            if any(k in name for k in ['head_', 'softplus', 'film', 'global_pool']):\n                params.append(param)\n        return params\n\n    def get_auxiliary_parameters(self):\n        return [param for name, param in self.named_parameters() if 'aux_' in name]\n`\nthis is my net.",
      "votes": null
    },
    {
      "id": "3399752",
      "postDate": "01/31/2026 08:52:00",
      "content": "<p>OK. Just now I saw the third person. He posted a message today. Maybe this method doesn't work. He didn't succeed either. Clearly, intuitively it seemed possible to succeed.🫠</p>",
      "rawMarkdown": "OK. Just now I saw the third person. He posted a message today. Maybe this method doesn't work. He didn't succeed either. Clearly, intuitively it seemed possible to succeed.🫠",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3399745,
      "author_name": "komilparmar",
      "author_url": "",
      "post_date": "01/31/2026 08:35:23",
      "content": "<p>There are many solution writeups that shows improvement using meta data. Please refer them.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3399752,
          "author_name": "waterjoe",
          "author_url": "",
          "post_date": "01/31/2026 08:52:00",
          "content": "<p>OK. Just now I saw the third person. He posted a message today. Maybe this method doesn't work. He didn't succeed either. Clearly, intuitively it seemed possible to succeed.🫠</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3399749,
      "author_name": "waterjoe",
      "author_url": "",
      "post_date": "01/31/2026 08:46:55",
      "content": "<p>`\nimport torch\nimport torch.nn as nn\nimport timm\nimport os\nfrom timm.models.vision_transformer import resample_abs_pos_embed</p>\n<p>class AuxBiomassModel(nn.Module):\n    def <strong>init</strong>(self, model_name, pretrained=False, dropout=0.2, num_states=4, num_seasons=4, local_path=None):\n        super().<strong>init</strong>()\n        self.backbone = timm.create_model(model_name, pretrained=False, num_classes=0, global_pool='')\n        nf = self.backbone.num_features</p>\n<pre><code>    if hasattr(self.backbone, 'set_grad_checkpointing'):\n        self.backbone.set_grad_checkpointing(True)\n\n    self.load_pretrained(local_path)   \n    self.global_pool = nn.AdaptiveAvgPool1d(1)\n    self.film = FiLM(nf)\n\n    def make_aux_head(out_dim):\n        return nn.Sequential(\n            nn.Linear(nf * 2, nf), nn.GELU(), nn.Dropout(dropout),\n            nn.Linear(nf, out_dim)\n        )\n\n    self.aux_ndvi_head = make_aux_head(1)\n    self.aux_height_head = make_aux_head(1)\n    self.aux_state_head = make_aux_head(num_states)\n    self.aux_season_head = make_aux_head(num_seasons)\n\n    def make_biomass_head():\n        return nn.Sequential(\n            nn.Linear(nf * 2, nf), nn.GELU(), nn.Dropout(dropout),\n            nn.Linear(nf, 1), nn.Softplus()\n        )\n\n    self.head_green_raw  = make_biomass_head()\n    self.head_clover_raw = make_biomass_head()\n    self.head_gdm_raw    = make_biomass_head()\n    self.head_total_raw  = make_biomass_head()\n    self.freeze_backbone()\n\ndef load_pretrained(self, local_path: str = None):\n    try:\n        if local_path and os.path.isfile(local_path):\n            ckpt = torch.load(local_path, map_location='cpu')\n            ckpt = ckpt.get('model', ckpt)\n            ckpt = ckpt.get('state_dict', ckpt)\n            bb_dict = {k: v for k, v in ckpt.items() if not k.startswith(('head', 'classifier'))}\n        else:\n            bb_dict = timm.create_model(\n                self.backbone.default_cfg['architecture'],\n                pretrained=True, num_classes=0, global_pool='').state_dict()\n\n        if 'pos_embed' in bb_dict and hasattr(self.backbone, 'pos_embed'):\n            pe_ck, pe_m = bb_dict['pos_embed'], self.backbone.pos_embed\n            if pe_ck.shape != pe_m.shape:\n                new_grid_size = self.backbone.patch_embed.grid_size[0]\n                new_num_patches = new_grid_size * new_grid_size\n                bb_dict['pos_embed'] = resample_abs_pos_embed(pe_ck, new_size=new_num_patches, num_prefix_tokens=1)\n        self.backbone.load_state_dict(bb_dict, strict=False)\n    except Exception as e:\n        print(f\"[WARN]: {e}\")\n\ndef freeze_backbone(self):\n    for p in self.backbone.parameters():\n        p.requires_grad = False\n\ndef unfreeze_backbone(self):\n    for p in self.backbone.parameters():\n        p.requires_grad = True\n\ndef forward(self, left, right, mode='biomass'):\n    lf_tokens = self.backbone(left)\n    rf_tokens = self.backbone(right)\n\n    lf = self.global_pool(lf_tokens.transpose(1, 2)).flatten(1)\n    rf = self.global_pool(rf_tokens.transpose(1, 2)).flatten(1)\n\n    ctx = (lf + rf) / 2.0\n    gamma, beta = self.film(ctx)\n    lf_mod = lf * (1 + gamma) + beta\n    rf_mod = rf * (1 + gamma) + beta\n\n    x = torch.cat([lf_mod, rf_mod], dim=1)\n\n    if mode == 'auxiliary':\n        return {\n            'ndvi': self.aux_ndvi_head(x),\n            'height': self.aux_height_head(x),\n            'state': self.aux_state_head(x),\n            'season': self.aux_season_head(x)\n        }\n    else:\n        green  = self.head_green_raw(x)\n        clover = self.head_clover_raw(x)\n        gdm    = self.head_gdm_raw(x)\n        total  = self.head_total_raw(x)\n        dead   = total - gdm\n        return total, gdm, green, clover, dead\n\ndef get_biomass_parameters(self):\n    params = []\n    for name, param in self.named_parameters():\n        if any(k in name for k in ['head_', 'softplus', 'film', 'global_pool']):\n            params.append(param)\n    return params\n\ndef get_auxiliary_parameters(self):\n    return [param for name, param in self.named_parameters() if 'aux_' in name]\n</code></pre>\n<p>`\nthis is my net.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3399731": "Has anyone tried this method? In the first stage, we predicted several sets of table data such as NDVI. In the second stage, we retrained the regression head, but the score R^2 remained stuck at around 0.55.🤔",
    "3399745": "There are many solution writeups that shows improvement using meta data. Please refer them.",
    "3399749": "`\nimport torch\nimport torch.nn as nn\nimport timm\nimport os\nfrom timm.models.vision_transformer import resample_abs_pos_embed\n\nclass AuxBiomassModel(nn.Module):\n    def __init__(self, model_name, pretrained=False, dropout=0.2, num_states=4, num_seasons=4, local_path=None):\n        super().__init__()\n        self.backbone = timm.create_model(model_name, pretrained=False, num_classes=0, global_pool='')\n        nf = self.backbone.num_features\n\n        if hasattr(self.backbone, 'set_grad_checkpointing'):\n            self.backbone.set_grad_checkpointing(True)\n\n        self.load_pretrained(local_path)   \n        self.global_pool = nn.AdaptiveAvgPool1d(1)\n        self.film = FiLM(nf)\n\n        def make_aux_head(out_dim):\n            return nn.Sequential(\n                nn.Linear(nf * 2, nf), nn.GELU(), nn.Dropout(dropout),\n                nn.Linear(nf, out_dim)\n            )\n        \n        self.aux_ndvi_head = make_aux_head(1)\n        self.aux_height_head = make_aux_head(1)\n        self.aux_state_head = make_aux_head(num_states)\n        self.aux_season_head = make_aux_head(num_seasons)\n        \n        def make_biomass_head():\n            return nn.Sequential(\n                nn.Linear(nf * 2, nf), nn.GELU(), nn.Dropout(dropout),\n                nn.Linear(nf, 1), nn.Softplus()\n            )\n        \n        self.head_green_raw  = make_biomass_head()\n        self.head_clover_raw = make_biomass_head()\n        self.head_gdm_raw    = make_biomass_head()\n        self.head_total_raw  = make_biomass_head()\n        self.freeze_backbone()\n\n    def load_pretrained(self, local_path: str = None):\n        try:\n            if local_path and os.path.isfile(local_path):\n                ckpt = torch.load(local_path, map_location='cpu')\n                ckpt = ckpt.get('model', ckpt)\n                ckpt = ckpt.get('state_dict', ckpt)\n                bb_dict = {k: v for k, v in ckpt.items() if not k.startswith(('head', 'classifier'))}\n            else:\n                bb_dict = timm.create_model(\n                    self.backbone.default_cfg['architecture'],\n                    pretrained=True, num_classes=0, global_pool='').state_dict()\n\n            if 'pos_embed' in bb_dict and hasattr(self.backbone, 'pos_embed'):\n                pe_ck, pe_m = bb_dict['pos_embed'], self.backbone.pos_embed\n                if pe_ck.shape != pe_m.shape:\n                    new_grid_size = self.backbone.patch_embed.grid_size[0]\n                    new_num_patches = new_grid_size * new_grid_size\n                    bb_dict['pos_embed'] = resample_abs_pos_embed(pe_ck, new_size=new_num_patches, num_prefix_tokens=1)\n            self.backbone.load_state_dict(bb_dict, strict=False)\n        except Exception as e:\n            print(f\"[WARN]: {e}\")\n    \n    def freeze_backbone(self):\n        for p in self.backbone.parameters():\n            p.requires_grad = False\n    \n    def unfreeze_backbone(self):\n        for p in self.backbone.parameters():\n            p.requires_grad = True\n    \n    def forward(self, left, right, mode='biomass'):\n        lf_tokens = self.backbone(left)\n        rf_tokens = self.backbone(right)\n        \n        lf = self.global_pool(lf_tokens.transpose(1, 2)).flatten(1)\n        rf = self.global_pool(rf_tokens.transpose(1, 2)).flatten(1)\n        \n        ctx = (lf + rf) / 2.0\n        gamma, beta = self.film(ctx)\n        lf_mod = lf * (1 + gamma) + beta\n        rf_mod = rf * (1 + gamma) + beta\n        \n        x = torch.cat([lf_mod, rf_mod], dim=1)\n        \n        if mode == 'auxiliary':\n            return {\n                'ndvi': self.aux_ndvi_head(x),\n                'height': self.aux_height_head(x),\n                'state': self.aux_state_head(x),\n                'season': self.aux_season_head(x)\n            }\n        else:\n            green  = self.head_green_raw(x)\n            clover = self.head_clover_raw(x)\n            gdm    = self.head_gdm_raw(x)\n            total  = self.head_total_raw(x)\n            dead   = total - gdm\n            return total, gdm, green, clover, dead\n\n    def get_biomass_parameters(self):\n        params = []\n        for name, param in self.named_parameters():\n            if any(k in name for k in ['head_', 'softplus', 'film', 'global_pool']):\n                params.append(param)\n        return params\n\n    def get_auxiliary_parameters(self):\n        return [param for name, param in self.named_parameters() if 'aux_' in name]\n`\nthis is my net.",
    "3399752": "OK. Just now I saw the third person. He posted a message today. Maybe this method doesn't work. He didn't succeed either. Clearly, intuitively it seemed possible to succeed.🫠"
  },
  "source": "meta"
}