The root cause is that traditional image regression models based on RMSE tend to rely too heavily on the label distribution. What if we used cross-entropy loss instead, to predict the probability of the output being closest to each possible value?\nI find this approach quite promising in the field of regression. However, in this challenge I tried decoding-based regression and the model I designed didn’t converge well due to the small amount of data. However, this method could be useful for other problems.
", "messages": [ { "id": "3393267", "postDate": "01/18/2026 16:40:21", "content": "The root cause is that traditional image regression models based on RMSE tend to rely too heavily on the label distribution. What if we used cross-entropy loss instead, to predict the probability of the output being closest to each possible value?\nI find this approach quite promising in the field of regression. However, in this challenge I tried decoding-based regression and the model I designed didn’t converge well due to the small amount of data. However, this method could be useful for other problems.
", "rawMarkdown": "Decoding-based Regression: https://arxiv.org/pdf/2501.19383. For example, tokens <1><.><2><3> can be decoded to represent 1.23\n- RegressLM: https://github.com/google-deepmind/regress-lm\n\nThe root cause is that traditional image regression models based on RMSE tend to rely too heavily on the label distribution. What if we used cross-entropy loss instead, to predict the probability of the output being closest to each possible value?\nI find this approach quite promising in the field of regression. However, in this challenge I tried decoding-based regression and the model I designed didn’t converge well due to the small amount of data. However, this method could be useful for other problems.", "votes": null }, { "id": "3393275", "postDate": "01/18/2026 17:10:43", "content": "Very interesting approach. Decoding-based regression using cross-entropy instead of RMSE is indeed promising, especially to reduce over-reliance on label distribution.
", "rawMarkdown": "Very interesting approach. Decoding-based regression using cross-entropy instead of RMSE is indeed promising, especially to reduce over-reliance on label distribution.", "votes": null }, { "id": "3393432", "postDate": "01/19/2026 02:47:04", "content": "Another issue is that some image labels are difficult to distinguish, or the images themselves have significant noise.
", "rawMarkdown": "Another issue is that some image labels are difficult to distinguish, or the images themselves have significant noise.", "votes": null }, { "id": "3393434", "postDate": "01/19/2026 02:53:26", "content": "Assuming in this competition, I think we would still use a vision encoder to extract features and then add an auto-regressive decoder. With this approach, we still have to face the same challenges on images anyway.
", "rawMarkdown": "Assuming in this competition, I think we would still use a vision encoder to extract features and then add an auto-regressive decoder. With this approach, we still have to face the same challenges on images anyway.", "votes": null } ], "comments": [ { "id": 3393275, "author_name": "mdferozahmedafm", "author_url": "", "post_date": "01/18/2026 17:10:43", "content": "Very interesting approach. Decoding-based regression using cross-entropy instead of RMSE is indeed promising, especially to reduce over-reliance on label distribution.
", "votes": null, "replies": [] }, { "id": 3393432, "author_name": "sweetyheehee", "author_url": "", "post_date": "01/19/2026 02:47:04", "content": "Another issue is that some image labels are difficult to distinguish, or the images themselves have significant noise.
", "votes": null, "replies": [ { "id": 3393434, "author_name": "quan0095", "author_url": "", "post_date": "01/19/2026 02:53:26", "content": "Assuming in this competition, I think we would still use a vision encoder to extract features and then add an auto-regressive decoder. With this approach, we still have to face the same challenges on images anyway.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3393267": "Decoding-based Regression: https://arxiv.org/pdf/2501.19383. For example, tokens <1><.><2><3> can be decoded to represent 1.23\n- RegressLM: https://github.com/google-deepmind/regress-lm\n\nThe root cause is that traditional image regression models based on RMSE tend to rely too heavily on the label distribution. What if we used cross-entropy loss instead, to predict the probability of the output being closest to each possible value?\nI find this approach quite promising in the field of regression. However, in this challenge I tried decoding-based regression and the model I designed didn’t converge well due to the small amount of data. However, this method could be useful for other problems.", "3393275": "Very interesting approach. Decoding-based regression using cross-entropy instead of RMSE is indeed promising, especially to reduce over-reliance on label distribution.", "3393432": "Another issue is that some image labels are difficult to distinguish, or the images themselves have significant noise.", "3393434": "Assuming in this competition, I think we would still use a vision encoder to extract features and then add an auto-regressive decoder. With this approach, we still have to face the same challenges on images anyway." }, "source": "meta" }