@deeppast Hello, t Is it possible to check our score/quality of our translation without submitting? The 5 submissions per day is inefficient for testing.
\nThanks
", "messages": [ { "id": "3392094", "postDate": "01/16/2026 09:24:25", "content": "@deeppast Hello, t Is it possible to check our score/quality of our translation without submitting? The 5 submissions per day is inefficient for testing.
\nThanks
", "rawMarkdown": "deeppast Hello, t Is it possible to check our score/quality of our translation without submitting? The 5 submissions per day is inefficient for testing.\n\nThanks", "votes": null }, { "id": "3392121", "postDate": "01/16/2026 10:11:12", "content": "You can calculate the score based on the competition metric, included in the Overview. Either that, or calculate the BLEU and chrf scores and compute their geometric mean.
When I use a validation test created from train.csv it gives a blue and chrf while do the training but when I submit it in kaggle it gives score like they are not even close to my calculated values that's the problem
", "rawMarkdown": "When I use a validation test created from train.csv it gives a blue and chrf while do the training but when I submit it in kaggle it gives score like they are not even close to my calculated values that's the problem", "votes": null }, { "id": "3392233", "postDate": "01/16/2026 14:13:30", "content": "That's everyone's problem. Many people have highlighted that the main challenge in this competition is to align the data processing pipeline with the one used for hidden test set.
\nSo, that's the case for everyone and organizers know that.
", "rawMarkdown": "That's everyone's problem. Many people have highlighted that the main challenge in this competition is to align the data processing pipeline with the one used for hidden test set.\n\nSo, that's the case for everyone and organizers know that.", "votes": null }, { "id": "3392247", "postDate": "01/16/2026 14:52:11", "content": "\n\nWhen I use a validation test created from train.csv it gives a blue and chrf while do the training but when I submit it in kaggle it gives score like they are not even close to my calculated values that's the problem
\n
You're referring to the difference between CV (cross-validation) and LB (Leaderboard) values. Since the test set is hidden you won't be able to calculate it completely accurate.
\nWhat you can do is calculate the score for the Ground Truth data from the train.csv with your translations and experiment with how differences in translation/transliteration impact the score.
A rule of thumb is to believe in your CV, not the LB as you don't have the full data and it's easy to overfit to the subset of the Public Test Leaderboard.
", "rawMarkdown": "> When I use a validation test created from train.csv it gives a blue and chrf while do the training but when I submit it in kaggle it gives score like they are not even close to my calculated values that's the problem\n\nYou're referring to the difference between CV (cross-validation) and LB (Leaderboard) values. Since the test set is hidden you won't be able to calculate it completely accurate.\n\nWhat you can do is calculate the score for the Ground Truth data from the `train.csv` with your translations and experiment with how differences in translation/transliteration impact the score.\n\nA rule of thumb is to believe in your CV, not the LB as you don't have the full data and it's easy to overfit to the subset of the Public Test Leaderboard.", "votes": null } ], "comments": [ { "id": 3392121, "author_name": "angantyr", "author_url": "", "post_date": "01/16/2026 10:11:12", "content": "You can calculate the score based on the competition metric, included in the Overview. Either that, or calculate the BLEU and chrf scores and compute their geometric mean.
When I use a validation test created from train.csv it gives a blue and chrf while do the training but when I submit it in kaggle it gives score like they are not even close to my calculated values that's the problem
", "votes": null, "replies": [ { "id": 3392233, "author_name": "acraevschi", "author_url": "", "post_date": "01/16/2026 14:13:30", "content": "That's everyone's problem. Many people have highlighted that the main challenge in this competition is to align the data processing pipeline with the one used for hidden test set.
\nSo, that's the case for everyone and organizers know that.
", "votes": null, "replies": [] }, { "id": 3392247, "author_name": "angantyr", "author_url": "", "post_date": "01/16/2026 14:52:11", "content": "\n\nWhen I use a validation test created from train.csv it gives a blue and chrf while do the training but when I submit it in kaggle it gives score like they are not even close to my calculated values that's the problem
\n
You're referring to the difference between CV (cross-validation) and LB (Leaderboard) values. Since the test set is hidden you won't be able to calculate it completely accurate.
\nWhat you can do is calculate the score for the Ground Truth data from the train.csv with your translations and experiment with how differences in translation/transliteration impact the score.
A rule of thumb is to believe in your CV, not the LB as you don't have the full data and it's easy to overfit to the subset of the Public Test Leaderboard.
", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3392094": "deeppast Hello, t Is it possible to check our score/quality of our translation without submitting? The 5 submissions per day is inefficient for testing.\n\nThanks", "3392121": "You can calculate the score based on [the competition metric, included in the Overview](https://www.kaggle.com/code/metric/dpi-bleu-chrf). Either that, or calculate the `BLEU` and `chrf` scores and compute their geometric mean.", "3392191": "When I use a validation test created from train.csv it gives a blue and chrf while do the training but when I submit it in kaggle it gives score like they are not even close to my calculated values that's the problem", "3392233": "That's everyone's problem. Many people have highlighted that the main challenge in this competition is to align the data processing pipeline with the one used for hidden test set.\n\nSo, that's the case for everyone and organizers know that.", "3392247": "> When I use a validation test created from train.csv it gives a blue and chrf while do the training but when I submit it in kaggle it gives score like they are not even close to my calculated values that's the problem\n\nYou're referring to the difference between CV (cross-validation) and LB (Leaderboard) values. Since the test set is hidden you won't be able to calculate it completely accurate.\n\nWhat you can do is calculate the score for the Ground Truth data from the `train.csv` with your translations and experiment with how differences in translation/transliteration impact the score.\n\nA rule of thumb is to believe in your CV, not the LB as you don't have the full data and it's easy to overfit to the subset of the Public Test Leaderboard." }, "source": "meta" }