{
  "id": 324608,
  "title": "Language of markdowns",
  "url": "/competitions/AI4Code/discussion/324608",
  "author_name": "",
  "post_date": "2022-05-12T12:44:18.543511400Z",
  "votes": 13,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Is the final test set going to be filtered for markdowns written in English?</p>\n<p>Didn't look into the data yet, just saw that the first training sample's (00001756c60be8.json) markdowns are written with cyrillic letters.<br>\nIs it expected that the test data will also have such non-English samples?</p>",
  "messages": [
    {
      "id": "1785819",
      "postDate": "05/12/2022 12:44:18",
      "content": "<p>Is the final test set going to be filtered for markdowns written in English?</p>\n<p>Didn't look into the data yet, just saw that the first training sample's (00001756c60be8.json) markdowns are written with cyrillic letters.<br>\nIs it expected that the test data will also have such non-English samples?</p>",
      "rawMarkdown": "Is the final test set going to be filtered for markdowns written in English?\n\nDidn't look into the data yet, just saw that the first training sample's (00001756c60be8.json) markdowns are written with cyrillic letters.\nIs it expected that the test data will also have such non-English samples?",
      "votes": null
    },
    {
      "id": "1786072",
      "postDate": "05/12/2022 15:31:50",
      "content": "<p>Yes, we wanted to leave the data as intact as possible for this challenge, so there will be multiple languages represented.</p>",
      "rawMarkdown": "Yes, we wanted to leave the data as intact as possible for this challenge, so there will be multiple languages represented.",
      "votes": null
    },
    {
      "id": "1803854",
      "postDate": "05/28/2022 08:47:26",
      "content": "<p><a href=\"https://www.kaggle.com/wcukierski\" target=\"_blank\">@wcukierski</a> <br>\nSorry but I don't get the ideal about how actual test set can be predicted when language is changed. If I have to use multi models for predicting order of cells, first model for language detection and second models for predicting order for each language, so I have so many models here: </p>\n<ul>\n<li>language detection</li>\n<li>predicting order for English cells.</li>\n<li>predicting order for Japanese cells</li>\n<li>predicting order for Chinese cells</li>\n<li>…<br>\nso how can kaggle can replace sampled test rows with actual rows and rerun correct model for predicting ? As an example, if we replace an English notebook with Japanese notebook, my code need to re-run language detection model, get Japanese model and then apply a Japanese cell model instead of English cell model, can kaggle do that ?</li>\n</ul>",
      "rawMarkdown": "wcukierski \nSorry but I don't get the ideal about how actual test set can be predicted when language is changed. If I have to use multi models for predicting order of cells, first model for language detection and second models for predicting order for each language, so I have so many models here: \n* language detection\n* predicting order for English cells.\n* predicting order for Japanese cells\n* predicting order for Chinese cells\n* ...\nso how can kaggle can replace sampled test rows with actual rows and rerun correct model for predicting ? As an example, if we replace an English notebook with Japanese notebook, my code need to re-run language detection model, get Japanese model and then apply a Japanese cell model instead of English cell model, can kaggle do that ?",
      "votes": null
    },
    {
      "id": "1809230",
      "postDate": "06/02/2022 14:30:00",
      "content": "<p><a href=\"https://www.kaggle.com/astrung\" target=\"_blank\">@astrung</a> Kaggle reruns your notebook with the new test data in place of the current test data. If your approach has multiple models, it has to decide which ones to use on its own.</p>",
      "rawMarkdown": "astrung Kaggle reruns your notebook with the new test data in place of the current test data. If your approach has multiple models, it has to decide which ones to use on its own.",
      "votes": null
    },
    {
      "id": "1822893",
      "postDate": "06/16/2022 19:30:25",
      "content": "<p>This decision is quite questionable. To mix multilanguage task with task of natural-programming language correspondence.<br>\nAt least can we expect that test set has same language distribution? Test and train sets should be drawn from the same distribution.</p>",
      "rawMarkdown": "This decision is quite questionable. To mix multilanguage task with task of natural-programming language correspondence.\nAt least can we expect that test set has same language distribution? Test and train sets should be drawn from the same distribution.",
      "votes": null
    },
    {
      "id": "1823061",
      "postDate": "06/17/2022 02:17:10",
      "content": "<blockquote>\n  <p>Kaggle reruns your notebook with the new test data in place of the current test data.</p>\n</blockquote>\n<p>Can we assume a comparable number of notebooks and the maximum/average notebook size in the new test set?</p>\n<p>To avoid the case the solution is working fine on the current test set but exceeds the allocated time if the new test set is twice as large or contains notebooks with 100k cells (I imagine it would fail for example if someone decides to use the lightweight pairwise model with quadratic complexity over the number of cells).</p>",
      "rawMarkdown": "> Kaggle reruns your notebook with the new test data in place of the current test data.\n\nCan we assume a comparable number of notebooks and the maximum/average notebook size in the new test set?\n\nTo avoid the case the solution is working fine on the current test set but exceeds the allocated time if the new test set is twice as large or contains notebooks with 100k cells (I imagine it would fail for example if someone decides to use the lightweight pairwise model with quadratic complexity over the number of cells).",
      "votes": null
    },
    {
      "id": "1868378",
      "postDate": "07/24/2022 00:03:43",
      "content": "<p>Is it safe to assume that there may be languages in the test set that are not present in the training set? Or will the set of languages present in the test set be a subset of the languages in the training set?</p>",
      "rawMarkdown": "Is it safe to assume that there may be languages in the test set that are not present in the training set? Or will the set of languages present in the test set be a subset of the languages in the training set?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1786072,
      "author_name": "wcukierski",
      "author_url": "",
      "post_date": "05/12/2022 15:31:50",
      "content": "<p>Yes, we wanted to leave the data as intact as possible for this challenge, so there will be multiple languages represented.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1803854,
          "author_name": "astrung",
          "author_url": "",
          "post_date": "05/28/2022 08:47:26",
          "content": "<p><a href=\"https://www.kaggle.com/wcukierski\" target=\"_blank\">@wcukierski</a> <br>\nSorry but I don't get the ideal about how actual test set can be predicted when language is changed. If I have to use multi models for predicting order of cells, first model for language detection and second models for predicting order for each language, so I have so many models here: </p>\n<ul>\n<li>language detection</li>\n<li>predicting order for English cells.</li>\n<li>predicting order for Japanese cells</li>\n<li>predicting order for Chinese cells</li>\n<li>…<br>\nso how can kaggle can replace sampled test rows with actual rows and rerun correct model for predicting ? As an example, if we replace an English notebook with Japanese notebook, my code need to re-run language detection model, get Japanese model and then apply a Japanese cell model instead of English cell model, can kaggle do that ?</li>\n</ul>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1809230,
          "author_name": "wcukierski",
          "author_url": "",
          "post_date": "06/02/2022 14:30:00",
          "content": "<p><a href=\"https://www.kaggle.com/astrung\" target=\"_blank\">@astrung</a> Kaggle reruns your notebook with the new test data in place of the current test data. If your approach has multiple models, it has to decide which ones to use on its own.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1822893,
          "author_name": "andreybazhan",
          "author_url": "",
          "post_date": "06/16/2022 19:30:25",
          "content": "<p>This decision is quite questionable. To mix multilanguage task with task of natural-programming language correspondence.<br>\nAt least can we expect that test set has same language distribution? Test and train sets should be drawn from the same distribution.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1823061,
          "author_name": "dmytropoplavskiy",
          "author_url": "",
          "post_date": "06/17/2022 02:17:10",
          "content": "<blockquote>\n  <p>Kaggle reruns your notebook with the new test data in place of the current test data.</p>\n</blockquote>\n<p>Can we assume a comparable number of notebooks and the maximum/average notebook size in the new test set?</p>\n<p>To avoid the case the solution is working fine on the current test set but exceeds the allocated time if the new test set is twice as large or contains notebooks with 100k cells (I imagine it would fail for example if someone decides to use the lightweight pairwise model with quadratic complexity over the number of cells).</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1868378,
          "author_name": "shreyasagarwal",
          "author_url": "",
          "post_date": "07/24/2022 00:03:43",
          "content": "<p>Is it safe to assume that there may be languages in the test set that are not present in the training set? Or will the set of languages present in the test set be a subset of the languages in the training set?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1785819": "Is the final test set going to be filtered for markdowns written in English?\n\nDidn't look into the data yet, just saw that the first training sample's (00001756c60be8.json) markdowns are written with cyrillic letters.\nIs it expected that the test data will also have such non-English samples?",
    "1786072": "Yes, we wanted to leave the data as intact as possible for this challenge, so there will be multiple languages represented.",
    "1803854": "wcukierski \nSorry but I don't get the ideal about how actual test set can be predicted when language is changed. If I have to use multi models for predicting order of cells, first model for language detection and second models for predicting order for each language, so I have so many models here: \n* language detection\n* predicting order for English cells.\n* predicting order for Japanese cells\n* predicting order for Chinese cells\n* ...\nso how can kaggle can replace sampled test rows with actual rows and rerun correct model for predicting ? As an example, if we replace an English notebook with Japanese notebook, my code need to re-run language detection model, get Japanese model and then apply a Japanese cell model instead of English cell model, can kaggle do that ?",
    "1809230": "astrung Kaggle reruns your notebook with the new test data in place of the current test data. If your approach has multiple models, it has to decide which ones to use on its own.",
    "1822893": "This decision is quite questionable. To mix multilanguage task with task of natural-programming language correspondence.\nAt least can we expect that test set has same language distribution? Test and train sets should be drawn from the same distribution.",
    "1823061": "> Kaggle reruns your notebook with the new test data in place of the current test data.\n\nCan we assume a comparable number of notebooks and the maximum/average notebook size in the new test set?\n\nTo avoid the case the solution is working fine on the current test set but exceeds the allocated time if the new test set is twice as large or contains notebooks with 100k cells (I imagine it would fail for example if someone decides to use the lightweight pairwise model with quadratic complexity over the number of cells).",
    "1868378": "Is it safe to assume that there may be languages in the test set that are not present in the training set? Or will the set of languages present in the test set be a subset of the languages in the training set?"
  },
  "source": "meta"
}