{
  "id": 332982,
  "title": "submission status is 'Notebook Exceeded Allowed Compute'",
  "url": "/competitions/AI4Code/discussion/332982",
  "author_name": "",
  "post_date": "2022-06-24T08:26:50.951671900Z",
  "votes": 4,
  "comment_count": 3,
  "views": 0,
  "content": "<p><a href=\"https://www.kaggle.com/code/kimnnm/roberta-base-inference/notebook\" target=\"_blank\">https://www.kaggle.com/code/kimnnm/roberta-base-inference/notebook</a></p>\n<p>When I submitted the file, I got the following message.<br>\n'Notebook Exceeded Allowed Compute'</p>\n<p>If I run it with four test files, the memory usage is about 5GB.</p>",
  "messages": [
    {
      "id": "1831532",
      "postDate": "06/24/2022 08:26:50",
      "content": "<p><a href=\"https://www.kaggle.com/code/kimnnm/roberta-base-inference/notebook\" target=\"_blank\">https://www.kaggle.com/code/kimnnm/roberta-base-inference/notebook</a></p>\n<p>When I submitted the file, I got the following message.<br>\n'Notebook Exceeded Allowed Compute'</p>\n<p>If I run it with four test files, the memory usage is about 5GB.</p>",
      "rawMarkdown": "https://www.kaggle.com/code/kimnnm/roberta-base-inference/notebook\n\nWhen I submitted the file, I got the following message.\n'Notebook Exceeded Allowed Compute'\n\nIf I run it with four test files, the memory usage is about 5GB.",
      "votes": null
    },
    {
      "id": "1831990",
      "postDate": "06/24/2022 15:08:38",
      "content": "<p>'Notebook Exceeded Allowed Compute' means it took too long to run against the test set, running out of memory's a different error.  </p>\n<p>I don't know if AutoTokenizer uses a Fast tokenizer, if not that's one place to look…</p>",
      "rawMarkdown": "'Notebook Exceeded Allowed Compute' means it took too long to run against the test set, running out of memory's a different error.  \n\nI don't know if AutoTokenizer uses a Fast tokenizer, if not that's one place to look...",
      "votes": null
    },
    {
      "id": "1837842",
      "postDate": "06/30/2022 01:56:28",
      "content": "<p><a href=\"https://www.kaggle.com/happycube\" target=\"_blank\">@happycube</a> <br>\nThanks for comment.<br>\nI'm looking for a way.</p>\n<p>FYI<br>\nAutoTokenizer used fast Tokenizer as default option<br>\n<a href=\"https://huggingface.co/docs/transformers/v4.20.1/en/model_doc/auto#transformers.AutoTokenizer.from_pretrained.tokenizer_type\" target=\"_blank\">https://huggingface.co/docs/transformers/v4.20.1/en/model_doc/auto#transformers.AutoTokenizer.from_pretrained.tokenizer_type</a></p>",
      "rawMarkdown": "happycube \nThanks for comment.\nI'm looking for a way.\n\nFYI\nAutoTokenizer used fast Tokenizer as default option\nhttps://huggingface.co/docs/transformers/v4.20.1/en/model_doc/auto#transformers.AutoTokenizer.from_pretrained.tokenizer_type",
      "votes": null
    },
    {
      "id": "1838889",
      "postDate": "07/01/2022 00:11:57",
      "content": "<p>problem<br>\nMaybe there is no memory space for tokenization or it took a lot of time</p>\n<p>solution<br>\nTokenize for each batch</p>\n<p><a href=\"https://www.kaggle.com/code/kimnnm/roberta-base-inference/notebook?scriptVersionId=99670175\" target=\"_blank\">https://www.kaggle.com/code/kimnnm/roberta-base-inference/notebook?scriptVersionId=99670175</a></p>",
      "rawMarkdown": "problem\nMaybe there is no memory space for tokenization or it took a lot of time\n\nsolution\nTokenize for each batch\n\nhttps://www.kaggle.com/code/kimnnm/roberta-base-inference/notebook?scriptVersionId=99670175",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1831990,
      "author_name": "happycube",
      "author_url": "",
      "post_date": "06/24/2022 15:08:38",
      "content": "<p>'Notebook Exceeded Allowed Compute' means it took too long to run against the test set, running out of memory's a different error.  </p>\n<p>I don't know if AutoTokenizer uses a Fast tokenizer, if not that's one place to look…</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1837842,
      "author_name": "kimnnm",
      "author_url": "",
      "post_date": "06/30/2022 01:56:28",
      "content": "<p><a href=\"https://www.kaggle.com/happycube\" target=\"_blank\">@happycube</a> <br>\nThanks for comment.<br>\nI'm looking for a way.</p>\n<p>FYI<br>\nAutoTokenizer used fast Tokenizer as default option<br>\n<a href=\"https://huggingface.co/docs/transformers/v4.20.1/en/model_doc/auto#transformers.AutoTokenizer.from_pretrained.tokenizer_type\" target=\"_blank\">https://huggingface.co/docs/transformers/v4.20.1/en/model_doc/auto#transformers.AutoTokenizer.from_pretrained.tokenizer_type</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1838889,
      "author_name": "kimnnm",
      "author_url": "",
      "post_date": "07/01/2022 00:11:57",
      "content": "<p>problem<br>\nMaybe there is no memory space for tokenization or it took a lot of time</p>\n<p>solution<br>\nTokenize for each batch</p>\n<p><a href=\"https://www.kaggle.com/code/kimnnm/roberta-base-inference/notebook?scriptVersionId=99670175\" target=\"_blank\">https://www.kaggle.com/code/kimnnm/roberta-base-inference/notebook?scriptVersionId=99670175</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1831532": "https://www.kaggle.com/code/kimnnm/roberta-base-inference/notebook\n\nWhen I submitted the file, I got the following message.\n'Notebook Exceeded Allowed Compute'\n\nIf I run it with four test files, the memory usage is about 5GB.",
    "1831990": "'Notebook Exceeded Allowed Compute' means it took too long to run against the test set, running out of memory's a different error.  \n\nI don't know if AutoTokenizer uses a Fast tokenizer, if not that's one place to look...",
    "1837842": "happycube \nThanks for comment.\nI'm looking for a way.\n\nFYI\nAutoTokenizer used fast Tokenizer as default option\nhttps://huggingface.co/docs/transformers/v4.20.1/en/model_doc/auto#transformers.AutoTokenizer.from_pretrained.tokenizer_type",
    "1838889": "problem\nMaybe there is no memory space for tokenization or it took a lot of time\n\nsolution\nTokenize for each batch\n\nhttps://www.kaggle.com/code/kimnnm/roberta-base-inference/notebook?scriptVersionId=99670175"
  },
  "source": "meta"
}