{
  "id": 416226,
  "title": "Want to know how to reduce scoring time",
  "url": "/competitions/benetech-making-graphs-accessible/discussion/416226",
  "author_name": "",
  "post_date": "2023-06-10T08:53:12.538960Z",
  "votes": 2,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I use deplot, but the current scoring time is too long, I would like to know how to reduce this time, thank you</p>",
  "messages": [
    {
      "id": "2294708",
      "postDate": "06/10/2023 08:53:12",
      "content": "<p>I use deplot, but the current scoring time is too long, I would like to know how to reduce this time, thank you</p>",
      "rawMarkdown": "I use deplot, but the current scoring time is too long, I would like to know how to reduce this time, thank you",
      "votes": null
    },
    {
      "id": "2294926",
      "postDate": "06/10/2023 13:07:49",
      "content": "<p>If you're not already utilizing the early stopping, you might consider it. </p>\n<pre><code>outputs = model.generate(\n    input_vals,\n    max_length=CFG.max_length,\n    early_stopping=,\n    pad_token_id=processor.tokenizer.pad_token_id,\n    eos_token_id=processor.tokenizer.eos_token_id\n)\n</code></pre>\n<p>or if your model has a different stop token:</p>\n<pre><code> transformers  StoppingCriteria, StoppingCriteriaList\n\n ():\n\n     ():\n        ().__init__()\n        self.stops = [stop.to(device)  stop  stops]\n\n     ():\n         stop  self.stops:\n             torch.((stop == input_ids[][-(stop):])).item():\n                 \n\n         \n\n\nmodel = \nprocessor = \nstop_words = []\nstop_words_ids = [processor.tokenizer(stop_word, return_tensors=, add_special_tokens=)[]  stop_word  stop_words]\nstopping_criteria = StoppingCriteriaList([StoppingCriteriaSub(stops=stop_words_ids)])\n\noutputs = model.generate(\n        input_vals,\n        max_length=CFG.max_length,\n        early_stopping=,\n        pad_token_id=processor.tokenizer.pad_token_id,\n        eos_token_id=processor.tokenizer.eos_token_id,\n        stopping_criteria=stopping_criteria \n)\n</code></pre>",
      "rawMarkdown": "If you're not already utilizing the early stopping, you might consider it. \n\n```python\noutputs = model.generate(\n    input_vals,\n    max_length=CFG.max_length,\n    early_stopping=True,\n    pad_token_id=processor.tokenizer.pad_token_id,\n    eos_token_id=processor.tokenizer.eos_token_id\n)\n```\n\nor if your model has a different stop token:\n\n```python\nfrom transformers import StoppingCriteria, StoppingCriteriaList\n\nclass StoppingCriteriaSub(StoppingCriteria):\n\n    def __init__(self, stops = [], encounters=1):\n        super().__init__()\n        self.stops = [stop.to(device) for stop in stops]\n\n    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor):\n        for stop in self.stops:\n            if torch.all((stop == input_ids[0][-len(stop):])).item():\n                return True\n\n        return False\n\n\nmodel = \"YOUR_GENERATIVE_MODEL\"\nprocessor = \"YOUR_PROCESSOR\"\nstop_words = [\"YOUR_STOP_TOKEN\"]\nstop_words_ids = [processor.tokenizer(stop_word, return_tensors=\"pt\", add_special_tokens=False)['input_ids'] for stop_word in stop_words]\nstopping_criteria = StoppingCriteriaList([StoppingCriteriaSub(stops=stop_words_ids)])\n\noutputs = model.generate(\n        input_vals,\n        max_length=CFG.max_length,\n        early_stopping=True,\n        pad_token_id=processor.tokenizer.pad_token_id,\n        eos_token_id=processor.tokenizer.eos_token_id,\n        stopping_criteria=stopping_criteria # <-- Add you own stopping criteria\n)\n\n\n```",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2294926,
      "author_name": "snnclsr",
      "author_url": "",
      "post_date": "06/10/2023 13:07:49",
      "content": "<p>If you're not already utilizing the early stopping, you might consider it. </p>\n<pre><code>outputs = model.generate(\n    input_vals,\n    max_length=CFG.max_length,\n    early_stopping=,\n    pad_token_id=processor.tokenizer.pad_token_id,\n    eos_token_id=processor.tokenizer.eos_token_id\n)\n</code></pre>\n<p>or if your model has a different stop token:</p>\n<pre><code> transformers  StoppingCriteria, StoppingCriteriaList\n\n ():\n\n     ():\n        ().__init__()\n        self.stops = [stop.to(device)  stop  stops]\n\n     ():\n         stop  self.stops:\n             torch.((stop == input_ids[][-(stop):])).item():\n                 \n\n         \n\n\nmodel = \nprocessor = \nstop_words = []\nstop_words_ids = [processor.tokenizer(stop_word, return_tensors=, add_special_tokens=)[]  stop_word  stop_words]\nstopping_criteria = StoppingCriteriaList([StoppingCriteriaSub(stops=stop_words_ids)])\n\noutputs = model.generate(\n        input_vals,\n        max_length=CFG.max_length,\n        early_stopping=,\n        pad_token_id=processor.tokenizer.pad_token_id,\n        eos_token_id=processor.tokenizer.eos_token_id,\n        stopping_criteria=stopping_criteria \n)\n</code></pre>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2294708": "I use deplot, but the current scoring time is too long, I would like to know how to reduce this time, thank you",
    "2294926": "If you're not already utilizing the early stopping, you might consider it. \n\n```python\noutputs = model.generate(\n    input_vals,\n    max_length=CFG.max_length,\n    early_stopping=True,\n    pad_token_id=processor.tokenizer.pad_token_id,\n    eos_token_id=processor.tokenizer.eos_token_id\n)\n```\n\nor if your model has a different stop token:\n\n```python\nfrom transformers import StoppingCriteria, StoppingCriteriaList\n\nclass StoppingCriteriaSub(StoppingCriteria):\n\n    def __init__(self, stops = [], encounters=1):\n        super().__init__()\n        self.stops = [stop.to(device) for stop in stops]\n\n    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor):\n        for stop in self.stops:\n            if torch.all((stop == input_ids[0][-len(stop):])).item():\n                return True\n\n        return False\n\n\nmodel = \"YOUR_GENERATIVE_MODEL\"\nprocessor = \"YOUR_PROCESSOR\"\nstop_words = [\"YOUR_STOP_TOKEN\"]\nstop_words_ids = [processor.tokenizer(stop_word, return_tensors=\"pt\", add_special_tokens=False)['input_ids'] for stop_word in stop_words]\nstopping_criteria = StoppingCriteriaList([StoppingCriteriaSub(stops=stop_words_ids)])\n\noutputs = model.generate(\n        input_vals,\n        max_length=CFG.max_length,\n        early_stopping=True,\n        pad_token_id=processor.tokenizer.pad_token_id,\n        eos_token_id=processor.tokenizer.eos_token_id,\n        stopping_criteria=stopping_criteria # <-- Add you own stopping criteria\n)\n\n\n```"
  },
  "source": "meta"
}