{
  "id": 50329,
  "title": "Weird submission exception \"Index was outside the bounds of the array.\"",
  "url": "/competitions/data-science-bowl-2018/discussion/50329",
  "author_name": "",
  "post_date": "2018-02-22T15:04:55.022617900Z",
  "votes": null,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hi all,</p>\n\n<p>I have suffered by this exception during submission for a while now, wonder if someone can delight me a little bit.</p>\n\n<p>I use proven rle encoding method as many of you do and have run additional like comparing mask pixel index with total number of pixels in the given test image, and it all went well.</p>\n\n<pre><code>def rle_encoding(x, total_pixel):\n    dots = np.where(x.T.flatten() == 1)[0]\n    run_lengths = []\n    prev = -2\n    for b in dots:\n        #make sure mask pixel index is less than total number of pixel.\n        assert(b&gt;=(total_pixel-1))\n        if (b&gt;prev+1): run_lengths.extend((b + 1, 0))\n           run_lengths[-1] += 1\n           prev = b\n    return run_lengths\n</code></pre>\n\n<p>How i calculate the total number of pixels in original test image, of course i have revert the resizing of the test image.</p>\n\n<p><code>total_pixel = original_image.shape[0]*original_image.shape[1]</code></p>\n\n<p>Thank you in advance!</p>",
  "messages": [
    {
      "id": "286642",
      "postDate": "02/22/2018 15:04:55",
      "content": "<p>Hi all,</p>\n\n<p>I have suffered by this exception during submission for a while now, wonder if someone can delight me a little bit.</p>\n\n<p>I use proven rle encoding method as many of you do and have run additional like comparing mask pixel index with total number of pixels in the given test image, and it all went well.</p>\n\n<pre><code>def rle_encoding(x, total_pixel):\n    dots = np.where(x.T.flatten() == 1)[0]\n    run_lengths = []\n    prev = -2\n    for b in dots:\n        #make sure mask pixel index is less than total number of pixel.\n        assert(b&gt;=(total_pixel-1))\n        if (b&gt;prev+1): run_lengths.extend((b + 1, 0))\n           run_lengths[-1] += 1\n           prev = b\n    return run_lengths\n</code></pre>\n\n<p>How i calculate the total number of pixels in original test image, of course i have revert the resizing of the test image.</p>\n\n<p><code>total_pixel = original_image.shape[0]*original_image.shape[1]</code></p>\n\n<p>Thank you in advance!</p>",
      "rawMarkdown": "Hi all,\n\nI have suffered by this exception during submission for a while now, wonder if someone can delight me a little bit.\n\nI use proven rle encoding method as many of you do and have run additional like comparing mask pixel index with total number of pixels in the given test image, and it all went well.\n\n    def rle_encoding(x, total_pixel):\n        dots = np.where(x.T.flatten() == 1)[0]\n        run_lengths = []\n        prev = -2\n        for b in dots:\n            #make sure mask pixel index is less than total number of pixel.\n            assert(b&gt;=(total_pixel-1))\n            if (b&gt;prev+1): run_lengths.extend((b + 1, 0))\n               run_lengths[-1] += 1\n               prev = b\n        return run_lengths\n\n\n\n\nHow i calculate the total number of pixels in original test image, of course i have revert the resizing of the test image.\n\n`total_pixel = original_image.shape[0]*original_image.shape[1]`\n\nThank you in advance!",
      "votes": null
    },
    {
      "id": "286658",
      "postDate": "02/22/2018 15:47:43",
      "content": "<p>I've had trouble with a few items that lead me to that error in the past: one was down to me indexing the run length encoding to start at 0 rather than 1. The other was down to me indexing left-right, top-bottom rather than top-bottom, left-right.</p>\n\n<p>If you want to try another version for sanity checking this is what I am using:</p>\n\n<p>Call run_length_encode(mask.T, index_offset = 1)</p>\n\n<pre><code>def run_length_encode(mask, index_offset = 0):\n\n    '''\n    img: numpy array, 1 - mask, 0 - background\n    Returns run length as string formated.\n    If index_offset = 1 we assume arrays start at 1.\n    '''\n\n    inds = np.append(np.insert(mask.flatten(), 0, 0), 0) \n    runs = np.where(inds[1:] != inds[:-1])[0]\n    runs[1::2] = runs[1::2] - runs[:-1:2] \n\n    if index_offset &gt; 0:\n        runs[0::2] += index_offset\n\n    rle = ' '.join([str(r) for r in runs])\n\n    return rle\n</code></pre>",
      "rawMarkdown": "I've had trouble with a few items that lead me to that error in the past: one was down to me indexing the run length encoding to start at 0 rather than 1. The other was down to me indexing left-right, top-bottom rather than top-bottom, left-right.\n\nIf you want to try another version for sanity checking this is what I am using:\n\nCall run_length_encode(mask.T, index_offset = 1)\n\n    def run_length_encode(mask, index_offset = 0):\n\n        '''\n        img: numpy array, 1 - mask, 0 - background\n        Returns run length as string formated.\n        If index_offset = 1 we assume arrays start at 1.\n        '''\n\n        inds = np.append(np.insert(mask.flatten(), 0, 0), 0) \n        runs = np.where(inds[1:] != inds[:-1])[0]\n        runs[1::2] = runs[1::2] - runs[:-1:2] \n    \n        if index_offset &gt; 0:\n            runs[0::2] += index_offset\n                \n        rle = ' '.join([str(r) for r in runs])\n    \n        return rle",
      "votes": null
    },
    {
      "id": "286675",
      "postDate": "02/22/2018 16:28:06",
      "content": "<p>Have you checked if there are any empty lines in your submission?  I don't know if this is your case, I encountered this same error message before and fixed it by removing the empty lines. </p>\n\n<p>E.g.</p>\n\n<pre><code>ImageID                             EncodePixels\n123abef3485902938576                1 5 2 1\n123abef3485902938576                                                     &amp;lt;-- Empty Line\n123abef3485902938576                459 2 500 8\n</code></pre>",
      "rawMarkdown": "Have you checked if there are any empty lines in your submission?  I don't know if this is your case, I encountered this same error message before and fixed it by removing the empty lines. \n\nE.g.\n\n    ImageID                             EncodePixels\n    123abef3485902938576                1 5 2 1\n    123abef3485902938576                                                     &lt;-- Empty Line\n    123abef3485902938576                459 2 500 8",
      "votes": null
    },
    {
      "id": "286838",
      "postDate": "02/22/2018 22:13:38",
      "content": "<p>Thx Abner and fergusoci, the pb was empty line as Abner pointed, not sure what such mask appears in the result although!</p>",
      "rawMarkdown": "Thx Abner and fergusoci, the pb was empty line as Abner pointed, not sure what such mask appears in the result although!",
      "votes": null
    },
    {
      "id": "286839",
      "postDate": "02/22/2018 22:14:11",
      "content": "<p>Thx fergusoci, good to have to implementation to cross check!</p>",
      "rawMarkdown": "Thx fergusoci, good to have to implementation to cross check!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 286658,
      "author_name": "fergusoci",
      "author_url": "",
      "post_date": "02/22/2018 15:47:43",
      "content": "<p>I've had trouble with a few items that lead me to that error in the past: one was down to me indexing the run length encoding to start at 0 rather than 1. The other was down to me indexing left-right, top-bottom rather than top-bottom, left-right.</p>\n\n<p>If you want to try another version for sanity checking this is what I am using:</p>\n\n<p>Call run_length_encode(mask.T, index_offset = 1)</p>\n\n<pre><code>def run_length_encode(mask, index_offset = 0):\n\n    '''\n    img: numpy array, 1 - mask, 0 - background\n    Returns run length as string formated.\n    If index_offset = 1 we assume arrays start at 1.\n    '''\n\n    inds = np.append(np.insert(mask.flatten(), 0, 0), 0) \n    runs = np.where(inds[1:] != inds[:-1])[0]\n    runs[1::2] = runs[1::2] - runs[:-1:2] \n\n    if index_offset &gt; 0:\n        runs[0::2] += index_offset\n\n    rle = ' '.join([str(r) for r in runs])\n\n    return rle\n</code></pre>",
      "votes": null,
      "replies": [
        {
          "id": 286839,
          "author_name": "theone4ever",
          "author_url": "",
          "post_date": "02/22/2018 22:14:11",
          "content": "<p>Thx fergusoci, good to have to implementation to cross check!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 286675,
      "author_name": "abnerchou",
      "author_url": "",
      "post_date": "02/22/2018 16:28:06",
      "content": "<p>Have you checked if there are any empty lines in your submission?  I don't know if this is your case, I encountered this same error message before and fixed it by removing the empty lines. </p>\n\n<p>E.g.</p>\n\n<pre><code>ImageID                             EncodePixels\n123abef3485902938576                1 5 2 1\n123abef3485902938576                                                     &amp;lt;-- Empty Line\n123abef3485902938576                459 2 500 8\n</code></pre>",
      "votes": null,
      "replies": [
        {
          "id": 286838,
          "author_name": "theone4ever",
          "author_url": "",
          "post_date": "02/22/2018 22:13:38",
          "content": "<p>Thx Abner and fergusoci, the pb was empty line as Abner pointed, not sure what such mask appears in the result although!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "286642": "Hi all,\n\nI have suffered by this exception during submission for a while now, wonder if someone can delight me a little bit.\n\nI use proven rle encoding method as many of you do and have run additional like comparing mask pixel index with total number of pixels in the given test image, and it all went well.\n\n    def rle_encoding(x, total_pixel):\n        dots = np.where(x.T.flatten() == 1)[0]\n        run_lengths = []\n        prev = -2\n        for b in dots:\n            #make sure mask pixel index is less than total number of pixel.\n            assert(b&gt;=(total_pixel-1))\n            if (b&gt;prev+1): run_lengths.extend((b + 1, 0))\n               run_lengths[-1] += 1\n               prev = b\n        return run_lengths\n\n\n\n\nHow i calculate the total number of pixels in original test image, of course i have revert the resizing of the test image.\n\n`total_pixel = original_image.shape[0]*original_image.shape[1]`\n\nThank you in advance!",
    "286658": "I've had trouble with a few items that lead me to that error in the past: one was down to me indexing the run length encoding to start at 0 rather than 1. The other was down to me indexing left-right, top-bottom rather than top-bottom, left-right.\n\nIf you want to try another version for sanity checking this is what I am using:\n\nCall run_length_encode(mask.T, index_offset = 1)\n\n    def run_length_encode(mask, index_offset = 0):\n\n        '''\n        img: numpy array, 1 - mask, 0 - background\n        Returns run length as string formated.\n        If index_offset = 1 we assume arrays start at 1.\n        '''\n\n        inds = np.append(np.insert(mask.flatten(), 0, 0), 0) \n        runs = np.where(inds[1:] != inds[:-1])[0]\n        runs[1::2] = runs[1::2] - runs[:-1:2] \n    \n        if index_offset &gt; 0:\n            runs[0::2] += index_offset\n                \n        rle = ' '.join([str(r) for r in runs])\n    \n        return rle",
    "286675": "Have you checked if there are any empty lines in your submission?  I don't know if this is your case, I encountered this same error message before and fixed it by removing the empty lines. \n\nE.g.\n\n    ImageID                             EncodePixels\n    123abef3485902938576                1 5 2 1\n    123abef3485902938576                                                     &lt;-- Empty Line\n    123abef3485902938576                459 2 500 8",
    "286838": "Thx Abner and fergusoci, the pb was empty line as Abner pointed, not sure what such mask appears in the result although!",
    "286839": "Thx fergusoci, good to have to implementation to cross check!"
  },
  "source": "meta"
}