{
  "id": 12327,
  "title": "one interesting observation",
  "url": "/competitions/datasciencebowl/discussion/12327",
  "author_name": "",
  "post_date": "2015-01-25T08:44:04.663Z",
  "votes": 8,
  "comment_count": 8,
  "views": 3149,
  "content": "<p>It seems the data are generated by a segmentation algorithm. You can see this from the attached two images.&nbsp;</p>\n<p>raw = 255 - img;</p>\n<p>times_by_100 = uint8(1000 * raw)</p>\n<p>However, I am not sure whether it is useful for the performance :(</p>",
  "messages": [
    {
      "id": "63131",
      "postDate": "01/25/2015 08:44:04",
      "content": "<p>It seems the data are generated by a segmentation algorithm. You can see this from the attached two images.&nbsp;</p>\n<p>raw = 255 - img;</p>\n<p>times_by_100 = uint8(1000 * raw)</p>\n<p>However, I am not sure whether it is useful for the performance :(</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63134",
      "postDate": "01/25/2015 09:38:56",
      "content": "<p>Thanks, very interesting. You mean that every picture is tiled by many blocks?&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63135",
      "postDate": "01/25/2015 09:57:01",
      "content": "<p>Yes.</p>\n\n<p>By scaling 1000 times , you can see the noise in the raw data. Interestingly, the noise are artificially block shaped, so I guess the data are generated by a segmentation algorithm.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63136",
      "postDate": "01/25/2015 10:03:31",
      "content": "<p>or maybe an artifact from jpg compression?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63137",
      "postDate": "01/25/2015 10:04:30",
      "content": "<p>Or it is a result of denoising the image. If a block had too little non zero values it was removed. It can be checked calculating the lower bound of non zero pixels in these blocks.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63221",
      "postDate": "01/27/2015 02:49:29",
      "content": "<p>I noticed that too when I tried some localized histogram normalization.&nbsp; My hunch is that in order to get a pure white background they did some trickery with preserving the &quot;central blob&quot; of interest and making the colour taper to white quickly.</p>\n<p>I would like to know (after the comp) if anyone trimmed those bits off and got better results because of it.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63228",
      "postDate": "01/27/2015 07:16:41",
      "content": "<p>This might be relevant to data augmentations, since&nbsp;the orientation and position of the almost-invisible blocks is going to correlate with orientation and position of other features in images that are translated or rotated.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63280",
      "postDate": "01/28/2015 16:33:06",
      "content": "<p>The data description states: &quot;...Each raw image was run through an automatic process to extract regions of interest, resulting in smaller images that contain a single organism/entity.&quot;</p>\n<p>So, they explicitly stated the data were generated by an automatic segmentation algorithm. I also read from some papers how they prepare the images and of course they are using automated segmentation due to the shear volume of the data captured each day.</p>\n<p>Good observation, regardless.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64155",
      "postDate": "02/13/2015 05:38:36",
      "content": "<p>Agreed</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 63134,
      "author_name": "liangpang",
      "author_url": "",
      "post_date": "01/25/2015 09:38:56",
      "content": "<p>Thanks, very interesting. You mean that every picture is tiled by many blocks?&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63135,
      "author_name": "xudongcao",
      "author_url": "",
      "post_date": "01/25/2015 09:57:01",
      "content": "<p>Yes.</p>\n\n<p>By scaling 1000 times , you can see the noise in the raw data. Interestingly, the noise are artificially block shaped, so I guess the data are generated by a segmentation algorithm.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63136,
      "author_name": "mjahrer",
      "author_url": "",
      "post_date": "01/25/2015 10:03:31",
      "content": "<p>or maybe an artifact from jpg compression?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63137,
      "author_name": "paweljankiewicz",
      "author_url": "",
      "post_date": "01/25/2015 10:04:30",
      "content": "<p>Or it is a result of denoising the image. If a block had too little non zero values it was removed. It can be checked calculating the lower bound of non zero pixels in these blocks.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63221,
      "author_name": "zerozero",
      "author_url": "",
      "post_date": "01/27/2015 02:49:29",
      "content": "<p>I noticed that too when I tried some localized histogram normalization.&nbsp; My hunch is that in order to get a pure white background they did some trickery with preserving the &quot;central blob&quot; of interest and making the colour taper to white quickly.</p>\n<p>I would like to know (after the comp) if anyone trimmed those bits off and got better results because of it.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63228,
      "author_name": "slobo777",
      "author_url": "",
      "post_date": "01/27/2015 07:16:41",
      "content": "<p>This might be relevant to data augmentations, since&nbsp;the orientation and position of the almost-invisible blocks is going to correlate with orientation and position of other features in images that are translated or rotated.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63280,
      "author_name": "deepcnn",
      "author_url": "",
      "post_date": "01/28/2015 16:33:06",
      "content": "<p>The data description states: &quot;...Each raw image was run through an automatic process to extract regions of interest, resulting in smaller images that contain a single organism/entity.&quot;</p>\n<p>So, they explicitly stated the data were generated by an automatic segmentation algorithm. I also read from some papers how they prepare the images and of course they are using automated segmentation due to the shear volume of the data captured each day.</p>\n<p>Good observation, regardless.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64155,
      "author_name": "iosentinel",
      "author_url": "",
      "post_date": "02/13/2015 05:38:36",
      "content": "<p>Agreed</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "63131": "",
    "63134": "",
    "63135": "",
    "63136": "",
    "63137": "",
    "63221": "",
    "63228": "",
    "63280": "",
    "64155": ""
  },
  "source": "meta"
}