{
  "id": 12843,
  "title": "Conv net's first layer of features ",
  "url": "/competitions/datasciencebowl/discussion/12843",
  "author_name": "HAL2014",
  "post_date": "2015-03-16T23:41:56.520000",
  "votes": 0,
  "comment_count": 0,
  "views": 690,
  "content": "<p>I've attached a snapshot of the first layer of features from&nbsp;my conv net in case anyone is interested. My conclusions from this are -</p>\n<p>- the conv net trained fairly successfully, since the filters are Gabor-like as would be expected</p>\n<p>- there are&nbsp;a couple of rather similar but very distinct spot patterns (row 3, #2 and #4), which I think are unique to the dataset, e.g. if compared to something like MNIST where such precise filters&nbsp;have less&nbsp;discriminative value (on loopy numerals)</p>\n<p>Beyond that, I wasn't sure whether the filters suggested I should be increasing the depth of my first layer, or doing something&nbsp;to help break the symmetry between features like the spot pattern at the same level...</p>\n<p>Anyone have any thoughts, or useful insights into first layer features? Is it better to have redundancy in the first layer, to make sure you aren't losing information so early on in the network? Is there any way to better inform your CNN architecture by examining these features?</p>\n<p>Well done to all those who learned something from the contest&nbsp;:) This was my first ever Kaggle contest and (despite all this controversy with the code releases at the end) was a lot of fun, so thanks to everyone who's been posting on the boards!</p>",
  "messages": [
    {
      "id": 66571,
      "postDate": "2015-03-16T23:41:56.520Z",
      "content": "<p>I've attached a snapshot of the first layer of features from&nbsp;my conv net in case anyone is interested. My conclusions from this are -</p>\n<p>- the conv net trained fairly successfully, since the filters are Gabor-like as would be expected</p>\n<p>- there are&nbsp;a couple of rather similar but very distinct spot patterns (row 3, #2 and #4), which I think are unique to the dataset, e.g. if compared to something like MNIST where such precise filters&nbsp;have less&nbsp;discriminative value (on loopy numerals)</p>\n<p>Beyond that, I wasn't sure whether the filters suggested I should be increasing the depth of my first layer, or doing something&nbsp;to help break the symmetry between features like the spot pattern at the same level...</p>\n<p>Anyone have any thoughts, or useful insights into first layer features? Is it better to have redundancy in the first layer, to make sure you aren't losing information so early on in the network? Is there any way to better inform your CNN architecture by examining these features?</p>\n<p>Well done to all those who learned something from the contest&nbsp;:) This was my first ever Kaggle contest and (despite all this controversy with the code releases at the end) was a lot of fun, so thanks to everyone who's been posting on the boards!</p>"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "66571": ""
  }
}