![\"Image_mean\"](\"https://dl.dropboxusercontent.com/u/16873884/data_mean.png\")
With all the deep nets on the leaderboard, I suspect there could be some pretty cool viz to come out of this. Will there be a viz competition?
\n\n
I'm a complete noob to deep learning, but I'm learning quickly. I thought the denoising autoencoder weights dumped as images were pretty interesting:
\n\n
", "messages": [ { "id": "36836", "postDate": "12/30/2013 22:55:56", "content": "
With all the deep nets on the leaderboard, I suspect there could be some pretty cool viz to come out of this. Will there be a viz competition?
\n\n
I'm a complete noob to deep learning, but I'm learning quickly. I thought the denoising autoencoder weights dumped as images were pretty interesting:
\n\n
", "rawMarkdown": "", "votes": null }, { "id": "36853", "postDate": "12/31/2013 04:48:13", "content": "
I'd love some good visualizations - we're sorely lacking some right now. I don't know that it will be a formal contest any time soon, but I promise careful consideration and exposure through the Zooniverse if anyone's interested.
\n\n
I like this first one! - can you explain it further?
", "rawMarkdown": "", "votes": null }, { "id": "36854", "postDate": "12/31/2013 04:55:34", "content": "[quote=Kyle Willett;36853]
\ncan you explain it further?
\n[/quote]
\n\n
Not very well, I'm afraid. I'm still very new to the idea of autoencoders, but I'll do my best to recite what I've learned:
\nThis is a sample of the weights in the autoencoder. The weights are a matrix trained to transform an image (a galaxy) into a hidden representation (encode), and then another set of weights transform the hidden representation back into the original image (decode). In this case, the decoder weights are just the transpose of the encoder weights (called tied weights). This sounds a lot like a matrix identity function, but you leave out a percentage of the input while still expecting the output, which causes it to learn a general representation of the set of images.
\nIf anyone else has a way to explain it that makes more sense or corrects any mistakes I made, I'd appreciate it.
", "rawMarkdown": "", "votes": null }, { "id": "36885", "postDate": "01/01/2014 01:12:17", "content": "I'll post some from my model soon.
Looks decent Zac, but you've got a lot of units that are stuck/not learned. If you're using ReLU try a small positive bias to start with, dropout also helps unstick units usually.
What did you use for converting the weights back to an rgb image? The current thing I've been using is gray as a baseline for a 0 weight and taking negative r,g,b weights as positive of their opposite. This is what Alex Krizhevsky used in his masters, which is one of the first papers I've read with an explanation of their color visualizations.
http://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf
I used this handy script to reshape the weights matrix into tiles and then rendered the image with PIL. To extract the three color channels, I wrote this little function:
\n\n
def dump_weights_as_image(da, title):
weights = da.weights.get_value().T
weights = weights.reshape(500, 424, 424, 3)
red = weights[:, :, :, 0]
blue = weights[:, :, :, 1]
green = weights[:, :, :, 2]
tiles = plot.tile_raster_images(
X=(red, blue, green, None),
img_shape=(424, 424), tile_shape=(10, 10),
tile_spacing=(1, 1)) image = PIL.Image.fromarray(tiles) image.save("images/%s.png" % title)
Roland Memisevic has some code here:
http://www.iro.umontreal.ca/~memisevr/cifar2013/dispims_color.py
Pretty cool, I like the colors. I'm currently working with grayscale pictures only.
\nHere is what my first denoising autoencoder layer learned after 7000 iterations on the images.
", "rawMarkdown": "", "votes": null }, { "id": "38896", "postDate": "02/16/2014 10:14:26", "content": "Hi, I have also plotted the mean of all the images (I used centre 224*224*3 patch which I deem will suffice). Even though mean does not mean anything, but by looking at the mean, there is a very clear round shape with some certain colour at the brink. I think I am plotting the right mean, but the image seems a bit too regular and the colour formation is a bit odd. Anyone have similar result or spot anything erroneous here?
\n
Apologies for the erroneous picture, in Python pylab.imshow, we need to normalize to (0,1). Attached with the correct (boring) mean:
\n![\"correct_mean\"](\"https://dl.dropboxusercontent.com/u/16873884/data_mean_python_correct.png\")
I'd love some good visualizations - we're sorely lacking some right now. I don't know that it will be a formal contest any time soon, but I promise careful consideration and exposure through the Zooniverse if anyone's interested.
\n\n
I like this first one! - can you explain it further?
", "votes": null, "replies": [] }, { "id": 36854, "author_name": "zacstewart", "author_url": "", "post_date": "12/31/2013 04:55:34", "content": "[quote=Kyle Willett;36853]
\ncan you explain it further?
\n[/quote]
\n\n
Not very well, I'm afraid. I'm still very new to the idea of autoencoders, but I'll do my best to recite what I've learned:
\nThis is a sample of the weights in the autoencoder. The weights are a matrix trained to transform an image (a galaxy) into a hidden representation (encode), and then another set of weights transform the hidden representation back into the original image (decode). In this case, the decoder weights are just the transpose of the encoder weights (called tied weights). This sounds a lot like a matrix identity function, but you leave out a percentage of the input while still expecting the output, which causes it to learn a general representation of the set of images.
\nIf anyone else has a way to explain it that makes more sense or corrects any mistakes I made, I'd appreciate it.
", "votes": null, "replies": [] }, { "id": 36885, "author_name": "alecradford", "author_url": "", "post_date": "01/01/2014 01:12:17", "content": "I'll post some from my model soon.
Looks decent Zac, but you've got a lot of units that are stuck/not learned. If you're using ReLU try a small positive bias to start with, dropout also helps unstick units usually.
What did you use for converting the weights back to an rgb image? The current thing I've been using is gray as a baseline for a 0 weight and taking negative r,g,b weights as positive of their opposite. This is what Alex Krizhevsky used in his masters, which is one of the first papers I've read with an explanation of their color visualizations.
http://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf
I used this handy script to reshape the weights matrix into tiles and then rendered the image with PIL. To extract the three color channels, I wrote this little function:
\n\n
def dump_weights_as_image(da, title):
weights = da.weights.get_value().T
weights = weights.reshape(500, 424, 424, 3)
red = weights[:, :, :, 0]
blue = weights[:, :, :, 1]
green = weights[:, :, :, 2]
tiles = plot.tile_raster_images(
X=(red, blue, green, None),
img_shape=(424, 424), tile_shape=(10, 10),
tile_spacing=(1, 1)) image = PIL.Image.fromarray(tiles) image.save("images/%s.png" % title)
Roland Memisevic has some code here:
http://www.iro.umontreal.ca/~memisevr/cifar2013/dispims_color.py
Pretty cool, I like the colors. I'm currently working with grayscale pictures only.
\nHere is what my first denoising autoencoder layer learned after 7000 iterations on the images.
", "votes": null, "replies": [] }, { "id": 38896, "author_name": "stevenwudi", "author_url": "", "post_date": "02/16/2014 10:14:26", "content": "Hi, I have also plotted the mean of all the images (I used centre 224*224*3 patch which I deem will suffice). Even though mean does not mean anything, but by looking at the mean, there is a very clear round shape with some certain colour at the brink. I think I am plotting the right mean, but the image seems a bit too regular and the colour formation is a bit odd. Anyone have similar result or spot anything erroneous here?
\n
Apologies for the erroneous picture, in Python pylab.imshow, we need to normalize to (0,1). Attached with the correct (boring) mean:
\n