{ "id": 12857, "title": "Data Augmentation Questions", "url": "/competitions/datasciencebowl/discussion/12857", "author_name": "", "post_date": "2015-03-17T13:22:02.863Z", "votes": null, "comment_count": 8, "views": 2049, "content": "

Hi everyone,

\n

Great competition! Since it is now over I would appreciate some advice on data augmentation. I was able to successfully implement augmentation on-demand during training but was only able to get reasonable results for the affine transformations of:

\n

rotation: [90,180,270]

\n

flipping: yes or no 

\n

Whenever I tried adding new transformations such as more rotations, or translations and shearing my results would quickly become much worse in both my training and validation log loss/accuracy.

\n

Are there general strategies that people follow when adding more and more transformations? I tried things like increase/decrease batch size, increase/decrease learning rate, increase/decrease number of epochs. I tried different combinations of these but to no success.

\n

Anyone have any ideas of how to make this work?

", "messages": [ { "id": "66697", "postDate": "03/17/2015 13:22:02", "content": "

Hi everyone,

\n

Great competition! Since it is now over I would appreciate some advice on data augmentation. I was able to successfully implement augmentation on-demand during training but was only able to get reasonable results for the affine transformations of:

\n

rotation: [90,180,270]

\n

flipping: yes or no 

\n

Whenever I tried adding new transformations such as more rotations, or translations and shearing my results would quickly become much worse in both my training and validation log loss/accuracy.

\n

Are there general strategies that people follow when adding more and more transformations? I tried things like increase/decrease batch size, increase/decrease learning rate, increase/decrease number of epochs. I tried different combinations of these but to no success.

\n

Anyone have any ideas of how to make this work?

", "rawMarkdown": "", "votes": null }, { "id": "66698", "postDate": "03/17/2015 13:25:04", "content": "

It should help. Here is my guess: one bug I had early on was that the background in the images is white (255) and opencv/scikit-image assume by default that it is black (0). 

", "rawMarkdown": "", "votes": null }, { "id": "66699", "postDate": "03/17/2015 13:36:45", "content": "

Thats interesting, thank you.

\n

Do you know if there is a parameter to pass to fix this? I was using scikit-image will a function call like

\n

tform_augment = transform.AffineTransform(scale=(1/zoom, 1/zoom),
                                                                        rotation=np.deg2rad(rotation),
                                                                        shear=np.deg2rad(shear),
                                                                        translation=translation)

", "rawMarkdown": "", "votes": null }, { "id": "66711", "postDate": "03/17/2015 14:41:58", "content": "

The solution is simple - just invert all images. Actually it is even more logical, when bg == no signal == 0, not otherwise.

", "rawMarkdown": "", "votes": null }, { "id": "66713", "postDate": "03/17/2015 14:48:47", "content": "

What old-ufo said :) We did the same thing. Also, when adding more augmentations, the problem becomes more complex to solve, so you might just need to train for longer.

", "rawMarkdown": "", "votes": null }, { "id": "66715", "postDate": "03/17/2015 14:51:49", "content": "

great, thanks for the feedback. 

", "rawMarkdown": "", "votes": null }, { "id": "66767", "postDate": "03/17/2015 18:12:25", "content": "

Hi mike1886, I tried out your code and get the same problem too.

\n

Not only that, even for four rotations, I wasn't able to get the loss below 0.9,
after a few epochs, the training loss keep dropping, but the validation loss start
to increase. I never encounter this kind of problem when I use caffe, where after some iterations the validation loss stoped dropping or slightly getting worse.
I even inverted the image in realtime augmentation, but still get the same issue.

\n

I saw you subtract the mean before you do realtime augmentation, and use 'nearest' for filling in the blank spaces created by affine transformation, that might be a issue. I just saved some sample images when they was feed into the convnet, they all look like this(attached), I am not sure if it was suppose to be like this or not?

", "rawMarkdown": "", "votes": null }, { "id": "66800", "postDate": "03/17/2015 19:27:53", "content": "

@SkyLibrary: thanks for the reply.

\n

How do you prep your images, do you do any mean/variance subtraction?

\n

How do you fill your space, with wrapping instead?

", "rawMarkdown": "", "votes": null }, { "id": "66806", "postDate": "03/17/2015 20:06:06", "content": "

I was using caffe, after I saved the raw image into leveldb(a format used by caffe), there is a function to calculated the image mean for that leveldb file. Then during training after I use data_transformer to do realtime augmentation, the mean was subtracted right before it feed into convnet with 

\n

transformed_data[top_index] = (datum_element - mean[mean_index]) * scale;

\n

As I said, I don't know how people subtract mean with lasagne/nolearn, it would be great if people who get good results with lasagne/nolearn can share their approach?

\n

I was also converting image into black. If you don't want to convert image into black, you can set the blank to constant value， say instead of 'nearest',  try 'constant' with cval = 0. or 1. depend on what color you want the blank to be.

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 66698, "author_name": "udibr1", "author_url": "", "post_date": "03/17/2015 13:25:04", "content": "

It should help. Here is my guess: one bug I had early on was that the background in the images is white (255) and opencv/scikit-image assume by default that it is black (0). 

", "votes": null, "replies": [] }, { "id": 66699, "author_name": "msegala", "author_url": "", "post_date": "03/17/2015 13:36:45", "content": "

Thats interesting, thank you.

\n

Do you know if there is a parameter to pass to fix this? I was using scikit-image will a function call like

\n

tform_augment = transform.AffineTransform(scale=(1/zoom, 1/zoom),
                                                                        rotation=np.deg2rad(rotation),
                                                                        shear=np.deg2rad(shear),
                                                                        translation=translation)

", "votes": null, "replies": [] }, { "id": 66711, "author_name": "oldufo", "author_url": "", "post_date": "03/17/2015 14:41:58", "content": "

The solution is simple - just invert all images. Actually it is even more logical, when bg == no signal == 0, not otherwise.

", "votes": null, "replies": [] }, { "id": 66713, "author_name": "sedielem", "author_url": "", "post_date": "03/17/2015 14:48:47", "content": "

What old-ufo said :) We did the same thing. Also, when adding more augmentations, the problem becomes more complex to solve, so you might just need to train for longer.

", "votes": null, "replies": [] }, { "id": 66715, "author_name": "msegala", "author_url": "", "post_date": "03/17/2015 14:51:49", "content": "

great, thanks for the feedback. 

", "votes": null, "replies": [] }, { "id": 66767, "author_name": "skylibrary", "author_url": "", "post_date": "03/17/2015 18:12:25", "content": "

Hi mike1886, I tried out your code and get the same problem too.

\n

Not only that, even for four rotations, I wasn't able to get the loss below 0.9,
after a few epochs, the training loss keep dropping, but the validation loss start
to increase. I never encounter this kind of problem when I use caffe, where after some iterations the validation loss stoped dropping or slightly getting worse.
I even inverted the image in realtime augmentation, but still get the same issue.

\n

I saw you subtract the mean before you do realtime augmentation, and use 'nearest' for filling in the blank spaces created by affine transformation, that might be a issue. I just saved some sample images when they was feed into the convnet, they all look like this(attached), I am not sure if it was suppose to be like this or not?

", "votes": null, "replies": [] }, { "id": 66800, "author_name": "msegala", "author_url": "", "post_date": "03/17/2015 19:27:53", "content": "

@SkyLibrary: thanks for the reply.

\n

How do you prep your images, do you do any mean/variance subtraction?

\n

How do you fill your space, with wrapping instead?

", "votes": null, "replies": [] }, { "id": 66806, "author_name": "skylibrary", "author_url": "", "post_date": "03/17/2015 20:06:06", "content": "

I was using caffe, after I saved the raw image into leveldb(a format used by caffe), there is a function to calculated the image mean for that leveldb file. Then during training after I use data_transformer to do realtime augmentation, the mean was subtracted right before it feed into convnet with 

\n

transformed_data[top_index] = (datum_element - mean[mean_index]) * scale;

\n

As I said, I don't know how people subtract mean with lasagne/nolearn, it would be great if people who get good results with lasagne/nolearn can share their approach?

\n

I was also converting image into black. If you don't want to convert image into black, you can set the blank to constant value， say instead of 'nearest',  try 'constant' with cval = 0. or 1. depend on what color you want the blank to be.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "66697": "", "66698": "", "66699": "", "66711": "", "66713": "", "66715": "", "66767": "", "66800": "", "66806": "" }, "source": "meta" }