{ "id": 12652, "title": "CNN", "url": "/competitions/datasciencebowl/discussion/12652", "author_name": "", "post_date": "2015-03-02T03:39:17.763Z", "votes": null, "comment_count": 30, "views": 6758, "content": "

is there is any CNN code for this competition using python lasagne + nolearn ?

", "messages": [ { "id": "65213", "postDate": "03/02/2015 03:39:17", "content": "

is there is any CNN code for this competition using python lasagne + nolearn ?

", "rawMarkdown": "", "votes": null }, { "id": "65243", "postDate": "03/02/2015 15:20:24", "content": "

Hi,

\n

I was trying to use lasagne+nolearn, but the results I obtained are very bad! moreover the values I found for MCL with the model on a test set are very different from the ones on the LB.

", "rawMarkdown": "", "votes": null }, { "id": "65246", "postDate": "03/02/2015 15:39:16", "content": "

I am using a lasagne + nolearn setup, you can check it out here:

\n

https://github.com/msegala/Kaggle-National_Data_Science_Bowl

", "rawMarkdown": "", "votes": null }, { "id": "65247", "postDate": "03/02/2015 15:48:27", "content": "

[quote=mike1886;65246]

\n

I am using a lasagne + nolearn setup, you can check it out here:

\n

https://github.com/msegala/Kaggle-National_Data_Science_Bowl

\n

[/quote]

\n

is this your code? what is the logloss do you get with this?

", "rawMarkdown": "", "votes": null }, { "id": "65250", "postDate": "03/02/2015 15:51:30", "content": "

This is not the final code I use for submission, I have spent a long time tweaking this for performance. 

\n

My current log loss is ~0.75. 

", "rawMarkdown": "", "votes": null }, { "id": "65253", "postDate": "03/02/2015 16:29:02", "content": "

Did you find overfitting? I tried with a similar net, without batchiterator, and Maxout, But I found a lot of overfitting.

", "rawMarkdown": "", "votes": null }, { "id": "65254", "postDate": "03/02/2015 16:36:22", "content": "

I was able to get pretty good results by training a deep network with large layers. Of course there is still overfitting but using dropout and maxout layers helps a lot. 

", "rawMarkdown": "", "votes": null }, { "id": "65255", "postDate": "03/02/2015 16:36:46", "content": "

I am just curious here, since Maxout was proposed as an activation function to enhance dropout's model averaging accuracy, why would you put dropout before Maxout? Did you observe any improvement than putting dropout after Maxout?

", "rawMarkdown": "", "votes": null }, { "id": "65258", "postDate": "03/02/2015 16:43:00", "content": "

@yr, do you mean why did I put Maxout before dropout?

\n

layers=[
('input', layers.InputLayer),
('conv1', Conv2DLayer),
('pool1', MaxPool2DLayer),
('dropout1', layers.DropoutLayer),

\n

.....

\n

right now my maxout layers feeds into my drouput layer

", "rawMarkdown": "", "votes": null }, { "id": "65262", "postDate": "03/02/2015 16:52:13", "content": "

Yes. The part you quoted is max-pooling (across spatial). I mean here:

\n

('hidden4', layers.DenseLayer),
('dropout4', layers.DropoutLayer),
('maxout4', Maxout),
('hidden5', layers.DenseLayer),
('dropout5', layers.DropoutLayer),
('maxout5', Maxout),
('output', layers.DenseLayer),

\n

maxout (across feature map) is after dropout. If you treat maxout as activation function, as I understand, it should be something like:

\n

('hidden4', layers.DenseLayer),
('maxout4', Maxout),

\n

('dropout4', layers.DropoutLayer),

", "rawMarkdown": "", "votes": null }, { "id": "65263", "postDate": "03/02/2015 16:56:06", "content": "

ah sorry, I looked to quickly. I thought it was supposed this way, I will try to reverse it and let you know. 

", "rawMarkdown": "", "votes": null }, { "id": "65272", "postDate": "03/02/2015 17:55:41", "content": "

I used Lasagne + NoLearn to quickly get a score < 1.0. Pretty standard CNN with several convolutional + max pooling layers followed by fully connected dropout layers. I used RMSprop to train it.

", "rawMarkdown": "", "votes": null }, { "id": "65278", "postDate": "03/02/2015 18:56:15", "content": "

Can someone post a simple code for the prediction? I wrote a training code using CNN similar to the examples  following the blog of Nouri

\n

http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/.

\n

With the first tests, I found on a test set which consist of a 30% of the shuffled training set a MCL of a value of 1.5~1.6.

\n

I saved the model on disk  and then load the model to make prediction.

\n

I'm attaching my code for prediction.

\n

When I post the prediction on LB I found values~10!!!

\n

Is there something wrong in my prediction code?

", "rawMarkdown": "", "votes": null }, { "id": "65288", "postDate": "03/02/2015 20:39:24", "content": "

Forgot my post| I realized that I may have swapped the classes names in the submission file!

", "rawMarkdown": "", "votes": null }, { "id": "65295", "postDate": "03/02/2015 21:36:08", "content": "

Mike1886

\n

How many hour it took you to train the model you posted?

\n

Do you still using lasagne+nolearn or you switched to Caffe or other frameworks?

", "rawMarkdown": "", "votes": null }, { "id": "65306", "postDate": "03/03/2015 00:52:08", "content": "

Hi Mike, 

\n

Do you get logloss ~0.75 just for a single model or is it with  model averaging? I'm kind of stuck in ~0.83 and was wondering whether I have to do model averaging to go beyond that? 

\n

Can anyone suggest me some resources about model averaging?

", "rawMarkdown": "", "votes": null }, { "id": "65309", "postDate": "03/03/2015 01:35:13", "content": "

@Akile: I get around 0.78 with my best single model and then when averaging three models together I get around 0.75. 

\n

The way I am doing the averaging (which may not be the best) is to simply average the predictions together. So imagine you had an array,df, that contained your model predictions. Just do something like

\n

combined = (np.array(df) + np.array(df1) + np.array(df2)) / 3

", "rawMarkdown": "", "votes": null }, { "id": "65310", "postDate": "03/03/2015 01:49:05", "content": "

Please can u check whats wrong with my code. I am trying to use the simplest one layer network

\n

pixelNo = 10
imageSize = pixelNo*pixelNo
rowsNo = 30336
l = range(rowsNo)

\n


def readTrainingImagesCNN(imagesPath):
directory_names = list(set(glob.glob(os.path.join(imagesPath, "train", "*"))\\
).difference(set(glob.glob(os.path.join(imagesPath, "train", "*.*")))))

\n

label = 0
images = list()
header = list()
targets = list()

\n

for folder in directory_names:
header.append(folder.split("\\\\")[-1])
for fileNameDir in os.walk(folder):
for fileName in fileNameDir[2]:
if fileName[-4:] != ".jpg":
continue
imageReading = "{0}{1}{2}".format(fileNameDir[0], os.sep, fileName)
images.append(imread(imageReading, as_grey=True))
targets.append(label)
label += 1
header = ",".join(header)
return images, targets, header

\n\n

def imageResizingCNN(images, size):
imagesResized = list()
for image in images:
imagesResized.append(resize(image, (size, size)))
del images
return imagesResized

\n\n

print "Image reading"
t = time()
images, targets, header = readTrainingImagesCNN(root)
print "Time: ", round(time()-t, 3), "\\n"

\n\n

print "Image resizing"
t = time()
imagesResized = imageResizingCNN(images, pixelNo)
print "Time: ", round(time()-t, 3), "\\n"

\n

del images

\n\n

print "Reshaping Images"
imageFeatures = np.zeros((rowsNo, imageSize), dtype=float)
t = time()
for i in l:
imageFeatures[i, :] = np.reshape(imagesResized[i], (1, imageSize))
print "Time: ", round(time()-t, 3), "\\n"

\n


features = imageFeatures
targets = np.asarray(targets)
del imagesResized, imageFeatures

\n

print "features ", features.shape
print "targets", targets.shape

\n


netOneLayer = NeuralNet(
layers=[
('input', layers.InputLayer),
('hidden', layers.DenseLayer),
('output', layers.DenseLayer),
],

\n

input_shape=(None, imageSize),
hidden_num_units=100,
output_nonlinearity=None,
output_num_units=121,

\n

update=nesterov_momentum,
update_learning_rate=0.01,
update_momentum=0.9,

\n

regression=False,
max_epochs=100,
verbose=1,
)

\n

features2 = features[0:1000, :]
targets2 = targets[0:1000]
netOneLayer.fit(features2, targets2)

\n


print "Total time: ", round(time()-totalTime, 3)

\n\n

The code is running well but the neural output is the following 

\n

I don't know where the bug

\n\n

InputLayer (None, 100) produces 100 outputs
DenseLayer (None, 100) produces 100 outputs
DenseLayer (None, 121) produces 121 outputs

\n

Epoch | Train loss | Valid loss | Train / Val | Valid acc | Dur
--------|--------------|--------------|---------------|-------------|-------
1 | nan | nan | nan | 5.86% | 0.0s
2 | nan | nan | nan | 5.86% | 0.0s
3 | nan | nan | nan | 5.86% | 0.0s
4 | nan | nan | nan | 5.86% | 0.0s
5 | nan | nan | nan | 5.86% | 0.0s
6 | nan | nan | nan | 5.86% | 0.0s
7 | nan | nan | nan | 5.86% | 0.0s
8 | nan | nan | nan | 31.25% | 0.0s
9 | nan | nan | nan | 30.86% | 0.0s
10 | nan | nan | nan | 30.86% | 0.0s
11 | nan | nan | nan | 30.86% | 0.0s
12 | nan | nan | nan | 30.86% | 0.0s
13 | nan | nan | nan | 30.86% | 0.0s
14 | nan | nan | nan | 30.86% | 0.0s
15 | nan | nan | nan | 30.86% | 0.0s
16 | nan | nan | nan | 30.86% | 0.0s
17 | nan | nan | nan | 30.86% | 0.0s
18 | nan | nan | nan | 30.86% | 0.0s
19 | nan | nan | nan | 30.86% | 0.0s
20 | nan | nan | nan | 30.86% | 0.0s
21 | nan | nan | nan | 30.86% | 0.0s
22 | nan | nan | nan | 30.86% | 0.0s
23 | nan | nan | nan | 30.86% | 0.0s
24 | nan | nan | nan | 30.86% | 0.0s
25 | nan | nan | nan | 30.86% | 0.0s
26 | nan | nan | nan | 30.86% | 0.0s
27 | nan | nan | nan | 30.86% | 0.0s
28 | nan | nan | nan | 30.86% | 0.0s
29 | nan | nan | nan | 30.86% | 0.0s
30 | nan | nan | nan | 30.86% | 0.0s
31 | nan | nan | nan | 30.86% | 0.0s
32 | nan | nan | nan | 30.86% | 0.0s
33 | nan | nan | nan | 30.86% | 0.0s
34 | nan | nan | nan | 30.86% | 0.0s
35 | nan | nan | nan | 30.86% | 0.0s
36 | nan | nan | nan | 30.86% | 0.0s
37 | nan | nan | nan | 30.86% | 0.0s
38 | nan | nan | nan | 30.86% | 0.0s
39 | nan | nan | nan | 30.86% | 0.0s
40 | nan | nan | nan | 30.86% | 0.0s
41 | nan | nan | nan | 30.86% | 0.0s
42 | nan | nan | nan | 30.86% | 0.0s
43 | nan | nan | nan | 30.86% | 0.0s
44 | nan | nan | nan | 30.86% | 0.0s
45 | nan | nan | nan | 30.86% | 0.0s
46 | nan | nan | nan | 30.86% | 0.0s
47 | nan | nan | nan | 30.86% | 0.0s
48 | nan | nan | nan | 30.86% | 0.0s
49 | nan | nan | nan | 30.86% | 0.0s
50 | nan | nan | nan | 30.86% | 0.0s
51 | nan | nan | nan | 30.86% | 0.0s
52 | nan | nan | nan | 30.86% | 0.0s
53 | nan | nan | nan | 30.86% | 0.0s
54 | nan | nan | nan | 30.86% | 0.0s
55 | nan | nan | nan | 30.86% | 0.0s
56 | nan | nan | nan | 30.86% | 0.0s
57 | nan | nan | nan | 30.86% | 0.0s
58 | nan | nan | nan | 30.86% | 0.0s
59 | nan | nan | nan | 30.86% | 0.0s
60 | nan | nan | nan | 30.86% | 0.0s
61 | nan | nan | nan | 30.86% | 0.0s
62 | nan | nan | nan | 30.86% | 0.0s
63 | nan | nan | nan | 30.86% | 0.0s
64 | nan | nan | nan | 30.86% | 0.0s
65 | nan | nan | nan | 30.86% | 0.0s
66 | nan | nan | nan | 30.86% | 0.0s
67 | nan | nan | nan | 30.86% | 0.0s
68 | nan | nan | nan | 30.86% | 0.0s
69 | nan | nan | nan | 30.86% | 0.0s
70 | nan | nan | nan | 30.86% | 0.0s
71 | nan | nan | nan | 30.86% | 0.0s
72 | nan | nan | nan | 30.86% | 0.0s
73 | nan | nan | nan | 30.86% | 0.0s
74 | nan | nan | nan | 30.86% | 0.0s
75 | nan | nan | nan | 30.86% | 0.0s
76 | nan | nan | nan | 30.86% | 0.0s
77 | nan | nan | nan | 30.86% | 0.0s
78 | nan | nan | nan | 30.86% | 0.0s
79 | nan | nan | nan | 30.86% | 0.0s
80 | nan | nan | nan | 30.86% | 0.0s
81 | nan | nan | nan | 30.86% | 0.0s
82 | nan | nan | nan | 30.86% | 0.0s
83 | nan | nan | nan | 30.86% | 0.0s
84 | nan | nan | nan | 30.86% | 0.0s
85 | nan | nan | nan | 30.86% | 0.0s
86 | nan | nan | nan | 30.86% | 0.0s
87 | nan | nan | nan | 30.86% | 0.0s
88 | nan | nan | nan | 30.86% | 0.0s
89 | nan | nan | nan | 30.86% | 0.0s
90 | nan | nan | nan | 30.86% | 0.0s
91 | nan | nan | nan | 30.86% | 0.0s
92 | nan | nan | nan | 30.86% | 0.0s
93 | nan | nan | nan | 30.86% | 0.0s
94 | nan | nan | nan | 30.86% | 0.0s
95 | nan | nan | nan | 30.86% | 0.0s
96 | nan | nan | nan | 30.86% | 0.0s
97 | nan | nan | nan | 30.86% | 0.0s
98 | nan | nan | nan | 30.86% | 0.0s
99 | nan | nan | nan | 30.86% | 0.0s
100 | nan | nan | nan | 30.86% | 0.0s
Total time: 21.955

", "rawMarkdown": "", "votes": null }, { "id": "65311", "postDate": "03/03/2015 01:55:11", "content": "

It may be to simple, try something more realistic like 60 pixels and a bit more complex network. I had issues like this when things were way off from ideal. 

", "rawMarkdown": "", "votes": null }, { "id": "65313", "postDate": "03/03/2015 02:27:06", "content": "

I am planning to do that but I need to make this network work to make sure am in the right direction

\n

I think may be the problem is with targets coz it suppose to be 121 but am sending it as int

", "rawMarkdown": "", "votes": null }, { "id": "65314", "postDate": "03/03/2015 02:35:25", "content": "

@Data-Lover, if you want your network to output probabilities, you should probably have a softmax (or similar) non-linearity on your output layer.

", "rawMarkdown": "", "votes": null }, { "id": "65335", "postDate": "03/03/2015 14:09:12", "content": "

Thanks Mike. I guess it better to start model averaging now. I tried to use large layers as you told. I couldn't get any improvement by doing that, may be I should try some more configurations. 

", "rawMarkdown": "", "votes": null }, { "id": "65353", "postDate": "03/03/2015 18:24:36", "content": "

This is a very similar version of CNN based on  no-learn, lasagne as Mike's one

\n

https://github.com/elenacuoco/Bowl

", "rawMarkdown": "", "votes": null }, { "id": "65357", "postDate": "03/03/2015 19:23:11", "content": "

@Elena, what logloss score are you currently getting for your single best model?

", "rawMarkdown": "", "votes": null }, { "id": "65361", "postDate": "03/03/2015 21:07:41", "content": "

[quote=mike1886;65357]

\n

@Elena, what logloss score are you currently getting for your single best model?

\n

[/quote]

\n

For the moment ~1.2

", "rawMarkdown": "", "votes": null }, { "id": "65363", "postDate": "03/03/2015 21:59:49", "content": "

Thanks Elena for sharing

\n

just one note don't do image reading and feature extraction at the same time because it make the program slow. read all images first then do feature extraction this will increase speed more than 10 times

", "rawMarkdown": "", "votes": null }, { "id": "65372", "postDate": "03/04/2015 01:05:35", "content": "

Ok guys model averaging would definitely help. I averaged over four models and got to 0.785861 from 0.830915

", "rawMarkdown": "", "votes": null }, { "id": "65380", "postDate": "03/04/2015 04:45:48", "content": "

@mike1886, where does your Maxout layer come from ? It seems that lasagne.layers have only provide the definition for dense, conv, maxpooling. And what does such maxout layer do ?

", "rawMarkdown": "", "votes": null }, { "id": "65428", "postDate": "03/04/2015 18:53:53", "content": "

Add this line of code above your model definition:
Maxout = layers.pool.FeaturePoolLayer

\n

http://stats.stackexchange.com/questions/129698/what-is-maxout-in-neural-network

", "rawMarkdown": "", "votes": null }, { "id": "65505", "postDate": "03/05/2015 13:05:30", "content": "

Many thanks Mike1886 and Elena Cuocu

\n

I got my first CNN working. it is far from being good but at least it is working  ,)

", "rawMarkdown": "", "votes": null }, { "id": "3482188", "postDate": "06/26/2026 22:37:55", "content": "

Spot on! This context underscores why automated plankton classification and tracking is so important. Historically, Kaggle has hosted amazing competitions focused on plankton imagery. Building accurate Computer Vision models to monitor these populations can directly help marine biologists detect ecological upheaval before it's too late. High-impact domain!

", "rawMarkdown": "Spot on! This context underscores why automated plankton classification and tracking is so important. Historically, Kaggle has hosted amazing competitions focused on plankton imagery. Building accurate Computer Vision models to monitor these populations can directly help marine biologists detect ecological upheaval before it's too late. High-impact domain!", "votes": null } ], "comments": [ { "id": 3482188, "author_name": "", "author_url": "", "post_date": "06/26/2026 22:37:55", "content": "

Spot on! This context underscores why automated plankton classification and tracking is so important. Historically, Kaggle has hosted amazing competitions focused on plankton imagery. Building accurate Computer Vision models to monitor these populations can directly help marine biologists detect ecological upheaval before it's too late. High-impact domain!

", "votes": null, "replies": [] }, { "id": 65243, "author_name": "elenacuoco", "author_url": "", "post_date": "03/02/2015 15:20:24", "content": "

Hi,

\n

I was trying to use lasagne+nolearn, but the results I obtained are very bad! moreover the values I found for MCL with the model on a test set are very different from the ones on the LB.

", "votes": null, "replies": [] }, { "id": 65246, "author_name": "msegala", "author_url": "", "post_date": "03/02/2015 15:39:16", "content": "

I am using a lasagne + nolearn setup, you can check it out here:

\n

https://github.com/msegala/Kaggle-National_Data_Science_Bowl

", "votes": null, "replies": [] }, { "id": 65247, "author_name": "abhishek", "author_url": "", "post_date": "03/02/2015 15:48:27", "content": "

[quote=mike1886;65246]

\n

I am using a lasagne + nolearn setup, you can check it out here:

\n

https://github.com/msegala/Kaggle-National_Data_Science_Bowl

\n

[/quote]

\n

is this your code? what is the logloss do you get with this?

", "votes": null, "replies": [] }, { "id": 65250, "author_name": "msegala", "author_url": "", "post_date": "03/02/2015 15:51:30", "content": "

This is not the final code I use for submission, I have spent a long time tweaking this for performance. 

\n

My current log loss is ~0.75. 

", "votes": null, "replies": [] }, { "id": 65253, "author_name": "elenacuoco", "author_url": "", "post_date": "03/02/2015 16:29:02", "content": "

Did you find overfitting? I tried with a similar net, without batchiterator, and Maxout, But I found a lot of overfitting.

", "votes": null, "replies": [] }, { "id": 65254, "author_name": "msegala", "author_url": "", "post_date": "03/02/2015 16:36:22", "content": "

I was able to get pretty good results by training a deep network with large layers. Of course there is still overfitting but using dropout and maxout layers helps a lot. 

", "votes": null, "replies": [] }, { "id": 65255, "author_name": "chenglongchen", "author_url": "", "post_date": "03/02/2015 16:36:46", "content": "

I am just curious here, since Maxout was proposed as an activation function to enhance dropout's model averaging accuracy, why would you put dropout before Maxout? Did you observe any improvement than putting dropout after Maxout?

", "votes": null, "replies": [] }, { "id": 65258, "author_name": "msegala", "author_url": "", "post_date": "03/02/2015 16:43:00", "content": "

@yr, do you mean why did I put Maxout before dropout?

\n

layers=[
('input', layers.InputLayer),
('conv1', Conv2DLayer),
('pool1', MaxPool2DLayer),
('dropout1', layers.DropoutLayer),

\n

.....

\n

right now my maxout layers feeds into my drouput layer

", "votes": null, "replies": [] }, { "id": 65262, "author_name": "chenglongchen", "author_url": "", "post_date": "03/02/2015 16:52:13", "content": "

Yes. The part you quoted is max-pooling (across spatial). I mean here:

\n

('hidden4', layers.DenseLayer),
('dropout4', layers.DropoutLayer),
('maxout4', Maxout),
('hidden5', layers.DenseLayer),
('dropout5', layers.DropoutLayer),
('maxout5', Maxout),
('output', layers.DenseLayer),

\n

maxout (across feature map) is after dropout. If you treat maxout as activation function, as I understand, it should be something like:

\n

('hidden4', layers.DenseLayer),
('maxout4', Maxout),

\n

('dropout4', layers.DropoutLayer),

", "votes": null, "replies": [] }, { "id": 65263, "author_name": "msegala", "author_url": "", "post_date": "03/02/2015 16:56:06", "content": "

ah sorry, I looked to quickly. I thought it was supposed this way, I will try to reverse it and let you know. 

", "votes": null, "replies": [] }, { "id": 65272, "author_name": "albeam", "author_url": "", "post_date": "03/02/2015 17:55:41", "content": "

I used Lasagne + NoLearn to quickly get a score < 1.0. Pretty standard CNN with several convolutional + max pooling layers followed by fully connected dropout layers. I used RMSprop to train it.

", "votes": null, "replies": [] }, { "id": 65278, "author_name": "elenacuoco", "author_url": "", "post_date": "03/02/2015 18:56:15", "content": "

Can someone post a simple code for the prediction? I wrote a training code using CNN similar to the examples  following the blog of Nouri

\n

http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/.

\n

With the first tests, I found on a test set which consist of a 30% of the shuffled training set a MCL of a value of 1.5~1.6.

\n

I saved the model on disk  and then load the model to make prediction.

\n

I'm attaching my code for prediction.

\n

When I post the prediction on LB I found values~10!!!

\n

Is there something wrong in my prediction code?

", "votes": null, "replies": [] }, { "id": 65288, "author_name": "elenacuoco", "author_url": "", "post_date": "03/02/2015 20:39:24", "content": "

Forgot my post| I realized that I may have swapped the classes names in the submission file!

", "votes": null, "replies": [] }, { "id": 65295, "author_name": "", "author_url": "", "post_date": "03/02/2015 21:36:08", "content": "

Mike1886

\n

How many hour it took you to train the model you posted?

\n

Do you still using lasagne+nolearn or you switched to Caffe or other frameworks?

", "votes": null, "replies": [] }, { "id": 65306, "author_name": "akilaw", "author_url": "", "post_date": "03/03/2015 00:52:08", "content": "

Hi Mike, 

\n

Do you get logloss ~0.75 just for a single model or is it with  model averaging? I'm kind of stuck in ~0.83 and was wondering whether I have to do model averaging to go beyond that? 

\n

Can anyone suggest me some resources about model averaging?

", "votes": null, "replies": [] }, { "id": 65309, "author_name": "msegala", "author_url": "", "post_date": "03/03/2015 01:35:13", "content": "

@Akile: I get around 0.78 with my best single model and then when averaging three models together I get around 0.75. 

\n

The way I am doing the averaging (which may not be the best) is to simply average the predictions together. So imagine you had an array,df, that contained your model predictions. Just do something like

\n

combined = (np.array(df) + np.array(df1) + np.array(df2)) / 3

", "votes": null, "replies": [] }, { "id": 65310, "author_name": "", "author_url": "", "post_date": "03/03/2015 01:49:05", "content": "

Please can u check whats wrong with my code. I am trying to use the simplest one layer network

\n

pixelNo = 10
imageSize = pixelNo*pixelNo
rowsNo = 30336
l = range(rowsNo)

\n


def readTrainingImagesCNN(imagesPath):
directory_names = list(set(glob.glob(os.path.join(imagesPath, "train", "*"))\\
).difference(set(glob.glob(os.path.join(imagesPath, "train", "*.*")))))

\n

label = 0
images = list()
header = list()
targets = list()

\n

for folder in directory_names:
header.append(folder.split("\\\\")[-1])
for fileNameDir in os.walk(folder):
for fileName in fileNameDir[2]:
if fileName[-4:] != ".jpg":
continue
imageReading = "{0}{1}{2}".format(fileNameDir[0], os.sep, fileName)
images.append(imread(imageReading, as_grey=True))
targets.append(label)
label += 1
header = ",".join(header)
return images, targets, header

\n\n

def imageResizingCNN(images, size):
imagesResized = list()
for image in images:
imagesResized.append(resize(image, (size, size)))
del images
return imagesResized

\n\n

print "Image reading"
t = time()
images, targets, header = readTrainingImagesCNN(root)
print "Time: ", round(time()-t, 3), "\\n"

\n\n

print "Image resizing"
t = time()
imagesResized = imageResizingCNN(images, pixelNo)
print "Time: ", round(time()-t, 3), "\\n"

\n

del images

\n\n

print "Reshaping Images"
imageFeatures = np.zeros((rowsNo, imageSize), dtype=float)
t = time()
for i in l:
imageFeatures[i, :] = np.reshape(imagesResized[i], (1, imageSize))
print "Time: ", round(time()-t, 3), "\\n"

\n


features = imageFeatures
targets = np.asarray(targets)
del imagesResized, imageFeatures

\n

print "features ", features.shape
print "targets", targets.shape

\n


netOneLayer = NeuralNet(
layers=[
('input', layers.InputLayer),
('hidden', layers.DenseLayer),
('output', layers.DenseLayer),
],

\n

input_shape=(None, imageSize),
hidden_num_units=100,
output_nonlinearity=None,
output_num_units=121,

\n

update=nesterov_momentum,
update_learning_rate=0.01,
update_momentum=0.9,

\n

regression=False,
max_epochs=100,
verbose=1,
)

\n

features2 = features[0:1000, :]
targets2 = targets[0:1000]
netOneLayer.fit(features2, targets2)

\n


print "Total time: ", round(time()-totalTime, 3)

\n\n

The code is running well but the neural output is the following 

\n

I don't know where the bug

\n\n

InputLayer (None, 100) produces 100 outputs
DenseLayer (None, 100) produces 100 outputs
DenseLayer (None, 121) produces 121 outputs

\n

Epoch | Train loss | Valid loss | Train / Val | Valid acc | Dur
--------|--------------|--------------|---------------|-------------|-------
1 | nan | nan | nan | 5.86% | 0.0s
2 | nan | nan | nan | 5.86% | 0.0s
3 | nan | nan | nan | 5.86% | 0.0s
4 | nan | nan | nan | 5.86% | 0.0s
5 | nan | nan | nan | 5.86% | 0.0s
6 | nan | nan | nan | 5.86% | 0.0s
7 | nan | nan | nan | 5.86% | 0.0s
8 | nan | nan | nan | 31.25% | 0.0s
9 | nan | nan | nan | 30.86% | 0.0s
10 | nan | nan | nan | 30.86% | 0.0s
11 | nan | nan | nan | 30.86% | 0.0s
12 | nan | nan | nan | 30.86% | 0.0s
13 | nan | nan | nan | 30.86% | 0.0s
14 | nan | nan | nan | 30.86% | 0.0s
15 | nan | nan | nan | 30.86% | 0.0s
16 | nan | nan | nan | 30.86% | 0.0s
17 | nan | nan | nan | 30.86% | 0.0s
18 | nan | nan | nan | 30.86% | 0.0s
19 | nan | nan | nan | 30.86% | 0.0s
20 | nan | nan | nan | 30.86% | 0.0s
21 | nan | nan | nan | 30.86% | 0.0s
22 | nan | nan | nan | 30.86% | 0.0s
23 | nan | nan | nan | 30.86% | 0.0s
24 | nan | nan | nan | 30.86% | 0.0s
25 | nan | nan | nan | 30.86% | 0.0s
26 | nan | nan | nan | 30.86% | 0.0s
27 | nan | nan | nan | 30.86% | 0.0s
28 | nan | nan | nan | 30.86% | 0.0s
29 | nan | nan | nan | 30.86% | 0.0s
30 | nan | nan | nan | 30.86% | 0.0s
31 | nan | nan | nan | 30.86% | 0.0s
32 | nan | nan | nan | 30.86% | 0.0s
33 | nan | nan | nan | 30.86% | 0.0s
34 | nan | nan | nan | 30.86% | 0.0s
35 | nan | nan | nan | 30.86% | 0.0s
36 | nan | nan | nan | 30.86% | 0.0s
37 | nan | nan | nan | 30.86% | 0.0s
38 | nan | nan | nan | 30.86% | 0.0s
39 | nan | nan | nan | 30.86% | 0.0s
40 | nan | nan | nan | 30.86% | 0.0s
41 | nan | nan | nan | 30.86% | 0.0s
42 | nan | nan | nan | 30.86% | 0.0s
43 | nan | nan | nan | 30.86% | 0.0s
44 | nan | nan | nan | 30.86% | 0.0s
45 | nan | nan | nan | 30.86% | 0.0s
46 | nan | nan | nan | 30.86% | 0.0s
47 | nan | nan | nan | 30.86% | 0.0s
48 | nan | nan | nan | 30.86% | 0.0s
49 | nan | nan | nan | 30.86% | 0.0s
50 | nan | nan | nan | 30.86% | 0.0s
51 | nan | nan | nan | 30.86% | 0.0s
52 | nan | nan | nan | 30.86% | 0.0s
53 | nan | nan | nan | 30.86% | 0.0s
54 | nan | nan | nan | 30.86% | 0.0s
55 | nan | nan | nan | 30.86% | 0.0s
56 | nan | nan | nan | 30.86% | 0.0s
57 | nan | nan | nan | 30.86% | 0.0s
58 | nan | nan | nan | 30.86% | 0.0s
59 | nan | nan | nan | 30.86% | 0.0s
60 | nan | nan | nan | 30.86% | 0.0s
61 | nan | nan | nan | 30.86% | 0.0s
62 | nan | nan | nan | 30.86% | 0.0s
63 | nan | nan | nan | 30.86% | 0.0s
64 | nan | nan | nan | 30.86% | 0.0s
65 | nan | nan | nan | 30.86% | 0.0s
66 | nan | nan | nan | 30.86% | 0.0s
67 | nan | nan | nan | 30.86% | 0.0s
68 | nan | nan | nan | 30.86% | 0.0s
69 | nan | nan | nan | 30.86% | 0.0s
70 | nan | nan | nan | 30.86% | 0.0s
71 | nan | nan | nan | 30.86% | 0.0s
72 | nan | nan | nan | 30.86% | 0.0s
73 | nan | nan | nan | 30.86% | 0.0s
74 | nan | nan | nan | 30.86% | 0.0s
75 | nan | nan | nan | 30.86% | 0.0s
76 | nan | nan | nan | 30.86% | 0.0s
77 | nan | nan | nan | 30.86% | 0.0s
78 | nan | nan | nan | 30.86% | 0.0s
79 | nan | nan | nan | 30.86% | 0.0s
80 | nan | nan | nan | 30.86% | 0.0s
81 | nan | nan | nan | 30.86% | 0.0s
82 | nan | nan | nan | 30.86% | 0.0s
83 | nan | nan | nan | 30.86% | 0.0s
84 | nan | nan | nan | 30.86% | 0.0s
85 | nan | nan | nan | 30.86% | 0.0s
86 | nan | nan | nan | 30.86% | 0.0s
87 | nan | nan | nan | 30.86% | 0.0s
88 | nan | nan | nan | 30.86% | 0.0s
89 | nan | nan | nan | 30.86% | 0.0s
90 | nan | nan | nan | 30.86% | 0.0s
91 | nan | nan | nan | 30.86% | 0.0s
92 | nan | nan | nan | 30.86% | 0.0s
93 | nan | nan | nan | 30.86% | 0.0s
94 | nan | nan | nan | 30.86% | 0.0s
95 | nan | nan | nan | 30.86% | 0.0s
96 | nan | nan | nan | 30.86% | 0.0s
97 | nan | nan | nan | 30.86% | 0.0s
98 | nan | nan | nan | 30.86% | 0.0s
99 | nan | nan | nan | 30.86% | 0.0s
100 | nan | nan | nan | 30.86% | 0.0s
Total time: 21.955

", "votes": null, "replies": [] }, { "id": 65311, "author_name": "msegala", "author_url": "", "post_date": "03/03/2015 01:55:11", "content": "

It may be to simple, try something more realistic like 60 pixels and a bit more complex network. I had issues like this when things were way off from ideal. 

", "votes": null, "replies": [] }, { "id": 65313, "author_name": "", "author_url": "", "post_date": "03/03/2015 02:27:06", "content": "

I am planning to do that but I need to make this network work to make sure am in the right direction

\n

I think may be the problem is with targets coz it suppose to be 121 but am sending it as int

", "votes": null, "replies": [] }, { "id": 65314, "author_name": "hendrikw", "author_url": "", "post_date": "03/03/2015 02:35:25", "content": "

@Data-Lover, if you want your network to output probabilities, you should probably have a softmax (or similar) non-linearity on your output layer.

", "votes": null, "replies": [] }, { "id": 65335, "author_name": "akilaw", "author_url": "", "post_date": "03/03/2015 14:09:12", "content": "

Thanks Mike. I guess it better to start model averaging now. I tried to use large layers as you told. I couldn't get any improvement by doing that, may be I should try some more configurations. 

", "votes": null, "replies": [] }, { "id": 65353, "author_name": "elenacuoco", "author_url": "", "post_date": "03/03/2015 18:24:36", "content": "

This is a very similar version of CNN based on  no-learn, lasagne as Mike's one

\n

https://github.com/elenacuoco/Bowl

", "votes": null, "replies": [] }, { "id": 65357, "author_name": "msegala", "author_url": "", "post_date": "03/03/2015 19:23:11", "content": "

@Elena, what logloss score are you currently getting for your single best model?

", "votes": null, "replies": [] }, { "id": 65361, "author_name": "elenacuoco", "author_url": "", "post_date": "03/03/2015 21:07:41", "content": "

[quote=mike1886;65357]

\n

@Elena, what logloss score are you currently getting for your single best model?

\n

[/quote]

\n

For the moment ~1.2

", "votes": null, "replies": [] }, { "id": 65363, "author_name": "", "author_url": "", "post_date": "03/03/2015 21:59:49", "content": "

Thanks Elena for sharing

\n

just one note don't do image reading and feature extraction at the same time because it make the program slow. read all images first then do feature extraction this will increase speed more than 10 times

", "votes": null, "replies": [] }, { "id": 65372, "author_name": "akilaw", "author_url": "", "post_date": "03/04/2015 01:05:35", "content": "

Ok guys model averaging would definitely help. I averaged over four models and got to 0.785861 from 0.830915

", "votes": null, "replies": [] }, { "id": 65380, "author_name": "pengpai", "author_url": "", "post_date": "03/04/2015 04:45:48", "content": "

@mike1886, where does your Maxout layer come from ? It seems that lasagne.layers have only provide the definition for dense, conv, maxpooling. And what does such maxout layer do ?

", "votes": null, "replies": [] }, { "id": 65428, "author_name": "stevenreitsma", "author_url": "", "post_date": "03/04/2015 18:53:53", "content": "

Add this line of code above your model definition:
Maxout = layers.pool.FeaturePoolLayer

\n

http://stats.stackexchange.com/questions/129698/what-is-maxout-in-neural-network

", "votes": null, "replies": [] }, { "id": 65505, "author_name": "", "author_url": "", "post_date": "03/05/2015 13:05:30", "content": "

Many thanks Mike1886 and Elena Cuocu

\n

I got my first CNN working. it is far from being good but at least it is working  ,)

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "65213": "", "65243": "", "65246": "", "65247": "", "65250": "", "65253": "", "65254": "", "65255": "", "65258": "", "65262": "", "65263": "", "65272": "", "65278": "", "65288": "", "65295": "", "65306": "", "65309": "", "65310": "", "65311": "", "65313": "", "65314": "", "65335": "", "65353": "", "65357": "", "65361": "", "65363": "", "65372": "", "65380": "", "65428": "", "65505": "", "3482188": "Spot on! This context underscores why automated plankton classification and tracking is so important. Historically, Kaggle has hosted amazing competitions focused on plankton imagery. Building accurate Computer Vision models to monitor these populations can directly help marine biologists detect ecological upheaval before it's too late. High-impact domain!" }, "source": "meta" }